Metrics that Matter

Firstly apologies to those of you that aren't quite as geeky about the numbers of recruitment as I am, I'll be back to ranting about the misuse of Pinterest for recruitment soon.  As I promised previously I wanted to give a little insight into those individual statistics that go to make up the metrics I use (or those I like to see) when recruiting.  Gathering this information isn't about producing a report simply to prove effort.  It is only the most unengaged stakeholder who can take solace in knowing that candidate and recruiters are somewhere in the building...  Gathering this seemingly disparate data points, in a consistent format (more on this later) is about creating a dataset that is alive and available to answer questions that may arise later... regardless of what those questions might be...

So what are the basics?  Those elements that you have to capture and whether that's in an ATS, a spreadsheet or typed up and popped in one of those old-timey filing cabinets.

Name, gender - All of your candidates will have a name, even if they have just one like a Brazilian footballer or Madonna they still have a name.  You should decide in advance on a format for writing these names capitalization, hyphenation etc this is to facilitate later use of names in mail merge or batch operations - candidates don't want to receive an email for "MAtthw BUCKLAND" so spell it right and you won't have to change 1000 name spellings at a later date.

Gender as a metric is of particular interest to me.  I've always worked in technical recruitment and it's an industry where females and transgendered people are under represented.  This metric can be combined with source to know which sources are productive for diversity goals and with the date ranges to know if and where candidates excel or fall down in your recruitment process.  This can facilitate later discussion and provide great evidence for changing processes later.

Role - the role the candidate applies for...this one really is basic to be able to slice numbers of total applicants by role, I hope everyone does at least this.  If not I guess they just tie CV's to the back of kittens and let them lose...

Gate Dates - Not Match.com for Farmers, this is the notation of the dates that a candidate moves through the hiring process.  Date of Application, Date of Phone Screen, Date of First On-site Interview all the way through to Date of Offer, Verbal Acceptance, Written Acceptance and Start Date.  GET ALL THE DATES!  So why track all these dates?  These date ranges can be used to answer a multitude of questions.  With values in these ranges reports can be compiled that show total length of process, drop-out ratios, expose bottlenecks in the process, expose waiting times and hold-ups, track notice periods... basically everything.  The date ranges and days elapsed are the bread and butter of recruitment reporting.  Do you currently know the average length of your interview process?  Does it vary a great deal?  Why is that? It's the interrogation of these dates that will give you those answers and perhaps when you have enough of an historical dataset predict time to hire of for future capacity planning... all for putting some dates in a spreadsheet or clicking those little calendar icons in your swanky new ATS!  Brilliant!

Source - Again a simple one, but it bears repeating, the source is how the candidate arrived in your recruitment process.  This should break down the source into broad categories that can tell at a glance what is a good source (a lot of quality candidates) a weak source (few candidates) or a bad source (lots of irrelevant candidates).  Example sources should differentiate between the "How" of the source too e.g. not just "LinkedIn" correct reporting should be "LinkedIn Search" and "LinkedIn Advert", this will enable you to distinguish between an active candidate application versus a directly sourced passive candidate.

Secondary Source - Some sources may require extra insight, you might need to know more for a later report.  If you have a primary source as "Event" this could be the particular Meetup, conference or pub you met them at.  A primary source of "Agency" might have the secondary source of the agency's name, for referrals it could be the refering employees name... remember they all have one...

Country of Residence -  I also like to track where a particular candidate is based this has multiple reasons, one might be for immigration purposes to highlight to internal teams where visa constraints may be an issue or delay a start date, a second reason could be to track individual sourcing efforts from a particular country... best of all most reports can include a lovely map showing where candidates came from...the prettiest metric :)

Contact Details - This should be the most obvious but still I see people finding value in the wrong things.  We all should know that a direct contact is better than a message delivered through a third party.  Simply put a telephone call or a direct email address are better than a LinkedIn Inmail.  If you only use LinkedIn to contact candidates and leaving it at that you're doing it wrong.

Last Employer - Want to know your pulling power?  Doing some competitor analysis?  Then you'll need to know where your candidates are currently working.

Recruiter - Who found the candidate and who is shepherding them through the process?  It's important that I'm not noting this to provide a productivity report for managerial consumption.  Unless all the members of the team are hiring for the same role in the same geography there is little to be gained from a direct comparison.  Raw numbers alone, stripped of context are not an aid.  They are a great example of one of the great flaws in gathering data - quantity isn't always preferable to quality.

Date of Last Contact - One of the consistent complaints and killers of candidate experience is the lack of timely feedback.  Even giving a candidate a short "no news yet" will pay dividends if you later wish to offer against a less communicative rival.  To overstate, if you track the last date you contacted a list of candidates you can very easily automate an email letting them know what's going on and when they'll get feedback.

Status - Decide on a glossary of terms that best fit your process, get the hiring managers involved in this process too.  Phone Screen, First Interview, Second Interview..etc.  Have as many of these as you feel you need.  Counting each of these each week will give you a very rapid view of the overall pipeline.  Hiring managers will love this, full on warm and fuzzy feelings.  Too often the work of the recruiter can look like a dark art - they go and stare at a screen and people magically appear for interviews - a weekly pipeline report just illustrating the numbers of potential candidates at each stage will calm even the most rabid of hiring manager.

There are more things to track of course and when real value can be derived from the collation of this data you'll find it quite addictive.  Best of all, when you start to move on from thinking the collection of data is just to describe the current status to instead thinking that you are creating a living, growing dataset that can be used to answer questions that haven't yet been thought of... you'll start to see why metrics really do matter.

The Itchy Security Blanket of Recruitment Metrics

The rise of more intuitive technology enabling the recruitment process has made for an interesting corollary - a rise in an organisation's ability to collect and report data connected to the recruitment process.  The increasing data driven programmatic approach to recruitment can do much to aid in the design and selection of a recruitment strategy.  Seemingly small changes can be tracked to measure their impact on the success or failure rates of a decision.

The growth in our ability to collect these metrics has been matched by a hunger within the stakeholder set as a whole.  Once a hiring manager has seen a report that gives seemingly scientific insight into the hiring process it will be almost impossible to revert to something which grants them less insight.  I'm not advocating that we take away metrics for these managers rather than we give them the access and supply the relevant context.  The greatest danger of data collection lies not in the information, but in its interpretation.  

So what metrics are appropriate to measure? What metrics can offer us certainty without falling into the the traps of selection or confirmation bias?  There are already a lot of hyperbolic blog posts like "The Top 10 Metrics You Must Have" or "7 Recruitment Metrics to Win" these miss the point.  The metrics of recruitment are best used for experimentation - tied to the continuous improvement of the team.  If you are producing metrics that will sit unopened in a spreadsheet to appease a hiring manager you are guilty of security blanket metrics.  Whilst you will feel all warm and fuzzy because you can prove that some *thing* is happening they will be of no real practical value, like butterflies pinned to a board underglass, nice to look at but not useful.

So whats the alternative?  When done correctly the term "metrics" is a misnomer.  The gathering of data around recruitment will give you a dataset which you can apply to provide insight into historical performance and to measure impact of the specific efficacy of projects the team undertakes.  In this way it's possible to see results in real time - does that new advert copy lead to more applications? You can see that! Which website is best to advertise on? You can test that! Did that rival companies announcement affect your response rate? You'll be able to see!  Did adding that photo of a cat to your website make it better?  Of course it did! You don't need metrics to tell you that!

What can't metrics do?  Predict the future.  In many of the articles I've read about recruitment metrics I've seen a large number of lofty claims about prediction.  All the while these claims are made without noting the limitations of the dataset we have access to.  It's the measurement of this dataset that will be the most effective use of business value not on fortune teller style inference of outcomes.  Statements like "we had 1000 applicants in 2013, so this year we will have 1500" are always going to be more wishful thinking than informed prediction.  Metrics can help in planning for the future but knowing the limitations of the basis of those predictions is key.  If we aren't aware of the limits of prediction we risk undoing the good that data can do and reaching for the crystal ball.  

In a future post I'll list the what and why of the metrics I like to measure.  Both for tracking team and individual performance within the team.  Hopefully you'll recognise it's a list high on building a dataset with experimentation in mind and low on fluffy feel goods and blame dodging.

Innovation in Job Hunting - Engaging the Recruiter

I always seem to harping on about what employers can do to encourage engagement from talented candidates.  Today I came across  reddit user Leah, who goes by Pastlightspeed, who posted photos of her recent application to two advertising agencies for an intern position.  It's hard to know how to standout in this increasingly competitive market and whilst Leah skirts the line between impressive and gimmicky I think the end result is both pleasing and communicates her potential well.

This isn't the first time I've seen this type of thing and whilst it lends itself well to creative professions I think there's scope to produce this kind of thing for other disciplines too.  In the past I've seen resumes submitted in LaTeX for researcher roles, as an API for an engineering role and a candidate at Facebook sent a single shoe - the accompanying message stating "...if the shoe fits".  All three stood out and all three got interviewed.  Of course you still have to interview well but thinking about the application process in a creative way could give you an advantage over other applicants and may help to pique the interest of even the most jaded in-house recruiters.

On "Culture" - “You keep using that word, I do not think it means what you think it means”.

How many job adverts currently advertise a "great culture", "a start-up culture" or a "Google-like culture"?  It seems as though the only company not shouting about how Google-like their culture is are Google themselves.  It's a particular bugbear of mine at the moment because it's not only a trite cliché it's also meaningless.

"Culture" as it is currently being used in job adverts has come to mean little more than a perk.  "Salary, Bonus, Life Insurance, Great Culture".  Whilst this doesn't make the top ten in my all time annoyances with how jobs are advertised it does make the mistake of entirely missing the point.  If the "culture" is a differentiator why wouldn't you tell a prospective candidate about it in lavish detail?  I think the issue here might be one of misunderstanding of the term.

So what is culture?  Broadly defined the culture of a company is the ideas, customs and social behaviour of a particular group or society.  These are the building blocks, the elemental stage of what we collectively called culture.  Without description of these ideas, customs and behaviours and why they are good bad or of no interest to a candidate mentioning it is redundant.

So what isn't culture?  Another facet of a lack of description in a job advert is a description of the wrong things a quick scan of well intentioned descriptions lists "beers in the office", "foosball" and "free food".  These things are not culture.  Just like empty pyramids and papyrus scrolls are not the sum total of Ancient Egypt any more so than the Parthenon and Feta cheese are the whole of Greece.  Whilst these things are of cultural significance as parts of a job description without more insight they are little more than window dressing, set up to be dismissed by all but the most earnest of job hunters.  Whilst a recruiter may think that they are choosing the most attractive attributes of a compensation package they must also ask themselves do they really want to attract the candidate who favours a free lunch over a technology choice or a chance for progression?

I think the answer lies in a system of first and second order signifiers when talking about culture.  Those elements you call attention to first should be the most pertinent to your audience.  In the case of a Developer role for example I think we should assume that a candidate would want to know what technologies are involved, how the company writes code, how the teams are organised etc.  I'd hope a great candidate would want to know all of this before hearing about the details of a benefits package...even if they include "onsite barber" and "free laundry".  These first order signifiers should be discovered when a recruiter qualifies a requisition.  This is the true insider knowledge and where the true indicators of culture lie, for example when saying the company has a flat-structure give the signifiers of this - small functional teams, 360 review process, accessibility to senior management.  If you say a company is innovative, tell the candidate how this is manifest - hackathons, internal discussion forum, cross functional collaboration etc.  Don't just say those Ancient Egyptians were "Good builders" tell me about the pyramids!  If you don't you're missing the best opportunity.  Make the sell of the role more compelling through authenticity, not just spewing the benefits package verbatim - don't be a perk-ulator.

Those second order signifiers are those items that apply to the general population of an organisation i.e. not role specific but company specific.  These are best used to reinforce the company's values, attitudes and beliefs.  If possible these should be coupled with assumptions that let the reader know about the thought behind them.  Google's "20% time" (despite it's rumoured death) and Zappo's "$2000 to quit" are great examples of this and offer a great stepping off point for later discussion with candidates.

Remember, the ideal job advert is not only attractive to those people you want to hire but also screens out those you do not.  If you write a generic job advertisement you will get a generic response.  A correctly worded ad to the right audience is a great first filter.  Candidates are not stupid, they will self select if they feel the role suits them and that is what should happen.  If you write a job description that everyone likes, everyone will apply but then of course you don't want to hire everyone...

Innovation in Sourcing - The Poaching Phone

I recently posted on the wealth of innovative techniques available to a forward thinking sourcing departments who are targeting known individuals in competitor organisations.  A Dubai based advertising agency, FP7, gives an object lesson in how to do this well and the direct return on investment they made from using this approach.

"We set out to expand our creative department, but hiring talent in the region is a constant struggle. Headhunters charge exuberant fees, so we did our homework and captured the attention of the region's best talent using the ultimate creative recruiter - The Poaching Phone. Faux industry Self help books were personalised to potential recruits and demonstrated how they could advance their career with us. Inside each book, an ordinary phone was concealed in die-cut pages and programmed with only one contact, our ECDs number. We then sent it out to infiltrate Dubai's top Ad Agencies. Within a week, we received the phone calls we were hoping for. A month later, we had 4 new members join our creative family. In the end, we saved 97% of our projected recruitment costs with a simple phone."

Four hires and a 97% reduction in projected costs make this a obvious success in the face of the "spray and pray" mentality of some sourcing strategies.

Advertising a Vacancy in the Key of C#

There is a problem with advertising a vacancy on a job board.  Not just the general problem of the decline in qualified candidates having to use job boards to find a new role but also the problem of standing out in a sea of other text all advertising the same type of vacancies.  How can you make plain text stand out when it's just the same as everything else?  Better yet how can you make it truly relevant to your target audience?  

If you take the time to look at what your competitors are putting on job boards you might notice some strange behaviours.  How many of the "adverts" are actually just job descriptions?  A job description and an job advertisement perform two very different functions and should look very different.  If you produce a job description and post that instead of telling a reader how amazing it would be for them to work for your company you're posting a list of demands in HR Speak.

This is the equivalent of a car manufacturer televising the turning pages of the technical manual, it's just so boring!  Stretching the analogy further an advert for a new job should be just as aspirational as for a new car - we want all the cornfields on fire, explosions and leather clad luxury of a car ad.  We want excitement, something that will appeal to the target audience and something that demonstrates that we, as an employer, understand them. 

Today I worked with one of our developers to write a job advertisement in C#.  What would have taken me an age obviously only took him a few seconds to write but the feedback was the best I've ever heard for any advertisement, after we finished he said - "I would apply".

We're currently trialing a number of different styles of advertising for our jobs over on our StackOverflow company page.  It's particularly useful because we can see both page views and applications so we're better able to judge the effectiveness of an ad.  I'm hoping this ad in code as well as other versions we're working on might encourage those that see them to explore a little further.

1. using System;
2. using System.Linq;
4. namespace CriteoQuestions
5. {
6.     class Program
7.     {
8.         static readonly uint THRESHOLD = 5;
10.         static uint Question(string text)
11.         {
12.             Console.WriteLine(text + " [y/N]");
13.             string answer = Console.ReadLine();
14.             return answer != null && answer.Equals("y") ? 1U : 0U;
15.         }
17.         static void Main()
18.         {
19.             string[] questionTexts =
20.                 {
21.                     "Looking for a new challenge?",
22.                     "Want to work in the heart of Paris?",
23.                     "Do you enjoy solving hard problems efficiently and creatively?",
24.                     "Would you like to work where Big Data is more than a buzz word?",
25.                     "Want to work on a product at true web scale with 30B HTTP requests and 2.5B unique banners displayed per day?",
26.                     "Would you like to know more?"
27.                 };
28.             uint score = questionTexts.Aggregate<string, uint>(0, (current, text) => current + Question(text));
29.             Console.WriteLine(score > THRESHOLD
30.                                   ? @"Contact m.buckland@criteo.com today"
31.                                   : @"That’s a shame, you can learn more at http://labs.criteo.com/ maybe we can change your mind?");
32.             Console.ReadLine();
33.         }
34.     }
35. }

What other ways are there to stand out when advertising jobs online?  How can you make the limitations of plain text on a job board into advantages that will make your adverts stand out from the crowd?

Innovation in Sourcing - Standing out from the crowd

The word "Sourcing" has come to be used in a particular way recently.  In an age of "social recruiting" the meaning of sourcing has become narrowed to the point that it really only relates to new ways of searching the internet or the latest in a long line of software tools to interrogate ever growing datasets.  However, as recruiters, often we already know who we want to target.  We know the companies they work for, we know the skills they possess, we know their titles, in some cases we even know their names.  The overly stalkerish amongst us sometimes even know their addresses...

In recent years there have been a number of landmark instances using more non-traditional tactics.  New companies wanting to make an impact, older organisations seeking out particular known individuals or just a grand gesture of recruitment, recruitment as an event or spectacle, existing to generate a larger story with the resulting publicity driving even more people to learn about the company. Further, frustrations over "access" to these candidates forces more innovative companies to imagine more and more innovative solutions to get their message across.  Some are clever, some confrontational, but all of them have made an impact beyond their original target audience.  Here are some of my favourites from over the years.

 In 2003 Electronic Arts in Canada took out some billboard space near the offices of rival games developer Radical Entertainment. Near enough to be read by the developers at Radical who had no problem working out that the message reads "We're Hiring".    The results of this obviously confrontational stance by EA didn't really do them much good - the team at Radical garnered a lot of positive press. The public love an underdog it seems.  Founder and CEO at Radical, Ian Wilkinson sums it up well "This has been far more aggressive than past attempts, but I have no reason to believe that this will be any more effective."

So overtly hostile attempts can often be jarring and work against you - at the very least they convey a lot more about the brand than was originally intended.  Here, EA were the giant trying to take down an independent success story, it didn't work but it has been done better.


Enter Google.  In 2004 this billboard appeared near the Ralston exit leading to Santa Clara, California.  A prime location for attracting the attention of the employees of Silicon Valley as they sat in traffic on their way to work.  Free from any branding the billboard itself is a challenge.  Perfectly aimed at their target audience of engineers and researchers who love to solve problems.   The problem itself led to a url that in turn led to another problem and eventually a pay-off and reveal that it was a Google recruiting strategy.  This is still talked about today as being ground-breaking and it certainly aided in the establishment of the mythical status of Google's hiring process.  Looking back it's easy to assume that "of course it's Google" but at the time they were pre-IPO, 1907 employees (as of March 2004) and they were already doing truly innovative things.  Interestingly, it also didn't stop them pursuing other more "grey" tactics too - at the same time they were winning hearts and minds, and enjoying massive viral publicity with their billboard they were also sponsoring job adverts in their own search results.  As well as sponsoring traditional job applicant search terms they also sponsored ads on the keyword/name "Udi Manber", who was then chief of Amazon.com's search technology unit, A9.  It would be just two years later that Udi joined Google...

These are both still broadcast messages, though it's true they act as a filter for talent, so the organisations only have to deal with those people who are able to answer the questions.  What if you already know who you want to talk to?  Not a type of person or a profile - what if you actually know the person?  

Video game start-up Red 5 Studios handpicked about 100 dream candidates, spent time learning about their backgrounds and interests from social networks and personal blogs, and airmailed each one a personalized iPod, inside 5 artistic nested boxes complete with a recorded message from CEO Mark Kern. More than 90 recipients responded to the pitch, three left their jobs to come on board, and many more potential hires discovered the company through word-of-mouth buzz generated by the search.  Whilst it is true that these types of initiatives have a higher initial cost for the more price-conscious organisation this can be mitigated by the quality of the potential audience - they targeted their "dream" employees. The saving in costs versus the same approaches made through an third party recruitment firm are not to be sniffed at.  Chances are a single hire made through an agency would have exceeded the total cost of this project.  There's also a third more intangible return on investment, the virality of this approach.  I am confident that there is a secondary impact of this type of approach the effect on other employees in the target organisation when told about the parcel and now the impact of this type of approach being shared on social media - the outlets of which have increased exponentially since Red 5 Studios did this in 2007.  

Facebook did something very similar in 2013 for hardware engineers.  As a pilot program they sent branded Raspberry Pi's to potential candidates they had identified as a good potential fit.  On connecting the credit-card-sized single-board computer they were presented with a personalized video giving them a tour of the working environment and a brief of where they would potentially fit in.  This type of approach is hard to ignore.

A mobile handset manufacturer could send their latest handset with a willing hiring manager's number pre-installed?  This would both show off the product and demonstrate the value the company see in the candidate.  Spotify already send tongue-in-cheek playlists to potential candidates, demonstrating the product in a fun way as well as letting the candidate know they are hiring.  There are dozens of these initiatives going on all the time.  Sitting back and waiting to resumes is unforgivable - what can your organisation do to differentiate itself?