Psychometrically brilliant, psychologically flawed

March 6, 2014

stethoscopeDoctor, doctor

First, imagine the scenario. You’re lying in a hospital bed. The doctor comes to see you. You are given a physical examination and asked a series of probing questions. A stethoscope is applied to your chest and to your back; and your temperature and blood pressure charts are scrutinized. Some blood tests are ordered and you’re sent for a scan. Later that day the doctor returns and provides a diagnosis.

The doctor has used the information in a particular way. A number of hypotheses about what may be wrong with you have been considered. Along the way, individual pieces of information may or may not have been useful; indeed may not have been relevant at all, or completely misleading. However the final diagnosis is based on a skilled integration of all the available information. It has been filtered through the doctor’s general understanding of how the body works, plus expert knowledge of a range of conditions and illnesses.

Now, a second scenario: You are an HR manager wanting to recruit a senior manager. You have an up-to-date job description and have compiled a thorough specification of the person you think is required to do the job well. You decide to use a variety of methods to gather information about the candidates. These include asking about their track record, and any particular knowledge and skills that are relevant to the job. You also decide to use psychometric tests to measure specific abilities, and a personality questionnaire to explore how candidates are likely to behave; plus an interview to gather evidence on key competencies, and on things like commitment and motivation.

The right blend

The blending of assessment information requires the same skills as those used by the doctor. The information from an interview depends on the candidate telling the truth, as does anything that is said about previous work history – although to a certain extent this can be verified by previous employers. The results of psychometric tests are of most utility if they are used to measure abilities that are directly relevant to the job – albeit that having a handle on a candidate’s general mental ability has been shown to be one of the best predictors of work performance.

Personality questionnaires fall into a different category, as despite sophisticated designs they are open to a candidate putting a positive spin on their answers. However, that being said, trait-based questionnaires can accurately predict behaviour, and as such they are genuinely useful in providing prompts for interview questions.

Of course no process is one hundred percent predictive, and there will be occasions when the wrong selection decision is made. This is inevitable, but a well designed process, using a balanced range of exercises, will significantly increase the odds of picking a winner. Indeed the odds will be increased to maximum if the person dealing with the information is appropriately trained and experienced: like a business psychologist. This is where the true value is added, especially when it comes to deftly combining information that ranges from the robust and objective, to other sources which may be completely ‘self-report’.

It doesn’t work (or does it!)

Criticisms when things go wrong are often of a binary nature, especially in the Press. For example, if tests or questionnaires have been used (journalists unfailingly lump them together), and the ‘wrong’ candidate is perceived to have been recruited, then it is all too easy to blame the tools. The argument is often that the person sailed through the psychometrics but turned out to be a psychological liability. Thus the answer to the – do ‘tests’ work? question – is obviously a resounding ‘no’. The problem with this logic is that it ignores everything else that has happened, including the due diligence that should have occurred before candidates completed any tests or questionnaires.  But let’s face it, poking fun at psychometrics makes a better story, and provides a golden opportunity to use a picture of an inkblot.

In reality psychometrics are the most predictive tools that are available, certainly many times better than merely relying on an interview. So it’s something of a paradox that the more senior the position the less likely the candidates are to be assessed fully. It’s a bit like going to hospital, announcing that you are the CEO of a bank, and the doctor sitting on the end of the bed, asking a couple of questions, and making an instant diagnosis. No need for all those irritating tests, I can tell what’s wrong with you by enquiring about your golf handicap.

To be serious again, perhaps this is where the attention should be focused, on getting the recruitment process for the ‘top jobs’ properly sorted out: and the case is actually made for more assessment, not less.

The research: Do tests work?

A dating agency for job hunters!

February 20, 2012

Looking for a job on the web is set to get more like using a dating site as reliable psychometrics are used to sift candidates… Keep an eye on this stuff as it’s a growing trend! And my thanks to Angus McDonald for drawing my intention to this article:

What are candidate assessment measures measuring?

September 8, 2011

Do candidates who are better at predicting what employers are looking for – who have a well-developed ‘ability to identify criteria’ – actually do better at assessment events? Yes… in fact research seems to show that this ability correlates more strongly with job performance than assessment centre scores themselves!

Read this BPS Research Digest article for the full story: What exactly are candidate selection measures measuring?

How do you weigh an elephant without using a weighing machine?

May 26, 2011

Isn’t it refreshing to know that people are still being asked ridiculous interview questions. Follow this link to find some real crackers:

And a couple of my personal favourites: ‘When was the last time you cried?’ and ‘Do you prefer cats or dogs?’ In the first one you are damned if you do and damned if you don’t. Admit to blubbing on a regular basis and you’ll be labelled as wet and over-sensitive, and of course if you don’t cry at all you’re just  a hard-nosed and uncaring sort of person. Cats or dogs? Well, it depends on which the interviewer prefers. Try and spot the pet hairs on his/her clothing and see if that provides a clue. Joking apart, I thought we had got over all this stuff – interview questions should relate directly, unambiguously and fairly to the job in question.

Picture credit: Tom Curtis/

The Apprentice selection process

May 20, 2011

The new season of The Apprentice is now firmly established on BBC1. However wouldn’t it be interesting to put the ‘top entrepreneurs’ through a rigorous assessment process before the series, to see who has what it takes, and then compare the eventual winner with their assessment profile? In the meantime here’s how the candidates are selected :)

(c) BBC 2011

Cats with thumbs

May 6, 2011

When I first went to University I studied Biology and I well remember being asked in some interview or other how you could breed a dog that could climb trees. Seriously. And in response I think I waffled on about the need for it to develop opposable thumbs or to grow claws like grappling hooks. I’m sure you get the idea. Anyhow this is a long winded way of introducing a great and funny advert about… cats with thumbs. Nothing to do with business, I just like it!

Google job mania

February 4, 2011

You think you’ve got problems! Google recently received 75,000 job applications in a single week. OK, they reckon to have 6000 vacancies coming up, but that’s still heck of a number. Why the recruitment drive? They’ve decided to take on both Facebook and Apple. Not sure who my money would be on, but I’m surprised they didn’t apply their considerable brain power to designing a better recruitment process. This one was surely crying out for rigorous ‘self-selection’, right at the start.

Check out this Bloomberg article.


Get every new post delivered to your Inbox.

Join 89 other followers