What are candidate assessment measures measuring?

Do candidates who are better at predicting what employers are looking for – who have a well-developed ’ability to identify criteria’ – actually do better at assessment events? Yes… in fact research seems to show that this ability correlates more strongly with job performance than assessment centre scores themselves!

Read this BPS Research Digest article for the full story: What exactly are candidate selection measures measuring?

Psychometrics & return on investment

It’s self-evident that selecting people who are better at doing a job, whatever that job is, will have an effect on your bottom line. There’s also plenty of research that supports the idea that psychometric tests and questionnaires, amongst other assessment techniques, can give a significant boost to your ability to spot winners. But, how can you show the benefits in terms of hard cash and/or reduced staff turn-over?

A number of psychometric test publishers and occupational psychology firms have started to produce Return on Investment (ROI) calculators. These can be used to estimate the financial gains achieved by increasing the quality of hires, and also provide a guide to the likely reduction in staff turnover. If you want to know how they work, or would like to try putting your own figures through, try one of these:

• SHL group: http://www.shl.com/uk/results/roi-calculator/
• PreVisor: http://www.previsor.com/results/calculator
• Learn Purple: http://www.learnpurple.com/free_stuff/calculators/
• Mentis Consulting: http://www.mentis-consulting.com/roi.asp

You’ll also find a very readable and insightful article from Talent Q here: Introduction to Measuring the ROI of Assessment.

Free psychometric tests & questionnaires

In my mission to try to steer people towards useful psychometric practice material, I now add interesting thoughts and snippets to this Facebook page:

www.facebook.com/psychometric.tests

This provides links to properly stimulating stuff (e.g. tests & questionnaires that can be used to prepare for selection events), amusing free surveys (e.g. do you have a male or female brain?), interesting articles (e.g. do entrepreneurs have lucky personalities?), and even test publishers who will pay you to try their latest products!

Now and again I also add links to the growing number of psychometric apps for smart phones, Facebook and the like. For instance, did you know that more than three million people have used the free (Big Five) myPersonality app at:

http://apps.facebook.com/mypersonality/consent.php

Facebook really is changing the ‘face’ of psychological research…

Do people cheat on psychometric tests?

Not as much as you might think… Read the latest research from Psylutions.

 

Fairy tales and predicting good leaders

“A common phenomenon and problem in leadership practice concerns undue reliance on popular fads without sufficient consideration given to the validity of those ideas…” Click on the link below to read a very good review from Amazure Consulting for the evidence on cognitive tests, personality, situational judgement, emotional intelligence and interviews being ‘effective’ predictors of leadership ability.

http://www.amazureconsulting.com/files/1/39199746/FairyTalesFactsPredictingLeadershipEffectiveness.pdf

Do situational judgement tests work?

Situational judgement tests (SJTs), or tests that assess a candidates ‘preferred’ responses to a range of work-based scenarios, are growing in popularity and are now commonplace in many graduate and management recruitment processes. But do they work? Well, it seems there’s pretty good evidence that SJTs do predict job related criteria such as sales performance or ratings by managers. The first really thorough analysis, conducted by McDaniel et al (2001) across 95 different studies, concluding that the correlation between SJTs and job performance is in the region of 0.34. Incidentally McDaniel also found that when SJTs were closely matched to the job in question – via a properly conducted job analysis – the figure rose to 0.38. 

The same figure was reported earlier this year by SHL Group, with a composite of 0.38 being achieved for a ‘relating & networking’ criterion and one of their SJTs which is being used by a global retailing organisation.

In addition various studies have looked at whether SJTs significantly add to the prediction of job performance over and above that which is achieved by using measures of cognitive ability (psychometric reasoning tests), job experience and personality. Again McDaniel et al (2007) have found that SJTs provide incremental validity over cognitive ability of between 3 and 5 per cent, i.e. they add something extra to an understanding of ‘thinking’ competencies; and of 6 to 7 per cent compared to personality questionnaires, i.e. they add even more to an understanding of how someone deploys their personality at work.

P.S. In the great scheme of things 0.3, which is a ‘moderate’ correlation, is the point at which things are starting to get particularly useful, especially if the assessment method in question is being used for volume recruitment.

Want to know more?

McDaniel, M.A. and Nguyen, N.T. (2001). Situational Judgment Tests: A Review of Practice and Constructs Assessed. International Journal of Selection and Assessment, 9(1-2), 103-113.

McDaniel, M.A., Hartman, N.S., Whetzel, D.L. and Grubb, W.L. (2007). Situational Judgment Tests, Response Instructions and Validity: A Meta-analysis. Personnel Psychology, 60(1), 63-91.

Lievens, F., Peeters, H. and Schollaert, E. (2008). Situational Judgement Tests: A Review of Recent Research. Personnel Review, 37 (4), 426-441.

How many successful entrepreneurs would fail an IQ test?

If you would like to be part of some research to find out, follow this link. Online survey designed by Tomas Chamorro-Premuzic of Goldsmiths College, University of London.

Interesting footnote: Tomas was the resident psychologist on Big Brother!

Free stuff @ www.markparkinson.co.uk

Graduates! Just another reminder that there’s a multitude of free psychometric tests and questionnaires on the ‘links’ page of my website at:

www.markparkinson.co.uk/psychometric_links.htm

Also a growing number of other sites that offer free taster tests, for example you will find verbal, numerical and inductive reasoning tests at:

www.graduatesfirst.com

Do psychometric tests work?

Good question. What people usually mean when they ask if they work is: do tests predict anything useful about future work performance? The short answer is a resounding ‘yes’. As long as a test is used to measure an ability that is actually required of a particular job, then predictive validities are often in the 0.5-0.6 range. What this means is that at the top end of the scale, a test (the predictor) explains 36% ((0.6 x 0.6) x100) of the variance in the criterion – the criterion being something like a measure of productivity. By way of contrast other assessment methods such as the interview are often far less effective. A semi-structured interview would weigh in at 0.38 (14%) or thereabouts. And to get the whole thing in perspective, just in case you’re not impressed, in other fields such as the drug industry, predictive-type validities are often lower. For example, the association between Ibuprofen (the well-known anti-inflammatory) and pain reduction is in the region of 0.14 (2%) – see Robert Hogan’s article, details below.

Want to know more, here are some key references:

• Bertua, C., Anderson, N., and Salgado, J.F. (2005). The Predictive Validity of Cognitive Ability Tests: A UK Meta-Analysis. Journal of Occupational and Organizational Psychology, 78(3), 387-410.
• Hogan, R. (2005). In Defense of Personality Measurement: New Wine in Old Whiners. Human Performance, 18, 331-341.
• Hunter, J.E, & Hunter, R.F. (1984). Validity and Utility of Alternative Predictors of Job Performance. Psychological Bulletin, 96, 72-98.
• Schmidt, F.L, & Hunter, J.E. (1998). The Validity and Utility of Selection Methods in Personnel Psychology: Practical Implications of 85 Years of Research Findings. Psychological Bulletin, 124, 262-274.

No work experience, no graduate job!

High Fliers Research conduct an annual survey of UK graduate job prospects. This year, whilst vacancies are set to rise, recruiters report that one-third of this year’s positions will be filled by graduates who they already know – either through work placements, internships or sponsorships. In addition two-thirds of employers say that graduates with no work experience at all are very unlikely to get through the selection process, and thus have little or no chance of receiving a job offer.

This is sobering stuff and highlights the need for all undergraduates to seek out placements or acquire meaningful work experience. It also means that many will need to work on their interview technique, and to practice psychometric and situational judgement tests (see my SJT post). All of these assessment methods are becoming increasingly popular for selecting placement students and interns.