“A common phenomenon and problem in leadership practice concerns undue reliance on popular fads without sufficient consideration given to the validity of those ideas…” Click on the link below to read a very good review from Amazure Consulting for the evidence on cognitive tests, personality, situational judgement, emotional intelligence and interviews being ‘effective’ predictors of leadership ability.
Situational judgement tests (SJTs), or tests that assess a candidates ‘preferred’ responses to a range of work-based scenarios, are growing in popularity and are now commonplace in many graduate and management recruitment processes. But do they work? Well, it seems there’s pretty good evidence that SJTs do predict job related criteria such as sales performance or ratings by managers. The first really thorough analysis, conducted by McDaniel et al (2001) across 95 different studies, concluding that the correlation between SJTs and job performance is in the region of 0.34. Incidentally McDaniel also found that when SJTs were closely matched to the job in question – via a properly conducted job analysis – the figure rose to 0.38.
The same figure was reported earlier this year by SHL Group, with a composite of 0.38 being achieved for a ‘relating & networking’ criterion and one of their SJTs which is being used by a global retailing organisation.
In addition various studies have looked at whether SJTs significantly add to the prediction of job performance over and above that which is achieved by using measures of cognitive ability (psychometric reasoning tests), job experience and personality. Again McDaniel et al (2007) have found that SJTs provide incremental validity over cognitive ability of between 3 and 5 per cent, i.e. they add something extra to an understanding of ‘thinking’ competencies; and of 6 to 7 per cent compared to personality questionnaires, i.e. they add even more to an understanding of how someone deploys their personality at work.
P.S. In the great scheme of things 0.3, which is a ‘moderate’ correlation, is the point at which things are starting to get particularly useful, especially if the assessment method in question is being used for volume recruitment.
Want to know more?
McDaniel, M.A. and Nguyen, N.T. (2001). Situational Judgment Tests: A Review of Practice and Constructs Assessed. International Journal of Selection and Assessment, 9(1-2), 103-113.
McDaniel, M.A., Hartman, N.S., Whetzel, D.L. and Grubb, W.L. (2007). Situational Judgment Tests, Response Instructions and Validity: A Meta-analysis. Personnel Psychology, 60(1), 63-91.
Lievens, F., Peeters, H. and Schollaert, E. (2008). Situational Judgement Tests: A Review of Recent Research. Personnel Review, 37 (4), 426-441.
High Fliers Research conduct an annual survey of UK graduate job prospects. This year, whilst vacancies are set to rise, recruiters report that one-third of this year’s positions will be filled by graduates who they already know – either through work placements, internships or sponsorships. In addition two-thirds of employers say that graduates with no work experience at all are very unlikely to get through the selection process, and thus have little or no chance of receiving a job offer.
This is sobering stuff and highlights the need for all undergraduates to seek out placements or acquire meaningful work experience. It also means that many will need to work on their interview technique, and to practice psychometric and situational judgement tests (see my SJT post). All of these assessment methods are becoming increasingly popular for selecting placement students and interns.
Over the last few days I’ve been asked if there’s anything on the web that can help someone prepare for a Situational Judgement Test (SJT). For the uninitiated an SJT is a type of psychometric test that presents the test-taker with scenario-based problems; then for each problem various actions are provided and the idea is to pick the one you think is the best – this is the judgement bit. SJTs are sometimes called ‘Management Competence Tests’.
SJTs are used by a number of large organisations and are a particular favourite of the UK and EU Civil Services. So here’s some practice material from the European Personnel Selection Office (EPSO) and the UK Ministry of Defence (MOD).
Picture credit: Francesco Marino/FreeDigitalPhotos.net
Here are the slides for a recent presentation I gave on preparing for psychometric tests:
It was delivered six times to groups of students preparing for tests. And putting modesty (and alliteration) to one side, was described by one punter as ‘the best lecture he had ever had’, at the university in question…
PS. If you want to know the answers, you’re going to have to ask me!
Slide-9. For the curious, if you guess the answers to 10 questions, in a test with five answer options per question, your chances of picking the correct answers for all the questions are 1 in 10,000,000. Your chances of being struck by lightening are also about 1 in 10,000,000. Guessing is not a good tactic!
Images. Slide photos are by Luigi Diamanti, Salvatore Vuono, Gregory Szarkiewicz, JS Creations & Suat Eman – www.freedigitalphotos.net