The Unsung Genius of the Structured Interview

 Check out what Laszlo Bock has to say about the UN-structured interview….

Laszlo is the former SVP of People Ops at Google who published “Work Rules”, which shares insights from his time at Google that are intended to help readers change the way they live and lead. Laszlo’s twitter account is a blend of personal thoughts and shared insights from the world of People Ops.

 

The Unsung Genius of the Structured Interview

In 1998, Frank Schmidt and John Hunter published a meta-analysis of 85 years of research on how well assessments predict performance. They looked at 19 different assessment techniques and found that typical, unstructured job interviews were pretty bad at predicting how someone would perform once hired.

Unstructured interviews have an r2 of 0.14, meaning that they can explain only 14 percent of an employee’s performance. This is somewhat ahead of reference checks (explaining 7 percent of performance), ahead of the number of years of work experience (3 percent).

The best predictor of how someone will perform in a job is a work sample test (29 percent). This entails giving candidates a sample piece of work, similar to that which they would do in the job, and assessing their performance at it. Even this can’t predict performance perfectly, since actual performance also depends on other skills, such as how well you collaborate with others, adapt to uncertainty, and learn.

People who score high on conscientiousness are more likely to feel responsibility for their teams.

And worse, many jobs don’t have nice, neat pieces of work that you can hand to a candidate. You can (and should) offer a work sample test to someone applying to work in a call center or to do very task- oriented work, but for many jobs there are too many variables involved day‑to‑day to allow the construction of a representative work sample. All our technical hires, whether in engineering or product management, go through a work sample test of sorts, where they are asked to solve engineering problems during the interview.

The second-best predictors of performance are tests of general cognitive ability (26 percent). In contrast to case interviews and brainteasers, these are actual tests with defined right and wrong answers, similar to what you might find on an IQ test. They are predictive because general cognitive ability includes the capacity to learn, and the combination of raw intelligence and learning ability will make most people successful in most jobs. The problem, however, is that most standardized tests of this type discriminate against non-white, non-male test takers (at least in the United States). The SAT consistently underpredicts how women and non- whites will perform in college. Reasons why include the test format (there is no gender gap on Advanced Placement tests, which use short answers and essays instead of multiple choice); test scoring (boys are more likely to guess after eliminating one possible answer, which improves their scores); and even the content of questions.

Tied with tests of general cognitive ability are structured interviews (26 percent), where candidates are asked a consistent set of questions with clear criteria to assess the quality of responses. There are two kinds of structured interviews: behavioral and situational. Behavioral interviews ask candidates to describe prior achievements and match those to what is required in the current job (i.e., “Tell me about a time . . . ?”). Situational interviews present a job-related hypothetical situation (i.e., “What would you do if . . . ?”). A diligent interviewer will probe deeply to assess the veracity and thought process behind the stories told by the candidate.

Structured interviews are predictive even for jobs that are themselves unstructured. We’ve also found that they cause both candidates and interviewers to have a better experience and are perceived to be most fair. So why don’t more companies use them? Well, they are hard to develop: You have to write them, test them, and make sure interviewers stick to them. And then you have to continuously refresh them so candidates don’t compare notes and come prepared with all the answers. It’s a lot of work, but the alternative is to waste everyone’s time with a typical interview that is either highly subjective, or discriminatory, or both.

There is a better way. Research shows that combinations of assessment techniques are better than any single technique. For example, a test of general cognitive ability when combined with an assessment of conscientiousness is better able to predict who will be successful in a job. My experience is that people who score high on conscientiousness “work to completion”—meaning they don’t stop until a job is done rather than quitting at good enough—and are more likely to feel responsibility for their teams and the environment around them.

Sure, it can be fun to ask ‘What song best describes your work ethic?’ but the point is not to indulge yourself with questions that trigger your biases.