Bryan Caplan  

Four Big Facts About Hiring and IQ

PRINT
A Spoonful of Sugar Helps the ... Meta-Measuring Signaling...
Many economists seem to think that IQ-based hiring is effectively illegal in the U.S.  O'Keefe and Vedder are two prominent voices, but plenty of mainstream labor economists say the same.  The more I read about this topic, though, the more legal IQ-based hiring seems to be.  Three big facts:

1. Though data is spotty, 10-30% of large U.S. employers freely admit that they use cognitive ability tests to make hiring decisions. (see pp.45-7 and references)

2. Cognitive ability testing is only modestly less common in the U.S. than other countries.  Ryan et al's "An International Look at Selection Practices: Nation and Culture as Explanations for Variability in Practice" (Personnel Psychology,1999) surveyed firms in 20 countries about their hiring methods.  On a 1-5 scale, where 1="never uses cognitive ability tests" and 5="almost always or always uses cognitive ability tests," the average U.S. firm scores 2.08, versus 2.98 for the whole sample.

3. When asked, only 16% of U.S. employers who don't test cognitive ability cite "legal concerns." (See Terpstra, David, and Elizabeth Rozell.  1997.  "Why Some Potentially Effective Staffing Practices Are Seldom Used."  Public Personnel Management 26, p.490).  The leading reason they don't hire based on IQ is that they doubt the usefulness of the tests.

A fourth fact that might explain all the confusion:

4. In the U.S., cognitive ability testing is the hiring tool most likely to be avoided for legal reasons.  The only reason it holds this distinction, though, is that U.S. employers worry even less about the legality of all the other hiring methods. (Terpstra and Rozell, p.490)


Comments and Sharing





COMMENTS (21 to date)
nazgulnarsil writes:

I don't buy it. The IQ issue is too entangled with racial shibboleths in the US to trust any straightforward polling data.

nerdbound writes:

The company I'm at requires all potential hires to take a 'logic test' which basically amounts to an IQ test. The test is only really useful to screen out a few outliers, however, and many of them additionally get screened out by something else we do, so the added value is pretty small.

At one point, I was asked to test out a system we were considering which was a much more extensive test product. My verdict was that it was a terrible product... Extremely irritating and menial problems that were far below the sort of skill level we wanted to test for.

My basic conclusion from this has been: no one has a test product which can reliably find high performers, which explains why IQ tests are rarely used in hiring.

nerdbound writes:

I believe this company has some of the products we were looking into: http://www.shl.com/us/

babar writes:

An IQ test might at best indicate some measure of raw ability but would say nothing about whether the person has specific skills or whether they would work hard or get along with other people...

Hugh writes:

Google seem to have developed a battery of (very difficult) questions to test new hires.

These questions are clearly a test of IQ, but I'm sure their HR department has thought up a much more attractive name.............

.........maybe IQ tests just need to rebrand, reposition and go to market with a new paint job.

John Thacker writes:

Hugh:

Except that Google seems to have abandoned the brainteaser approach as not useful. Note that they also say that degrees and GPA seem to be worthless as well.

Bryan:

It's interesting considering that face-to-face interviews are also worthless according to all research, and also seem to be very vulnerable to legal challenges, yet they persist.

Bostonian writes:

typo -- in "employers worry even else about the legality", "else" should be "less"

Anthony writes:

It's a problem legally if you're the local government and you're hiring police or fireman. There, if you use hiring mechanisms that lead to disparate impact, as cognitive ability tests and even reasoning tests with fire/police content do, you are likely to run into problems with the DoJ. This is especially the case for big-city departments. But of course that's a very small slice of employers: otherwise, you're fine.

I don't know why these are the employers who are the exception. It's routine for teachers to get content knowledge tests which produce disparate impact. The military certainly uses them. The civilian civil service uses them in some cases, although to a far lesser degree. But police and fireman in big cities basically can't, and there have been major legal challenges and significant effort to design tests that don't cause disparate impact. The result is that cities have moved to tests which only disqualify a very small proportion of people, and then use other mechanisms to decide among them. They are still using tests, yes. But they're a much smaller piece of the hiring process than they used to be.

Glen Smith writes:

John,

What alternatives were tried? Without knowing the alternatives, my conclusion would be that face2face interviews are bad but not worse than the alternatives.

Glen Smith writes:

Anthony,

For over 10 years I was involved in developing systems to do exactly what you describe. While certain non-predictive variables (for example, race) you don't want to collect for production (there was correlation but it was not predictive), we never ran into any legal issues.

Hugh writes:

@ John Thacker

Point taken that Google no longer uses IQ tests (or brainteasers).

The Google guy freely admitted that they had used them, but had given them up as "a complete waste of time".

This suggests that they are not too worried about the legal angle from past use of these tests.

Anthony writes:

Glen,
I overstated my case on this. If you're a small department, I don't know how this goes. If you're New York, Los Angeles, Chicago, or Boston, you've been sued and/or come under consent decrees from the DoJ and have responded.

Eric Falkenstein writes:

I would guess that a good third of college graduates find IQ tests morally abhorrent(eg, the tests are systematically biased against non-asian minorities and merely used to keep these people out). Therefore, if you don't use them ultimately because it's legally dubious, the firm and its many stakeholders have a strong incentive to say they don't use such tests because they do not want to, not because they can't.

Daublin writes:

Curiously, drug testing has the same issues as IQ testing: it's correlated with performances but is a poor direct predicter. Yet, drug testing is common among large U.S. companies.

Count me in as thinking neither is very helpful. I'd guess that drug tests are much more convenient for selective enforcement. If you want to get rid of someone, then forcing them to take a drug test can give you a way to do it.

Ricardo writes:

Employers refrain from saying "we don't do that for legal reasons" for legal reasons.

Yancey Ward writes:

If you do something basically half-assed because you are worried about the various legal ramifications, it is likely to reveal itself as useless.

In any case, I would rather, if I were doing hiring decisions, to have a specific skills test for a particular position, along with a very good personality/psychological screen.

Isegoria writes:

H. Beam Piper's Day of the Moron points to union resistance.

johnson85 writes:

I agree that you shouldn't take companies professed reasons at self-value. Most companies will be hesitant to come out and say we'd like to do something except it will result in criminal/civil liability. But on top of that, I'd guess a lot of companies don't use IQ tests because they don't use IQ tests. The person that made the decision, and the reason therefore, are probably gone for a lot of companies. And it's not like it's a hot trend. There's no upside to an HR professional to suggesting IQ testing for applicants when most companies don't do it and there is any concern regarding liability.

But probaly more important, IQ tests aren't that helpful because it's less controversial to use grades and college reputation (which basically gives the same or better information), but at a cheaper cost to the employer (if not to society).

And if you were dealing with a position for which getting a college graduate isn't practical (and this applies to fewer jobs everyday), I'm not sure how much IQ testing would be helpful. You could probably pick out the people on the low side through an interview and if a candidate really did well, wouldn't you be just as likely to think there must be a behavioral based reason that he/she didn't go to/graduate from college when we already send lots of people to college that don't really have the aptitude for it?

John Thacker writes:

@Glen Smith:

If face to face interviews are "bad, but not worse," seems like on their face they'd have trouble with the law, since face to face interviews make it much easier to discriminate on the basis of impermissible categories (even compared to say, phone or written interviews.)

However, in practice companies don't seem to worry about the extra discrimination lawsuits that they could be inviting by conducting face to face interviews.

Google apparently believes these days that the best hiring practice is just to try people out.

Garth Zietsman writes:

A number of commenters seem to suggest that IQ tests do not work and are systematically biased against minorities. IQ tests work pretty well - better than any other measure. There is a ton of validity research on them. Sure they don't measure lots of important stuff like how hard someone will work but that's no reason not to use them for things they do measure - like ability to think and learn.

They aren't biased against minorities. Disparate impact is not a sign of bias. Bias would be where a particular test score - an IQ of 90 say - means something different with respect to job performance or ability to learn, across groups. Bias research - and there has been a lot of it - consistently shows that test scores mean the same thing across groups.

Some employers may not be using tests because they don't believe they work but I'm inclined to think they haven't actually looked at the research or they don't understand restriction of range or think 30% better than chance sounds trivial and don't get how much benefit they could get from that sort of performance.

Glen Smith writes:

Anthony,

I worked on what was, at the time, the biggest hiring performance projection model in the world. Problem was only that we couldn't use certain higher level variables (such as marital status) and had to research for the actual predictive variables behind them. Our production models could not be allowed to see those variables at all but the research models could use them for reporting, training and testing.

Comments for this entry have been closed
Return to top