The Poor Man's Polygraph

From the Rational Enquirer, Vol 3, No. 1, Jul 89.

Dale Beyerstein

The BC Skeptics continues its cooperation with the BC Civil Liberties Association to eliminate pseudoscientific tests that violate the rights of job seekers to a fair evaluation. The BCCLA has been informed that the Real Canadian Superstore chain has been using the Reid Report, a paper-and-pencil honesty test devised by Reid Psychological Services of Chicago, in their hiring. Reid, originally a firm specializing in polygraphs, has marketed this test for a number of years for firms that are too cheap to use the polygraph. (I have referred to the Reid Report as the "Poor Man's Polygraph"). However, as of the end of 1988, the use of polygraphs by private employers is illegal in the United States; and the `downside' of this otherwise commendable legislation is that it has resulted in graphology and tests such as the Reid Report being used to fill the vacuum formerly occupied by the polygraph. And, of course, it is only a matter of time until trends from the US make their way into Canada.

The Reid Report attempts to predict honesty partly on the basis of two unjustified assumptions about traits correlated with honesty. One is that an honest person thinks that others are honest, while a dishonest one assumes that "everyone else is doing it too, so why shouldn't I?". Thus, a question on the test asks people to estimate the amount of thievery amongst their colleagues. The higher the estimate, the more likely to be dishonest, Reid assumes. Pity the honest person who leaves his previous job because he's continually getting his wallet stolen, and who decides to answer this question honestly!

The second assumption is that honest people are more likely than dishonest ones to recommend harsh punishments for thievery. Several questions ask the candidate to choose suitable punishments for people caught with their hands in the till: the meaner the punishment the candidate circles, the more honest she is, Reid assumes. I wonder how they deal with the surveys that show that convicted murderers are even more pro-capital punishment than the average person.

Since it is a paper-and-pencil test, measuring the length of the candidate's nose is not part of the study.

John Dixon of the BCCLA roundly condemned the use of the Reid Report in a Vancouver Sun interview on April 7. On April 26, Dr David Arnold, Vice President, Research & Development of the Reid organization, responded to Dixon in a Sun letter to the editor. In that letter, Dr Arnold cited a validity study in the Journal of Applied Psychology as proof that the Reid Report is a fair test for detecting dishonest job applicants. Of course, I examined that article, and discovered that the criterion in this study was the polygraph!

I then wrote Bruce Kent, Industrial Relations Manager of the Real Canadian Superstore, pointing out that such a validation study is scientifically worthless. That letter was obviously passed on to Dr Arnold, who kindly sent me a copy of the response to my charges that he wrote to Mr Kent.

Dr Arnold's response astounded me. Here is the business part:

Furthermore, due to the unreliability of the polygraph, any validity estimate of a paper-and-pencil test gathered using the polygraph as a criterion would of course be an underestimate of the written test's actual validity.

First, it is interesting that he so readily admits that the polygraph is unreliable, after years of arguing the exact opposite, before his company was prevented by US law from using it as a personnel tool.

Second, I was astounded by the obviousness of this fallacy. Here's the way I explained it to Dr Arnold:

[S]uppose that I hold that weight is positively correlated with honesty (an assumption I hope you will grant is false). Now, suppose I attempt to establish this claim by validating my weight test against a test of subject's height, and further foolishly assume that height is positively correlated with honesty. Now since there is a positive correlation between weight and height (less than 1, but greater than the .40 you claim in your study -- suppose it is .50), I would have a reliability coefficient of .50 for the weight test when the criterion is height, and 0 when the criterion is honesty. This is exactly the same blunder you have made in the quoted passage above.

To date, the BCCLA has not heard from the Real Canadian Superstore about its policy of using the Reid Report, despite my pointing out to them the quality of argument Reid Psychological Systems uses to defend its test. We raised several other criticisms of the evidence Reid has presented in favour of the validity of its test. However, at this point, the only bright side to this story is that we have no evidence that they are using graphology.

Home pages: BC Skeptics, PSG.