We use cookies to support features like login and allow trusted media partners to analyse aggregated site usage. Keep cookies enabled to enjoy the full site experience. By browsing our site with cookies enabled, you are agreeing to their use. Review our cookies information for more details.
We use cookies to support features like login and allow trusted media partners to analyse aggregated site usage. Keep cookies enabled to enjoy the full site experience. By browsing our site with cookies enabled, you are agreeing to their use. Review our cookies information for more details.
We use cookies to support features like login and allow trusted media partners to analyse aggregated site usage. Keep cookies enabled to enjoy the full site experience. By browsing our site with cookies enabled, you are agreeing to their use. Review our cookies information for more details.
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details
This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Review our cookies information for more details
Babbage

Science and technology

Animal behaviour

Clever hounds

Feb 15th 2011, 9:05 by M.K.

IN THE early 20th century, a horse named Clever Hans was believed capable of counting and other impressive mental tasks. After years of great performance, psychologists put the ruse to rest by demonstrating that though Hans was certainly clever, he was not clever in the way that everyone expected. The horse was cunningly picking up on tiny, unintentional bodily and facial cues given out not only by his trainer, but also by the audience. Aware of the “Clever Hans” effect, Lisa Lit at the University of California, Davis, and her colleagues, wondered whether the beliefs of professional dog handlers might similarly affect the outcomes of searches for drugs and explosives. Remarkably, Dr Lit found, they do.

Dr Lit asked 18 professional dog handlers and their mutts to complete two sets of four brief searches. Thirteen of those who participated worked in drug detection, three in explosives detection, and two worked in both. The dogs had been trained to use one of two signals to indicate to their handlers that they had detected something. Some would bark, others would sit.

The experimental searches took places in the rooms of a church, and each team of dog and human had five minutes allocated to each of the eight searches. Before the searches, the handlers were informed that some of the search areas might contain up to three target scents, and also that in two cases those scents would be marked by pieces of red paper.

What the handlers were not told was that two of the targets contained decoy scents, in the form of unwrapped, hidden sausages, to encourage the dogs' interest in a false location. Moreover, none of the search areas contained the scents of either drugs or explosives. Any “detections” made by the teams thus had to be false. Recorders, who were blind to the study, noted where handlers indicated that their dogs had raised alerts.

The findings, which Dr Lit reports in Animal Cognition, reveal that of 144 searches, only 21 were clean (no alerts). All the others raised one alert or more. In total, the teams raised 225 alerts, all of them false. While the sheer number of false alerts struck Dr Lit as fascinating, it was where they took place that was of greatest interest.

When handlers could see a red piece of paper, allegedly marking a location of interest, they were much more likely to say that their dogs signalled an alert. Indeed, in the two rooms where red paper was present and sausages were not, 32 of a possible 36 alerts were raised. In the two where both red paper and sausages were present that figure was 30–not significantly different. In contrast, in search areas where a sausage was hidden but no red piece of paper was there for handlers to see, it was only 17. 

The dogs, in other words, were distracted only about half the time by the stimulus aimed at them. The human handlers were not only distracted on almost every occasion by the stimulus aimed at them, but also transmitted that distraction to their animals–who responded accordingly. To mix metaphors, the dogs were crying “wolf” at the unconscious behest of their handlers.

How much that matters in the real world is unclear. But it might. If a handler, for example, unconsciously “profiled” people being sniffed by a drug- or explosive-detecting dog at an airport, false positives could abound. That is not only bad for innocent travellers, but might distract the team from catching the guilty. Handlers' expectations may be stopping sniffer dogs doing their jobs properly.

Readers' comments

The Economist welcomes your views. Please stay on topic and be respectful of other readers. Review our comments policy.

Perturbed

If this description is correct the test is completely flawed.
Paragraph 3 explicitly says: handlers have been told that scents (of drugs or explosives) would be placed, and that some (but not all) of these scents were marked with red paper. In other words, all of the red paper found indicates scents that should be detected.
The handler then goes looking around, and see a little piece of red paper tucked in somewhere. Not alerting means that the dog has failed the test. The handler is supposed to, when red paper is seen, examine the dog if the dog shows marking behaviour, and answer honestly "Yes, it shows marking behaviour, my dog has confirmed the positive", or, "No, it does not show marking behaviour, my dog has failed to mark at a positive."
This gives the handler an enormous incentive to interpret whatever behaviour the dog shows as "confirming behaviour". The dog doesn't have to notice anything at all.
But this situation is completely unrealistic and says little about the real world. Because in the real world there are not "known positives" lying around. Handlers don't go around having been told where they true positives are. They might unwarranted SUSPECT where a positive is, but that's different from having been TOLD where a positive is.

Most Daft Sir in reply to Perturbed

I think the concern this raises is that if a handler wants to get his dog to pretend to detect his target scent he is able to.

'If a handler, for example, unconsciously “profiled” people being sniffed by a drug- or explosive-detecting dog at an airport, false positives could abound.'

I don't think that this study was flawed, it just shows that handlers perception have a strong effect on what kind of response a dog gives.

I would like to see them do a similar study where there are scents of explosives and drugs but the handlers are told that there are none, and see if their perception similarly affected the response to a positive scent.

MalvolioSF in reply to Perturbed

If this description is correct the test is completely flawed.That's not a flaw in the test; that's the point of the test: the handler's (in this case mistaken) beliefs are projected on to the dog.

The handler might not be told where contraband is, but he sees two black guys in a car, maybe he thinks he knows.

jpatt in reply to Perturbed

It seems you're missing the point, Perturbed. The study aimed to demonstrate the affect the dog handler's bias has on a dog's performance. In this case the handlers were biased by believing red paper marked positive targets, and due to this bias they reported that the dogs identified these targets.
This could affect the real world if handlers lead sniffer dogs to targets that they incorrectly believe to be positive, perhaps because the suspected individual looks suspicious or is of a certain race. Such behavior could result in false positives and, at the very least, distracts the dogs from locating actual drugs or explosives.

Perturbed in reply to jpatt

Except that the situation that the handler knows something is hidden has many and material differences from a situation when the handler only suspects something is hidden (even if the suspicion is due to an unconscious bias). That means the conclusion isn't transferable.
Here's one: When the handler "knows" something is hidden, the dog failing to mark means that handler and his dog has failed badly at their job. The handler therefore has a very strong incentive to say that the dog marked right there and then if the handler wants to come across as competent.
If the handler "suspects" someone on the street, the dog failing to mark does not mean that either has failed at their job. The handler doesn't have anything to gain by saying that the dog has marked. To the contrary, being affected by unwarranted suspicion and getting a lot of false positives would be failing at his job.
It seems several people frame the standard for dog handlers as that of scientific apparatuses that must be able to be repeatedly calibrated precisely under the most unusual conditions. They aren't. They are only supposed to work right under actual, practical, live, real conditions. And there are far better ways to test that.

Perturbed in reply to Most Daft Sir

"I think the concern this raises is that if a handler wants to get his dog to pretend to detect his target scent he is able to."

The handler was the one who reported that the dog had marked. The observed behaviour could be explained without attributing anything to the dog at all, just by saying that the problem was at the handler side, since the handler was primed to interpret any behaviour the dog showed as confirming behaviour.

Jive Dadson

In the US, the Fourth Amendment to the Constitution bars unreasonable searches and seizures. There is nothing reasonable about a search by a trained dog. The same amendment says that a judge must issue a warrant before a search can take place. The dogs are routinely used to search belongings and persons, and then if the handler sees, or thinks he sees, or claims he sees the dog "alert," then they will then go to a judge to get a warrant for a second search.

The whole thing is a travesty.

Aside to Pansapiens: I do not see how a study could be designed to differentiate between the dogs alerting and the handlers reporting an alert that did not happen.

steviesteveo

@LexHumana

Of course false positives matter in law enforcement. The more innocent people that get dragged out of the check in line when going on their holidays and searched for explosives or drugs the more innocent people who don't trust the police there will be.

The police needs law abiding people to think they're at least slightly good at their job. You can't just look through everyone's house or bag on the off chance.

firsys

There was a recent study on the ability of mice to detect
trace odours; they were shown to be much several times more
sensitive than dogs. Whether they would yield less false positives
is another question. Would mouse handlers convey their prejudices
to the mouse?
I chuckle at the thought of a bevy of mice on leads being led
around a busy airport.
John F

Julio Ra

Can't expect that dogs are perfect trackers, but they can be very useful especially at airports where there is considerable flow of people.

Also, we must think they are animals with instincts and feelings. Respect their existence here.

Michael Dunne

Interesting points raised both in the article as well as in the subsequent comments.

I am for having more dog/handler teams, both for narcotics and explosives. I think when all is said and done, they would be more effective than technology for surveillance of transportations networks.

Unless sniffer technologies improve dramatically (a la the urine/people sniffers experimented in Vietnam), I think good dog teams could prove quite effective (think of the "Midnight Express"-esque nerviousness it could generator from potential perps...)

Plus, I would rather get aggresively sniffed by a Giant Schnauzer or German Shephard or hound than suffer long lines held up by dubious technology/processes.

As for the side comments about the drug war - We will always have crime, so does that mean we should stop funding the police? Also, how likely is it that white bread america would concede to legalizing cocaine, heroin or other drugs (Charlie Sheen party favors?). Aside from weed, I doubt legalization would extend to many other substances (psychotropic varieties for mental health research?)..

Nirvana-bound

I guess to err is assinine, to be influenced canine!

But the whole narcotics act is such a brazen fiasco. An exercise in abject futility. A blatant charade, meant to hoodwink the public into thinking that drug trafficking is being addressed & under control.

Far from the truth! The so called Drug War has been a lost cause from the get go. A heinous trillion dollar game, being played out by the Cartels & the Cabals in close cahoots with the corrupt & sycophantic powers-that-be & their grovelling minions, salivating in the drug alleys for lucrative hand-outs.

So let the sick mind games continue, unabashedly! Afterall they can always blame it on the poor canines, when things go wrong!

Duganinja

It's too difficult to read a simplified layman's report of someone else's scientific process and remark accurately on the accuracy of the data, even so far as to comment on the process in which the data was collected. The gentleman gives a broad strokes report of the research, which inevitably produces an "I'd have done it better" mentality. Indeed, even scientists have their Monday morning quarterbacks.

@LexHumana Kudos. Quantity over quality will always cast a wider and thus more successful net in law enforcement. With packages vs. people, the bias is a completely different type of preformed ideal. Racial, class, or religious bias are far more damaging to innocent people than to innocent packages.

MarkHarrisonUK

Surely what the study has actually shown is that the dog-handler team have to be regarded as a pair of items with feedback between them, rather than dog being purely an "input" to the handler's assessment.

Whether profiling is unconscious or conscious, however, seems to be nothing to do with this study, and I'm not sure why it was reported as such.

Of course, whether or not profiling is "good" or "bad" depends on whether you think it is more important not the "oppress" certain groups, or actually catch bombs and drugs.

651columbia

How does the author extrapolate a test where handlers are given conscious inputs immediately before a search into evidence that "unconsciously profiling" would produce the same false positives?

Dr. Lit's test doesn’t seem designed to reach those conclusions?

trustbutverify

False Alarm indeed! What a silly article that turns logic on its head by claiming the handlers are interrupting the dogs' work. It basically concludes that the famed 19th century horse should be allowed to do the math.

The dogs are not meant to do the reasoning. They merely lend their sharp noses to the handlers. The handlers combine the dogs' noses with their other four senses to assess the situation. If the handlers are being prejudiced, surely they can be trained too.

On a side note, Israeli security forces, which use profiling unapologetically, have shown that it can be quite effective when used with purpose.

LexHumana

Zambino wrote: Feb 15th 2011 11:30 GMT
"I am greatly in favour of using more dogs in the fight against terrorism, particularly for air cargo. I know it costs about £25k (about $50k in the good 'ol days)to train the dogs, but I did not realise that the handlers would also require training not to let their prejudices get in the way. And most of these handlers are policeman - not exactly a group profiled as the least racist/prejudiced lot around."

All canine units are actually trained as a team. The handler and the canine are paired up and trained together, and a canine is never simply handed over to a random policeman. This is so the handler gets to know the unique behaviors of his or her detector dog, how they react, and how they behave when alerting. The handler is also trained to make sure the canine is not placed in a position where you might get ambiguous alerts (e.g. bags are separated out, rather than put in a great big pile). Additionally, canine teams are trained to alert for specific odors, and generally are not used for multipurpose sniffs -- a narcotics canine searches only for narcotics, an explosives canine searches only for explosives, a currency canine searches only for currency, etc. etc.

CalvinBama

In my small town I saw and heard of police dogs "Alerting" on cars many times without the cops finding anything. They always made the case that "there must have been drugs in there earlier, but they already smoked it all", but obviously it was due to the Police biases and passing them on to the dogs

LexHumana

Dan abroad wrote: Feb 16th 2011 5:07 GMT
"In the context of airport or border checks, the sheer volume of checking required means that excessive false positives will cause the system to fail. The Mexican example cited by @LaContra is one good example among many."

Technically, LaContra was not describing a false positive -- the dogs were in fact alerting to the actual residue of narcotics. The ruse by the smugglers was to taint as many bags as possible so that they APPEARED to be false positives to the handlers. The dogs were right, the handlers were wrong.

Again, false positives can happen for a variety of reasons, and they generally don't pose a huge problem -- the offending bag or parcel is searched, and if there is nothing there, it is sent on its way. As far as "causing the system to fail," you are assuming that the canines' false alerts are somehow frequent and random, but that is not the case -- the study only shows that if a handler is suspicious about a package, it is more likely for the dog to alert to that package. Of course, a handler is not going to be suspicious about EVERY package or bag, and if a handler is suspicious, it is usually for a reason (origin of the package, where it is being sent to, the weight and density, how it is wrapped, etc.)

As a thought experiment, assume that a handler is deliberately profiling bags and packages that are coming from Colombia as suspicious for narcotics (which is not an unreasonable search parameter). In a planeload of bags coming from Colombia, there is no reason to suspect one bag over the other (all are coming from Colombia) and therefore the dog would not be picking up on any particular "tells" from the handler -- any alerts would be because the dog smells something. Compare this to a search of inbound foreign mail from South America. The same handler is deliberately profiling Colombian parcels. The shipments of mail are from different locations, but the handler is particularly interested in Colombian parcels, so the dog may actually pick up on "tells" from the handler and alert to Colombian parcels. This, too, is not a problem - the quantity of Colombian parcels is already less in a mixed shipment, so there is not going to be an overwhelming number of alerts to deal with. Also, the parcels will either have narcotics in them or they won't, and if they don't they will be sent on their way without further fuss.

People seem to be under the mistaken impression that alerts must be 100% efficient. This is false. In fact, from a law enforcement perspective, if you are conducting searches and all of them come up 100% positive, that is actually a sign that your search parameters are not wide enough -- you are likely letting in packages or internal smugglers that are not meeting your narrow search criteria (i.e. you are only searching for the obvious "sure thing"). In the real world, law enforcement stops and inspects bags, parcels, and passengers and often comes up with a negative search, and this is simply considered par for the course.

Dr. Frost

@LexHumana

Strictly speaking, Lex, the dogs are not making mistakes, they are just trying to excell at what the always do: trying to please their handlers.

The dogs are trained to recognize certain things and are rewarded for their success with affection or treats. I doubt they understand that their masters do not have the same ability to detect as they do - in fact, their training may even prevent just such an insight, if the handlers are informed in advance where the positive test samples are.

So the dogs are looking to their handlers to see if they are successful - and are obviously clever enough to spot telltale body language indicating their handlers confidence in a positive (Cal Lightman would be proud).

The red paper turns out to have been a necessary setup to control for profiling, and I am amazed at the result.

Dan abroad

@LexHumana >> "False positives are no big deal from a law enforcement perspective".

I respectfully disagree.

In the context of airport or border checks, the sheer volume of checking required means that excessive false positives will cause the system to fail. The Mexican example cited by @LaContra is one good example among many.

LexHumana

I'm not sure what this study is attempting to prove, other than dogs can make mistakes, which is something every law enforcement agency already knows.

False positives are no big deal from a law enforcement perspective -- the only thing they result in is a more detailed search of the bag or parcel, and if nothing shows up in that search, the package is sent on its way. What would bother law enforcement is a false negative -- a contraband package that goes undetected.

I think it is also interesting that the packages were deliberately market as items of interest so that the handlers knew and could see them. Most drug dealers or terrorists don't mark their packages that way.

PSH

An interesting study, but no doubt courtesy of chronically short research funding, mathematically pretty weak. 18 dogs in a total of 144 tests, 72 of them apparently unique. 2-3 targets each to test several variables of a complex man-dog interaction. Sort of double blind, but no repeat runs, or evidence of deliberate randomness, much less controls.

Not the sort of thing to warm the heart of any statistician. More a test of the experimenters expectations perhaps?

On the other hand TSA has spent billions based on far weaker "proof". For equipment far less effective than dogs. Pity TSA didn't throw a few funding crumbs off their table toward UC Davis. But then they don't want to admit their fancy new scanners and such can't compare to a single mutt's nose.

LaContra

Dogs are only as good as their handlers and their accuracy limited by logistical constraints.

In the 1990's at the busiest border crossing in the US between Juarez and El Paso dogs were often used to sniff the luggage unloaded from buses coming from Mexico.

In some classic 'outside the box' thinking the smugglers focused on the dogs as the weak link in the busy, fast paced border operation and instead of trying to mask the scent of drugs they used the dogs talents against border agents themselves.

The young boy who loaded the bags onto the buses on the Mexican side for a few pesos, was also well paid by the smugglers to rub cocaine reside onto every bag loaded onto the chosen bus.

When the bags were unloaded at the border and placed in a line, some dogs would act confused, identifying every bag as a target, barking and chasing their tail, whilst others would simply sit there looking confused. The busy handlers with a line of vehicles backing up would invariably chide the dog for acting crazy and move on to the next vehicle..after all the dogs are not foolproof machines and the handlers recognise this.

With different handlers and dogs, at different times, and on different days of the month, it took several months before someone noticed a possible pattern regarding these incidents, investigated and thus uncovering the ruse.....but only after scores of kilograms of cocaine were shipped across the border right under their noses (pardon the pun).

Proving the old adage that 'where there is a will there is a way', and demonstrating why America cannot win its war on drugs.

pansapiens

"Recorders, who were blind to the study, noted where HANDLERS INDICATED that their dogs had raised alerts."
...so it is not clear if the dogs were signaling, or if the handlers were erroriously interpreting the dogs as signaling. Bad study design.

About Babbage

In this blog, our correspondents report on the intersections between science, technology, culture and policy. The blog takes its name from Charles Babbage, a Victorian mathematician and engineer who designed a mechanical computer.

Advertisement

Economist video

Explore trending topics

Comments and tweets on popular topics

Advertisement

Products & events

Advertisement