A case-control study of the effectiveness of bicycle safety helmets

Thompson, Rivara & Thompson. New England Journal of Medicine 1989, Vol 320 No 21 p1361-7.

Same data set used in this other paper by the same authors:
A case-control study on the effectiveness of bicycle safety helmets in preventing facial injury.
American Journal of Public Health, 1990; 80(12):1471-1474.

 

Download or print this paper

Original authors' abstract

Bicycling accidents cause many serious injuries and, in the United States, about 1,300 deaths per year, mainly from head injuries. Safety helmets are widely recommended for cyclists, but convincing evidence of their effectiveness is lacking. Over one year we conducted a case-control study in which the case patients were 235 persons with head injuries received while bicycling, who sought emergency care at one of five hospitals. One control group consisted of 433 persons who received emergency care at the same hospitals for bicycling injuries not involving the head. A second control group consisted of 558 members of a large health maintenance organization who had had bicycling accidents during the previous year.

Seven percent of the case patients were wearing helmets at the time of their head injuries, as compared with 24 percent of the emergency room controls and 23 percent of the second control group. Of the 99 cyclists with serious brain injury only 4 percent wore helmets. In regression analyses to control for age, sex, income, education, cycling experience, and the severity of the accident, we found that riders with helmets had an 85 percent reduction in their risk of head injury (odds ratio, 0.15; 95 percent confidence interval, 0.07 to 0.29) and an 88 percent reduction in their risk of brain injury (odds ratio, 0.12; 95 percent confidence interval, 0.04 to 0.40).

We conclude that bicycle safety helmets are highly effective in preventing head injury. Helmets are particularly important for children, since they suffer the majority of serious head injuries from bicycling accidents.

BHRF Commentary

This paper (TRT89) is by far the most frequently cited research paper in support of the promotion of cycle helmets. It is referred to by most other papers on helmets. In fact, many other papers, and nearly all helmet promotion policies, rely fundamentally upon the validity of its conclusions.

The claims that helmets reduce head injuries by 85% and brain injuries by 88% come only from this source, yet are quoted widely as fact. For example, a policy statement on bicycle helmets by the American Academy of Pediatrics in 2001 states: “The bicycle helmet is a very effective device that can prevent the occurrence of up to 88% of serious brain injuries.” [1]

The prospect of achieving such massive reductions in injuries to cyclists lies at the root of helmet promotion and helmet laws around the world. Those who have taken the trouble to analyse the paper in detail, however, have found it to be seriously flawed and its conclusions untenable. Moreover, by making different - but no less valid - assumptions, the conclusions change radically.

Groups of cyclists are too dissimilar
The most serious criticism concerns the considerable differences between the two main groups of cyclists upon which the research is based. Case-control studies are valid only if the 'control' group is representative of the population at risk (the cyclists who might suffer head injuries).

In this study, a comparison was made between 145 children treated in hospitals in Seattle for a head injury (the 'cases'), and a 'community control' group of 480 children who had, in one way or another, simply fallen from their bikes. A comparison of the two groups based mainly on helmet use of children under 15 years (21.1% of ‘control’ vs 2.1% of ‘case’ children) leads to the frequently quoted claim that the reduction in head injury due to helmets is 85%.

However, at the same time as this research was being carried out, there was a much more extensive survey of helmet use in the city of Seattle [2]. Of 4,501 child cyclists observed cycling around Seattle, just 3.2% wore helmets. This is not statistically different from the 2.1% of the hospital cases who were wearing helmets.

As well as having a helmet wearing rate 7 times that of the cyclists riding round Seattle, the ‘community control’ group came from higher income households and had parents with higher educational levels. The observational survey of child cyclists riding in Seattle found that helmet wearers were predominantly white, middle class, riding with their parents in parks, whereas the non-wearers were more often black or other races riding alone on busy city streets. The risk profile of these two groups would be quite different.

Why did the community control group fall from their bikes?
Before any claim is made about the efficacy of helmets, it is necessary to consider which group is more representative of the population of cyclists who might suffer head injury. If the sole difference between the two groups was that the former had fallen off their bikes, then the most significant conclusion would be that helmet wearing was associated with a 7-fold increase in the risk in accidents! Such a large increase in risk would negate any benefits of helmets.

However, the children who fell off their bikes were members of a group healthcare cooperative that may have promoted helmet wearing to its members. Children may also be persuaded to wear helmets when first learning to ride and likely to experience low-speed falls that rarely result in injury. Rather than a tendency for helmet wearers to take more risks, factors such as helmet promotion, higher income levels, or a natural tendency to wear helmets when learning to ride, may explain the differences.

With alternative control group, helmets show no benefit
As discussed above, it seems more likely that the 4,501 cyclists actually observed riding around Seattle were more typical of the norm than the 'community control' group, leading to the conclusion that helmets make no significant difference. This is also the conclusion from whole-population data around the world.

Mis-use of odds ratios – impossible benefit or none?
Although the authors call odds ratios “percentage reductions in risk”, it is more informative to use risk ratios (RR) = %HIH / %HIN where %HIH and %HIN are the percentages of helmeted and non-helmeted cyclists with head injuries. TRT89 reported data for cyclists attending the emergency department for non-head injuries. For this group, which can be considered as an alternative control, the risk ratio was 0.36, suggesting that helmets prevented 64% of head injuries.

McDermott et al.[3] obtained more information and reported numbers of head injuries excluding forehead lacerations in the TRT89 study. The risk ratio excluding forehead lacerations was 0.39, a small reduction in the estimated benefit of helmets.

McDermott’s data on hospital admissions also illustrates the folly of labelling odds ratios as risk ratios. 28.6% of adult cyclists who wore helmets still had head injuries. If helmets prevented 85% of head injuries, an impossible 191% of non-helmeted cyclists would have head injuries. The actual figure (38%) was higher than for helmet wearers, but the difference due to helmet wearing was not statistically significant. [3]

Involvement of motor vehicles – how similar were the collisions suffered?
Numerous studies show that virtually all cyclist deaths and the vast majority of debilitating brain injuries are caused by collisions with motor vehicles. According to one source, not a single helmeted cyclist in the TRT89 study was in collision with a motor vehicle [4]. This would suggest that helmeted cyclists were not representative of the population at risk; they may have had fewer head injuries because they were in lower impact collisions than non-wearers.

A study in Tucson, Arizona, of bike/motor vehicle collisions found that helmet wearers had less severe non-head injuries. The authors concluded: "This implies that nonusers of helmets tend to be in higher impact crashes than helmet users. It is possible that at least some of the 'protection' afforded helmet wearers in previous studies may be explained by safer riding habits rather than solely a direct effect of the helmets themselves" [5].

Age adjustment
The authors reported only 3 age categories: <15, 15-24 and 25+. However, a subsequent analysis of a subset of the same data [6] showed that 83% of children aged 0-4 suffered head injury, compared to 42% of 5-9 year olds and 23% of 10-14 year olds. Such large differences suggest that age adjustment in the original study may have been inadequate and hence the conclusions invalid.

General limitations of case-control studies
Recent research has shown that it is very difficult to determine the benefit of a treatment by comparing groups who choose to take it with those who do not. A combined analysis of 30 studies, all or which compared women who chose to use Hormone Replacement Therapy (HRT) with non-users, estimated that HRT reduced the risk of heart disease by 50%. But when the bias of self-selection was avoided by choosing at random who would take HRT, it was found that HRT may even increase the risk of heart disease! The higher socioeconomic status of women who chose HRT may have been associated with other factors such as better diet and more exercise. This, not HRT, was the real cause of their lower rate of heart disease [7].

As noted above, cyclists who choose to wear helmets differ considerably from non-wearers. The study of bike/motor vehicle collisions found that helmet wearers also had much less serious non-head injuries. These differences, rather than helmets, may also explain their lower rate of head injuries.

Conclusion
The study compares groups of cyclists who chose to wear helmets with those who did not. Many variables, such as the reasons for wearing a helmet and attitudes to risk, were not controlled for by the researchers and may have influenced the results.

Postscript
As already noted, numerous papers and helmet promotion policies cite this paper as justification for helmet laws. After claiming in 2001, that "The bicycle helmet is a very effective device that can prevent the occurrence of up to 88% of serious brain injuries", the first recommendation of the American Academy of Pediatrics was: "All bicyclists should wear properly fitted bicycle or multisport helmets each time they ride." [1]

It is important to recognise the flaws in this logic. Even if helmets are effective, it does not follow that all cyclists should wear them. Racing car drivers wear helmets, but not people driving to work. The difference is the level of risk. For the same reason, racing cyclists and mountain bikers often choose to wear helmets, but riding down a quiet road to the corner shop is a generally safe activity. Driving and cycling have similar risks per hour of serious head injury.

To be consistent and just, the public health benefit of mandatory helmet wearing for cycling needs to be viewed in the context of similar requirements for other road users, especially as research has suggested that motoring helmets are more effective than cycle helmets, and motor vehicle occupants suffer many more head injuries [8].

The three authors of this study, individually or as a team, are responsible for many other papers on cycle helmets, including meta anaylses of research (in which their own research has sometimes been dominant) such as the influential Cochrane Review of helmet effectiveness. The authors appear to have a deep personal commitment to the wearing of cycle helmets, and have also written outspoken campaigning articles in, for example, the British Medical Journal pressing strongly for helmet legislation, claiming that “helmets reduced the risk by 63-88% for head, brain, and severe brain injury among cyclists of all ages.” [9] The claimed 88% reduction, observed only in the TRT89 study, is valid only if the helmet wearing rate of the 'community control' group (and not the thousands of children observed cycling around Seattle) was representative of the population of cyclists at risk of head injury.

Commentaries on other papers by these authors :
  Effectiveness of bicycle safety helmets in preventing head injuries: a case-control study
  Cochrane Review: Helmets for preventing head and facial injuries in bicyclists

See also: Why it is wrong to claim that cycle helmets prevent 85% of head injuries and 88% of brain injuries
  Limitations of case-control studies – Contradictory evidence about the effectiveness of cycle helmets
  Conflicting evidence - a concern not only for cycle helmet research

References

[1] American Academy of Pediatrics. Bicycle Helmets. Pediatrics 2001;108(4):1030-1032.
Other similar claims:
“The National Highway Traffic Safety Administration (NHTSA) supports the enactment of bicycle helmet usage laws. … Bicycle helmets are 85 to 88 percent effective in mitigating head and brain injuries, making the use of helmets the single most effective way to reduce head injuries and fatalities resulting from bicycle crashes.” NHTSA. Traffic Safety Facts. Bicycle Helmet Use Laws. 2005.
“Wearing a bicycle helmet is an important countermeasure, since it reduces the risk of serious brain injury by up to 85%” Schieber, RA, Sacks, JA, Measuring community bicycle helmet use among children. Public Health Reports, 2001;116:113-121.
Canada.com’s body and health page makes an even more extravagant claim that over 85% of all serious head injuries from bike falls could have been prevented by wearing a helmet.
[2] DiGuisseppi CG, Rivara FP, Koepsell TD. Bicycle helmet use by children. Evaluation of a community-wide helmet campaign. JAMA 1989;262:2256-61.
[3] McDermott FT, Lane JC, Brazenor GA, Debney EA. The effectiveness of bicyclist helmets: a study of 1710 casualties. J Trauma 1993;34(6):834-44.
[4] Private communication with Snell, the helmet standards body that funded the TRT89 study.
[5] Spaite DW, Murphy M, Criss EA, Valenzuela TD, Meislin HW. A prospective analysis of injury severity among helmeted and non helmeted bicyclists involved in collisions with motor vehicles. Journal of Trauma 1991;31:1510-1516.
[6] Thompson DC, Thompson RS, Rivara FP. Incidence of bicycle-related injuries in a defined population. Am J Public Health 1990;80(11):1388-90.
[7] Lawlor DA, Davey Smith G, Ebrahim S. Commentary: The hormone replacement-coronary heart disease conundrum: is this the death of observational epidemiology? Int. J. Epidemiol. 2004;33(3):464-467.
[8] McLean AJ, Fildes BN, Kloeden CN, Digges KH, Anderson RWG, Moore VM. Prevention of head injuries to car occupants: an investigation of interior padding options. 1998. Federal Office of Road Safety: Report CR160
[9] Thompson DC, Rivara FP, Thompson RS. Bicycle helmets: it's time to use them. British Medical Journal, 2000 v321 p1035-6.