I recently published a letter to the editor in the journal Environmental Health Perspectives in response to a commentary that got my attention. The commentary raised a provocative and fundamental question that has come up from time to time since I began consulting for the legal profession:
What is the standard of proof for a scientist to accept a specific theory?
Stated another way: Is there a standard a scientist can rely upon to quantify how certain he is that a given scientific hypothesis is true? Is he 51% certain, 75% certain, 95% certain? And, importantly, should this level of certainty be more rigorous for a scientist than for a juror in a civil court case?
I have heard judges ask this question of counsel in various contexts, including cases involving toxic torts and pharmaceutical and medical device litigation. Even more troubling, I have heard plaintiff counsel confidently assert that the scientiifc standard of proof is more rigorous than the legal standard of proof.
Indeed, some have tried to impose the statistical standard for hypothesis testing (commonly agreed to be 95% certainty) as synonymous with a quantitative standard for accepting a specific scientific hypothesis. And by invoking this standard, they argue that the scientific standard is more rigorous than the legal standard of "more probable than not." In other words, the 95% certainty that we should reject the null hypothesis is compared to the 51% certainty required in a civil action.
But this is clearly nonsense. The scientific standard for hypothesis testing is not a surrogate for the certainty that a scientific theory is correct. Rather, the often-utilized scientific standard for hypothesis testing is used as a cut-off for how certain a scientist is that he would obtain data as extreme as he did, given the null hypothesis is correct. When the probability is sufficiently low (commonly agreed to be below 5%), the scientist rejects the null hypothesis and accepts the alternative hypothesis.
So, what is the quantitative standard that scientists use to determine if their theory is correct? What is the burden of proof in science?
The truth is that scientists don't think this way, nor do I think there is a quantitative method for defining this level certainty in science or in a legal setting. Despite the well recognized "more probable than not" standard in the courtroom, I am not even sure that jurors think this way either. In both cases - in the courtroom and in the laboratory - it seems that we make binary decisions as to whether we accept a theory or reject it. When the evidence gets to a certain point, we accept the theory. To pretend that scientists (or jurors) have some sort of calculus to determine how certain they are that a scientific theory is correct seems contrary to how people make decisions.
My friend and colleague Nathan Schachtman has a recent blog post that addresses this topic quite rigorously.