Higgs boson

12 replies [Last post]
tonyohagan
Offline
Joined: 2009-10-19

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC needed convincing evidence before they would announce that a particle had been found that looks like (in the sense of having some of the right characteristics of) the elusive Higgs boson. Specifically, the news referred to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an extreme significance level. Five standard deviations, assuming normality, means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian perspective that this only makes sense if (a) the existence of the Higgs boson (or some other particle sharing some of its properties) has extremely small prior probability and/or (b) the consequences of erroneously announcing its discovery are dire in the extreme. Neither seems to be the case, so why 5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to do a proper Bayesian analysis. Are the particle physics community completely wedded to frequentist analysis? If so, has anyone tried to explain what bad science that is?

3. We know that given enough data it is nearly always possible for a significance test to reject the null hypothesis at arbitrarily low p-values, simply because the parameter will never be exactly equal to its null value. And apparently the LNC has accumulated a very large quantity of data. So could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

jeffmo2334
Offline
Joined: 2010-01-29
RE: Higgs boson

If someone can get these data to do a Bayesian analysis, I would be interested in seeing the results (and evaluating the underlying assumptions!)

Jeff Morris

--

---Original Message-----
From: ISBA Webmaster [mailto:hans@stat.duke.edu]
Sent: Tuesday, July 10, 2012 8:47 PM
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC needed convincing evidence before they would announce that a particle had been found that looks like (in the sense of having some of the right characteristics of) the elusive Higgs boson. Specifically, the news referred to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an extreme significance level. Five standard deviations, assuming normality, means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian perspective that this only makes sense if (a) the existence of the Higgs boson (or some other particle sharing some of its properties) has extremely small prior probability and/or (b) the consequences of erroneously announcing its discovery are dire in the extreme. Neither seems to be the case, so why 5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to do a proper Bayesian analysis. Are the particle physics community completely wedded to frequentist analysis? If so, has anyone tried to explain what bad science that is?

3. We know that given enough data it is nearly always possible for a significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are receiving this message as you have opted-in to receive email from bayes-news, the Valencia list or the ISBA forums. To opt out or change your email subscriptions please login at http://bayesian.org/user. Go to the My Account menu and select Forum Email Integration. Check the forums to which you wish to subscribe, then save your settings. You may view past and present content at any time by visiting http://bayesian.org/forum

Guest
RE: Higgs boson

I'd offer the following partial answers:

1. A 5-sigma rule has become a tradition in particle physics, but it is not clear why.
2. Yes, there have been Bayesian monographs written by and for physicists. Enrico Fermi seems to have recognized the issue. I cite two publications at http://for-sci-law-now.blogspot.com/2012/07/probability-that-higgs-boson...
3. Given the confluence of theory and other, less compelling experiments, it looks like there is a particle with the predicted mass producing the observed decay products. At any rate, that is the consensus of the physics community.

Cavaet: I am no physicist. I am just synthesizing what I have read.

David Kaye
http://www.personal.psu.edu/dhk3

________________________________________
From: ISBA Webmaster [hans@stat.duke.edu]
Sent: Tuesday, July 10, 2012 9:46 PM
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

HARRY@HEP.FSU.EDU
Offline
Joined: 2011-03-01
Re: Higgs boson

[Moderator's note: this message is from
Harrison B. Prosper
Kirby W. Kemper Professor of Physics
Distinguished Research Professor
Florida State University
harry@hep.fsu.edu
]

Dear Tony,

First some general remarks, then I'll try to answer your questions.

I am in an interesting position regarding the "Higgs" boson discovery: I am thrilled to be an insider with respect to the discovery and I happen also to be one of the relatively few particle physicists who actually regard Bayesian reasoning as "exactly what is needed" to make sense of what we do. The vast majority of my colleagues believe that p-values are objective and therefore "scientific". Therefore, many of my colleagues move mountains, or at any rate consume prodigious amounts of computing power, to check that some (typically ad hoc) procedure covers.

For your edification I've attached a (PUBLIC!) plot [Moderator's note: available at http://bayesian.org/webfm_send/274 ] of (slightly massaged) binned data from my collaboration (CMS) that shows a spectrum arising from proton-proton collisions that resulted in the creation of a pair of photons (gammas in high energy argot). The Standard Model predicts that the Higgs boson should decay (that is break up) into a pair of photons. (The Higgs is predicted to decay in other ways too, such as a pair of Z bosons.) The bump in the plot at around 125 GeV is evidence for the existence of some particle of a definite mass that decays into a pair of photons. That something, as far as we've been able to ascertain, is likely to be the Higgs boson. These data, along with data in which proton-proton collisions yield two Z bosons are the basis of our 5-sigma claim.

These data can be modeled with the function

f(x) = exp(a0 + a1*x + a2*x^2) + s * Gaussian(x|m, w)

where "x" is the mass the di-photon (pair of photons), and the first term describes the smoothly falling (background) spectrum, while the second term models the bump. "s" is the total expected signal, "m" is the mass of the new particle and "w" is the width of the bump. The total background, that is, the "noise" is just the integral of the first term. "a0", "a1", "a2" are nuisance parameters. This is therefore a 6-parameter problem for which we have no prior information (or choose to act as if this is so) for the six parameters. The analysis of this spectrum has caused a lot of angst about the "look-elsewhere-effect" (multiple hypothesis testing), which I think is a red herring in this context.

Now for your questions. See below.
Harrison

Quoting ISBA Webmaster <hans@stat.duke.edu>:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

The "5-sigma" (p-value = 3.0e-7) is an historical artifact. Over the past several decades, we have made many a "discovery" that turned out not to be so. As a consequence, we gradually settled on a p-value thought to be small enough to reduce the chance that we are fooling ourselves. In fact, we do have high standards because in our view we are trying to arrive at "true" statements about the world in the pragmatic sense that these statements yield predictions that turn out to be correct. Given that the search for the Higgs took some 45 years, tens of thousands of scientists and engineers, billions of dollars, not to mention numerous divorces, huge amounts of sleep deprivation, tens of thousands of bad airline meals, etc., etc., we want to be sure as is humanly possible that this is real.

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

I for one would be delighted to see a Bayesian analysis of these data from you guys! Unfortunately, however, I am forbidden from e-mailing you the 50,000 "x"s I have right here on my laptop...very frustrating...

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

As noted above, small p-value-based "discoveries" have come and gone.

However, the reason I am convinced this is real is not because of the p-value, nor frankly because of the (pseudo)-frequentist method of analysis showcased on July 4th during the announcement at CERN, a method that I have repeatedly criticized within my collaboration. Rather it is because when I study the profile likelihood in the variables "s", "m", and "w" for the 2011 dataset and for the 2012 dataset, I find visually convincing structures in the profile likelihood at m ~ 125 GeV in both independent datasets, obtained at different proton beam energies (7 TeV and 8 TeV). Of course, I would rather have preferred to have done a Bayesian analysis, marginalizing over "a0", "a1", "a2", and "w", and to study the posterior density in the variables "s" and "m", but constructing a non-evidence-based prior in 4-dimensions that would pass muster seems quite a chore. Any advice from You would be welcome. (I favor the recursive reference prior algorithm of Bernardo, but this would have to be done numerically and I have not yet figured out how to do so efficiently, while taking into account the "nested compact sets". The whole thing seems rather daunting.)

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

--
Harrison B. Prosper
Kirby W. Kemper Professor of Physics
Distinguished Research Professor
Florida State University
harry@hep.fsu.edu
Tel: 850 644 6760
Fax: 850 644 6735

In the best character there is an element of pride - not the sort of
pride that despises others, but the sort that will not be deflected
from what it thinks good by outside pressure.

"The Value of Free Thought", Bertrand Russell

----------------------------------------------------------------

davd95
davd95's picture
Offline
Joined: 2011-01-27
Re: Higgs boson

Hi Tony,

There is a group of high energy physicists that think very hard of source detection, model comparison, and statistical foundations. They host periodic workshops (https://plone4.fnal.gov:4430/P0/phystat/ and most recently http://www-conf.slac.stanford.edu/statisticalissues2012/) and regularly invite prominent statistician (Bayesian or otherwise) to participate. These meetings are great fun because of the active and informed dialogue on exactly the sorts of issues that you mention. Some physicists in this group remain skeptical about Bayesian procedures while others advocate for Bayes.

Unfortunately, this group does not get the final word on the statistical procedures that are actually used. The search for the Higgs boson in both incredibly high profile and incredibly expensive. There is a bureaucracy that determines in advance the procedures (statistical and otherwise) that will be implemented. Like most bureaucracies, this one is rather conservative and is unlikely to be far out-of-step with the general consensus of its scientific community, a community that is much larger than group that participates in Phystat workshops. So far, the official statistical methods have not been Bayesian.

As for why high-energy physicists require "5 sigma" to claim a discovery, there have been high profile cases where seemingly highly significant detections have had to be recalled. The response of editors at the leading journals in the field has been to require "more sigma". My personal opinion on the matter is that the problem is not with the significance level of the tests, but that there are statistical effects that are nor accounted for properly by the procedures (e.g., model misspecification or "systematic errors", unaccounted for effects of uncertainty in instrument calibration, and the so-called "look elsewhere effect", a sort of multiple testing).

If you are interested in reading more, there are a couple of good places to start:

1. An overview of statistical issues in particle physics<http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aoas/1223908045> by Louis Lyons published in The Annals of Applied Statistics (Vol 2, pp. 887-915).

2. An older discussion pape<http://www2.imperial.ac.uk/~dvandyk/presentations-astrostat.php>r in Statistical Science by Mark Mandelkern (Vol 17, pp. 149-172)

3. I discussed the question of "why 5 sigma" at a talk in a 2010 BIRS Workshop: Talk #15 at http://www2.imperial.ac.uk/~dvandyk/presentations-astrostat.php<http://w...

David

David A. van Dyk
Chair in Statistics
Department of Mathematics
Imperial College London SW7 2AZ

URL: http://www2.imperial.ac.uk/~dvandyk/
phone: +44 20 7594 8574
office: 535 Huxley Building

On Jul 11, 2012, at 2:46 AM, ISBA Webmaster wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk<mailto:a.ohagan@sheffield.ac.uk>
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

GIULIO.DAGOSTIN...
Offline
Joined: 2011-03-01
Re: Higgs boson

Dear Tony, dear all,

this paper provides some hints

http://arxiv.org/abs/1112.3620
(see also http://www.roma1.infn.it/~dagos/badmath/index.html#added )

Moreover

- The "higgs p-values" does not seem to be what a professional
(frequentistic) statistician would mean by that term:
-> there is no serious null hypothesis without Higgs, because
a Standard Model without Higgs mechanism loses completely meaning.
-> what has the meaning of a p-value depending on the mass?
(a number, calculated in the hypothesis that the Higgs does not
exist, reported in function of its mass...)

- Also the "95% CL exclusion" regions have dubious meaning
because they are derived by "prescriptions" that do not
provide a quantitative statement of how much we should
be confidence on something.

Regards, and best greetings to Dennis,

Giulio

On Tue, 10 Jul 2012, ISBA Webmaster wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------



************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

l.lyons1@physic...
Offline
Joined: 2011-03-02
RE: Higgs boson

Hello Bayesians,
Below are a few answers. Thnaks for the interest
Louis Lyons (organiser of PHYSTAT series of meetings, and member of CMS Collaboration at CERN.)
________________________________________
From: ISBA Webmaster [hans@stat.duke.edu]
Sent: 11 July 2012 02:46
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson.
************ The test statistic we use for looking at p-values is basically the likelihood
ratio for the two hypotheses (H_0 = Standard Model (S. M.) of Particle Physics, but no Higgs;
H_1 = S.M with Higgs). A small p_0 (and a reasonable p_1) then implies that H_1 is a better
description of the data than H_0. This of course does not prove that H_1 is correct, but
maybe Nature corresponds to some H_2, which is more like H_1 than it is like H_0. Indeed
in principle data will never prove a theory is true, but the more experimental tests it survives,
the happier we are to use it - e.g. Newtonian mechanics was fine for centuries till the arrival
of Relativity.
************* In the case of the Higgs, it can decay to different sets of particles, and these rates
are defined by the S.M. We measure these ratios, but with large uncertainties with the present data.
They are consistent with the S.M. predictions, but it could be much more convincing with more
data. Hence the caution about saying we have discovered the Higgs of the S. M..

Specifically, the news referred
to a confidence interval with 5-sigma limits.
************** 5-sigma really refers to p_0

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?
********************** This is an unfortunate tradition, that is used more readily
by journal editors than by Particle Physicists. Reasons are
a) Historically we have had 3 and 4 sigma effects that have gone away
b) The 'Look Elsewhere Effect' (LEE). We are worried about the chance of a
statistical fluctuation mimicking our observation, not only at the given mass
of 125 GeV but anywhere in the spectrum. The quoted p-values are
'local' i.e. the chance of a fluctuation at the observed mass. Unfortunately
the LEE correction factor is not very precisely defined, because of ambiguities about
what is meant by 'elsewhere'
c) The possibility of some systematic effect (characterised by a nuisance parameter)
being more important than allowed for in the analysis, or even overlooked - see the
recent experiment at CERN which claimed that neutrinos travelled faster than the speed of
light.
d) A subconscious use of Bayes Theorem to turn p-values into probabilities about the
hypotheses.
All the above vary from experiment to experiment, so we realise that it is a bit unfair to
use the same standard for discovery for all analyses. We prefer just to quote the p-values
(or whatever).

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis?
************** No we are not anti-Bayesian, and indeed our test statistics is a likelihood ratio.
If you like, you can regard our p-values as an attempt to calibrate the meaning of a
particular value of the likelihood ratio.
************** We actually recommend that for parameter determination at the LHC, it is
useful to compare Bayesian and Frequentist methods. But for comparing hypotheses
(e.g. an experimental distribution is fitted by H_0 = a smooth distribution; or by H_1 = a smooth
distribution plus a localised peak), we are worried about what priors to use for the extra
parameters that occur in the alternative hypothesis. ******* We would welcome advice.**********

If so, has anyone tried to explain what bad
science that is?
*************** Comment ignored

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LHC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?
************** We are aware of this. But in fact, although the LHC has accumulated enormous
amounts of data, the Higgs search is like looking for a needle in a haystack. The final samples
of events that are used to look for the Higgs contain only tens to thousands of events.

If anyone has any answers to these or related questions,
***************** These and related issues are discussed to some extent in my
article "Open statistical issues in Particle Physics", Ann. Appl. Stat. Volume 2, Number 3 (2008),
887-915. It is supposed to be statistician-friendly

I'd be interested to know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

David Spiegelhalter
Offline
Joined: 2011-03-02
Re: Higgs boson

Dear Tony

I have written a bit about the explanation of the P-value here

http://understandinguncertainty.org/explaining-5-sigma-higgs-how-well-di...

The CERN teams' reports also discuss what they would expect were the
Higgs there, so there seems a real possibility of a likelihood ratio
being computed, which would be a start. Not sure why they don't do this.

d

On 11/07/2012 02:46, ISBA Webmaster wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------



************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

--


David Spiegelhalter
Winton Professor for the Public Understanding of Risk
Statistical Laboratory
Centre for Mathematical Sciences
Wilberforce Road
Cambridge
CB3 0WB
UK

Tel: +44 (0)1223 337945
Fax: +44 (0)1223 337956

www.statslab.cam.ac.uk/Dept/People/Spiegelhalter/davids.html
www.understandinguncertainty.org

larrywasserman
Offline
Joined: 2012-05-22
Re: Higgs boson

Hi Tony

I was on the statistics committee for one of the detectors
at Fermi-Lab for a year. These physicists think about these things pretty
carefully.

The 5-sigma rule is for two-reasons; first, the stakes are very high
and protecting against a false positive is very important. Billions of
dollars
are at stake not to mention Nobel prizes. Also, the entire future direction
of particle physics research depends on getting it right.
Second, there is much more to the analysis than just a single
hypothesis test. Using a small p-value cut-off protects from
implicit multiple testing problems.

They are aware of Bayesian methods. There are some particle physicists
who write about them. But they mainly use frequentist methods for good
reasons: they are objective. (Niether you or Dennis will agree with that
but I do.)

The p-value is not illusory. This is not social science where the
null is always false and it is only a matter fo time until we reject.
In this case, the physics defines the null and alternatively clearly.

I hope this helps

Larry

On Tue, Jul 10, 2012 at 9:46 PM, ISBA Webmaster <hans@stat.duke.edu> wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news
referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously
announcing
its discovery are dire in the extreme. Neither seems to be the case, so
why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better
to
do a proper Bayesian analysis. Are the particle physics community
completely
wedded to frequentist analysis? If so, has anyone tried to explain what
bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low
p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested
to
know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------



************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

--


Larry Wasserman
Professor, Department of Statistics
and Machine Learning Department
Carnegie Mellon University
larry@stat.cmu.edu
www.stat.cmu.edu/~larry <http://www.stat.cmu.edu/%7Elarry>

-------

Please DO NOT SEND Microsoft Word attachments to me. Word is a
secret proprietary format, so I cannot read it. If you send me pdf,
plain text or HTML, then I can read it.

See
www.gnu.org/philosophy/no-word-attachments.html
for more information.

JPRATT@HBS.EDU
Offline
Joined: 2011-03-01
RE: Higgs boson

Is there any possibility of getting a Savage (et al.) - Berger (et al.) type of bound for a realistic version of this problem? It would be to the point and "objective".
Best to you all,
John

________________________________________
From: ISBA Webmaster [hans@stat.duke.edu]
Sent: Tuesday, July 10, 2012 9:46 PM
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

--

--
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

bill@astro.as.u...
Offline
Joined: 2009-10-05
Re: Higgs boson

[Moderator's note: this message is from
Bill Jefferys
bill@astro.as.utexas.edu

Department of Astronomy
The University of Texas at Austin

Department of Mathematics and Statistics
The University of Vermont
]

Dear Tony,

A CERN physicist wrote the following blog entry, which directly
addresses your third point:

http://www.science20.com/quantum_diaries_survivor/blog/keeplooking_bias

I am not a high-energy physicist, although I subscribe to a list that is
inhabited by a lot of them and where these issues are discussed almost
every day. As others have noted, and from my discussions with these
high-energy physicists, the reason for 5 sigmas is just that they have
been burned too often using less stringent criteria. And yes, it is a
problem in this expensive field when you announce results that you may
have to withdraw later. Better to be cautious seems to be the attitude.

It does seem that there is a preference in this field for
frequentist/likelihoodist statistics, though there are some who do use
Bayesian methods; the journals want frequentist statistics, however.

[Moderator's note: the following link was supplied by Bill in a follow-up comment:

http://www.technologyreview.com/view/428428/higgs-boson-may-be-an-impost...

This is one of the reasons for the very cautious words that you heard in
the press conference!

]

Bill Jefferys

Department of Astronomy
The University of Texas at Austin

Department of Mathematics and Statistics
The University of Vermont

--

--

On 7/10/12 9:46 PM, ISBA Webmaster wrote:

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------



************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

HRUBIN@STAT.PUR...
Offline
Joined: 2011-06-08
RE: Higgs boson

[Moderator's note: the following message was sent from

hrubin@stat.purdue.edu

and references a message sent by:

JPRATT@HBS.EDU (John below)

]

On Wed, 11 Jul 2012, ISBA Webmaster wrote:

Is there any possibility of getting a Savage (et al.) - Berger (et al.) type
of bound for a realistic version of this problem? It would be to the point
and "objective".
Best to you all,
John

I do not agree with the pseudo-objective Berger approach, but I consider
the proper use of statistics to be in the balancing of risks. In most
cases, I do not believe it can be put in the form of a Bayes factor,
and alas, this is one of them.

In the assumed null model, there is a distribution of certain types
of events. If, and here are several ifs, there is a particle in a
fairly small mass range, which has the presumed properties of the
Higgs boson, the distribution due to it will be of a slightly different
form. So here we can get the likelihood ratio, or a good approximation
thereof, as a function of the frequency of Higgs decays, which we seem
not to have a good idea about; if we had, we could just use a likelihood
ratio test and get the Bayes factor.

So we are again in the situation of testing a point null versus
a non-null effect. Here we have a one-sided test, but I have
found (not yet written up) that the asymptotic results in my
paper with Sethuraman (Sankhya 1965) does work even for fairly
small samples. What counts is the density under the alternative
relative to the mass of the null in the units used. If one
does not wish to make this constant locally, but have it behave
like a power or something else, that change should be made.

If the Type II risk is constant, the result calculates the
posterior probability of the alternative; the prior probability
is unimportant, as large values of the alternative parameter are
going to reject with very high probability. These large values
definitely have not been found. And while the calculations in
our paper technically have an infinite prior integral for the
alternative, that is irrerlevant, again, even for moderate
sized samples.

________________________________________
From: ISBA Webmaster [hans@stat.duke.edu]
Sent: Tuesday, July 10, 2012 9:46 PM
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of
answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC
needed convincing evidence before they would announce that a particle had
been found that looks like (in the sense of having some of the right
characteristics of) the elusive Higgs boson. Specifically, the news referred
to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an
extreme significance level. Five standard deviations, assuming normality,
means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian
perspective that this only makes sense if (a) the existence of the Higgs
boson (or some other particle sharing some of its properties) has extremely
small prior probability and/or (b) the consequences of erroneously announcing
its discovery are dire in the extreme. Neither seems to be the case, so why
5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to
do a proper Bayesian analysis. Are the particle physics community completely
wedded to frequentist analysis? If so, has anyone tried to explain what bad
science that is?

3. We know that given enough data it is nearly always possible for a
significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So
could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to
know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are
receiving this message as you have opted-in to receive email from
bayes-news, the Valencia list or the ISBA forums. To opt out or change
your email subscriptions please login at http://bayesian.org/user. Go
to the My Account menu and select Forum Email Integration. Check the
forums to which you wish to subscribe, then save your settings. You
may view past and present content at any time by visiting
http://bayesian.org/forum

P.DIGGLE@LANCAS...
Offline
Joined: 2011-03-01
RE: Higgs boson

[Moderator's note: the following message was sent by

Peter Diggle
P.DIGGLE@LANCASTER.AC.UK

]

Well, the extremity of the physicists' p-values pales into insignificance (sorry!) when compared with p-values routinely quoted in genetics papers. I'm sure Nature has published p-values as small as 10^(-31).

Peter Diggle

--

---Original Message-----
From: ISBA Webmaster [mailto:hans@stat.duke.edu]
Sent: 11 July 2012 02:47
To: news@bayesian.org
Subject: Higgs boson

Dear Bayesians,

A question from Dennis Lindley prompts me to consult this list in search of answers.

We've heard a lot about the Higgs boson. The news reports say that the LHC needed convincing evidence before they would announce that a particle had been found that looks like (in the sense of having some of the right characteristics of) the elusive Higgs boson. Specifically, the news referred to a confidence interval with 5-sigma limits.

Now this appears to correspond to a frequentist significance test with an extreme significance level. Five standard deviations, assuming normality, means a p-value of around 0.0000005. A number of questions spring to mind.

1. Why such an extreme evidence requirement? We know from a Bayesian perspective that this only makes sense if (a) the existence of the Higgs boson (or some other particle sharing some of its properties) has extremely small prior probability and/or (b) the consequences of erroneously announcing its discovery are dire in the extreme. Neither seems to be the case, so why 5-sigma?

2. Rather than ad hoc justification of a p-value, it is of course better to do a proper Bayesian analysis. Are the particle physics community completely wedded to frequentist analysis? If so, has anyone tried to explain what bad science that is?

3. We know that given enough data it is nearly always possible for a significance test to reject the null hypothesis at arbitrarily low p-values,
simply because the parameter will never be exactly equal to its null value.
And apparently the LNC has accumulated a very large quantity of data. So could even this extreme p-value be illusory?

If anyone has any answers to these or related questions, I'd be interested to know and will be sure to pass them on to Dennis.

Regards,

Tony

----
Professor A O'Hagan Email: a.ohagan@sheffield.ac.uk
Department of Probability and Statistics
University of Sheffield Phone: +44 114 222 3773
Hicks Building
Sheffield S3 7RH, UK Fax: +44 114 222 3759
----------- http://www.tonyohagan.co.uk/ ------------

************************************************************************
ISBA now maintains the bayes-news and Valencia email lists; you are receiving this message as you have opted-in to receive email from bayes-news, the Valencia list or the ISBA forums. To opt out or change your email subscriptions please login at http://bayesian.org/user. Go to the My Account menu and select Forum Email Integration. Check the forums to which you wish to subscribe, then save your settings. You may view past and present content at any time by visiting http://bayesian.org/forum