1
Methods of Experimental Particle Physics
Alexei Safonov
Lecture #25
2
Today• Brief reminder on upper limits discussion
from last time• More on parameter estimation• Combining measurements• Alternatives to ML• Walk through and interpret the CMS
results
3
One Sided Limits• It is typical in HEP to look
for things that are at the edge of your sensitivity• You frequently can’t “see”
the signal, but you still want to say that the cross-section for that new process can’t be larger than X
• Also very useful information for theorists and model builders as your results can rule out a whole bunch of models or parameter space of the models
• Can do it for either Bayesian or Frequentist methods • Most of the time fairly straightforward – either
construct a one-sided intervals with known coverage or calculate the integral from 0 to x in Bayesian case
4
Some good to remember numbers• Upper 90% and 95%
limit on the rate of a Poisson process in the absence of background when you observe n events• If you observe no
events, the limit is 3 events
• In Bayesian case, this would also be true for any expected B rate
5
Practical Parameter Estimation• You usually calculate –log(L)
• Assuming you are doing a measurement, the minimum of –log(L) is the maximum of L, so that gives the most likely parameter value
• Changing – log(L) by +/-1/2 (think of taking a log of a gaussian distribution – it will give you (x-x0)2/2s2 so x shifted by sigma gives ½) gives you 1 sigma deviation in the parameter (68% C.L.)
• With the MLS method, you vary it by 1 instead:
• MINUIT is the most used minimization package in HEP (it is part of ROOT), easy to use in simple cases, some experience required for more complex ones
6
Upper Limits Calculations• In case of upper limits, you will need to
integrate the distribution to find the point where 95% of the integral is accumulated• Any numeric integration method will do as the
integral is usually not too complicated
• I usually just keep halving the bin size until I get accuracy well below a fraction of a percent• Surely can be done more
efficiently, but if it takes no time who cares
7
Combining Measurements• Having likelihood function
makes combining measurements very easy• For a combined measurement,
the joint likelihood is a product of the likelihoods:
• Obvious if you consider one of the measurements as a prior and take into account that both measure the same value
• Correlated systematics may need to be accounted for • The product/limit will
be a factor sqrt(2) more narrow/lower• Kind of like doubling the
sample
8
Correlated Systematics• Having correlated systematics is not
unusual• E.g. the luminosity (L) measurement is
usually 100% correlated
• If the correlation is not 100%, include a prior correlation matrix
• The prior P in this case should be constructed to include the correlation matrix
• Can be easily done using Monte Carlo generation: generate pairs of x1 and x2 following the required correlation
9
Un-Binned Likelihood• Binned likelihood is easy to
interpret and gives consistent and predictable outcomes• But there is a choice of bin size
and strictly speaking you are loosing information by clamping things together
• Can you avoid that?• Decrease the size of the bins, in the extreme
limit - unbinned likelihood:
• where nu is the function value at the point where the data point happened to occur
• Is there a problem with this?
10
Un-Binned Likelihood• Key difference: binned likelihood knows about
areas where there is no data• It will try to push the fit function down to account for
the lack of data points
• Unbinned likelihood doesn’t account for empty “bins”• If you have areas in the histogram that are included
in the fit and which are
• One can correct for it (modified unbinned likelihood):
• The last term accounts for the empty bins
11
WALK THROUGH AND COMMENT ON THE CMS OBSERVATION PAPER
http://arxiv.org/pdf/1303.4571.pdf
12
Expected Upper Limits
• Mean expected • 95% C.L. upper limit on s/s(SM) in the
absence of Higgs • The p-value if Higgs exists versus its mass
13
Signal in Two Photon and ZZ Channels
• Pretty compelling, right?
14
Observed Upper Limits
• Three main channels• What can you tell about upward downward
fluctuations that happened in data based on these plots?
15
Two Less Important Channels
• How do you interpret these?
16
Upper Limit and Hypothesis Testing
• This includes all channels• Left is upper limit• Right is the “p-value” for the Higgs
hypothesis (not null hypothesis)
17
Local p-value
• Two channels
18
Combined Channels
19
Signal Strength and Mass
• q on the right is the test statistic (it’s the likelihood ratio based analysis)
20
Signal Strength: Channels
21
Next Time• Review ongoing and future analyses at
the LHC