Bayesian model comparison |
The posterior probability of a model given data, P(H|D), is given by Bayes' theorem:
- P(H|D) = P(D|H)P(H)/P(D)
The key data-dependent term P(D|H) is a likelihood, and is sometimes called
the evidence for model H; evaluating it correctly is the key to Bayesian model comparison.
The evidence is usually the normalizing constant or
partition function of another inference, namely the inference
of the parameters of model H given the data D.
The plausibility of two different models H1 and H2, parametrised by model parameter vectors theta1 and theta2 is assessed by
the Bayes factor given by
P(D|H2) / P(D|H1)
= integral{P(theta2|H2)P(D|theta2,H2)dtheta2}/{integral{P(theta1|H1)P(D|theta1,H1)dtheta1}}
References
- Richard O. Duda, Peter E. Hart, David G. Stork (2000) Pattern classification (2nd edition), Section 9.6.5, p.
487-489, Wiley, ISBN 0471056693
- Chapter 24 in Probability Theory - The logic of science by E. T. Jaynes, 1994.
- David J.C. MacKay
(2003) Information theory, inference and learning algorithms, CUP, ISBN 0521642981, (also available online )
External links
|