|
Bayesianism is the philosophical tenet that the
mathematical theory of probability applies to the degree of plausibility of
statements, or to the degree of belief of rational agents in the truth of statements; when used with Bayes theorem, it then becomes Bayesian inference. This is in contrast to frequentism, which rejects degree-of-belief interpretations of mathematical probability, and assigns
probabilities only to random events according to their relative frequencies of occurrence. The Bayesian interpretation of
probability allows probabilities assigned to random events, but also allows the assignment of probabilities to any other kind of
statement.
Whereas a frequentist and a Bayesian might both assign probability 1/2 to the event of getting a head when a coin is tossed,
only a Bayesian might assign probability 1/1000 to personal belief in the proposition that there was life on Mars a billion years
ago, without intending to assert anything about any relative frequency.
History of Bayesian probability
"Bayesian" probability is named after Thomas Bayes, who proved a special
case of what is called Bayes' theorem. (However, the term "Bayesian"
came into use only around 1950, and in fact it is not clear that Bayes would have endorsed
the very broad interpretation of probability now called "Bayesian".) Laplace independently proved a more general version of Bayes' theorem and put it to good use in
solving problems in celestial mechanics, medical statistics and, by some accounts, even jurisprudence.
For instance, Laplace estimated the mass of Saturn,
given orbital data that were available to him from various astronomical observations. He presented the result together with an
indication of its uncertainty, stating it like this: `It is a bet of 11000 to 1 that the error in this result is not within
1/100th of its value'. He would have won the bet, as another 150 years' accumulation of data has changed the estimate by only
0.63%.
The general outlook of Bayesian probability, promoted by Laplace and several later authors, has been that the laws of
probability apply equally to propositions of all kinds. Several attempts have been made to ground this intuitive notion in formal
demonstrations. One line of argument is based on betting, as expressed by Bruno de Finetti and others. Another line of argument is based on
probability as an extension of ordinary logic to degrees of belief other than 0 and 1.
This argument has been expounded by Harold Jeffreys, Richard T. Cox,
and Edwin Jaynes. Other well-known proponents of Bayesian
probability have included L. J.
Savage, Frank P. Ramsey, John Maynard Keynes, B.O. Koopman.
The frequentist interpretation of probability was preferred by some of the most influential figures in statistics during the
first half of the twentieth century, including R.A. Fisher,
Egon Pearson, and Jerzy Neyman. Thus for some decades the
Bayesian interpretation fell out of favor. Beginning about 1950 and continuing into the
present day, the work of Savage, Koopman, Abraham Wald, and others has led to broader acceptance.
Varieties of Bayesian probability
The terms subjective probability, personal probability, epistemic probability and logical
probability describe some of the schools of thought which are customarily called "Bayesian". These overlap but there are
differences of emphasis. There are also Bayesians who do not accept the subjectivity. The main protagonists of the objectivist
school are Edwin Thompson Jaynes (who died in 1998) and
Harold Jeffreys. Perhaps
the main protagonist now living is James Berger of Duke University. There are still others, such as Jose Bernardo, who accept
some degree of subjectivity but who believe there is a need for "reference priors" in many practical situations.
Advocates of logical probability would like to codify techniques whereby if two people have the same
information relevant to the truth of an uncertain proposition, then they would assign the same probability. No one has any idea
how to do that except in simple cases, and then the validity of proposed methods is subject to philosophical controversy. The
proponents of this view include Sir Harold Jeffreys, Richard Threlkeld
Cox, and Edwin Jaynes. Its critics challenge the
suggestion that it is possible or necessary in the absence of information to start with an objective prior belief which would be
acceptable to all.
Subjective probability is supposed to measure how sure an individual is of an uncertain proposition.
Bayesian and frequentist probability
The Bayesian approach is in contrast to the concept of frequency probability where probability is held to be derived from observed or imagined
frequency distributions or proportions of populations. The difference has many implications for the methods by which statistics is practiced when following one model or the other, and also for the way in
which conclusions are expressed. When comparing two hypotheses and using some information, frequency methods would typically
result in the rejection or non-rejection of the original hypothesis with a particular degree of confidence, while Bayesian
methods would suggest that one hypothesis was more probable than the other or that the expected loss associated with one was less
than the expected loss of the other.
Bayes' theorem is often used to update the plausibility of a given statement in light of new evidence. For example, Laplace
estimated the mass of Saturn (described above) in this way. According to the frequency probability definition, however, the laws of probability are not applicable to this problem. This is because the mass of Saturn is a constant
and not a random variable, therefore, it has no frequency distribution and so the laws of probability cannot be used.
Applications of Bayesian probability
Today, there are a variety of applications of personal probability that have gained wide acceptance. Some schools of thought
emphasise Cox's theorem and Jaynes' principle of maximum entropy as cornerstones of the
theory, while others may claim that Bayesian methods are more general and give better results in practice than frequency probability. See Bayesian inference for applications and Bayes' Theorem for the mathematics.
Bayesian inference is proposed as a model of the scientific method. It is claimed that updating probabilities via Bayes' theorem is similar to the scientific method, in which one starts with
an initial set of beliefs about the relative plausiblity of various hypotheses,
collects new information (for example by conducting an experiment), and adjusts
the original set of beliefs in the light of the new information to produce a more refined set of beliefs of the plausibility of
the different hypotheses. See Bayesian inference for more
information in this regard.
See also
External links and references
- David Howie: Interpreting Probability, Controversies and Developments in the Early Twentieth Century, Cambridge
University Press, 2002, ISBN
0521812518
|