|
The prosecutor's fallacy is a fallacy commonly
occurring in criminal trials and elsewhere. A prosecutor has collected some
evidence (for instance a DNA match) and has an
expert testify that the probability of finding this evidence if the accused
were innocent is tiny. The fallacy is committed if one then proceeds to claim that the probability of the accused being innocent
is comparably tiny.
Why this is fallacious: a concrete counterexample
A concrete example can make it clear why this reasoning is fallacious. Suppose there is a one-in-a-million chance of a match
given that the accused is innocent. The prosector says that means there is only a one-in-a-million chance of innocence. But in a
community of 10 million people, one expects 10 matches, and the accused is just one of those ten. That would indicate only a
one-in-ten chance of guilt, if no other evidence is available.
Why this is fallacious: an abstract reason
The fallacy lies in the fact that the a priori probability of guilt is not taken into account. If this probability is
small, then the only effect of the presented evidence is to increase that probability somewhat, but not necessarily
dramatically.
Examples
Consider for instance the case of Sally Clark, who was accused in 1998 of having killed her first child at 11 weeks of age, then conceived another child and killed
it at 8 weeks of age. The prosecution had expert witness Sir Roy Meadow testify that the probability of two children dying from sudden infant death syndrome is about 1 in 73
million. To provide proper context for this number, the probability of a mother killing one child, conceiving another and killing
that one too, should have been estimated and compared to the 1 in 73 million figure, but it wasn't. Ms. Clark was convicted in
1999, resulting in a press release by the Royal Statistical Society which pointed out the mistake. (See link at end of article.) Sally
Clark's conviction was eventually quashed on other grounds on appeal on 29th January 2003.
In another scenario, assume a rape has been committed in a town, and 20,000 men in the town have their DNA compared to a
sample from the crime. One of these men has matching DNA, and at his trial, it is testified that the probability that two DNA
profiles match by chance is only 1 in 10,000. This does not mean the probability that the suspect is innocent is 1 in
10,000. Since 20,000 men were tested, there were 20,000 opportunities to find a match by chance; the probability that there was
at least one DNA match is 1-(1-1/10000)^20000, about 86% -- considerably more than 1 in 10,000. (The probability that
exactly one of the 20,000 men has a match is 20000*(1/10000)*(1-1/10000)^19999, or about 27%, which is still rather
high.)
Another instance of the prosecutor's fallacy is sometimes encountered when discussing the origins of life: the probability of life arising at random out of the physical laws is estimated to be tiny, and this is
presented as evidence for a creator, without regard for the possibility that the probability of such a creator could be even
tinier.
Mathematical explanation of the general prosecutor's fallacy
We can view finding a person innocent or guilty in mathematical terms as a form of binary classification.
We start with a thought experiment. I have a big bowl with
one thousand balls, some of them made of wood, some of them made of plastic. I know that 100% of the wooden balls are white, and
only 1% of the plastic balls are white, the others being red. Now I pull a ball out at random, and observe that it is actually
white. Given this information, how likely is it that the ball I pulled out is made of wood? Is it 99%? No! Maybe the bowl
contains only 10 wooden and 990 plastic balls. Without that information (the a priori probability), we cannot make any
statement. In this thought experiment, you should think of the wooden balls as "accused is guilty" or "life originated from a
creator", the plastic balls as "accused is innocent" or "life emerged without a creator", and the white balls as "the evidence is
observed" or "life developed".
The fallacy can be analyzed using conditional
probability: Suppose E is the evidence, and G stands for "guilt". We are interested in Odds(G|E) (the odds that the accused
is guilty, given the evidence) and we know that P(E|~G) (the probability that the evidence would be observed if the accused were
innocent) is tiny. One formulation of Bayes' theorem then states:
- Odds(G|E) = Odds(G) · P(E|G)/P(E|~G)
Without knowledge of the a priori odds of G, the small value of P(E|~G) does not necessarily imply that Odds(G|E) is
large.
The prosecutor's fallacy is therefore no fallacy if the a priori odds of guilt are assumed to be 1:1. In an Bayesian approach to personal probabilities, where probabilities
represent degrees of belief of reasonable persons, this assumption can be justified as follows: a completely unbiased person,
without having been shown any evidence and without any prior knowledge, will estimate the a priori odds of guilt as
1:1.
In this picture then, the fallacy consists in the fact that the prosecutor claims an absolutely low probability of innocence,
without mentioning that the information he conveniently omitted would have led to a different estimate.
In legal terms, the prosecutor is operating in terms of a presumption of guilt, something which is contrary to the normal
presumption of innocence where a person is assumed
to be innocent unless found guilty. A more reasonable value for the prior odds of guilt might be a value estimated from the
overall frequency of the given crime in the general population.
Defendant's fallacy
The defendant's fallacy (taking the earlier example) says "We would expect 10 matches in this city of 10 million people, so
this particular piece of evidence suggests there is 90% chance that the accused is innocent. So this evidence cannot be used to
point to a conclusion of guilt, and should be excluded."
The problem with the defendant's argument is that there may be other available evidence which on its own is also not
conclusive. For example if CCTV cameras surrounding the scene of the crime spotted all one
hundred people there at the relevant time, one of which was the accused, then the defendant could claim: "The photograph suggests
a 99% chance that the defendant is innocent. The match suggested a 90% chance of innoncence. So the conclusion should be a
finding of innocence."
When the photographic evidence is combined with the match, the two together point strongly towards guilt, since (assuming the
chance of being in the photograph and having the match are independent) the chance that the accused is innocent falls to about
0.01%.
See also
External links
|