Categories
Uncategorized

Probabilistic fallacies in law

The Prosecutor’s fallacy

This is the classic illustration: a murder takes place. The forensics team identify hair from the crime scene. A DNA test is carried out, indicating that someone with a previous conviction who lives in the same town is a match.

The suspect is arrested. An expert report is produced: it states that there is only a 1 in 10,000 chance that the DNA of the former convict and the hair sample will be a positive match. Naturally the Crown Court’s jury is persuaded: here is a scientific expert, stating that there is only a one-in-10,000 chance that the accused is not responsible for the DNA trace.

But of course, after a moment’s reflection, we ought to realise that, if the DNA database were to contain tens of thousands of entries, that would be expected to lead not to one but perhaps to a number of other potential matches. Should we ignore those innocent matches?

Algebraic expression of the fallacy

The prosecutor’s fallacy is when the probability of innocence, given the evidence, is wrongly conflated with the very tiny probability that the evidence would occur if the defendant were innocent. In terms of probabilistic formula, it is wrongly assuming that the probability of

P(Innocence | Evidence) [the probability of the accused being innocent given he matches the DNA]

is no different to

P(Evidence | Innocence) [the probability of an innocent person having a DNA match)

So: coming back to the example set out above, the probabilistic fallacy occurs if the jury confuse the (apparently) really tiny chance that there should be a DNA match if the accused were innocent, with the probability that the accused is in fact innocent in the light of the evidence.

They are not the same thing at all, although unfortunately this distinction does not jump out at us when verbally formulated: regrettably the two grammatical structures sound more like rephrased versions of each other. Instead, the confusing situation is best represented graphically, in the tree diagram as follows –

[Source: Norman Fenton and Martin Neil: ‘Avoiding Probabilistic Reasoning Fallacies in Legal Practice Using Bayesian Networks, presentation at 7th International Conference on Forensic Inference and Statistics (ICFIS) Lausanne, 21 August 2008]

In the example, only one of the 10,000 potential suspects can be the actual source of the DNA (i. e. the guilty person actually present at the scene and leaving a trace). However, because there is a 1/1,000 probability of there being a positive match among the larger 10,000-strong population of potential suspects, this means that about 10 (i.e. 10,000/1,000) innocent suspects would be a positive match.

The Bayesian party trick

The shocking, literally unbelievable conclusion we are forced to assent to, applying Bayes’ Theorem, is that you have to incorporate the objective statistical information and integrate it with the knowledge of the accused testing positive in order to interpret that properly.

The probability of the accused’s innocence, conditional on a positive DNA match, is not 1/10,000 (as would appear if you only considered the top ‘branch’ of the tree diagram above and as ‘common sense’ points to) but instead that there is a 10/11 (i.e. 90.1%) probability that the accused is innocent, based on the DNA match: in other words, out of the suspect population, around 10 would be expect to be DNA matches, so the accused, being a match, simply has a roughly 1/10 chance of guilt before we consider other evidence. Not guilty.

In finance the prosecutor’s fallacy is referred to as the base rate fallacy – the tendency in judgment to give preeminence to information specific to an individual and neglect the broader statistical context. It is, it would seem, a common problem in our habits of probabilistic reasoning across domains.

Bayes in the employment law context?

The use of Bayesian reasoning has been deprecated by the UK’s senior criminal courts for reasons Professor Fenton takes exception to. As an employment practitioner my concern is that our failure to apply rigorous probabilistic reasoning results in numerous errors of judgment, in circumstances where the nature of the judge’s decision-making process remains unexamined.

Leave a Reply

Your email address will not be published. Required fields are marked *