This is the classic illustration: a murder takes place. The forensics team identify hair from the crime scene. A DNA test is carried out, indicating that someone with a previous conviction who lives in the same town is a match.
The suspect is arrested. An expert report is produced: it states that there is only a 1 in 10,000 chance that the DNA of the former convict and the hair sample will be a positive match. Naturally the Crown Court’s jury is persuaded: here is a scientific expert, stating that there is only a one-in-10,000 chance that the accused is not responsible for the DNA trace.
But of course, after a moment’s reflection, we ought to realise that, if the DNA database were to contain tens of thousands of entries, that would be expected to lead not to one but perhaps to a number of other potential matches. Should we ignore those innocent matches?
Algebraic expression of the fallacy
The prosecutor’s fallacy is when the probability of innocence, given the evidence, is wrongly conflated with the very tiny probability that the evidence would occur if the defendant were innocent. In terms of probabilistic formula, it is wrongly assuming that the probability of
P(Innocence | Evidence) [the probability of the accused being innocent given he matches the DNA]
is no different to
P(Evidence | Innocence) [the probability of an innocent person having a DNA match)
So: coming back to the example set out above, the probabilistic fallacy occurs if the jury confuse the (apparently) really tiny chance that there should be a DNA match if the accused were innocent, with the probability that the accused is in fact innocent in the light of the evidence.
They are not the same thing at all, although unfortunately this distinction does not jump out at us when verbally formulated: regrettably the two grammatical structures sound more like rephrased versions of each other. Instead, the confusing situation is best represented graphically, in the tree diagram as follows –
[Source: Norman Fenton and Martin Neil: ‘Avoiding Probabilistic Reasoning Fallacies in Legal Practice Using Bayesian Networks, presentation at 7th International Conference on Forensic Inference and Statistics (ICFIS) Lausanne, 21 August 2008]
In the example, only one of the 10,000 potential suspects can be the actual source of the DNA (i. e. the guilty person actually present at the scene and leaving a trace). However, because there is a 1/1,000 probability of there being a positive match among the larger 10,000-strong population of potential suspects, this means that about 10 (i.e. 10,000/1,000) innocent suspects would be a positive match.
The Bayesian party trick
The shocking, literally unbelievable conclusion we are forced to assent to, applying Bayes’ Theorem, is that you have to incorporate the objective statistical information and integrate it with the knowledge of the accused testing positive in order to interpret that properly.
The probability of the accused’s innocence, conditional on a positive DNA match, is not 1/10,000 (as would appear if you only considered the top ‘branch’ of the tree diagram above and as ‘common sense’ points to) but instead that there is a 10/11 (i.e. 90.1%) probability that the accused is innocent, based on the DNA match: in other words, out of the suspect population, around 10 would be expect to be DNA matches, so the accused, being a match, simply has a roughly 1/10 chance of guilt before we consider other evidence. Not guilty.
In finance the prosecutor’s fallacy is referred to as the base rate fallacy – the tendency in judgment to give preeminence to information specific to an individual and neglect the broader statistical context. It is, it would seem, a common problem in our habits of probabilistic reasoning across domains.
Bayes in the employment law context?
The use of Bayesian reasoning has been deprecated by the UK’s senior criminal courts for reasons Professor Fenton takes exception to. As an employment practitioner my concern is that our failure to apply rigorous probabilistic reasoning results in numerous errors of judgment, in circumstances where the nature of the judge’s decision-making process remains unexamined.
You don’t have to pore through the pages of the Ministry of Justice’s statistics publications to be struck by the stark imbalance between the tally of employment cases that are filed each year, and the tribunal service’s ability to hear them. But if you do take a moment to review the statisticians’ report, it tells of a growing backlog, snowballing as though it were a cumulative debt being run up against the justice system’s tab.
Litigation risk and ‘administrative risk’
That debt will need to be settled at some point, if the system is to retain its credibility. There comes a point at which, if a complainant knows that getting a hearing will take many years, it ceases to be desirable to pursue a claim at all. Parties always have to deal with the vicissitudes of trial – ‘litigation risk‘. But the risk we are talking about is of a different character – essentially there is now ‘administrative risk‘, being the risk of not being heard within a satisfactory timeframe due to underfunding of the tribunal system. At that point of despair, while the laws on employment rights continue to exist on paper, they cease being relevant because unenforced.
Instead of looking at the figures about the annual shortfall in serving justice, you could also simply talk with a sample of litigants about their experiences, as I commonly need to do in my role. Litigants commonly have the unfortunate experience of having to wait a year – or more commonly several years – to have their cases heard, and tribunal correspondence often suffers from the same lag-time caused by the imbalance between case load and resourcing. Tribunal cases that have taken a year of more to come to trial can be put off at the last minute, sometimes for a whole year or more, due to a lack of judicial resources. Anecdotally, I have experienced this situation more than once in recent months.
Time series forecasting for the MoJ
The graphic I have included at the head of this article is a time series forecast I have made, based on MoJ historical data and using statistical software, suggesting that over the course of the next five years at current rates we can expect the backlog to rise as high as 420,000 cases by 2026.
If the Ministry of Justice were a business with supply chain problems, you would expect an efficient market to satisfy demand by the entry of alternative service providers. What alternative is there to the justice system? There is, of course, alternative dispute resolution (ADR) in the form of conciliation via ACAS. And there have been great efforts by the tribunal service to encourage mediation. But the statistics clearly indicate that, however praiseworthy such initiatives have been, as currently implemented they remain inadequate to the task of clearing the backlog.
I previously wrote about the benefits of remote hearings in terms of speed, convenience, and cost reduction, and suggested that these benefits would also translate well to enriching the conciliation process. Earlier this year, the Presidents of the Employment Tribunals in England & Wales and Scotland (Judge Barry Clarke P and Judge Shona Simon, respectively) wrote about the advantages of the new Cloud Video Platform (CVP) in their Roadmap for 2022:
Video technology allowed us to move at pace to retain listed cases in response to the restrictions announced in January 2021, and it will do so again if coronavirus restrictions return on a seasonal or regional basis. It allows for flexible use of judiciary, as the virtual region will demonstrate. It has facilitated a steady increase in the rate at which we can adjudicate upon the claims that have been presented to the Employment Tribunals – what we call our “disposal rate”. The disposal rate had returned to its pre-pandemic rate by the Autumn of 2020, and it continues to improve. It does so because, freed of the constraints of the physical estate, more hearings can be listed and remain effective. No hearings were lost in recent months for want of a hearing room. That is because CVP has effectively tripled the size of our estate.
While I share this enthusiasm for the technology, and the giddy feeling of technology suddenly tripling the tribunal’s estate, my feeling after some reflection is this message focuses too much on freedom from the constraints of the physical estate as though that were a panacea. But it cannot be, otherwise we would be seeing a different picture in the data.
Why have things got so bad?
While reference to an improved ‘disposal rate’ is hopeful, the fact remains disposals are outstripped by the rate of new ‘receipts’. Below is my data visualisation indicating what the statistics actually report about the relative rates of disposals: you can see while disposals and receipts went in lockstep until 2015, something appeared to break at that point.
Employment law is certainly not alone in feeling this justice gap. Consider the following graphic drawing on CPS statistics (source: https://commonslibrary.parliament.uk/is-the-criminal-justice-system-fit-for-purpose/ ‘Is the Criminal Justice System Fit for Purpose?’ by Georgina Sturge and Sally Lipscombe) – in the criminal justice system, there is also a marked aggravation in the gap between recorded crime and prosecutions around 2015.
The explanation provided in the Commons Insight paper which I have just cited draws on updated calculations from the Institute for Fiscal Studies:
In 2017, the Institute for Fiscal Studies calculated that in the decade from 2010/11, the Ministry of Justice’s (MoJ) budget would be cut by around 40%. Spending plans have been revised upwards since then, so that in 2019/20 the total MoJ budget was only around 25% lower than in 2010/11.
A second explanation which I should address is the abolition of tribunal fees in July 2017 following the Supreme Court’s judgment in R (on the application of UNISON) (Appellant) v Lord Chancellor (Respondent)  UKSC 51. The background secondary legislation was the Lord Chancellor’s Employment Tribunals and the Employment Appeal Tribunal Fees Order 2013 (SI 2013/1893).
The bar chart below (from the House of Commons Briefing 7081, 18 Dec 2017, ‘Employment tribunal fees’ https://researchbriefings.files.parliament.uk/documents/SN07081/SN07081.pdf ) reflects that fees led to a sudden near 70% reduction in cases being filed. Upon the abolition of fees 4 years later in July 2017 there was a partial recovery with a 30% increase in receipts felt in 2017/Q3.
If we return to a detailed view of the original area chart I created to examine the backlog, you can now see how the previous cuts in justice spending introduced by the 2010 Budget (as noted, with as an impact MoJ funding being at least 25% below 2010 funding levels by 2017), aligned with the post-abolition mini-renaissance in employment litigation in late 2017 to create a growing mountain of cases in the backlog.
We can now see why the current situation developed. What other solutions – beyond the current steps around the use of video technology and promoting or mandating judicial mediation and conciliation – can we identify? I have three suggestions which I may return to later:
The most obvious solution is political: reverse the MoJ funding cuts to eliminate the structural justice deficit. But any such 25% MoJ funding surge would require UK voters considering justice a service worth spending money on. At present, I am unaware of it being a vote-winner. The media don’t like it, and the trope of the fat-cat lawyer is easily dusted off to rubbish claims that spending money on justice is in peoples’ interests. Such spending increases may simply not be politically viable within the coming decade, and both Brexit and the pandemic have conspired to give UK GDP a substantial haircut of 4-6%. In the case of Brexit, that 4% GDP loss is ongoing until/unless better trade terms are negotiated. There is no headroom for discretionary spending, let alone for big structural spending.
Tool-up Acas. I have previously written about Acas’s services being plainly under-funded. Most tribunal claims settle, or are withdrawn, or get struck out. Many of the remaining 10% make it to a full hearing at which both parties would be better served by quality conciliation services. The simple truth is that current services are commonly just box-ticking exercises. The tribunal rules require a claim to be intimated to Acas by the Claimant. Nothing more need be done – hence for many the box gets ticked, but matters proceed to trial. I am not suggesting that rules should mandate more steps being taken – but better-funded, and better-equipped conciliators would result in substantial savings because, quite simply, a day spent with a trained conciliator would cost much less than a judge, wing members and courtroom with attendant administrative staff. At present parties sometimes get an email passed on by a conciliator, and if they respond to the conciliator there is no guarantee of having a reply of any sort.
This one is more controversial: Commercial B2B contracts often mandate arbitration to avoid the cost and delay of court proceedings. Put simply, per s203 ERA 1996, in the UK employment agreements cannot provide for the employee ‘contracting out’ of their rights to bring a claim to enforce their legal rights. But if the employment tribunal system will not be made ready perhaps a shadow employment arbitration service should be allowed to serve the growing need for speedy remedy. To do so, Parliament would need to amend the ERA to introduce an arbitration service. It would be a radical, and controversial solution and would essentially risk claimants contracting out of their right to remedy for all manner of statutory torts arising in the course of employment; it might also water down the value of current legal protections. But subject to careful scrutiny and limited trials, I expect it would be a cheap solution, and it would largely push many of the costs of tribunals onto employers.