The hungry judge, revisited

You have seen the chart. Parole approvals start the morning at 65 percent, slide toward zero, and snap back to 65 percent right after the judges eat. Daniel Kahneman put it in a book. Every talk about bias in AI and law opens with it. And within six months of publication two researchers had a much more boring explanation that fits the data at least as well.

The claim

In 2011 Shai Danziger (Ben-Gurion University), Jonathan Levav (then Columbia Business School) and Liora Avnaim-Pesso (Ben-Gurion) published “Extraneous factors in judicial decisions” in PNAS. They followed eight Israeli judges through 1,112 parole board rulings over ten months. Each day had two food breaks, which split it into three sessions. Within each session, the probability of a favourable ruling fell from about 65 percent at the start to nearly 0 percent at the end. After a break it jumped back to 65 percent. The authors suggested mental depletion. Tired, hungry judges default to the easy answer, which is no.

How they found it

This is observational data, not an experiment. The team recorded the order of cases within each session, the ruling, and whether a break came before it. They controlled for the prisoner’s offence, prior incarcerations, and whether a rehabilitation programme was in place. The pattern survived those controls. The judges averaged 22 years of experience, and the cases represented about 40 percent of all Israeli parole requests in the period.

The key assumption is buried in that description. For a break to cause a change in rulings, the order of cases has to be unrelated to anything that predicts the outcome. If easy denials get scheduled near breaks for some other reason, the chart would look identical and hunger would have nothing to do with it.

The mechanism

The authors’ story is ego depletion. Making decisions is effortful, effort runs down over a session, and a depleted decision maker falls back on the status quo. Denying parole keeps the prisoner where he is, so it is the default. Food and rest restore the capacity to deliberate. This is the same theory behind the famous chocolate cookie and radish experiments, which is worth remembering, because that literature has had a rough decade.

How much to believe it

One dataset, no replication. There is no second parole board, no second country, and no experiment. Every subsequent paper uses the same 1,112 rulings or simulations of them.

The strongest criticism came fast. Keren Weinshall-Margel and John Shapard, in a PNAS letter the same year, went to the actual hearings and interviewed staff. They observed 12 hearing days and 227 decisions. Case order was not random. The board finishes all prisoners from one prison before taking a break and starts the next prison after. Within a prison’s block, prisoners with a lawyer go first and prisoners without one go last. Unrepresented prisoners are far less likely to win parole. So the cases right before a break are systematically the weakest ones, for reasons that have nothing to do with anyone’s blood sugar. Danziger and colleagues replied that adding representation as a control did not kill the effect, but they never disputed that ordering was non-random.

Then the effect size was attacked. Andreas Glöckner (2016, Judgment and Decision Making) built a simulation. Favourable rulings take longer to write than denials. If judges can see the clock, they will rationally avoid starting a long case right before a scheduled break. Plug that one behaviour into a model with no hunger at all and you reproduce a sharp fall in approvals before each break. Glöckner’s conclusion is that the original paper overestimates the magnitude of any real depletion effect, possibly by a lot.

Then the sanity check. Several commentators, most sharply Konstantin Chatziathanasiou in the German Law Journal (2022), pointed out that a 65 point swing is not the size of a psychological effect. It is the size of a scheduling artefact. Nothing in the psychology literature moves a real-world decision from near-certain yes to near-certain no. If the effect were that large, judges and lawyers would have noticed decades ago.

Does it matter in real life? The scheduling and representation story has an actual policy implication, which is that prisoners without lawyers lose, and that is worth fixing. The hunger story mostly gets used as a rhetorical opener for whatever the speaker wanted to sell anyway.

What it means for you

When a chart shows a huge effect from a small cause, ask how the rows got sorted before you ask what the cause was. Ordering effects, batch boundaries and time-of-day artefacts show up in product analytics constantly, and they look exactly like this.

Suggested chart

Reproduce the original figure 1 from Danziger et al., favourable ruling rate by ordinal position within session, then overlay the simulated curve from Glöckner’s model with zero depletion. Data for the first is in the PNAS paper’s supplementary material. The second would need to be re-simulated from the parameters in Glöckner’s paper.

Sources

1. Danziger, S., Levav, J. and Avnaim-Pesso, L., 2011, “Extraneous factors in judicial decisions,” PNAS, 108(17), 6889-6892. pnas.org.
2. Weinshall-Margel, K. and Shapard, J., 2011, “Overlooked factors in the analysis of parole decisions,” PNAS, 108(42), E833. pnas.org.
3. Danziger, S., Levav, J. and Avnaim-Pesso, L., 2011, “Reply to Weinshall-Margel and Shapard: Extraneous factors in judicial decisions persist,” PNAS, 108(42), E834. pnas.org.
4. Glöckner, A., 2016, “The irrational hungry judge effect revisited: Simulations reveal that the magnitude of the effect is overestimated,” Judgment and Decision Making, 11(6), 601-610. cambridge.org.
5. Chatziathanasiou, K., 2022, “Beware the Lure of Narratives: ‘Hungry Judges’ Should Not Motivate the Use of ‘Artificial Intelligence’ in Law,” German Law Journal, 23(4). cambridge.org.