Emily, Greg, Lakisha and Jamal

Most entries in this series end with the finding shrinking. This one does not. Marianne Bertrand and Sendhil Mullainathan ran the simplest experiment in social science: write resumes, staple a random name to each, send them to real jobs, count the calls. Twenty years and one much larger replication later, the answer has not moved in sign and barely in story.

The claim

Bertrand and Mullainathan (2004, American Economic Review) sent about 4,870 fictitious resumes to roughly 1,300 help-wanted adverts in Boston and Chicago, for sales, administrative, clerical and customer service roles. Each resume was randomly assigned either a name common among white Americans (Emily Walsh, Greg Baker) or one common among Black Americans (Lakisha Washington, Jamal Jones). Nothing else about the resume was tied to the name.

White-sounding names got a callback on about 9.65 percent of resumes. Black-sounding names got about 6.45 percent. Put another way, the same applicant needed to send about fifteen resumes instead of ten to get the same number of calls.

How they found it

Randomisation is doing all of the work, and it is why this design is worth studying even if you never think about hiring.

In observational data, you cannot separate the effect of a name from everything correlated with having that name: schools, neighbourhoods, networks, the wording of the resume itself. Here the name is assigned by coin flip after the resume is written. Any difference in outcomes between the two groups is caused by the name, because the name is the only thing the coin touched. There is no confounder to argue about, no lag to choose, no specification to fork.

The cost is that you learn about one specific thing: the callback stage, in those cities, in those occupations, at that time. You do not learn about offers, wages, promotion, or what happens once a human being is in the room.

The mechanism

The paper is careful about what it can and cannot distinguish. A name on a resume can trigger a judgment about race directly, or it can act as a signal for something a recruiter believes is correlated with race, or it can simply be unfamiliar to a screener spending eight seconds per page. Taste-based and statistical discrimination look the same in callback counts.

One result narrows it. The experiment also randomised resume quality, adding experience, certifications and fewer employment gaps. The better resume raised callbacks for the white names by roughly 30 percent. For the Black names it moved almost nothing. A pure statistical-inference story has trouble with that: if employers were rationally updating on limited information, more information should matter more, not less.

How much to believe it

The design is the strongest one available. This is a field experiment with real employers, real adverts and random assignment. It is not a survey, a lab game, or a regression on administrative data. On the internal validity axis there is very little to attack.

It replicated at a scale the original could not dream of. Patrick Kline, Evan Rose and Christopher Walters sent roughly 83,000 applications to 108 of the largest employers in the United States, and found a contact gap of a couple of percentage points on a much higher base rate. Smaller in relative terms than 2004, still clearly there. When a twenty-year-old finding survives a replication two orders of magnitude larger, that is about as good as social science gets.

The large study found something new, which is the interesting part. With that many applications per firm, they could estimate a gap for each individual employer. Discrimination turned out to be highly concentrated: a minority of firms account for most of it, and many show no gap at all. “American employers discriminate” and “your employer discriminates” are different claims, and only the first is established by the average.

The criticisms are about interpretation, not validity. The most substantive is that names common among Black Americans also signal social class, so part of the gap may be class rather than race. Later audit work has tried to separate the two with mixed success, and it is a real limitation rather than a debunking. The second is scope: callbacks are the first gate, not the outcome anyone cares about, and the relationship between callback gaps and eventual employment gaps is not one to one.

Nobody has produced a null. The audit literature is now large, spans many countries, and has its own meta-analyses. Effects vary in size by context and occupation. The sign does not move.

What it means for you

If you run hiring anywhere, the firm-level result is the actionable one. The average is a fact about the economy; your own callback rates by applicant group are a fact about you, and the 2022 study demonstrates that they are measurable. Most organisations have never looked, and the distribution says a meaningful share would not like the answer.

More generally: this is what the top of the evidence ladder looks like, and it is worth holding the other entries in this series against it. Random assignment, pre-specified outcome, field setting, large replication. When you next see a headline finding, ask which of those four it has. Most have none.

Suggested chart

Two panels side by side. Left, the 2004 callback rates as two dot grids of a hundred squares each, lit proportionally, which makes the gap legible without a single axis. Right, the distribution of per-firm contact gaps from the 2022 replication as a sorted bar chart, so the concentration is visible. The second panel needs the published firm-level estimates, which are in the paper’s appendix.

Sources

1. Bertrand, M. and Mullainathan, S., 2004, “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination,” American Economic Review, 94(4), 991-1013. aeaweb.org.
2. Kline, P., Rose, E. K. and Walters, C. R., 2022, “Systemic Discrimination Among Large U.S. Employers,” Quarterly Journal of Economics, 137(4), 1963-2036. nber.org.
3. Quillian, L., Pager, D., Hexel, O. and Midtbøen, A. H., 2017, “Meta-analysis of field experiments shows no change in racial discrimination in hiring over time,” PNAS, 114(41), 10870-10875. pnas.org.