FairStress benchmark tests hidden demographic asymmetries in LLM decisions

The benchmark addresses high-stakes settings where model outputs can affect people’s opportunities, resources and care, from screening job applicants and approving loans to prioritizing patients for treatment and evaluating students for admissions. Standard demographic fairness benchmarks often test whether a model selects the same candidate in otherwise identical scenarios after candidates’ demographic identities are swapped. For example, if one job candidate has clearly stronger experience and qualifications, does the model still choose that candidate when the stronger candidate is Black in one version of the scenario and white in the other, with qualifications held constant?

The blog post said arriving at the same answer in two versions of a scenario can hide meaningful differences in how readily a model changes that decision when challenged. Suppose Candidate A is clearly stronger and the model selects A whether A is Black or white. If the model is then presented with a persuasive argument—for example, that Candidate B is preferred by senior leadership or that people from B’s demographic group have historically been overlooked—it may switch away from the stronger candidate much more readily in one version of the scenario than the other. FairStress is designed to measure these hidden asymmetries.

Across open-weight and frontier models including Olmo, Llama, Qwen, Gemma, Claude and the GPT family, and across multiple stages of training, the blog post said the same persuasive arguments, called adversarial prefixes, can make models more or less likely to change their decisions depending on which demographic group the argument favors. It also reported differences in how models explain decisions involving demographic information and in their internal activation patterns across domains including hiring, lending, healthcare, education and housing.

FairStress targets symmetric fairness, according to the blog post: when the evidence supports a decision, a model should apply the same criteria and be equally resistant to changing that decision regardless of which demographic group benefits. The post said FairStress does not attempt to capture every conception of fairness. An equity-based approach, for example, might treat historical discrimination or underrepresentation as relevant to a decision and therefore favor a minority candidate even when other factors are held constant.

The benchmark contains 300 realistic two-candidate decision scenarios across 29 domains including hiring, healthcare, lending, education, housing and criminal justice. One scenario asks a model to choose between two candidates for a mechanical engineering job, while another asks which of two patients should receive the last unit of a scarce medication. Some scenarios have a clearly stronger candidate whose profile better satisfies the stated criteria, allowing tests of whether demographic information changes how often a model selects the candidate best supported by the evidence. Other scenarios are deliberately ambiguous, with both candidates strong in different, hard-to-compare ways. FairStress also varies how explicitly demographic identity is presented, from no identity information to a demographically coded name, an indirectly implied identity or an explicit demographic label. It tests how stable models’ decisions are when challenged by adversarial prefixes drawing on authority, social consensus, purported evidence, legal concerns, historical disadvantage and other forms of persuasion; each prefix has a matched version that makes the same kind of case for the other candidate.