One Example Is Enough to Pass Fairness Benchmarks

Rethinking Fairness Evaluation for Aligned LLMs — EMNLP 2026 (Main Conference)

Naihao Deng¹, Samee Arif¹, Shuaichen Chang², Yulong Chen³⁴, Rada Mihalcea¹
¹University of Michigan · ²The Ohio State University · ³University of Aberdeen · ⁴University of Cambridge

Project page · Paper · Adapters

Content warning. This work studies stereotypes and social bias, and reproduces toxic and offensive examples for illustration purposes only.

---

What this shows

BBQ is the primary — often the only — fairness benchmark in model reports from GPT-4.5/5/o1/o3/o4/OSS, Gemini, and Claude. We show it can be saturated from a single example, and that saturating it does not make a model fairer.

Qwen 2.5 7B	BBQ
Base	79.9
+ one-shot GRPO (1 example)	92.9
+ one-shot ICL (same example, no training)	99.0
Instruct (large-scale RLHF)	96.1

The effect holds across Qwen 3 8B, Gemma 2 9B, Llama 3.1 8B, and Mistral 7B v0.3 — Mistral goes from 0.0 to 97.8. A cross-conditioning analysis shows the gain is carried by the model's reasoning trace, not its weights. On RealToxicityPrompts the same models show no toxicity improvement, so the shortcut does not transfer to generative fairness.

---

Released adapters

We release one-shot GRPO models as LoRA adapters (rank 32, α = 32), not merged weights. The adapter is the point: a ~160 MB low-rank delta is all that separates 79.9 from 92.9 on BBQ.

One repository per (base model, training example). Every GRPO step is a git revision, and main is the step the paper reports — so a plain load reproduces the published number.

Base model	Repositories	Revisions	main
Qwen/Qwen2.5-7B	…-qwen2.5-7b-z1, -z2, -z251, -z501, -z751, -z876, -z999, -z1000	step10 … step100	step30
Qwen/Qwen3-8B-Base	…-qwen3-8b-base-z1	step10 … step100	step30
google/gemma-2-9b	…-gemma-2-9b-z1	step50, step100	step50
meta-llama/Llama-3.1-8B	…-llama-3.1-8b-z1	step100, step125, step200	step125
mistralai/Mistral-7B-v0.3	…-mistral-7b-v0.3-z1	step50, step100	step50

All under the MichiganNLP org, prefixed hacking-fairness-benchmarks-.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", torch_dtype="bfloat16")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")

# main == step30, the checkpoint reported in the paper
model = PeftModel.from_pretrained(
    base, "MichiganNLP/hacking-fairness-benchmarks-qwen2.5-7b-z1")

# any other point on the training trajectory
model = PeftModel.from_pretrained(
    base, "MichiganNLP/hacking-fairness-benchmarks-qwen2.5-7b-z1", revision="step100")

Each adapter_config.json pins the exact base-model revision it was trained against. Models answer in <think>…</think><answer>A</answer> format.

---

Repository layout

src/src/          Python root (set PYTHONPATH here)
  data/           Dataset download + preprocessing (BBQ, StereoSet, CrowS-Pairs, WinoQueer, genMO, ...)
  verl_supplementary/  Reward functions, accuracy-over-steps, training-dynamics extraction
  eval/           Loglikelihood MCQ scoring, ICL eval, mechanistic analysis, RTP/BOLD, paper tables
  nli/            NLI reasoning-consistency analysis
  fair_prm/       FairPRM step-wise reasoning quality scoring
  verl/           Vendored copy of the verl RLHF framework used for training
src/scripts/      Launchers: train_verl/, eval/, nli/, fair_prm/, slurm_jobs/, singularity/
data/             The 8 released training examples + the z_1..z_1000 variance ranking
docs/             GitHub Pages project page

Large artifacts (checkpoints/, results/, datasets/, images/) are gitignored — datasets are rebuilt from source, and trained models are released as adapters on the Hub. The paper source is not distributed here; see the project page.

---

Reproducing

Training runs inside a Singularity container (images/verl_vllm018.dev1.sif).

1. Build the datasets

cd src/src && export PYTHONPATH=$(pwd)
python3 -m data.download_bbq            # -> datasets/bbq/processed/
python3 -m data.download_commonsenseqa
python3 -m data.combine_test_data       # -> combined_test.parquet (20,393 examples)

2. Train on one example

sbatch src/scripts/slurm_jobs/run_slurm_four_gpu.sh \
       src/scripts/train_verl/train_lora_qwen2.5-7b_single_example.sh

Reward (verl_supplementary/evaluator_score_think_template.py): +0.5 format, +0.5 valid choice, +2.0 correct answer, with a penalty for emitting multiple tag pairs.

3. Evaluate

Base models parse <answer> unreliably, so benchmark numbers come from answer-choice loglikelihood scoring. Drivers are in src/scripts/eval/, run via run_slurm_loglik_eval.sh (vLLM) or run_slurm_hf_loglik.sh (HF transformers).

4. Extract an adapter from a checkpoint

bash src/scripts/eval/extract_adapter_step.sh <ckpt_dir> <step> \
     Qwen/Qwen2.5-7B <base_revision> <out_dir> 32

verl's model_merger writes lora_alpha=0 into adapter_config.json regardless of the training-time value, which makes the adapter inert. extract_adapter_step.sh patches alpha and pins the base-model revision. Do not use raw merger output.

5. Rebuild paper tables and figures

cd src/src && export PYTHONPATH=$(pwd)
python3 -m eval.build_main_results_table
python3 -m eval.make_paper_figures

These write LaTeX/CSV into a local paper/ directory. The paper source itself is not distributed here — see the project page for the PDF.

See CLAUDE.md for the full command reference and environment notes.

---

Citation

@inproceedings{deng2026one,
  title     = {One Example Is Enough to Pass Fairness Benchmarks:
               Rethinking Fairness Evaluation for Aligned {LLM}s},
  author    = {Deng, Naihao and Arif, Samee and Chang, Shuaichen and
               Chen, Yulong and Mihalcea, Rada},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
               Natural Language Processing},
  year      = {2026}
}

License

Code is released under the MIT License. Released adapters are LoRA deltas and remain subject to the licenses of their respective base models.