An open harness for measuring AI resume screening bias — built to find it, honest when it doesn't.
Winnow generates controlled experiments that test whether AI resume screeners change their scores for reasons that have nothing to do with qualifications. It builds synthetic resumes with constructed ground truth, introduces one variable at a time (style, name, university, authorship), and measures whether scores change. Everything is reproducible: frozen inputs, deterministic scoring, open code.
We ran four bias experiments across two production models (gpt-4o-mini and deepseek-chat). None produced robust bias signals. Full write-ups in findings/.
| Experiment | What we tested | Result | Finding |
|---|---|---|---|
| Polish penalty | Does rough formatting lower scores for identical qualifications? | Null | 2026-06-27 |
| Self-preference | Does a model prefer resumes it wrote itself? | Null | 2026-06-27 |
| Demographic bias | Does swapping only the candidate's name (race/gender/ethnicity) change the score? | Null | 2026-06-28 |
| Education prestige | Does swapping only the university (elite/mid-tier/bootcamp) change the score? | Null | 2026-06-28 |
The honest summary: on the models and conditions we tested, we could not detect the biases we set out to find. That is a useful result — and it comes with real limitations (see below).
I wrote about what this means and what it doesn't: I wrote about what this means — and what it doesn't.
Polish bias. Claude Haiku generates polished and rough twins from the same structured data, fact-verified for content parity. A different model scores both. If the rough twin scores lower, that's a polish penalty.
Self-preference. Two models each generate a resume from the same data. Each model scores both versions. If a model scores its own version higher, that's self-preference. Tested under both absolute scoring and order-counterbalanced pairwise comparison.
Demographic bias. One pronoun-free baseline per resume (placeholder name, counterfactual substitution). Swap only the candidate's name across 12 demographic groups (validated core from Bertrand & Mullainathan 2004, plus exploratory groups). Every other character identical. Within-group noise floors from 5 names per group.
Education prestige. Same counterfactual method — swap only the institution across elite, mid-tier, and bootcamp tiers (3 institutions each for noise floor). Community college as an exploratory tier.
These are real and should be read before citing any result:
- Tech roles only. All 12 resumes are tech positions. Finance, law, and medicine — where prestige signals carry more weight — are untested.
- Two models. gpt-4o-mini and deepseek-chat. Results do not generalize to other models.
- Pilot scale. 12 base resumes. A pilot detects whether an effect exists; it does not confirm magnitude.
- Coarse scoring. Effects under ~2-3 points on a 0-100 scale may be unresolvable.
- Absolute scoring. The self-preference literature (Xu et al. 2025, Panickssery et al. 2024) found effects primarily under pairwise comparison. Our pairwise test also showed null, but on a small sample.
- Exploratory groups. Middle-Eastern, Indian, East-Asian, and East-European name groups are not from validated audit name lists — directional signal only.
git clone https://github.com/navox-labs/winnow.git
cd winnow
pip install -r requirements.txt
# Run the test suite
python -m pytest tests/ -v
# Example: generate and score polish twins
export ANTHROPIC_API_KEY=your-key
export OPENAI_API_KEY=your-key
python data_engine.py generate
python data_engine.py score --model gpt-4o-miniFull command reference: docs/usage.md
data_engine.py CLI entry point for all experiments
demographic_engine.py Demographic and education-prestige experiment
self_pref_engine.py Self-preference experiment (absolute scoring)
pairwise_engine.py Self-preference experiment (pairwise comparison)
scorer.py Model-agnostic scorer (OpenAI, DeepSeek, HuggingFace)
verify.py Fact-parity verification
resume_data_borderline.py 12 borderline synthetic resumes
findings/ Dated experiment results
docs/usage.md Full command reference
tests/ Test suite (130+ tests)
Navox also builds candidate-side career tools. What keeps this project honest isn't my word: every number is reproducible by anyone with API keys, every metric is open code, and no Navox product will ever appear in official comparisons. Trust the reproducibility, not the author.
MIT
A Navox Labs project. Findings log. References.