Park Bench

Park Bench is a Docker-first benchmark runner for agentic coding workflows. It stores results in SQLite, exposes a Litestar REST API, serves a React UI, and runs Docker task images selected from a database catalog.

The current runtime is:

• task-image catalog based — task images are rows in task_images
• task-configuration driven — runs, suites, and collections refer to an immutable task configuration
• manifest driven — task images provide /task/task-manifest.json
• score first — renderable values are normalized into score_results

Quick start

git clone <repository-url>
cd park-bench

cp .env.example .env
# edit .env with provider and judge credentials

docker compose up -d --build

Open:

• UI: http://localhost:8080/ui
• API: http://localhost:8080/api

A fresh database is bootstrapped to schema v10 and starts with an empty task-image catalog. Add task-image rows before creating task configurations.

Typical workflow

1. Register a task image

curl -X POST http://localhost:8080/api/task-images \
  -H "Content-Type: application/json" \
  -d '{
    "name": "oas-generation",
    "catalog_image_ref": "ghcr.io/danballance/park-bench-task-exercise-oas-generation:v1",
    "enabled": true
  }'

2. Create a task configuration

curl -X POST http://localhost:8080/api/task-configurations \
  -H "Content-Type: application/json" \
  -d '{
    "name": "oas-generation-pytest",
    "task_image_id": 1,
    "scorers": ["pytest", "complexity"]
  }'

3. Trigger a benchmark run

curl -X POST http://localhost:8080/api/runs/benchmark \
  -H "Content-Type: application/json" \
  -d '{
    "task_configuration_id": 1,
    "iterations": 3,
    "provider": "anthropic-api",
    "model": "claude-sonnet-4-6"
  }'

4. Create and run a suite

curl -X POST http://localhost:8080/api/suites \
  -H "Content-Type: application/json" \
  -d '{
    "name": "python-suite",
    "task_configuration_id": 1,
    "iterations": 3,
    "max_workers": 1,
    "providers": [
      {"provider": "anthropic-api", "model": "claude-sonnet-4-6"}
    ]
  }'

curl -X POST http://localhost:8080/api/runs/suite \
  -H "Content-Type: application/json" \
  -d '{"suite_id": 1}'

Local development

curl -LsSf https://astral.sh/uv/install.sh | sh
uv sync

uv run pytest --cov park_bench --cov-report term-missing tests
uv run ruff check --fix park_bench
uv run ruff format park_bench
uv run python .github/scripts/check_complexity.py

Build the UI manually when working in ui/:

cd ui
npm install
npm run build

Local task-image registry mode

docker compose \
  -f docker-compose.yaml \
  -f docker-compose.local-task-registry.yaml \
  up -d --build

This starts a registry on 127.0.0.1:5000 and rewrites catalog refs in memory by preserving the repository tail and replacing the registry/tag.

Documentation

See docs/index.md.

Useful entry points:

• Getting Started
• Architecture
• API Reference
• Task Images
• Scores
• Database Schema
• Deployment
• Web UI
• Contributing