repogym¶
Turn any git repository into a verifiable RL environment for coding agents.
Frontier coding agents are trained with reinforcement learning on real software-engineering work: fixing bugs, adding features, refactoring, optimising. The scarce resource is no longer model capacity but environments: tasks that are reproducible, executable, verified to be solvable, and resistant to reward hacking.
repogym is the library layer for that. A task is a folder with a task.yaml, a
repository snapshot, optional hidden tests and a golden solution. repogym validates
it, wraps it in a gym-style reset() / step() API, grades submissions with
composable graders, records every episode as a trajectory, and can generate new tasks
from git history or by fault injection.
git clone https://github.com/shi1720/repogym.git
cd repogym
python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
repogym validate tasks
repogym run tasks --agent golden --report report.html
Where to go next¶
- Installation - requirements, extras, Docker.
- Quickstart - run, validate and inspect the example tasks in five minutes.
- Concepts - tasks, workspaces, graders, environments, agents, trajectories.
- Authoring tasks - write a bug-fix, feature, refactor or perf task by hand.
- Mining tasks - generate hundreds of verified tasks from one repository.
- Architecture - how the pieces fit and why.
The pitch in one table¶
| Hand-rolled harness | repogym |
|---|---|
| One Dockerfile + script per repository | task.yaml + repo/; local or Docker sandbox |
| "It worked on my machine" tasks | repogym validate: baseline fails, golden passes, noop = 0, deterministic with --repeat |
| Agents game the tests | Hidden tests, protected files, scope and complexity constraints |
| Hand-curated issues only | repogym mine (git history) and repogym mutate (fault injection) |
| One agent framework | Any agent: Python class, LLM tool loop, or external CLI |
| Python-only parsing | JUnit XML contract: pytest, node, cargo, jest, gradle, go |
| Ad-hoc logs | Trajectories, leaderboards, HTML reports, JSONL export |