Reports and export¶
Results¶
repogym run ... -o results.json writes a list of RunResult records:
{
"task_id": "bugfix-median", "agent": "golden", "score": 1.0, "passed": true,
"steps": 1, "duration": 0.9, "error": null,
"grade": {"score": 1.0, "passed": true, "components": [...], "patch": "...", "files_changed": ["stats.py"]},
"trajectory_path": "trajectories/bugfix-median__golden__20260907T125754.json"
}
Reports¶
repogym run tasks --agent golden --report report.html # standalone HTML
repogym report results.json -o report.md # Markdown tables
repogym report results.json # terminal table + leaderboard
The HTML report has summary cards, a per-agent leaderboard with pass-rate bars, and per-run rows with grader messages and expandable patches. It has no external dependencies and can be attached to CI runs.
Trajectories¶
With --record DIR (or RepoEnv(record_dir=...)), every episode is written as
JSON:
{
"task_id": "bugfix-median", "agent": "claude:claude-opus-5",
"steps": [
{"index": 1, "action": {"kind": "read", "path": "stats.py"}, "observation": "stats.py (33 lines)\n 1| ...", "reward": 0.0, "done": false, "info": {...}},
{"index": 2, "action": {"kind": "edit", ...}, ...},
{"index": 3, "action": {"kind": "submit"}, "observation": "submitted. PASS 1.000 ...", "reward": 1.0, "done": true}
],
"final_reward": 1.0, "passed": true, "grade": {...}
}
Export¶
repogym export tasks -o tasks.jsonl # one task per line, SWE-bench-style keys
repogym export tasks -o tasks.jsonl --include-repo # embed repository files
repogym export --trajectories trajectories -o sft.jsonl --only-passed
Task records use instance_id, problem_statement, FAIL_TO_PASS,
PASS_TO_PASS, patch, test_cmd and friends so existing SWE-bench tooling can
read them. Trajectory records contain messages (alternating assistant JSON
actions and user observations), reward, passed and patch, ready for SFT or
rejection sampling.