Skip to content

Changelog

All notable changes to this project are documented here. The format follows Keep a Changelog and the project adheres to Semantic Versioning.

0.1.0 - 2026-09-07

Initial release.

Added

  • Declarative tasks (task.yaml + repo snapshot or git commit + hidden tests + golden solution) for four task types: bugfix, feature, refactor, perf.
  • repogym validate: hidden tests not leaked, fail-to-pass fail at baseline, pass-to-pass pass at baseline, empty submission scores exactly 0, golden solution scores 1, optional determinism check.
  • Gym-style RepoEnv (reset / step / state) with Read, Write, Edit, Run, Test, ListFiles, Submit actions, step limits, quote-aware command allowlist, optional reward shaping and JSON trajectory recording.
  • Graders with objective/gate roles: TestGrader (JUnit XML from any runner), PerfGrader (externally timed benchmark speed-up, log-scaled partial credit), QualityGrader (cyclomatic complexity and function length), ConstraintGrader (scope, protected files, patterns, diff size), combined by CompositeGrader.
  • Reward-hacking defences: protected test files restored before grading, hidden tests injected only at grade time, git metadata kept outside the agent-writable tree with fail-closed grading, wall-clock benchmark timing, secrets scrubbed from the sandbox environment.
  • Agents: GoldenAgent, NoopAgent, ScriptedAgent, FunctionAgent, ShellAgent (any CLI such as Claude Code, Aider, Codex) and ClaudeAgent (Anthropic SDK tool loop).
  • Task mining: HistoryMiner (verified bug-fix tasks from git history) and MutationMiner (AST-guided fault injection with test verification).
  • LocalSandbox and DockerSandbox execution backends.
  • Reports (terminal, Markdown, standalone HTML with leaderboard) and JSONL export of tasks (SWE-bench-compatible keys) and trajectories (chat-style messages).
  • CLI: list, info, validate, run, report, mine, mutate, export, new, solution, doctor.
  • Five validated example tasks (Python and JavaScript), MkDocs documentation with API reference, landing page, CI on Linux/macOS for Python 3.9 to 3.13.