Concepts¶

Task¶

A directory with a task.yaml and a repository snapshot. The spec declares what the agent is told (problem_statement, hints), how the work is verified (tests, perf, constraints), how components combine (grading), the golden solution, and interaction limits (env). Tasks have a type:

Type	Typical reward signal
`bugfix`	`fail_to_pass` tests that reproduce the bug
`feature`	`fail_to_pass` tests for the new behaviour, often partly hidden
`refactor`	all tests `pass_to_pass` + complexity/length limits + scope
`perf`	benchmark speed-up + all tests `pass_to_pass`

Workspace¶

An isolated copy of the task repository with a git baseline commit. Agents edit files here; the workspace turns their work into a unified diff, can reset in milliseconds, injects hidden tests at grading time and restores protected files.

Sandbox¶

The thing that runs shell commands inside a workspace: LocalSandbox on the host (scrubbed environment) or DockerSandbox in a container (network off, resource limits).

Environment¶

RepoEnv exposes a task as a gym-style loop: reset() returns the initial observation, step(action) returns an Observation with text, reward, done and info. Actions are data (Read, Write, Edit, Run, Test, ListFiles, Submit). Reward is sparse by default: 0 until Submit, then the graded score.

Graders¶

Pure functions from a workspace to a GradeComponent (score in [0, 1], passed, role, details). CompositeGrader combines objectives by weight and lets any failing gate zero the score; passed requires every component to pass. Built-ins: TestGrader, PerfGrader, QualityGrader, ConstraintGrader.

Agent¶

Anything that drives an environment to done. Agent.run(env) handles reset and auto-submit. Built-ins cover the golden solution, a no-op baseline, scripted replays, external CLIs and an Anthropic tool loop.

Trajectory¶

The record of an episode: every (action, observation, reward) step, the final grade and the patch. Written as JSON when record_dir is set; exported to chat-style JSONL for training.

Validation¶

validate_task proves a task is a sound environment: baseline fails, golden passes, noop is not rewarded, hidden tests are hidden, and grading is deterministic.

Mining¶

HistoryMiner extracts verified bug-fix tasks from git commits; MutationMiner injects faults into a healthy repository and keeps the ones the tests catch.