API reference¶

The public surface is re-exported from repogym; the modules below are the canonical homes.

Tasks¶

`repogym.task` ¶

Task discovery and loading.

`Task(spec: TaskSpec, path: Path)` ¶

A task on disk: the parsed task.yaml plus resolved paths.

`load(path: Union[str, Path], *, require_solution: bool = True) -> Task` `classmethod` ¶

Load a task directory (or its task.yaml).

require_solution=False skips the check that the solution file exists, which repogym solution needs in order to create it.

`TaskError` ¶

Bases: ValueError

Raised when a task directory is malformed.

`load_tasks(root: Union[str, Path], *, include: Optional[Sequence[str]] = None, types: Optional[Sequence[str]] = None, tags: Optional[Sequence[str]] = None, strict: bool = True) -> List[Task]` ¶

Load all tasks under root with optional filtering.

include accepts glob patterns matched against task ids.

`iter_task_dirs(root: Union[str, Path]) -> Iterator[Path]` ¶

Yield every directory under root (inclusive) that contains a task.yaml.

Task specification¶

`repogym.models` ¶

Pydantic models that define a repogym task and the results it produces.

A task is a directory containing task.yaml (validated into :class:TaskSpec), a repository snapshot (or a pointer to a git commit), an optional golden solution, and optional hidden tests. Everything the library does - materialising a workspace, grading, validating, mining - is expressed in terms of these models, so they are deliberately explicit and heavily documented.

`TaskSpec` ¶

Bases: BaseModel

Top-level task.yaml schema.

`RepoSpec` ¶

Bases: BaseModel

Where the code under test comes from.

`TestSpec` ¶

Bases: BaseModel

How to run the test-suite and which tests carry the reward signal.

`PerfSpec` ¶

Bases: BaseModel

Benchmark definition for performance-optimisation tasks.

`ConstraintSpec` ¶

Bases: BaseModel

Static, execution-free rules a solution must satisfy.

These make refactoring tasks gradeable and stop reward hacking such as deleting tests, editing files outside the intended scope or shelling out to eval.

`GradingSpec` ¶

Bases: BaseModel

How individual grader scores are combined into a single reward in [0, 1].

`SolutionSpec` ¶

Bases: BaseModel

The golden reference solution used by repogym validate.

`EnvSpec` ¶

Bases: BaseModel

Interaction limits for the gym-style environment.

`TaskType` ¶

Bases: str, Enum

The four software-engineering skills a task can exercise.

`Difficulty` ¶

Bases: str, Enum

Environment¶

`repogym.env` ¶

Gym-style environment for coding agents.

The API mirrors the Gymnasium / OpenEnv convention -- reset() then a loop of step(action) until done -- so repogym drops into existing RL training loops. Actions are plain data (:class:Action), which keeps the environment transport-agnostic: an agent can be an in-process Python function, an LLM tool-calling loop, or a remote process sending JSON.

Rewards are sparse by default: 0 until submit, then the graded score in [0, 1]. Optional shaping (env.reward_shaping: true in the task) adds a small dense signal from the visible test-suite.

`RepoEnv(task: Task, sandbox: Optional[Sandbox] = None, *, agent_name: str = 'unknown', record_dir: Optional[Union[str, Path]] = None, keep_workspace: bool = False, workspace_root: Optional[Union[str, Path]] = None, show_tree: bool = True)` ¶

A single-task environment.

Example::

env = RepoEnv(Task.load("tasks/bugfix-median"))
obs = env.reset()
obs = env.step(Read("src/stats.py"))
obs = env.step(Edit("src/stats.py", "n // 2 + 1", "n // 2"))
obs = env.step(Submit())
print(obs.reward, obs.done)

`prompt() -> str` ¶

The initial observation: problem statement plus repository outline.

`Action` ¶

Bases: BaseModel

A single agent action.

Use the helper constructors (:func:Read, :func:Write, ...) or build one from a dict (Action.model_validate({"kind": "run", "command": "ls"})).

`Observation` ¶

Bases: BaseModel

What the agent sees after reset or step.

`EnvState` ¶

Bases: BaseModel

`Read(path: str, start_line: Optional[int] = None, end_line: Optional[int] = None) -> Action` ¶

`Write(path: str, content: str) -> Action` ¶

`Edit(path: str, old: str, new: str) -> Action` ¶

`Run(command: str) -> Action` ¶

`Test() -> Action` ¶

`ListFiles(pattern: Optional[str] = None) -> Action` ¶

`Submit() -> Action` ¶

Workspace and sandboxes¶

`repogym.workspace` ¶

Workspaces: isolated, git-tracked copies of a task's repository.

Every episode gets its own workspace. A baseline commit is created right after materialisation so that the agent's work can always be expressed as a unified diff (git diff <baseline>), reset in milliseconds, or replayed elsewhere.

The git metadata lives outside the working tree (<workspace>.scratch/git) and every git call passes GIT_DIR/GIT_WORK_TREE explicitly, so an agent cannot blind the graders by deleting or rewriting .git. If the metadata does go missing, git calls fail closed with :class:WorkspaceError and the submission is graded 0.

`Workspace(task: Task, sandbox: Optional[Sandbox] = None, root: Optional[Union[str, Path]] = None, keep: bool = False)` ¶

A materialised task repository with git-based change tracking.

`scratch: Path` `property` ¶

Directory outside the repo for junit files etc. (never part of diffs).

`resolve(rel: Union[str, Path]) -> Path` ¶

Resolve a path inside the workspace, refusing escapes (../symlinks).

`diff() -> str` ¶

Unified diff of everything the agent changed since the baseline.

`apply_solution() -> None` ¶

Overlay the task's golden solution onto the workspace.

`restore(files: List[str]) -> None` ¶

Restore specific files to their baseline content (anti-tampering).

`reset() -> None` ¶

Discard all changes and return to the baseline commit.

`inject_hidden_tests() -> List[str]` ¶

Copy hidden tests in; anything they overwrite is restored by :meth:remove_hidden_tests (so an agent's file at that path survives).

`WorkspaceError` ¶

Bases: RuntimeError

`repogym.sandbox` ¶

Execution backends.

A :class:Sandbox runs shell commands inside a workspace directory. The :class:LocalSandbox is the default and needs nothing but a shell. The :class:DockerSandbox mounts the workspace into a container so that each task can pin its own toolchain (python:3.12, rust:1.80, node:22 ...) without polluting the host.

Both share the same interface so graders and environments never care which one they are talking to.

`Sandbox` ¶

Bases: Protocol

Minimal interface every execution backend implements.

`LocalSandbox(extra_env: Optional[Mapping[str, str]] = None, inherit_env: bool = False)` ¶

Runs commands with subprocess on the host.

The environment is scrubbed to an allowlist so that secrets in the parent process (API keys, tokens) never leak into agent-controlled commands.

`DockerSandbox(image: str = 'python:3.12-slim', network: str = 'none', memory: str = '2g', cpus: str = '2', docker_bin: str = 'docker', extra_args: Optional[List[str]] = None, workdir: str = '/workspace')` ¶

Runs each command in a fresh container with the workspace bind-mounted.

Parameters:

Name	Type	Description	Default
`image`	`str`	Docker image to use, e.g. `python:3.12-slim`.	`'python:3.12-slim'`
`network`	`str`	Docker network mode. `none` (default) blocks all network access, which keeps graded runs hermetic. Use `bridge` for `repo.setup` steps that need to download dependencies.	`'none'`
`memory`	`str`	Resource limits passed straight to `docker run`.	`'2g'`

`get_sandbox(name: str = 'local', **kwargs: object) -> Sandbox` ¶

Factory used by the CLI: local or docker[:image].

Graders¶

`repogym.graders.base` ¶

`Grader` ¶

Bases: ABC

Base class for all graders.

A grader inspects a workspace (after the agent has finished) and returns a :class:GradeComponent whose score is in [0, 1].

`repogym.graders.tests` ¶

Execution-based grading via the task's test-suite.

`TestGrader(use_hidden: bool = True, restore_protected: bool = True)` ¶

Bases: Grader

Scores fail_to_pass and pass_to_pass tests.

score = fraction of fail_to_pass tests passing (or 0/1 if partial credit is disabled)
any failing pass_to_pass test zeroes the score when require_all_pass_to_pass is set (the default)
with no fail_to_pass tests (refactor / perf tasks) the component is a gate: it contributes no credit but zeroes the total when regressions appear

`run_tests(workspace: Workspace, command: Optional[str] = None, timeout: Optional[int] = None, *, hidden: bool = False, restore_protected: bool = False) -> TestRun` ¶

Run the task's test command in the workspace and parse the JUnit output.

Parameters:

Name	Type	Description	Default
`hidden`	`bool`	Inject the task's hidden tests before running and remove them afterwards.	`False`
`restore_protected`	`bool`	Reset the visible test files referenced by `fail_to_pass`/`pass_to_pass` to their baseline content first, so agents cannot pass by editing tests.	`False`

`protected_test_files(workspace: Workspace) -> List[str]` ¶

Visible test files that carry reward and therefore must not be edited.

`repogym.graders.perf` ¶

Benchmark-based grading for performance-optimisation tasks.

Timing is noisy, and RL rewards must not be. Three measures keep the perf signal honest:

the benchmark process is timed externally (wall clock) by default, so agent-reachable code cannot fake the number;
baseline and candidate are measured interleaved (B, C, B, C, ...) in the same grading call and the minimum of each is used, which cancels machine drift and additive noise;
speed-ups below perf.noise_floor (default 1.15x) count as no improvement.

`PerfGrader(use_cache: bool = True)` ¶

Bases: Grader

Rewards speed-ups relative to the baseline implementation.

score = clamp(log(speedup / noise_floor) / log(min_speedup / noise_floor), 0, 1): zero until the noise floor, smooth partial credit up to the target, full credit at or beyond it. passed requires speedup >= min_speedup.

`parse_seconds(stdout: str) -> Optional[float]` ¶

Extract the benchmark time from stdout.

Accepts {"seconds": 1.23} JSON (possibly surrounded by other output) or a bare float on the last non-empty line.

`measure(workspace: Workspace, runs: Optional[int] = None) -> List[float]` ¶

Run the benchmark runs times and return the timings.

`baseline_seconds(workspace: Workspace, use_cache: bool = True) -> float` ¶

Minimum benchmark time of the unmodified task repository.

`repogym.graders.constraints` ¶

Static (execution-free) grading: scope, patterns and code-quality limits.

`ConstraintGrader` ¶

Bases: Grader

Gate: scope and pattern rules from the task's constraints section.

Checks protected files (test files carrying reward are always protected), allowed_files, max_files_changed, max_lines_changed, forbidden_patterns and required_patterns. The score is the fraction of rules satisfied; as a gate any violation zeroes the composite score.

`QualityGrader` ¶

Bases: Grader

Per-function size and complexity limits (Python AST).

Each configured limit (max_function_length, max_cyclomatic_complexity) is one rule; the score is the fraction of rules with zero violations, so a refactor that fixes complexity but not length earns 0.5. Measured on constraints.paths.

For refactor tasks the limits are the objective. For every other task type they are a gate: new code must stay within them, but meeting them earns no credit (otherwise an empty submission would be rewarded).

`repogym.graders.composite` ¶

Combine grader components into a single reward.

`CompositeGrader(graders: Sequence[Grader], weights: Optional[Dict[str, float]] = None)` ¶

Bases: Grader

Weighted combination of graders.

objectives (target tests, benchmarks, quality limits) contribute their weighted scores;
gates (regression tests, scope/pattern rules) contribute nothing, but a failing gate zeroes the score;
passed requires every component to pass and the score to reach grading.pass_threshold.

Reward hacking on one axis (a fast but wrong solution, a correct fix that also rewrites the tests) therefore never yields a passing grade, and an empty submission always scores 0.

`build_grader(workspace_or_task: object) -> CompositeGrader` ¶

Construct the standard grader stack for a task from its spec.

`grade_workspace(workspace: Workspace) -> GradeResult` ¶

One-liner: grade a workspace with the task's default grader stack.

Results¶

`repogym.models` ¶

Pydantic models that define a repogym task and the results it produces.

A task is a directory containing task.yaml (validated into :class:TaskSpec), a repository snapshot (or a pointer to a git commit), an optional golden solution, and optional hidden tests. Everything the library does - materialising a workspace, grading, validating, mining - is expressed in terms of these models, so they are deliberately explicit and heavily documented.

`GradeComponent` ¶

Bases: BaseModel

Score produced by one grader.

role decides how the composite treats it: an objective contributes its weighted score, a gate contributes nothing but zeroes the total when it fails. Regression tests, scope rules and pattern rules are gates; target tests, benchmarks and code-quality limits are objectives. This is what makes "doing nothing scores 0" hold for every task type.

`GradeResult` ¶

Bases: BaseModel

Final reward for a submission.

`TestRun` ¶

Bases: BaseModel

Parsed result of one execution of the test command.

`TestCaseResult` ¶

Bases: BaseModel

`TestStatus` ¶

Bases: str, Enum

`ValidationReport` ¶

Bases: BaseModel

Output of repogym validate for one task.

`ValidationCheck` ¶

Bases: BaseModel

`Trajectory` ¶

Bases: BaseModel

A full episode: the primary training-data artefact repogym produces.

`TrajectoryStep` ¶

Bases: BaseModel

One (action, observation, reward) tuple recorded during an episode.

`RunResult` ¶

Bases: BaseModel

Result of running one agent on one task.

`CommandResult` ¶

Bases: BaseModel

Outcome of a shell command executed in a sandbox.

Agents¶

`repogym.agents.base` ¶

`Agent` ¶

Bases: ABC

Anything that can solve a task by acting on an environment.

Subclass and implement :meth:solve; it receives the environment after reset() and the initial observation, and must return the final observation (done=True). The helper :meth:run wraps the whole episode lifecycle.

`GoldenAgent` ¶

Bases: Agent

Applies the task's golden solution and submits.

Used by repogym validate to prove that a task is solvable, and as the upper bound in reports.

`NoopAgent` ¶

Bases: Agent

Submits immediately. A sanity baseline: every task must score 0 here.

`ScriptedAgent(actions: Iterable[Union[Action, Dict[str, Any]]], name: str = 'scripted')` ¶

Bases: Agent

Replays a fixed list of actions (handy for tests and trajectory replay).

`FunctionAgent(fn: Callable[[RepoEnv, Observation], Observation], name: str = 'function')` ¶

Bases: Agent

Wraps a plain callable fn(env, obs) -> Observation.

`ShellAgent(command: str, timeout: int = 1800, name: Optional[str] = None, env: Optional[Dict[str, str]] = None)` ¶

Bases: Agent

Runs an external coding agent CLI inside the workspace, then submits.

The command is a shell template; {prompt} is replaced with a shell-quoted problem statement. The environment variables REPOGYM_TASK_ID, REPOGYM_PROBLEM and REPOGYM_WORKSPACE are set as well, so any tool (Claude Code, Aider, Codex, OpenHands, your own script) can be plugged in::

ShellAgent('claude -p {prompt} --allowedTools Edit,Write,Bash')
ShellAgent('aider --message {prompt} --yes')
ShellAgent('python my_agent.py')

`get_agent(spec: str, **kwargs: Any) -> Agent` ¶

Resolve an agent from a CLI-style spec.

golden / noop
shell:<command> - e.g. shell:aider --message {prompt} --yes
claude or claude:<model> - the built-in Anthropic tool-loop agent
module.path:ClassName - any importable :class:Agent subclass

`repogym.agents.claude` ¶

A reference LLM agent built on the Anthropic SDK.

It exposes the environment's actions as tools and runs a standard tool-use loop. It is intentionally small: the point of repogym is to make any agent gradeable, and this one exists so that repogym run --agent claude works out of the box and so that trajectories can be generated for training data.

Install with pip install "repogym[anthropic]" and set ANTHROPIC_API_KEY.

`ClaudeAgent(model: str = 'claude-opus-5', max_turns: int = 40, max_tokens: int = 16000, effort: Optional[str] = None, client: Any = None, system_prompt: str = SYSTEM_PROMPT)` ¶

Bases: Agent

Tool-use loop over the Anthropic Messages API.

`tool_call_to_action(name: str, args: Dict[str, Any]) -> Action` ¶

Validation and running¶

`repogym.validate` ¶

Task validation: prove that a task is a sound RL environment.

A task is only useful for training if

the baseline (unmodified repo) does not already pass -> reward 0 for doing nothing
the golden solution does pass every grader -> reward 1 is attainable
pass-to-pass tests hold on both sides -> the regression guard is meaningful
hidden tests are truly hidden -> no leakage into the agent's view
results are deterministic -> reward is a function of the patch, not of luck

validate_task runs all of those checks and returns a report.

`repogym.runner` ¶

Run agents against tasks, serially or in parallel, and collect results.

`run_task(task: Task, agent: Agent, sandbox: Optional[Sandbox] = None, *, record_dir: Optional[Union[str, Path]] = None, keep_workspace: bool = False) -> RunResult` ¶

Run one agent on one task and return a :class:RunResult.

`run_suite(tasks: Iterable[Task], agent_factory: Callable[[], Agent], sandbox: Optional[Sandbox] = None, *, workers: int = 1, record_dir: Optional[Union[str, Path]] = None, on_result: Optional[ProgressCallback] = None) -> List[RunResult]` ¶

Run an agent over many tasks.

agent_factory is called once per task so that stateful agents (LLM conversations) never share state across episodes.

Mining¶

`repogym.mine.history` ¶

Mine real bug-fix tasks from git history (the SWE-bench recipe, automated).

For each candidate commit that touches both tests and source:

check out the parent commit and overlay the commit's test changes -> this is the task's starting state (bug present, new tests visible)
run the tests; tests that fail here are candidates for fail_to_pass
check out the full commit and run the tests again; the intersection of "failed before" and "passes after" is fail_to_pass, tests passing on both sides become pass_to_pass
the source-only part of the commit diff becomes solution.patch

Only commits where every step succeeds produce a task, so mined tasks are verified by construction.

`HistoryMiner(repo: Path, *, test_command: str = 'python -m pytest -q -p no:cacheprovider --junitxml={junit}', setup: Optional[List[str]] = None, sandbox: Optional[Sandbox] = None, timeout: int = 600, max_failing_fraction: float = 0.5, language: str = 'python')` ¶

`evaluate(cand: MinedCandidate, log: Callable[[str], None] = lambda s: None) -> MinedCandidate` ¶

Verify a candidate by executing tests before/after the fix.

`MinedCandidate(commit: str, parent: str, subject: str, body: str, test_files: List[str], source_files: List[str], fail_to_pass: List[str] = list(), pass_to_pass: List[str] = list(), solution_patch: str = '', failure_messages: Dict[str, str] = dict(), reason: Optional[str] = None)` `dataclass` ¶

`repogym.mine.mutate` ¶

Synthesise verified bug-fix tasks by injecting faults into a healthy repository.

This is the cheapest way to get thousands of executable, verified tasks from a single codebase: each mutation that makes some (but not all) tests fail is, by construction, a bug with a known one-line fix and an executable oracle.

Mutation operators (applied textually, so formatting and comments survive):

comparison flips < <-> <=, > <-> >=, == <-> !=
arithmetic swaps + <-> -, * <-> /
boolean swaps and <-> or
off-by-one constants n -> n + 1 / n - 1
boundary literals True <-> False
negation removal not x -> x

`MutationMiner(repo: Path, *, test_command: str = 'python -m pytest -q -p no:cacheprovider --junitxml={junit}', setup: Optional[List[str]] = None, sandbox: Optional[Sandbox] = None, timeout: int = 600, include: Sequence[str] = ('**/*.py',), exclude: Sequence[str] = (), max_failing_fraction: float = 0.5, min_failing: int = 1, seed: int = 0)` ¶

`mine(out_root: Path, *, max_tasks: int = 10, max_per_file: int = 50, sample: Optional[int] = None, overwrite: bool = False, log: Callable[[str], None] = lambda s: None) -> List[Path]` ¶

Generate up to max_tasks verified tasks.

sample randomly subsamples the candidate mutations (deterministic via seed) so that large repositories do not take hours.

`Mutation(file: str, line: int, col: int, end_line: int, end_col: int, original: str, replacement: str, operator: str)` `dataclass` ¶

`generate_mutations(source: str, file: str, max_per_file: int = 50) -> List[Mutation]` ¶

Enumerate candidate mutations for one Python file.

Export and reports¶

`repogym.export` ¶

Export tasks and trajectories to training-friendly formats.

`task_to_record(task: Task, include_repo: bool = False) -> Dict[str, Any]` ¶

SWE-bench-compatible record (instance_id, problem_statement, ...).

`trajectory_to_messages(traj: Trajectory) -> List[Dict[str, Any]]` ¶

Flatten a trajectory into a chat-style message list for SFT pipelines.

Each action becomes an assistant message carrying a JSON action and each observation a user message, so the sequence can be tokenised directly.

`export_trajectories_jsonl(trajectories: Iterable[Union[Trajectory, Path, str]], path: Union[str, Path], *, only_passed: bool = False, min_reward: float = 0.0) -> int` ¶

Write trajectories as JSONL (one episode per line) with a chat rendering.

`repogym.report` ¶

Render run results as terminal tables, Markdown and a standalone HTML report.

`LeaderboardRow(agent: str, tasks: int, passed: int, pass_rate: float, mean_score: float, mean_steps: float, mean_duration: float, errors: int)` `dataclass` ¶

`leaderboard(results: List[RunResult]) -> List[LeaderboardRow]` ¶

Aggregate results per agent: pass rate, mean score, mean steps.

`to_markdown(results: List[RunResult]) -> str` ¶

`to_html(results: List[RunResult]) -> str` ¶

`write_report(results: List[RunResult], path: Union[str, Path]) -> Path` ¶

Metrics¶

`repogym.metrics` ¶

Lightweight, dependency-free static metrics for Python source.

Used by the constraint grader to make refactoring tasks objectively gradeable: "reduce process_order below complexity 10" is a checkable statement, while "make the code cleaner" is not.

`function_metrics(source: str, filename: str = '<string>') -> List[FunctionMetrics]` ¶

Return metrics for every function/method in source (nested included).

API reference¶

Tasks¶

repogym.task ¶

Task(spec: TaskSpec, path: Path) ¶

load(path: Union[str, Path], *, require_solution: bool = True) -> Task classmethod ¶

TaskError ¶

load_tasks(root: Union[str, Path], *, include: Optional[Sequence[str]] = None, types: Optional[Sequence[str]] = None, tags: Optional[Sequence[str]] = None, strict: bool = True) -> List[Task] ¶

iter_task_dirs(root: Union[str, Path]) -> Iterator[Path] ¶

Task specification¶

repogym.models ¶

TaskSpec ¶

RepoSpec ¶

TestSpec ¶

PerfSpec ¶

ConstraintSpec ¶

GradingSpec ¶

SolutionSpec ¶

EnvSpec ¶

TaskType ¶

Difficulty ¶

Environment¶

repogym.env ¶

RepoEnv(task: Task, sandbox: Optional[Sandbox] = None, *, agent_name: str = 'unknown', record_dir: Optional[Union[str, Path]] = None, keep_workspace: bool = False, workspace_root: Optional[Union[str, Path]] = None, show_tree: bool = True) ¶

prompt() -> str ¶

Action ¶

Observation ¶

EnvState ¶

Read(path: str, start_line: Optional[int] = None, end_line: Optional[int] = None) -> Action ¶

Write(path: str, content: str) -> Action ¶

Edit(path: str, old: str, new: str) -> Action ¶

Run(command: str) -> Action ¶

Test() -> Action ¶

ListFiles(pattern: Optional[str] = None) -> Action ¶

Submit() -> Action ¶

Workspace and sandboxes¶

repogym.workspace ¶

Workspace(task: Task, sandbox: Optional[Sandbox] = None, root: Optional[Union[str, Path]] = None, keep: bool = False) ¶

scratch: Path property ¶

resolve(rel: Union[str, Path]) -> Path ¶

diff() -> str ¶

apply_solution() -> None ¶

restore(files: List[str]) -> None ¶

reset() -> None ¶

inject_hidden_tests() -> List[str] ¶

WorkspaceError ¶

repogym.sandbox ¶

Sandbox ¶

LocalSandbox(extra_env: Optional[Mapping[str, str]] = None, inherit_env: bool = False) ¶

DockerSandbox(image: str = 'python:3.12-slim', network: str = 'none', memory: str = '2g', cpus: str = '2', docker_bin: str = 'docker', extra_args: Optional[List[str]] = None, workdir: str = '/workspace') ¶

get_sandbox(name: str = 'local', **kwargs: object) -> Sandbox ¶

Graders¶

repogym.graders.base ¶

Grader ¶

repogym.graders.tests ¶

TestGrader(use_hidden: bool = True, restore_protected: bool = True) ¶

run_tests(workspace: Workspace, command: Optional[str] = None, timeout: Optional[int] = None, *, hidden: bool = False, restore_protected: bool = False) -> TestRun ¶

protected_test_files(workspace: Workspace) -> List[str] ¶

repogym.graders.perf ¶

PerfGrader(use_cache: bool = True) ¶

parse_seconds(stdout: str) -> Optional[float] ¶

measure(workspace: Workspace, runs: Optional[int] = None) -> List[float] ¶

baseline_seconds(workspace: Workspace, use_cache: bool = True) -> float ¶

repogym.graders.constraints ¶

ConstraintGrader ¶

QualityGrader ¶

repogym.graders.composite ¶

CompositeGrader(graders: Sequence[Grader], weights: Optional[Dict[str, float]] = None) ¶

build_grader(workspace_or_task: object) -> CompositeGrader ¶

grade_workspace(workspace: Workspace) -> GradeResult ¶

Results¶

repogym.models ¶

GradeComponent ¶

GradeResult ¶

TestRun ¶

TestCaseResult ¶

TestStatus ¶

ValidationReport ¶

ValidationCheck ¶

Trajectory ¶

TrajectoryStep ¶

`repogym.task` ¶

`Task(spec: TaskSpec, path: Path)` ¶

`load(path: Union[str, Path], *, require_solution: bool = True) -> Task` `classmethod` ¶

`TaskError` ¶

`load_tasks(root: Union[str, Path], *, include: Optional[Sequence[str]] = None, types: Optional[Sequence[str]] = None, tags: Optional[Sequence[str]] = None, strict: bool = True) -> List[Task]` ¶

`iter_task_dirs(root: Union[str, Path]) -> Iterator[Path]` ¶

`repogym.models` ¶

`TaskSpec` ¶

`RepoSpec` ¶

`TestSpec` ¶

`PerfSpec` ¶

`ConstraintSpec` ¶

`GradingSpec` ¶

`SolutionSpec` ¶

`EnvSpec` ¶

`TaskType` ¶

`Difficulty` ¶

`repogym.env` ¶

`RepoEnv(task: Task, sandbox: Optional[Sandbox] = None, *, agent_name: str = 'unknown', record_dir: Optional[Union[str, Path]] = None, keep_workspace: bool = False, workspace_root: Optional[Union[str, Path]] = None, show_tree: bool = True)` ¶

`prompt() -> str` ¶

`Action` ¶

`Observation` ¶

`EnvState` ¶

`Read(path: str, start_line: Optional[int] = None, end_line: Optional[int] = None) -> Action` ¶

`Write(path: str, content: str) -> Action` ¶

`Edit(path: str, old: str, new: str) -> Action` ¶

`Run(command: str) -> Action` ¶

`Test() -> Action` ¶

`ListFiles(pattern: Optional[str] = None) -> Action` ¶

`Submit() -> Action` ¶

`repogym.workspace` ¶

`Workspace(task: Task, sandbox: Optional[Sandbox] = None, root: Optional[Union[str, Path]] = None, keep: bool = False)` ¶

`scratch: Path` `property` ¶

`resolve(rel: Union[str, Path]) -> Path` ¶

`diff() -> str` ¶

`apply_solution() -> None` ¶

`restore(files: List[str]) -> None` ¶

`reset() -> None` ¶

`inject_hidden_tests() -> List[str]` ¶

`WorkspaceError` ¶

`repogym.sandbox` ¶

`Sandbox` ¶

`LocalSandbox(extra_env: Optional[Mapping[str, str]] = None, inherit_env: bool = False)` ¶

`DockerSandbox(image: str = 'python:3.12-slim', network: str = 'none', memory: str = '2g', cpus: str = '2', docker_bin: str = 'docker', extra_args: Optional[List[str]] = None, workdir: str = '/workspace')` ¶

`get_sandbox(name: str = 'local', **kwargs: object) -> Sandbox` ¶

`repogym.graders.base` ¶

`Grader` ¶

`repogym.graders.tests` ¶

`TestGrader(use_hidden: bool = True, restore_protected: bool = True)` ¶

`run_tests(workspace: Workspace, command: Optional[str] = None, timeout: Optional[int] = None, *, hidden: bool = False, restore_protected: bool = False) -> TestRun` ¶

`protected_test_files(workspace: Workspace) -> List[str]` ¶

`repogym.graders.perf` ¶

`PerfGrader(use_cache: bool = True)` ¶

`parse_seconds(stdout: str) -> Optional[float]` ¶

`measure(workspace: Workspace, runs: Optional[int] = None) -> List[float]` ¶

`baseline_seconds(workspace: Workspace, use_cache: bool = True) -> float` ¶

`repogym.graders.constraints` ¶

`ConstraintGrader` ¶

`QualityGrader` ¶

`repogym.graders.composite` ¶

`CompositeGrader(graders: Sequence[Grader], weights: Optional[Dict[str, float]] = None)` ¶

`build_grader(workspace_or_task: object) -> CompositeGrader` ¶

`grade_workspace(workspace: Workspace) -> GradeResult` ¶

`repogym.models` ¶

`GradeComponent` ¶

`GradeResult` ¶

`TestRun` ¶

`TestCaseResult` ¶

`TestStatus` ¶

`ValidationReport` ¶

`ValidationCheck` ¶

`Trajectory` ¶

`TrajectoryStep` ¶

`RunResult` ¶

`CommandResult` ¶

`repogym.agents.base` ¶

`Agent` ¶

`GoldenAgent` ¶

`NoopAgent` ¶

`ScriptedAgent(actions: Iterable[Union[Action, Dict[str, Any]]], name: str = 'scripted')` ¶