Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -789,6 +789,7 @@ Supported third-party environment integrations include:
- **`ReasoningGymEnv`** — wraps [reasoning-gym](https://github.com/open-thought/reasoning-gym) procedural datasets
- **`BrowserEnv`** — unified browser automation via [Browserbase](https://browserbase.com) with DOM and CUA modes
- **`OpenEnvEnv`** — wraps OpenEnv gym and MCP contracts using Prime Sandboxes with prebuilt images referenced from `.build.json`
- **`NemoGymEnv`** — wraps [NVIDIA NeMo Gym](https://github.com/NVIDIA-NeMo/Gym) environments.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NemoGymEnv install instructions missing from docs

Low Severity

NemoGymEnv is added to the integrations list but the follow-up install paragraph at line 794 only mentions extras for TextArena, BrowserEnv, and OpenEnvEnv. Since NemoGymEnv requires a different install flow (editable local clone, no pip extra defined in pyproject.toml), users reading "These require additional dependencies installed via extras" will be confused about how to install the NeMo Gym dependency.

Fix in Cursor Fix in Web

Triggered by project rule: BugBot Instructions

Reviewed by Cursor Bugbot for commit bb22743. Configure here.


These require additional dependencies installed via extras (e.g., `uv add 'verifiers[ta]'` for TextArena, `uv add 'verifiers[browser]'` for BrowserEnv, `uv add 'verifiers[openenv]'` for OpenEnvEnv). For OpenEnv environments, build the bundled project image with `prime env build <env-id>` before evaluation or training.

Expand Down
10 changes: 10 additions & 0 deletions environments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,15 @@ This folder contains installable example environments that showcase common usage
- **opencode_harbor**: Runs the OpenCode CLI agent on Harbor tasks with API interception via Prime Tunnel.
- **terminus_harbor**: Runs the Terminus agent on Harbor tasks with API interception via Prime Tunnel.

- **NeMo Gym**: wraps NVIDIA NeMo Gym environments. A few examples to get started:
- **nemo-gym-workplace-assistant**: Multi-step tool use workplace assistant
- **nemo-gym-reasoning-gym**: Reasoning Gym tasks with a simple agent
- **nemo-gym-reasoning-gym-reflection**: Reasoning Gym with a LangGraph reflection agent
- **nemo-gym-structured-outputs**: Structured output tasks
- **nemo-gym-code-gen**: Code generation tasks
- **nemo-gym-mcqa**: Science multiple choice question answering
- **nemo-gym-arc-agi**: ARC-AGI 1 and 2

Comment thread
cursor[bot] marked this conversation as resolved.
### Composition
- **EnvGroup**
- **math_group**: Groups two `SingleTurnEnv` tasks (GSM8K + Math) into one environment with shared interface.
Expand All @@ -73,6 +82,7 @@ This folder contains installable example environments that showcase common usage
- **GymEnv integration**: `gem_wordle`
- **OpenEnv integration (gym + MCP)**: `openenv_textarena`, `openenv_echo`
- **CLI agent sandboxes**: `opencode_harbor`, `terminus_harbor`
- **NeMo Gym integration**: `nemo-gym-reasoning-gym`, `nemo-gym-workplace-assistant`
- **MCP integration**: `mcp_search_env`
- **RLM (recursive LLM)**: `rlm_secrets`
- **Environment and rubric composition**: `math_group`, `math_python`, `wiki_search`
Expand Down
20 changes: 20 additions & 0 deletions environments/nemo_gym/nemo_gym_arc_agi/nemo_gym_arc_agi.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("arc_agi", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("arc_agi")],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_arc_agi/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-arc-agi"
description = "NeMo Gym ARC AGI 1 and 2 environment"
tags = ["nemo-gym", "knowledge", "single-turn"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_arc_agi.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
20 changes: 20 additions & 0 deletions environments/nemo_gym/nemo_gym_code_gen/nemo_gym_code_gen.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("code_gen", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("code_gen")],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_code_gen/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-code-gen"
description = "NeMo Gym code generation environment"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_code_gen.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
20 changes: 20 additions & 0 deletions environments/nemo_gym/nemo_gym_mcqa/nemo_gym_mcqa.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("mcqa", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("mcqa")],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_mcqa/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-mcqa"
description = "NeMo Gym mcqa environment"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_mcqa.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("reasoning_gym", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("reasoning_gym")],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_reasoning_gym/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-reasoning-gym"
description = "NeMo Gym Reasoning Gym simple agent environment"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_reasoning_gym.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("reasoning_gym", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("reasoning_gym", "parallel_thinking_agent")],
dataset=dataset,
**kwargs,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-reasoning-gym-parallel-thinking"
description = "NeMo Gym Reasoning Gym environment with LangGraph parallel thinking agent"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_reasoning_gym_parallel_thinking.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset("reasoning_gym", "example", dataset_path=dataset_path)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("reasoning_gym", "reflection_agent")],
dataset=dataset,
**kwargs,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-reasoning-gym-reflection"
description = "NeMo Gym Reasoning Gym environment with LangGraph reflection agent"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_reasoning_gym_reflection.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset(
"structured_outputs", "example", dataset_path=dataset_path
)
return NemoGymEnv(
gym_configs=[
_resolve_gym_config("structured_outputs", "structured_outputs_json")
],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_structured_outputs/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-structured-outputs"
description = "NeMo Gym structured outputs environment"
tags = ["nemo-gym"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_structured_outputs.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from typing import Any

import verifiers as vf
from verifiers.envs.integrations.nemo_gym import (
NemoGymEnv,
_build_dataset,
_resolve_gym_config,
)


def load_environment(
dataset_path: str | None = None,
**kwargs: Any,
) -> vf.Environment:
dataset, _ = _build_dataset(
"workplace_assistant", "example", dataset_path=dataset_path
)
return NemoGymEnv(
gym_configs=[_resolve_gym_config("workplace_assistant")],
dataset=dataset,
**kwargs,
)
21 changes: 21 additions & 0 deletions environments/nemo_gym/nemo_gym_workplace_assistant/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[project]
name = "nemo-gym-workplace-assistant"
description = "NeMo Gym workplace assistant environment"
tags = ["nemo-gym", "agent", "tools", "multi-turn"]
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
"verifiers>=0.1.11.dev1",
"nemo-gym>=0.2.0",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
include = ["nemo_gym_workplace_assistant.py", "pyproject.toml"]

[tool.verifiers.eval]
num_examples = 5
rollouts_per_example = 1
Loading