Skip to content

Commit 5f3240b

Browse files
authored
test: pin the srt-macos probe PATH and scale the BYOK serial timeout budget (#397)
* test(sandbox): pin the probe PATH instead of inheriting the ambient one The real sandbox-exec probes resolve their command through PATH, so inheriting the caller's let any wrapper ahead of /bin decide what `rm` means. On a machine that has run the agent, that is the recoverable-delete shim MCode installs into agent shells: `rm` resolves to the shim, the shim execs mavis-trash, the sandbox denies that trash move as a write outside the policy under test, and a correct allow-probe exits 1. The suite then reports a sandbox regression where the sandbox behaved correctly. Pin all three probe environments to the system binaries the probes actually use, matching the wrapper-argument test above them. This also drops the unguarded `process.env.PATH` in wrapProfile, which had no fallback and could hand a real sandbox spawn an undefined PATH. Fixes #394. * test(byok): scale the serial BYOK budget by platform The first case is 30+ serial CLI spawns, each booting the whole runtime, so its wall-clock cost tracks machine speed rather than the transport it asserts. The flat 90s budget only held with roughly 1.5x headroom on an idle machine, so a parallel build, a laptop under load, or a shared CI runner turned a passing transport into a `testTimeoutFailure` that named a product failure no assertion had actually observed. Budget it the way smoke.test.mjs already budgets runtime startup: a base allowance plus a Windows multiplier for slower process spawn. The test is not slowed down by a larger ceiling; it only stops failing for reasons unrelated to what it checks. Fixes #395.
1 parent 1e136bf commit 5f3240b

2 files changed

Lines changed: 23 additions & 4 deletions

File tree

‎packages/local-runtime-v2/src/service/sandbox/backend/srt-macos.test.ts‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1487,6 +1487,17 @@ async function expectMandatoryDenySurface(
14871487
}
14881488
}
14891489

1490+
/**
1491+
* Probes must measure the sandbox, not the caller's shell. Every probe spawns a
1492+
* real process and resolves its command through `PATH`, so inheriting the
1493+
* ambient one lets a wrapper ahead of `/bin` decide what `rm` means — for
1494+
* example the recoverable-delete shim MCode installs into agent shells, whose
1495+
* trash move the sandbox then denies, turning a correct allow-probe into a
1496+
* reported sandbox regression. Pin the probes to the system binaries they
1497+
* actually use (`cat`, `rm`, `rmdir`, `find`, `mv`, `printf`, `git`, `true`).
1498+
*/
1499+
const PROBE_PATH = "/usr/bin:/bin";
1500+
14901501
async function expectProbeResult(
14911502
command: string,
14921503
filesystem: SandboxInvocationFilesystemPolicy,
@@ -1520,7 +1531,7 @@ async function runProbe(
15201531
cwd: fixture.workspace,
15211532
baseEnv: {
15221533
HOME: process.env.HOME ?? "/var/empty",
1523-
PATH: process.env.PATH ?? "/usr/bin:/bin",
1534+
PATH: PROBE_PATH,
15241535
},
15251536
sandboxTempDir: fixture.sessionTemp,
15261537
commandId,
@@ -1593,7 +1604,7 @@ function createGitProbeFixture(): GitProbeFixture {
15931604
function gitProbeEnv(fixture: GitProbeFixture): Record<string, string> {
15941605
return {
15951606
HOME: fixture.home,
1596-
PATH: process.env.PATH ?? "/usr/bin:/bin",
1607+
PATH: PROBE_PATH,
15971608
GIT_CONFIG_NOSYSTEM: "1",
15981609
GIT_AUTHOR_NAME: "probe",
15991610
GIT_AUTHOR_EMAIL: "probe@example.invalid",
@@ -1712,7 +1723,7 @@ async function wrapProfile(
17121723
await profileBackend.wrap({
17131724
command: "true",
17141725
cwd: profileFixture,
1715-
baseEnv: { PATH: process.env.PATH },
1726+
baseEnv: { PATH: PROBE_PATH },
17161727
sandboxTempDir: profileFixture,
17171728
commandId: opaqueId,
17181729
commandText: opaqueId,

‎test/byok.test.mjs‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,18 @@ import { parse as parseYaml, stringify as stringifyYaml } from "yaml";
2222
import { withoutProxyEnvironment } from "./offline-environment.mjs";
2323

2424
const cli = process.env.MCODE_TEST_CLI ?? fileURLToPath(new URL("../dist/cli.js", import.meta.url));
25+
// This case is 30+ serial CLI spawns, each booting the whole runtime, so its
26+
// wall-clock cost tracks machine speed rather than the transport it asserts.
27+
// A flat budget sized for an idle laptop turns any busy runner (parallel build,
28+
// shared CI host) into a `testTimeoutFailure` that reports a product failure
29+
// while every assertion would have passed. Budget it the way smoke.test.mjs
30+
// budgets runtime startup: a base allowance plus a Windows multiplier for
31+
// slower process spawn, rather than a number that only holds on one machine.
32+
const byokSerialTimeoutMs = process.platform === "win32" ? 360000 : 180000;
2533
// This fixture validates BYOK transport and real Runtime persistence, not model quality.
2634
test(
2735
"BYOK runs without managed login and resumes its saved conversation",
28-
{ timeout: 90000 },
36+
{ timeout: byokSerialTimeoutMs },
2937
async (t) => {
3038
const fixtureDir = mkdtempSync(path.join(tmpdir(), "minimax-code-byok-"));
3139
const dataDir = path.join(fixtureDir, "data");

0 commit comments

Comments
 (0)