Add sandbox reliability and feature matrix benchmark by HeyGarrison · Pull Request #146 · computesdk/benchmarks

HeyGarrison · 2026-06-25T19:35:59Z

Summary

add a lean sandbox reliability benchmark mode for high-frequency outage tracking
run reliability every 10 minutes, sampling providers every 30 seconds, and store timestamped results plus latest.json under results/sandbox-reliability
add a separate daily sandbox feature matrix benchmark for richer in-box capabilities and lightweight performance probes
record default tools/package managers, Docker CLI/daemon usability, CPU, memory, FS IO, TCP/UDP, localhost TCP, network, filesystem, and isolation signals under results/sandbox-features
add PR smoke runs and PR comments for both reliability and feature matrix results
add merge support for sandbox reliability and sandbox feature artifacts
harden the new workflows with scoped permissions, validated workflow_dispatch inputs, no persisted bench checkout credentials, and no npm cache in the secret-handling bench job

Validation

npm ci
npx tsc --noEmit --ignoreConfig --target es2020 --module es2020 --moduleResolution bundler --strict --esModuleInterop --skipLibCheck src/sandbox/reliability.ts src/run.ts src/merge-results.ts
git diff --check

open-cla · 2026-06-25T19:36:06Z

Contributor License Agreement

All contributors are covered by a CLA.

superagent-security

Superagent found 3 security concern(s).

superagent-security · 2026-06-25T19:38:03Z

+          - upstash
+          - vercel
+    steps:
+      - uses: actions/checkout@v4


P1: Workflow actions use mutable tags instead of commit SHAs

Multiple steps use mutable action tags (e.g. actions/checkout@v4) instead of immutable commit SHAs.

Replace each tag with the full 40-character commit SHA of the intended version, plus a # vX.Y.Z comment.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:52"> <priority>P1</priority> <title>Workflow actions use mutable tags instead of commit SHAs</title> <evidence>The workflow uses actions/checkout@v4, actions/setup-node@v4, actions/upload-artifact@v4, and actions/download-artifact@v4. Tags are mutable and can be force-pushed by an attacker who compromises the action repository.</evidence> <recommendation>Pin each action to a specific 40-character commit SHA and add a version comment, e.g. uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 # v4.2.0.</recommendation> </violation> </file>

superagent-security · 2026-06-25T19:38:03Z

+        run: |
+          SAMPLE_FLAG=""
+          if [ -n "${{ github.event.inputs.samples }}" ]; then
+            SAMPLE_FLAG="--samples ${{ github.event.inputs.samples }}"


P0: Workflow interpolates user-controlled workflow_dispatch inputs directly into shell commands

Shell script interpolates workflow_dispatch inputs directly, enabling command injection.

Pass inputs through environment variables and reference them as shell variables.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:90"> <priority>P0</priority> <title>Workflow interpolates user-controlled workflow_dispatch inputs directly into shell commands</title> <evidence>The run block interpolates ${{ github.event.inputs.samples }}, ${{ github.event.inputs['duration-seconds'] }}, and ${{ github.event.inputs['interval-seconds'] }} directly into the shell script. An attacker with workflow dispatch access can inject arbitrary shell commands via these inputs.</evidence> <recommendation>Pass the inputs through environment variables instead of interpolating them into the script. For example, set env: SAMPLES: ${{ github.event.inputs.samples }}, DURATION_SECONDS: ${{ github.event.inputs['duration-seconds'] || '600' }}, INTERVAL_SECONDS: ${{ github.event.inputs['interval-seconds'] || '30' }}, and reference $SAMPLES, $DURATION_SECONDS, and $INTERVAL_SECONDS in the run block.</recommendation> </violation> </file>

superagent-security · 2026-06-25T19:38:03Z

+        default: ''
+
+permissions:
+  contents: write


P2: Overly broad top-level permissions grant contents: write to all jobs

Top-level permissions: contents: write applies to both the bench and collect jobs, but only collect needs it.

Move contents: write to the collect job and set permissions: {} at the workflow level.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:22"> <priority>P2</priority> <title>Overly broad top-level permissions grant contents: write to all jobs</title> <evidence>The workflow declares permissions: contents: write at the top level, which grants write access to the GITHUB_TOKEN for both the bench and collect jobs. The bench job only runs benchmarks and uploads artifacts; it does not need contents: write.</evidence> <recommendation>Set permissions: {} at the workflow level and move contents: write to the collect job only, so each job receives the least privilege required.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:45:20Z

+jobs:
+  bench:
+    name: Reliability ${{ matrix.provider }}
+    runs-on: namespace-profile-default


P0: Self-hosted runner is reachable by fork PRs

Workflow runs on a self-hosted runner triggered by pull_request, allowing fork PRs to execute arbitrary code on it.

Use GitHub-hosted runners for PR jobs or restrict the self-hosted runner from fork PRs.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:35"> <priority>P0</priority> <title>Self-hosted runner is reachable by fork PRs</title> <evidence>The workflow triggers on `pull_request` and uses `runs-on: namespace-profile-default`, which is a self-hosted runner. Fork PR authors can execute arbitrary code on this runner, potentially compromising other jobs, accessing credentials stored on the runner filesystem, or reaching internal network resources.</evidence> <recommendation>Use GitHub-hosted runners (e.g., `ubuntu-latest`) for the `pull_request` trigger, or restrict the self-hosted runner so it cannot be assigned to workflows triggered by fork PRs. Consider splitting into two workflows: one for PR validation using GitHub-hosted runners, and another for scheduled benchmarking using the self-hosted runner.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:52:27Z

+          - upstash
+          - vercel
+    steps:
+      - uses: actions/checkout@v4


P2: actions/checkout@v4 leaves credentials persisted in bench job that does not push back

The bench job checkout step uses persist-credentials: true but never pushes back. On a self-hosted runner this can leak the GITHUB_TOKEN across jobs.

Add persist-credentials: false to the bench job checkout step.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:60"> <priority>P2</priority> <title>actions/checkout@v4 leaves credentials persisted in bench job that does not push back</title> <evidence>The bench job checkout step at line 60 uses actions/checkout@v4 with the default persist-credentials: true. This job only runs benchmarks and uploads artifacts; it does not push back to the repository. On the self-hosted runner namespace-profile-default, leftover credentials could leak across jobs if workspaces are not fully cleaned between runs.</evidence> <recommendation>Add persist-credentials: false to the bench job checkout step.</recommendation> </violation> </file>

superagent-security

Superagent found 5 security concern(s).

superagent-security · 2026-06-25T20:56:37Z

+jobs:
+  bench:
+    name: Reliability ${{ matrix.provider }}
+    runs-on: namespace-profile-default


P0: Self-hosted runner on pull_request trigger, reachable by fork PRs

Self-hosted runner runs on pull_request trigger, reachable by fork PRs.

Restrict self-hosted runner to non-PR triggers or scope runner group to disallow forks.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:35"> <priority>P0</priority> <title>Self-hosted runner on pull_request trigger, reachable by fork PRs</title> <evidence>The workflow triggers on pull_request and runs on self-hosted runner namespace-profile-default. Fork PRs can execute on self-hosted runners, which share state across jobs and have been the source of critical compromises.</evidence> <recommendation>Restrict the self-hosted runner to non-PR triggers, or ensure the runner is not reachable by fork PRs via GitHub runner group settings.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:56:37Z

+        required: false
+        default: ''
+
+permissions:


P1: Overly broad top-level permissions: contents: write

Top-level contents:write grants unnecessary write access to the bench job.

Set top-level permissions to minimal and grant contents:write only to the collect job.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="2" location=".github/workflows/sandbox-reliability.yml:29"> <priority>P1</priority> <title>Overly broad top-level permissions: contents: write</title> <evidence>The workflow declares permissions: contents: write at the top level, granting every job write access to repository contents. The bench job only needs to upload artifacts and does not need contents: write.</evidence> <recommendation>Move permissions: contents: write to only the collect job that performs git push, and set top-level permissions to minimal defaults.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:56:37Z

+          VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
+        run: |
+          SAMPLE_FLAG=""
+          if [ -n "${{ github.event.inputs.samples }}" ]; then


P1: Template injection via workflow_dispatch inputs

workflow_dispatch inputs are interpolated directly into shell run block with secret access.

Pass inputs through env variables instead of direct template interpolation.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="3" location=".github/workflows/sandbox-reliability.yml:97"> <priority>P1</priority> <title>Template injection via workflow_dispatch inputs</title> <evidence>workflow_dispatch inputs (github.event.inputs.samples, github.event.inputs['duration-seconds'], github.event.inputs['interval-seconds']) are directly interpolated into the shell run block of the bench job, which has access to 20+ secrets.</evidence> <recommendation>Pass inputs via env: variables instead of interpolating them directly into shell scripts, e.g. env: SAMPLES: ${{ github.event.inputs.samples }} and reference $SAMPLES in the script.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:56:37Z

+          - upstash
+          - vercel
+    steps:
+      - uses: actions/checkout@v4


P1: Actions pinned to mutable tags instead of commit SHAs

Actions are pinned to mutable tags instead of immutable commit SHAs.

Pin all actions to full commit SHAs with version comments.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="4" location=".github/workflows/sandbox-reliability.yml:60"> <priority>P1</priority> <title>Actions pinned to mutable tags instead of commit SHAs</title> <evidence>Multiple actions use mutable tag refs: actions/checkout@v4, actions/setup-node@v4, actions/upload-artifact@v4, actions/download-artifact@v4, actions/github-script@v7. A compromised action repository can force-push a tag to inject malicious code.</evidence> <recommendation>Pin every action to a specific full-length commit SHA with a version comment, e.g. uses: actions/checkout@<sha> # v4.1.1.</recommendation> </violation> </file>

superagent-security · 2026-06-25T20:56:37Z

+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+          cache: 'npm'


P1: Cache used in workflow job handling multiple secrets

npm cache used in job with access to 20+ provider secrets.

Remove npm caching from the secret-handling benchmark job.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="5" location=".github/workflows/sandbox-reliability.yml:64"> <priority>P1</priority> <title>Cache used in workflow job handling multiple secrets</title> <evidence>The bench job uses cache: 'npm' in actions/setup-node while handling numerous provider API secrets. Cache poisoning could affect subsequent runs or be used as a cross-job attack vector, especially on a self-hosted runner.</evidence> <recommendation>Remove cache: 'npm' from the benchmark job, or scope cache keys explicitly and validate cache integrity.</recommendation> </violation> </file>

github-actions · 2026-06-25T21:03:07Z

Sandbox Reliability Results

No sandbox reliability results were generated.

View full run · Results available as build artifacts

github-actions · 2026-06-25T21:29:32Z

Sandbox Feature Matrix Results

No sandbox feature matrix results were generated.

View full run · Results available as build artifacts

Copilot

Pull request overview

Adds two new sandbox benchmark modes—high-frequency reliability sampling and a daily feature-matrix probe—and wires them into the CLI, result merging, and GitHub Actions automation to persist and report results.

Changes:

Introduces src/sandbox/reliability.ts to run reliability and feature-matrix probes, summarize outcomes, and write timestamped + latest.json artifacts.
Extends src/run.ts and package.json scripts to run bench:reliability and bench:features, writing results under results/sandbox-reliability and results/sandbox-features.
Adds merge support and two workflows to run benchmarks on schedule/PRs, upload artifacts, merge them, and post PR comments.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/sandbox/reliability.ts	Implements reliability + feature-matrix probe execution, summarization, and JSON output formatting.
src/run.ts	Adds CLI modes for reliability/features and writes results to new results subdirectories.
src/merge-results.ts	Adds merge mode support for sandbox reliability/features result artifacts.
package.json	Adds `bench:reliability` and `bench:features` scripts.
.github/workflows/sandbox-reliability.yml	Adds scheduled + PR-triggered reliability runs, artifact upload, merge, PR comment, and scheduled commit/push.
.github/workflows/sandbox-features.yml	Adds scheduled + PR-triggered feature-matrix runs, artifact upload, merge, PR comment, and scheduled commit/push.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+async function runReliability(toRun: typeof providers): Promise<void> {
+  const samples = samplesArg ? parseInt(samplesArg, 10) : undefined;
+  const durationMs = !samples && durationSecondsArg ? parseInt(durationSecondsArg, 10) * 1000 : undefined;
+  const intervalMs = intervalSecondsArg ? parseInt(intervalSecondsArg, 10) * 1000 : undefined;


+  const processMarker = `computesdk_reliability_process_${markerToken}`;
+  const previousProcessMarker = previousMarkerToken ? `computesdk_reliability_process_${previousMarkerToken}` : '';
+  const token = shellQuote(markerToken);


+    'printf "dnsExample=%s\\n" "$dns_example"',
+    'printf "httpExample=%s\\n" "$http_example"',
+    'printf "fsIsolationOk=%s\\n" "$([ -z "$previous_fs_marker_path" ] && printf true || printf false)"',
+    'printf "processIsolationOk=%s\\n" "$([ "$process_marker_seen" = false ] && printf true || printf false)"',


+  const deduped = Array.from(seen.values()).map(e => e.result);
+  console.log(`\nMerging ${deduped.length} provider results for mode: ${resultsSubdir}`);


+    concurrency:
+      group: sandbox-reliability-${{ matrix.provider }}
+      cancel-in-progress: true


+    concurrency:
+      group: sandbox-features-${{ matrix.provider }}
+      cancel-in-progress: true


+      - name: Run reliability benchmark
+        env:
+          COMPUTESDK_API_KEY: ${{ secrets.COMPUTESDK_API_KEY }}
+          ARCHIL_API_KEY: ${{ secrets.ARCHIL_API_KEY }}
+          ARCHIL_REGION: ${{ secrets.ARCHIL_REGION }}
+          ARCHIL_DISK_ID: ${{ secrets.ARCHIL_DISK_ID }}


+      - name: Run feature matrix benchmark
+        env:
+          COMPUTESDK_API_KEY: ${{ secrets.COMPUTESDK_API_KEY }}
+          ARCHIL_API_KEY: ${{ secrets.ARCHIL_API_KEY }}
+          ARCHIL_REGION: ${{ secrets.ARCHIL_REGION }}
+          ARCHIL_DISK_ID: ${{ secrets.ARCHIL_DISK_ID }}


+  return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d{3}Z$/, 'Z');
+}
+
 const args = process.argv.slice(2);


superagent-security

Superagent found 6 security concern(s).

superagent-security · 2026-06-25T21:45:51Z

+jobs:
+  bench:
+    name: Features ${{ matrix.provider }}
+    runs-on: namespace-profile-default


P0: Self-hosted runner reachable by fork PRs in sandbox-features workflow

Self-hosted runner used with pull_request trigger exposes runner to fork PRs.

Verify runner scoping or use GitHub-hosted runners for PR jobs.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-features.yml"> <violation number="1" location=".github/workflows/sandbox-features.yml:21"> <priority>P0</priority> <title>Self-hosted runner reachable by fork PRs in sandbox-features workflow</title> <evidence>The workflow triggers on pull_request and runs on a self-hosted runner labeled namespace-profile-default. Fork PRs can execute code on this runner, which shares state across jobs and has been the root cause of critical supply-chain compromises (e.g., PyTorch). The bench job runs arbitrary benchmark code from the checked-out repository on this runner.</evidence> <recommendation>Restrict the self-hosted runner to non-fork PRs by verifying scoping in repository Settings > Actions > Runners, or switch PR jobs to GitHub-hosted runners (e.g., ubuntu-latest). If the runner must be used, add an explicit if: github.event.pull_request.head.repo.full_name == github.repository guard or move the PR job to a separate workflow that uses GitHub-hosted runners.</recommendation> </violation> </file>

superagent-security · 2026-06-25T21:45:51Z

+          - upstash
+          - vercel
+    steps:
+      - uses: actions/checkout@v4


P1: Actions pinned to mutable tags instead of commit SHAs in sandbox-features workflow

Workflow actions use mutable version tags instead of immutable commit SHAs.

Pin all actions to commit SHA plus version comment.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-features.yml"> <violation number="1" location=".github/workflows/sandbox-features.yml:48"> <priority>P1</priority> <title>Actions pinned to mutable tags instead of commit SHAs in sandbox-features workflow</title> <evidence>actions/checkout@v4, actions/setup-node@v4, actions/upload-artifact@v4, actions/download-artifact@v4, and actions/github-script@v7 are all pinned to mutable tags. A compromised action repository can force-push a tag to inject malicious code into the workflow.</evidence> <recommendation>Pin every third-party and official action to a full 40-character commit SHA and add a version comment, e.g., uses: actions/checkout@<sha> # v4.1.1.</recommendation> </violation> </file>

superagent-security · 2026-06-25T21:45:51Z

+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/checkout@v4


P2: Missing persist-credentials on collect job checkout in sandbox-features workflow

Collect job checkout persists credentials unnecessarily.

Add persist-credentials: false to collect job checkout.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-features.yml"> <violation number="1" location=".github/workflows/sandbox-features.yml:105"> <priority>P2</priority> <title>Missing persist-credentials on collect job checkout in sandbox-features workflow</title> <evidence>The collect job uses actions/checkout@v4 with fetch-depth: 0 but does not set persist-credentials: false. This leaves the repository token persisted in the git config for the remainder of the job, expanding the blast radius if any step is compromised.</evidence> <recommendation>Add persist-credentials: false to the collect job checkout step to match the bench job hardening.</recommendation> </violation> </file>

superagent-security · 2026-06-25T21:45:51Z

+jobs:
+  bench:
+    name: Reliability ${{ matrix.provider }}
+    runs-on: namespace-profile-default


P0: Self-hosted runner reachable by fork PRs in sandbox-reliability workflow

Self-hosted runner used with pull_request trigger exposes runner to fork PRs.

Verify runner scoping or use GitHub-hosted runners for PR jobs.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:34"> <priority>P0</priority> <title>Self-hosted runner reachable by fork PRs in sandbox-reliability workflow</title> <evidence>The workflow triggers on pull_request and runs on namespace-profile-default, a self-hosted runner. Fork PR authors can execute arbitrary code on this runner, which shares state across jobs and has been the root cause of critical supply-chain compromises.</evidence> <recommendation>Restrict the self-hosted runner to non-fork PRs by verifying scoping in repository Settings > Actions > Runners, or switch PR jobs to GitHub-hosted runners (e.g., ubuntu-latest). If the runner must be used, add an explicit if: github.event.pull_request.head.repo.full_name == github.repository guard or move the PR job to a separate workflow that uses GitHub-hosted runners.</recommendation> </violation> </file>

superagent-security · 2026-06-25T21:45:51Z

+          - upstash
+          - vercel
+    steps:
+      - uses: actions/checkout@v4


P1: Actions pinned to mutable tags instead of commit SHAs in sandbox-reliability workflow

Workflow actions use mutable version tags instead of immutable commit SHAs.

Pin all actions to commit SHA plus version comment.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:61"> <priority>P1</priority> <title>Actions pinned to mutable tags instead of commit SHAs in sandbox-reliability workflow</title> <evidence>actions/checkout@v4, actions/setup-node@v4, actions/upload-artifact@v4, actions/download-artifact@v4, and actions/github-script@v7 are all pinned to mutable tags. A compromised action repository can force-push a tag to inject malicious code into the workflow.</evidence> <recommendation>Pin every third-party and official action to a full 40-character commit SHA and add a version comment, e.g., uses: actions/checkout@<sha> # v4.1.1.</recommendation> </violation> </file>

superagent-security · 2026-06-25T21:45:51Z

+    needs: bench
+    if: always()
+    permissions:
+      contents: write


P2: Missing persist-credentials on collect job checkout in sandbox-reliability workflow

Collect job checkout persists credentials unnecessarily.

Add persist-credentials: false to collect job checkout.

AI prompt

Check if this security scanner issue is valid. If so, understand the root cause and fix it. If appropriate, update or add tests. Keep the change focused and preserve intended behavior. <file name=".github/workflows/sandbox-reliability.yml"> <violation number="1" location=".github/workflows/sandbox-reliability.yml:133"> <priority>P2</priority> <title>Missing persist-credentials on collect job checkout in sandbox-reliability workflow</title> <evidence>The collect job uses actions/checkout@v4 with fetch-depth: 0 but does not set persist-credentials: false. This leaves the repository token persisted in the git config for the remainder of the job, expanding the blast radius if any step is compromised.</evidence> <recommendation>Add persist-credentials: false to the collect job checkout step to match the bench job hardening.</recommendation> </violation> </file>

feat: add sandbox reliability benchmark

2e68bfc

This comment was marked as outdated.

Sign in to view

superagent-security Bot reviewed Jun 25, 2026

View reviewed changes

superagent-security Bot added the pr:flagged PR flagged for review by security analysis. label Jun 25, 2026

This comment was marked as outdated.

Sign in to view

ci: run sandbox reliability smoke on pull requests

986e96a

superagent-security Bot reviewed Jun 25, 2026

View reviewed changes

ci: post sandbox reliability results on PRs

4ffb285

superagent-security Bot reviewed Jun 25, 2026

View reviewed changes

fix: encode reliability perf probe command

8f884c5

superagent-security Bot reviewed Jun 25, 2026

View reviewed changes

ci: harden sandbox reliability workflow

2030a5f

refactor: split sandbox reliability and feature probes

5580015

HeyGarrison requested a review from Copilot June 25, 2026 21:31

HeyGarrison changed the title ~~Add sandbox reliability benchmark~~ Add sandbox reliability and feature matrix benchmark Jun 25, 2026

Copilot started reviewing on behalf of HeyGarrison June 25, 2026 21:32

View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

fix: address sandbox workflow review feedback

01c5a1f

superagent-security Bot reviewed Jun 25, 2026

View reviewed changes

HeyGarrison closed this Jun 29, 2026

		const deduped = Array.from(seen.values()).map(e => e.result);
		console.log(`\nMerging ${deduped.length} provider results for mode: ${resultsSubdir}`);

Conversation

HeyGarrison commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Uh oh!

open-cla Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contributor License Agreement

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sandbox Reliability Results

Uh oh!

github-actions Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sandbox Feature Matrix Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

superagent-security Bot left a comment

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

superagent-security Bot Jun 25, 2026

Choose a reason for hiding this comment

HeyGarrison commented Jun 25, 2026 •

edited

Loading

open-cla Bot commented Jun 25, 2026 •

edited

Loading

github-actions Bot commented Jun 25, 2026 •

edited

Loading

github-actions Bot commented Jun 25, 2026 •

edited

Loading