Skip to content

Set up TestDriver with sample tests for the ports CLI - #4

Closed
testdriverai[bot] wants to merge 1 commit into
mainfrom
testdriver/set-up-testdriver-with-sample-tests-for-the-port
Closed

testdriverai[bot] wants to merge 1 commit into
mainfrom
testdriver/set-up-testdriver-with-sample-tests-for-the-port

Conversation

@testdriverai

@testdriverai testdriverai Bot commented Aug 4, 2026 •

Copy link
Copy Markdown

Closes #3.

Sets up TestDriver in the repo and adds short sample tests for Kiri (the ports CLI).

What this app is

Kiri is a Rust-based command-line tool (ports) for inspecting and managing local development ports — there's no web or GUI surface and no hosted "production environment" with credentials. So instead of a URL, these tests target the real production artifact: the published npm release @gaossr/kiri@latest.

Each test provisions a fresh Linux sandbox, installs the released binary, runs a real ports command, and asserts on its actual output (ANSI colors are stripped before matching). No fixtures or credentials were needed — the sandbox's own live processes/ports provide real data for the CLI to report on.

Sample tests (tests/kiri-cli.test.mjs)

  1. --help usage — installs the published CLI and verifies the usage banner (Kiri - inspect local listening ports, Usage:, ports --all, ports kill).
  2. ports overview — verifies the "Kiri is watching …" banner and the table headers (Port, Process, PID, Framework, Status).
  3. ports ps — verifies the developer-process view (Kiri - running processes, and the PID / Process / What columns).

Result — 3/3 passing

📊 Full test run report

installs the published CLI and prints usage via --help — ✅ passed
Test Recording

shows the ports overview with the Kiri banner and table headers — ✅ passed
Test Recording

lists developer background processes via ports ps — ✅ passed
Test Recording

Running in CI

The included .github/workflows/testdriver.yml runs these tests. The recommended way to authenticate is the published testdriverai/action with GitHub OIDC — it mints a short-lived token and exchanges it for your team's key at run time, so there's no TD_API_KEY secret to store or rotate (one-time: authorize the TestDriver GitHub App for the org at https://console.testdriver.ai). A stored TD_API_KEY repo secret is only needed as a fallback on runners without OIDC.

Run locally with npx vitest run tests/kiri-cli.test.mjs.

Summary by Sourcery

Add a TestDriver-powered Vitest suite and CI workflow to validate the published Kiri ports CLI via end-to-end CLI tests.

New Features:

  • Introduce end-to-end Vitest tests that install and exercise the published @gaossr/kiri CLI, covering help output, overview, and process listing commands.

Build:

  • Add a Node-based project configuration with scripts and dependencies needed to run Vitest and TestDriver tests against the Kiri CLI.

CI:

  • Add a GitHub Actions workflow that runs the Vitest TestDriver suite on pushes and pull requests using GitHub OIDC authentication for TestDriver.

Tests:

  • Configure Vitest with TestDriver integration and reporting, including timeouts and setup hooks for running the CLI tests against provisioned sandboxes.

@sourcery-ai

sourcery-ai Bot commented Aug 4, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Sets up a Vitest-based TestDriver.ai test harness and CI workflow to exercise the published Kiri ports CLI binary end-to-end in a Linux sandbox, with three sample CLI-output assertions and supporting Node/Vitest configuration.

Sequence diagram for GitHub Actions CI run with TestDriver.ai and Kiri ports CLI

sequenceDiagram
  actor Dev as Developer
  participant GH as GitHubActions
  participant TDAction as testdriverai_action_test
  participant Vitest as npx_vitest_run
  participant TDSDK as TestDriver_vitest_reporter
  participant Kiri as ports_CLI

  Dev->>GH: push_or_pull_request
  GH->>TDAction: Authenticate_to_TestDriver(channel, api-key)
  TDAction-->>GH: export_TD_API_KEY
  GH->>Vitest: run_tests
  Vitest->>TDSDK: initialize_TestDriver()
  Vitest->>Kiri: install_@gaossr_kiri_latest
  Vitest->>Kiri: run_ports_--help
  Vitest->>Kiri: run_ports
  Vitest->>Kiri: run_ports_ps
  TDSDK-->>GH: upload_TestDriver_results
Loading

File-Level Changes

Change Details Files
Add Vitest sample tests that exercise the published Kiri ports CLI via TestDriver.ai sandboxes.
  • Initialize Vitest test suite using TestDriver hooks to provision a fresh Linux Chrome sandbox per test.
  • Define shell script to install @gaossr/kiri globally in a user-writable npm prefix and report its exit code.
  • Prefix CLI invocations with a PATH setup so the installed ports binary is discoverable.
  • Normalize CLI output by stripping ANSI escape sequences before text assertions.
  • Add three tests covering ports --help, ports overview table, and ports ps process listing with specific banner/header expectations.
tests/kiri-cli.test.mjs
Configure GitHub Actions CI job to run TestDriver.ai-driven Vitest tests with GitHub OIDC authentication.
  • Create workflow that runs on pushes, pull requests, and manual dispatch on main/master.
  • Set up Node.js 20 with npm caching and install dependencies via npm ci.
  • Authenticate to TestDriver using the testdriverai/action with OIDC, optionally falling back to TD_API_KEY secret.
  • Run Vitest test suite and upload test results artifact for later inspection.
.github/workflows/testdriver.yml
Introduce Node/Vitest project configuration required for TestDriver.ai integration.
  • Add package.json defining a minimal workspace with vitest and testdriverai as devDependencies, and dotenv as a runtime dependency.
  • Define test scripts for running Vitest in batch, watch, and UI modes.
  • Constrain Node engine to >=20.19.0 to match workflow setup.
  • Add vitest.config.js that wires in TestDriver reporter, setup file, and extended test/hook timeouts tailored to TestDriver runs.
  • Generate package-lock.json to lock dependency versions for reproducible CI runs.
package.json
vitest.config.js
package-lock.json

Assessment against linked issues

Issue Objective Addressed Explanation
https://github.com/GaoSSR/Kiri/issues/3 Set up TestDriver in the repo, including any required dependencies, configuration, and CI integration so tests can be run automatically. ✅
https://github.com/GaoSSR/Kiri/issues/3 Create short sample tests for the app represented by the repo, targeting the production environment/artifact and using example credentials or fixtures when applicable. ✅

Possibly linked issues

  • #3: The PR fully implements TestDriver setup, sample Kiri CLI tests, and CI workflow requested by the issue.

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The INSTALL script uses Bash-specific features like PIPESTATUS but runs under sh, which may not be Bash in all environments; consider explicitly invoking bash -lc or rewriting the script to be POSIX-compliant to avoid subtle runtime issues.
  • The setup steps (provision.chrome, install via INSTALL, PATH export) are repeated verbatim across all tests; factoring this into a shared helper or beforeAll/beforeEach hook would simplify maintenance and reduce the risk of the flows diverging over time.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The INSTALL script uses Bash-specific features like PIPESTATUS but runs under `sh`, which may not be Bash in all environments; consider explicitly invoking `bash -lc` or rewriting the script to be POSIX-compliant to avoid subtle runtime issues.
- The setup steps (`provision.chrome`, install via INSTALL, PATH export) are repeated verbatim across all tests; factoring this into a shared helper or `beforeAll`/`beforeEach` hook would simplify maintenance and reduce the risk of the flows diverging over time.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@GaoSSR GaoSSR closed this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant