test: live E2E smoke test; docs for alpha testers

.fernignore

@@ -20,3 +20,6 @@ assets/
 # Hand-written: the generated types crate ships without a .gitignore, so running
 # its test suite leaves untracked build artifacts.
 elevenlabs-types/.gitignore
+
+# Hand-written live E2E smoke test (tests/wire_test.rs next to it is generated).
+tests/e2e_smoke.rs

README.md

@@ -24,6 +24,7 @@ The CLI does two things:
   - [Environment variables](#environment-variables)
   - [Output formats](#output-formats)
   - [Shell completion](#shell-completion)
+- [Development](#development)
 
 ## Installation
 
@@ -41,16 +42,36 @@ powershell -ExecutionPolicy ByPass -c "irm https://github.com/elevenlabs/cli/rel
 
 ### Build from source
 
-If you prefer to build from source, install the [Rust toolchain](https://rustup.rs/) and run:
+Install the [Rust toolchain](https://rustup.rs/), then:
 
 ```bash
+git clone https://github.com/elevenlabs/cli.git
+cd cli
 cargo build --release
-./target/release/elevenlabs --help
+```
+
+The binary lands at `./target/release/elevenlabs`. To run it as `elevenlabs`
+from anywhere, install it with cargo, which copies it into `~/.cargo/bin`:
+
+```bash
+cargo install --path .
 ```
 
 ## Authentication
 
-This API requires authentication. Run `elevenlabs --help` for details.
+Every request is authenticated with an [ElevenLabs API key](https://elevenlabs.io/app/settings/api-keys), sent as the `xi-api-key` header. Set it in the environment:
+
+```bash
+export ELEVENLABS_API_KEY=xi-...
+```
+
+A `.env` file in the working directory is loaded automatically, so putting `ELEVENLABS_API_KEY=xi-...` there works too — handy for keeping a key scoped to one project.
+
+For a one-off call, pass it per command instead:
+
+```bash
+elevenlabs user get --xi-api-key xi-...
+```
 
 ## Quick start
 
@@ -112,10 +133,23 @@ elevenlabs agents widget embed <agent_id>
 elevenlabs agents test <agent_id>
 ```
 
-These sit alongside the generated API commands in the same group, which own the
-primitives — `elevenlabs agents create | get | list | update | delete`, and the
-`elevenlabs agents branches ...` subgroup for branch management.
+### Two surfaces in one namespace
+
+`elevenlabs agents` holds two kinds of command, and it helps to know which you're using:
 
+| | Workflow commands | API commands |
+|---|---|---|
+| **What** | `init`, `add`, `status`, `push`, `pull`, `test`, `templates`, `widget embed` | `create`, `get`, `list`, `update`, `delete`, `duplicate`, `run_tests`, and subgroups like `branches`, `tools`, `tests`, `conversations` |
+| **Operates on** | Your local project files, syncing them with ElevenLabs | The API directly — one command, one request |
+| **Arguments** | Positional, e.g. `agents test <agent_id>` | Flags, e.g. `agents get --agent-id <id>` |
+| **Output** | Progress text | The API response (`--format json\|table\|yaml\|csv`) |
+
+The API commands own the primitive names, so the workflow only adds verbs the API doesn't have. Two consequences worth knowing:
+
+- **Listing** — `agents list` is the API's list of agents in your workspace. For what's configured *locally*, use `agents status`.
+- **Deleting** — `agents delete --agent-id <id>` deletes remotely (API command). It does **not** remove the local config file or its `agents.json` entry; delete those yourself.
+
+> **Migrating from v0 (`@elevenlabs/cli`):** v0's `agents list` showed local config and `agents delete <id>` took a positional and cleaned up locally. In v1 both names belong to the API surface — use `agents status` for the local view, and `--agent-id` for delete. v0's `agents widget <id>` is now `agents widget embed <id>`, since `agents widget` is an API subgroup.
 
 ### Tools
 
@@ -259,3 +293,37 @@ Generate shell completion scripts:
 elevenlabs completion <bash|zsh|fish|powershell>
 ```
 
+
+## Development
+
+The CLI is generated by [Fern](https://buildwithfern.com) from the ElevenLabs OpenAPI spec, with the agents-as-code workflow layered on top as hand-written commands in `cli/elevenlabs/workflow/`. Those files, along with this README and `.github/workflows/ci.yml`, are listed in `.fernignore` so regeneration can't overwrite them.
+
+### Tests
+
+```bash
+cargo test                                              # framework + workflow + wire tests
+cargo test --manifest-path elevenlabs-sdk/Cargo.toml    # generated SDK crate
+cargo test --manifest-path elevenlabs-types/Cargo.toml  # generated types crate
+```
+
+The generated crates are path dependencies rather than workspace members, so a plain `cargo test` skips them — hence the separate invocations (CI runs all three).
+
+| Suite | What it covers |
+|-------|----------------|
+| `cargo test --test wire_test` | Generated wire tests: each stands up an in-process mock server, drives one endpoint through the CLI, and asserts the request and rendered response. No network. |
+| `cargo test --bin elevenlabs` | Unit tests for the hand-written workflow (config round-tripping, templates, pull planning, residency, …) |
+| `cargo test --test e2e_smoke` | Live end-to-end smoke test — **opt-in**, see below |
+
+### Live end-to-end smoke test
+
+`tests/e2e_smoke.rs` runs the full workflow against a real account: `init` → `add` → `push` → `pull` → `delete`, asserting that a config survives a push/pull round-trip byte-for-byte.
+
+> ⚠️ **It creates and deletes agents. Use a dedicated, empty test account — never a production one.**
+
+It's opt-in and skips unless `ELEVENLABS_E2E_API_KEY` is set. That variable is deliberately *not* `ELEVENLABS_API_KEY`, so a shell with your normal credentials exported can't accidentally mutate a live workspace:
+
+```bash
+ELEVENLABS_E2E_API_KEY=xi-... cargo test --test e2e_smoke -- --nocapture
+```
+
+It cleans up the agent it creates, including on failure; if deletion fails it prints the id to remove by hand.

tests/e2e_smoke.rs

@@ -0,0 +1,202 @@
+//! End-to-end smoke test for the agents-as-code workflow, run against a real
+//! ElevenLabs account.
+//!
+//! # ⚠ DESTRUCTIVE — use a dedicated, empty test account
+//!
+//! This test **creates and deletes agents**. Point it at a throwaway account,
+//! never a production one. It is opt-in for exactly that reason: it does
+//! nothing unless `ELEVENLABS_E2E_API_KEY` is set, so `cargo test` on a
+//! developer machine or in CI skips it by default. Note it deliberately does
+//! *not* read `ELEVENLABS_API_KEY` — otherwise a normal shell with credentials
+//! already exported would start mutating a live workspace.
+//!
+//! ```bash
+//! ELEVENLABS_E2E_API_KEY=xi-... cargo test --test e2e_smoke -- --nocapture
+//! ```
+//!
+//! # What it proves
+//!
+//! The thing unit and wire tests can't: that a config survives a real
+//! round-trip through the API. `push` then `pull` twice and compare — the
+//! second and third snapshots must be byte-identical, which is the guarantee
+//! the whole raw-JSON design exists to provide (see `workflow/api.rs`). It also
+//! exercises init → add → push → pull → delete against live endpoints.
+
+use std::path::Path;
+use std::process::{Command, Output};
+use std::time::{SystemTime, UNIX_EPOCH};
+
+const OPT_IN_VAR: &str = "ELEVENLABS_E2E_API_KEY";
+
+/// Run the CLI in `dir` with the E2E credentials, returning the output.
+///
+/// `HOME` is redirected at the temp dir so the run can't pick up (or write) the
+/// developer's `~/.elevenlabs` residency setting.
+fn cli(dir: &Path, home: &Path, api_key: &str, args: &[&str]) -> Output {
+    Command::new(env!("CARGO_BIN_EXE_elevenlabs"))
+        .args(args)
+        .current_dir(dir)
+        .env("ELEVENLABS_API_KEY", api_key)
+        .env("HOME", home)
+        .env("NO_COLOR", "1")
+        // Never let an ambient override point the test somewhere unexpected.
+        .env_remove("ELEVENLABS_BASE_URL")
+        .output()
+        .expect("failed to spawn the elevenlabs binary")
+}
+
+fn stdout_of(out: &Output) -> String {
+    String::from_utf8_lossy(&out.stdout).to_string()
+}
+
+fn assert_ok(out: &Output, what: &str) {
+    assert!(
+        out.status.success(),
+        "{what} failed (exit {:?})\n--- stdout ---\n{}\n--- stderr ---\n{}",
+        out.status.code(),
+        String::from_utf8_lossy(&out.stdout),
+        String::from_utf8_lossy(&out.stderr),
+    );
+}
+
+/// The agent id recorded in `agents.json` after `agents add`.
+fn agent_id_from_registry(dir: &Path) -> Option<String> {
+    let raw = std::fs::read_to_string(dir.join("agents.json")).ok()?;
+    let json: serde_json::Value = serde_json::from_str(&raw).ok()?;
+    json.get("agents")?
+        .as_array()?
+        .first()?
+        .get("id")?
+        .as_str()
+        .map(String::from)
+}
+
+fn config_path_from_registry(dir: &Path) -> Option<String> {
+    let raw = std::fs::read_to_string(dir.join("agents.json")).ok()?;
+    let json: serde_json::Value = serde_json::from_str(&raw).ok()?;
+    json.get("agents")?
+        .as_array()?
+        .first()?
+        .get("config")?
+        .as_str()
+        .map(String::from)
+}
+
+#[test]
+fn agents_as_code_round_trips_against_a_live_account() {
+    let Ok(api_key) = std::env::var(OPT_IN_VAR) else {
+        eprintln!("skipping: set {OPT_IN_VAR} to run the live E2E smoke test");
+        return;
+    };
+
+    let workdir = tempfile::tempdir().expect("temp workdir");
+    let homedir = tempfile::tempdir().expect("temp home");
+    let dir = workdir.path();
+    let home = homedir.path();
+
+    // Unique name so parallel or repeated runs can't collide.
+    let nonce = SystemTime::now()
+        .duration_since(UNIX_EPOCH)
+        .expect("clock")
+        .as_secs();
+    let agent_name = format!("cli-e2e-{nonce}");
+
+    // 1. Scaffold a project.
+    assert_ok(&cli(dir, home, &api_key, &["agents", "init"]), "agents init");
+    assert!(dir.join("agents.json").exists(), "init wrote agents.json");
+    assert!(dir.join("agent_configs").is_dir(), "init made agent_configs/");
+
+    // 2. Create the agent remotely and register it locally.
+    let add = cli(
+        dir,
+        home,
+        &api_key,
+        &["agents", "add", &agent_name, "--template", "minimal"],
+    );
+
+    // From here on the agent exists remotely, so always clean up.
+    let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+        assert_ok(&add, "agents add");
+
+        let agent_id = agent_id_from_registry(dir)
+            .unwrap_or_else(|| panic!("no agent id in agents.json:\n{}", stdout_of(&add)));
+        let config_path = config_path_from_registry(dir).expect("config path in agents.json");
+        let config_file = dir.join(&config_path);
+        assert!(config_file.exists(), "add wrote {config_path}");
+
+        // 3. Status should now see it as created.
+        let status = cli(dir, home, &api_key, &["agents", "status"]);
+        assert_ok(&status, "agents status");
+        assert!(
+            stdout_of(&status).contains(&agent_id),
+            "status should report the agent id, got:\n{}",
+            stdout_of(&status)
+        );
+
+        // 4. Push the local config, then pull it back — this is the snapshot the
+        //    round-trip is measured from (the server fills defaults on first
+        //    write, so the pre-push file is not a fair comparison point).
+        assert_ok(&cli(dir, home, &api_key, &["agents", "push"]), "agents push");
+        assert_ok(
+            &cli(dir, home, &api_key, &["agents", "pull", "--agent", &agent_id, "--all"]),
+            "agents pull (first)",
+        );
+        let first = std::fs::read_to_string(&config_file).expect("read config after first pull");
+
+        // 5. Push that exact config back and pull again. A lossless round-trip
+        //    means nothing changed.
+        assert_ok(&cli(dir, home, &api_key, &["agents", "push"]), "agents push (second)");
+        assert_ok(
+            &cli(dir, home, &api_key, &["agents", "pull", "--agent", &agent_id, "--all"]),
+            "agents pull (second)",
+        );
+        let second = std::fs::read_to_string(&config_file).expect("read config after second pull");
+
+        assert_eq!(
+            first, second,
+            "config changed across a push/pull round-trip — the raw-JSON path is dropping or \
+             rewriting fields.\n--- first ---\n{first}\n--- second ---\n{second}"
+        );
+
+        // 6. The widget snippet should reference the live agent.
+        let widget = cli(
+            dir,
+            home,
+            &api_key,
+            &["agents", "widget", "embed", &agent_id],
+        );
+        assert_ok(&widget, "agents widget embed");
+        assert!(
+            stdout_of(&widget).contains(&agent_id),
+            "widget snippet should embed the agent id"
+        );
+
+        agent_id
+    }));
+
+    // 7. Always remove the agent we created, then surface any failure.
+    //    Deletion is a generated API command, so the path parameter is a flag
+    //    (`--agent-id`) rather than a positional.
+    if let Some(agent_id) = agent_id_from_registry(dir) {
+        let cleanup = cli(dir, home, &api_key, &["agents", "delete", "--agent-id", &agent_id]);
+        if !cleanup.status.success() {
+            eprintln!(
+                "WARNING: could not delete E2E agent {agent_id} — remove it by hand.\n{}",
+                String::from_utf8_lossy(&cleanup.stderr)
+            );
+        }
+    }
+
+    if let Err(panic) = outcome {
+        std::panic::resume_unwind(panic);
+    }
+}
+
+/// Guards against the opt-in being wired to the wrong variable — if this ever
+/// starts reading `ELEVENLABS_API_KEY`, a normal developer shell would begin
+/// mutating a live workspace on `cargo test`.
+#[test]
+fn the_live_test_is_opt_in_via_a_dedicated_variable() {
+    assert_eq!(OPT_IN_VAR, "ELEVENLABS_E2E_API_KEY");
+    assert_ne!(OPT_IN_VAR, "ELEVENLABS_API_KEY");
+}