diff --git a/README.md b/README.md index 18ec0b1..40fb565 100644 --- a/README.md +++ b/README.md @@ -26,14 +26,21 @@ rewrites the timeline to keep only the good segments. project.tscproj while a document is open, so the edit cycle is `close (saves) → edit JSON → open (reloads)`. Throws on other platforms. - **@camkit/cli** — the `camkit` binary: info, clips, sources, rebuild, - export-audio, captions, silences, transcribe, status, close, open, docs. + export-audio, export-video, captions, silences, transcribe, status, close, + open, docs. Rebuild always backs up to `.bak` and refuses to run with a `~project.tscproj` lock or an existing backup unless `--force`. Always `--dry-run` first. `export-audio` flat-mixes the timeline's audio to one file (m4a/wav/flac/…) for cleanup in Audacity/Auphonic — pure ffmpeg, honours track mute and per-clip gain (`--raw` to bypass). `captions` injects an animated Dynamic Caption track straight into the project from a transcript (same - backup/lock safety as rebuild). + backup/lock safety as rebuild). `export-video` renders the current timeline to + a ProRes 422 `.mov` by driving Camtasia's GUI export (Camtasia stays the + renderer — effects, transitions, Dynamic Captions and `.trec` video streams + need its engine, no ffmpeg re-encode). macOS-only; needs Accessibility + permission. **Brittle:** it drives Camtasia's export menus via UI scripting, + verified only on Camtasia 2026.1.3 — menu renames or a Welcome window can + break it. Prefer a short smoke export after Camtasia updates. - **@camkit/mcp** — placeholder; will wrap core later. ## Prerequisites @@ -42,57 +49,24 @@ rewrites the timeline to keep only the good segments. - **ffmpeg** on PATH — required by `camkit export-audio`, `silences`, and `transcribe` (`brew install ffmpeg`) - **A transcription engine** — required by `camkit transcribe` only; see - *Transcription engines* below. `OPENAI_API_KEY` (cloud), + [Transcription engines](docs/transcription.md). `OPENAI_API_KEY` (cloud), `REPLICATE_API_TOKEN` (hosted), or `whisper-cpp` (local, `brew install whisper-cpp`). -- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` only; - everything else is cross-platform - -## Transcription engines - -`camkit transcribe` resolves an engine by precedence (highest wins): an -explicit `--engine openai|replicate|whisper-cpp` flag, then environment, then -the `auto` default. `auto` picks: - -1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is - pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models - don't return the word-level timestamps the rebuild step needs. -2. **Else `REPLICATE_API_TOKEN` set → Replicate** hosted - `vaibhavs10/incredibly-fast-whisper` (word-level timestamps). Beats local - whisper-cpp whenever the token is present — use `--engine whisper-cpp` to - force local. Model version is pinned in code; refresh via Replicate's - `/v1/models/.../versions` API if predictions start failing. -3. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the - `ggml` model Camtasia downloads to - `Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized — - fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a - larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`. -4. **None → an error** telling you to set `OPENAI_API_KEY` or - `REPLICATE_API_TOKEN`, or run `brew install whisper-cpp`. camkit never - auto-installs (no silent `brew`). - -camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib` -(private, code-signed, undocumented ABI) — you bring your own `whisper-cli` -runner. The tiny local model has coarser word timestamps, so cross-checking -with `camkit silences` matters even more on the local path. - -## Captions in the Camtasia UI - -To get higher-quality captions than Camtasia's built-in tiny model: transcribe -with camkit (OpenAI or a larger local model), then either bring the result into -Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to -inject an animated **Dynamic Caption** track straight into the project. Do -**not** swap Camtasia's bundled model file — it's redownloaded on update and -unsupported. - -`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"` -writes the word-level stream onto the source and adds a styled caption track via -the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The -style comes from a Camtasia Dynamic Caption preset, resolved on demand from -Camtasia's app-support dir — list them with `camkit captions --list-presets`, -including any custom presets you've saved. Classic (non-animated) captions -aren't supported; they can't do the per-word highlight, and you can promote a -Dynamic track's styling further in Camtasia's UI. +- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` and + `export-video`; everything else is cross-platform +- **Accessibility permission** — required by `camkit export-video` only. It + drives Camtasia's export GUI via System Events, so grant your terminal under + System Settings ▸ Privacy & Security ▸ Accessibility. + +## Docs + +- [Transcription engines](docs/transcription.md) — engine precedence (OpenAI, + Replicate, local whisper.cpp), model overrides. +- [Captions in the Camtasia UI](docs/captions.md) — higher-quality captions, + Dynamic Caption presets, SRT import. +- [Why `export-video` drives the GUI](docs/export-video.md) — why rendering + can't be done from JSON, the export approaches tried, and why UI scripting is + the only working path. ## Use diff --git a/docs/captions.md b/docs/captions.md new file mode 100644 index 0000000..db490c7 --- /dev/null +++ b/docs/captions.md @@ -0,0 +1,17 @@ +# Captions in the Camtasia UI + +To get higher-quality captions than Camtasia's built-in tiny model: transcribe +with camkit (OpenAI or a larger local model), then either bring the result into +Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to +inject an animated **Dynamic Caption** track straight into the project. Do +**not** swap Camtasia's bundled model file — it's redownloaded on update and +unsupported. + +`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"` +writes the word-level stream onto the source and adds a styled caption track via +the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The +style comes from a Camtasia Dynamic Caption preset, resolved on demand from +Camtasia's app-support dir — list them with `camkit captions --list-presets`, +including any custom presets you've saved. Classic (non-animated) captions +aren't supported; they can't do the per-word highlight, and you can promote a +Dynamic track's styling further in Camtasia's UI. diff --git a/docs/export-video.md b/docs/export-video.md new file mode 100644 index 0000000..ce1fd9a --- /dev/null +++ b/docs/export-video.md @@ -0,0 +1,58 @@ +# Why `export-video` drives the GUI + +**This command is brittle.** It does not call a stable export API — it clicks +Camtasia's Export menus through macOS Accessibility (System Events). Verified +only on **Camtasia 2026.1.3**. After a Camtasia update, or if the Welcome window +steals focus, menu labels change, or Accessibility is missing, expect failures +that look like "Export dialog did not appear" rather than clean errors. Smoke-test +a short project after upgrades; do not treat this as a headless CI-grade pipeline. + +Every other camkit command works on `project.tscproj` directly — that file +*describes* the timeline, and a description is just data we can read and +rewrite. **Rendering is different**: turning that description into a `.mov` +means compositing every track, effect, transition and Dynamic Caption, +decoding the `.trec` `tscc2` screen-recording stream (which ffmpeg can't), and +encoding ProRes. Only Camtasia's engine does that. No file you can write makes +the rendered video appear — something has to run the renderer. So `export-video` +has to invoke Camtasia, not edit JSON. + +That left the question of *how* to invoke it. Options tried, in order of +preference, verified against **Camtasia 2026.1.3**: + +1. **AppleScript `export` verb — the right way. Dead.** `sdef + /Applications/Camtasia.app` shows `project` only `responds-to "export"` + (`exportToFile:`) — there is no `` block, so the verb + declares **no parameters** (no file, no codec, no preset). Tested live: + `export (project 1 …) file "…"` → syntax error (destination has no grammar + to attach to); `tell (project 1 …) export file "…"` → `doesn't understand + "export" (-1708)`; bare `export (project 1 …)` → dispatches then dies `Can't + continue export. (-1708)`. It routes to `exportToFile:` but there's no way to + supply the file argument, and it won't run without one. Consistent with the + already-documented broken suite (`-10000` on media elements). Re-check this + each Camtasia release — if TechSmith fixes the suite, it should replace UI + scripting. + +2. **A headless render/CLI binary in the app bundle.** Ruled out: + `Camtasia.app/Contents/MacOS` ships only `Camtasia` and `CamtasiaSupport.app` + — no standalone exporter. The render code is internal dylibs + (`libCSRenderLib.dylib`, `libCSEncodeLib.dylib`) driven by GUI-only Obj-C + controllers (`ExportMenuControllerProtocol`, …). No documented command-line + entry point. + +3. **Re-encode the `.tscproj` ourselves with ffmpeg.** Rejected by design — see + above: `.trec` `tscc2` is undecodable and Camtasia's effects/transitions + aren't reproducible. camkit's principle is "Camtasia stays the final + renderer." + +4. **UI scripting via macOS Accessibility (what shipped).** Drive the real + on-screen export UI through `osascript` → `System Events`: Export ▸ Local + File → File format *QuickTime Movie* → Options → Compression Type *Apple + ProRes 422* → set name/folder → Export. This is the **only** path that + actually produces a file today. + +Consequences of (4): **not headless** and **version-fragile.** It needs +Camtasia running and frontmost on a logged-in GUI session (no `ssh`/daemon), +the controlling terminal must have **Accessibility** permission (System +Settings ▸ Privacy & Security ▸ Accessibility), and the render blocks the GUI +while it runs. Element paths and menu labels are verified only on 2026.1.3 — +re-probe after each Camtasia release. diff --git a/docs/transcription.md b/docs/transcription.md new file mode 100644 index 0000000..79d35a3 --- /dev/null +++ b/docs/transcription.md @@ -0,0 +1,27 @@ +# Transcription engines + +`camkit transcribe` resolves an engine by precedence (highest wins): an +explicit `--engine openai|replicate|whisper-cpp` flag, then environment, then +the `auto` default. `auto` picks: + +1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is + pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models + don't return the word-level timestamps the rebuild step needs. +2. **Else `REPLICATE_API_TOKEN` set → Replicate** hosted + `vaibhavs10/incredibly-fast-whisper` (word-level timestamps). Beats local + whisper-cpp whenever the token is present — use `--engine whisper-cpp` to + force local. Model version is pinned in code; refresh via Replicate's + `/v1/models/.../versions` API if predictions start failing. +3. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the + `ggml` model Camtasia downloads to + `Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized — + fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a + larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`. +4. **None → an error** telling you to set `OPENAI_API_KEY` or + `REPLICATE_API_TOKEN`, or run `brew install whisper-cpp`. camkit never + auto-installs (no silent `brew`). + +camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib` +(private, code-signed, undocumented ABI) — you bring your own `whisper-cli` +runner. The tiny local model has coarser word timestamps, so cross-checking +with `camkit silences` matters even more on the local path. diff --git a/packages/cli/src/camkit.ts b/packages/cli/src/camkit.ts index 3396ded..2b78df0 100755 --- a/packages/cli/src/camkit.ts +++ b/packages/cli/src/camkit.ts @@ -10,7 +10,7 @@ * up to .bak and refuses to run with a ~project.tscproj lock or an existing * backup unless --force. */ -import { copyFileSync, existsSync, readFileSync, writeFileSync } from "node:fs"; +import { copyFileSync, existsSync, readFileSync, statSync, writeFileSync } from "node:fs"; import { dirname, join, resolve } from "node:path"; import { applyRebuild, @@ -34,7 +34,7 @@ import { wordsInRange, type KeepSeg, } from "@camkit/core"; -import { camtasiaDocPaths, closeProject, openProject, projectStatus } from "@camkit/darwin"; +import { camtasiaDocPaths, closeProject, exportVideo, openProject, projectStatus } from "@camkit/darwin"; import { exportAudio, runSilencedetect, transcribeRecording } from "./media.ts"; import { listPresets, resolvePreset } from "./presets.ts"; import { version } from "../package.json"; @@ -127,6 +127,28 @@ const HELP: Record = { " --raw ignore mute/solo/gain — dry unity sum of every clip", ], }, + "export-video": { + usage: "camkit export-video [--project PATH] [--out FILE] [--codec prores422]", + about: [ + "Render an open project's timeline to a ProRes 422 .mov by driving", + "Camtasia's GUI export (Export ▸ Local File ▸ QuickTime ▸ Apple ProRes", + "422). Camtasia stays the renderer — effects, transitions and Dynamic", + "Captions need its engine, and .trec screen recordings carry a video", + "stream ffmpeg can't decode. Run after rebuild to render the edited cut.", + "", + "macOS-only and needs Accessibility permission for your terminal (System", + "Settings ▸ Privacy & Security ▸ Accessibility). The project must already", + "be open in Camtasia. With several open, pass --project to pick which", + "window to raise (export always drives the UI of that document). Verified", + "on Camtasia 2026.1.3 only — menu renames, locale, or the Welcome window", + "can break it (brittle by design until TechSmith ships a real export", + "verb). Blocks until the output file appears and its size stops growing.", + "", + " --project PATH project that must be open (required if several are open)", + " --out FILE output path (default ./.mov)", + " --codec NAME only prores422 for now (default)", + ], + }, captions: { usage: "camkit captions [--project PATH] --from FILE.transcript.json (--preset NAME | --preset-file PATH) [--list-presets] [--src ID] [--strip-punctuation] [--strip-apostrophes] [--dry-run] [--force]", @@ -244,6 +266,7 @@ function printHelp(cmd?: string): void { sources: "list media-bin sources, placed or not", rebuild: "rewrite timeline to kept segments (rough cut)", "export-audio": "mix the timeline's audio to one file (m4a/wav/…)", + "export-video": "render the timeline to a ProRes 422 .mov via Camtasia", captions: "inject an animated Dynamic Caption track from a transcript", silences: "ffmpeg silencedetect on a recording", transcribe: "word-level Whisper transcript of a recording", @@ -371,6 +394,88 @@ async function cmdExportAudio(argv: string[]) { await exportAudio({ segs, projectPath: path, out, durationSeconds }); } +/** Pick which open Camtasia document export-video should drive. */ +function resolveOpenExportDoc(argv: string[]): { path: string; name: string } { + const open = camtasiaDocPaths(); + if (!open.length) { + throw new Error( + "No project open in Camtasia. Open it first (camkit open) — export drives the open timeline.", + ); + } + const explicit = flag(argv, "--project"); + if (explicit) { + const tscproj = resolveProjectPath(explicit); + const name = bundleName(tscproj); + const bundleDir = resolve(dirname(tscproj)); + const match = open.find( + (d) => d.name === name || resolve(d.path) === bundleDir || resolve(d.path) === resolve(tscproj), + ); + if (!match) { + throw new Error( + `${name} is not open in Camtasia (open: ${open.map((d) => d.name).join(", ")}). ` + + "Open it first — export cannot target a closed project.", + ); + } + return { path: match.path, name: match.name }; + } + if (open.length === 1) return { path: open[0].path, name: open[0].name }; + throw new Error( + `Several projects are open in Camtasia; pass --project to pick one:\n` + + open.map((d) => ` --project "${d.path}"`).join("\n"), + ); +} + +/** Wait until `path` exists and its size is stable (export finished writing). */ +async function waitForExportFile( + path: string, + opts: { appearMs?: number; stableMs?: number; overallMs?: number } = {}, +): Promise { + const appearMs = opts.appearMs ?? 3 * 60 * 1000; + const stableMs = opts.stableMs ?? 2500; + const overallMs = opts.overallMs ?? 2 * 60 * 60 * 1000; + const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); + + const appearDeadline = Date.now() + appearMs; + while (!existsSync(path)) { + if (Date.now() >= appearDeadline) { + throw new Error( + `Export file did not appear within ${appearMs / 1000}s: ${path}. ` + + "Camtasia may still be showing a dialog, or the UI path failed.", + ); + } + await sleep(500); + } + + const overallDeadline = Date.now() + overallMs; + let lastSize = -1; + let stableSince = Date.now(); + while (true) { + if (Date.now() >= overallDeadline) { + throw new Error(`Export still writing after ${overallMs / 1000}s: ${path}`); + } + const size = statSync(path).size; + if (size > 0 && size === lastSize) { + if (Date.now() - stableSince >= stableMs) return; + } else { + lastSize = size; + stableSince = Date.now(); + } + await sleep(500); + } +} + +async function cmdExportVideo(argv: string[]) { + const doc = resolveOpenExportDoc(argv); + const base = doc.name.replace(/\.(cmproj|tscproj)$/, ""); + const out = flag(argv, "--out") ? resolve(flag(argv, "--out")!) : resolve(`${base}.mov`); + const codec = flag(argv, "--codec") ?? "prores422"; + console.log(`Starting Camtasia export of ${doc.name} → ${out} (${codec})…`); + exportVideo({ out, codec, documentName: doc.name }); + console.log(`→ export started; waiting for ${out} to finish writing…`); + await waitForExportFile(out); + console.log(`✓ exported ${out}`); +} + function cmdCaptions(argv: string[]) { if (has(argv, "--list-presets")) { const presets = listPresets(); @@ -561,6 +666,7 @@ const COMMANDS: Record void | Promise> = { sources: cmdSources, rebuild: cmdRebuild, "export-audio": cmdExportAudio, + "export-video": cmdExportVideo, captions: cmdCaptions, silences: cmdSilences, transcribe: cmdTranscribe, diff --git a/packages/darwin/src/index.ts b/packages/darwin/src/index.ts index f811e56..c31fc50 100644 --- a/packages/darwin/src/index.ts +++ b/packages/darwin/src/index.ts @@ -19,6 +19,11 @@ function assertDarwin(): void { } } +/** Escape a string for embedding in a double-quoted AppleScript literal. */ +function asLiteral(s: string): string { + return s.replace(/\\/g, "\\\\").replace(/"/g, '\\"'); +} + function osascript(script: string): string { assertDarwin(); const r = spawnSync("osascript", ["-e", script], { encoding: "utf8" }); @@ -103,8 +108,143 @@ export function closeProject(projectPath: string): CloseResult { /** Reopen a project bundle in Camtasia (e.g. after a rebuild). */ export function openProject(projectPath: string): void { const bundle = dirname(projectPath); + const b = asLiteral(bundle); osascript(`tell application "Camtasia" activate - open POSIX file "${bundle}" + open POSIX file "${b}" end tell`); } + +/** Map a camkit --codec value to Camtasia's "Compression Type" menu label. + * Only prores422 is required for v1; add rows here for prores4444/h264. */ +const CODEC_LABELS: Record = { + prores422: "Apple ProRes 422", +}; + +/** + * Render a Camtasia document's timeline to a .mov via UI scripting. + * + * The AppleScript suite has NO working export verb on 2026.1.3 (export is a + * bare exportToFile: stub with no file/codec parameters — see issue #10 spike), + * so this drives the GUI: Export ▸ Local File… → File format "QuickTime Movie" + * → Options… → Compression Type "Apple ProRes 422" → set name+folder → Export. + * Verified against Camtasia 2026.1.3. Brittle across versions and needs + * Accessibility permission for the controlling terminal (System Events). + * + * Only kicks off the export UI — does not wait for the render to finish. + * Callers should poll for `opts.out` if they need a completion signal. + * + * ponytail: UI scripting is the only path Camtasia exposes; no abstraction + * layer until a second codec/version actually needs different element paths. + */ +export function exportVideo(opts: { + out: string; + codec?: string; + /** Open document / .cmproj bundle name to raise before exporting (e.g. "foo.cmproj"). */ + documentName?: string; +}): void { + assertDarwin(); + const codec = opts.codec ?? "prores422"; + const label = CODEC_LABELS[codec]; + if (!label) { + throw new Error(`Unsupported --codec "${codec}". Supported: ${Object.keys(CODEC_LABELS).join(", ")}.`); + } + if (camtasiaDocs().length === 0) { + throw new Error("No project open in Camtasia. Open it first (camkit open) — export renders the front timeline."); + } + const dir = asLiteral(dirname(opts.out)); + const base = asLiteral(opts.out.split("/").pop() ?? opts.out); + const labelLit = asLiteral(label); + // Window titles usually include the project title without .cmproj; match both. + const raiseNeedle = asLiteral( + (opts.documentName ?? "").replace(/\.cmproj$/i, "") || (opts.documentName ?? ""), + ); + + const raiseBlock = + raiseNeedle.length > 0 + ? ` + -- raise the target editor window (Welcome steals focus and disables Export) + try + perform action "AXRaise" of (first window whose name contains "${raiseNeedle}") + end try + set frontmost to true + delay 0.4 +` + : ` + set frontmost to true + delay 0.4 +`; + + osascript(` +tell application "Camtasia" to activate +delay 0.5 +tell application "System Events" + if not (UI elements enabled) then error "Accessibility permission required — grant your terminal under System Settings ▸ Privacy & Security ▸ Accessibility." + tell process "Camtasia" +${raiseBlock} + click menu item "Local File..." of menu 1 of menu bar item "Export" of menu bar 1 + delay 1.5 + -- the save sheet attaches to whichever window; find it + set swin to 0 + repeat with i from 1 to (count of windows) + try + if (count of sheets of window i) > 0 then set swin to i + end try + end repeat + if swin = 0 then error "Export dialog did not appear." + set sg to splitter group 1 of sheet 1 of window swin + + -- File format ▸ QuickTime Movie (.mov) + set fp to pop up button 2 of sg + click fp + delay 0.4 + click menu item "Export to QuickTime Movie (.mov)" of menu 1 of fp + delay 0.6 + + -- Options… ▸ Advanced Export Options ▸ Compression Type + click (item 1 of (buttons of sg whose name is "Options...")) + delay 1.2 + set adv to window "Advanced Export Options" + set picked to false + repeat with p in (pop up buttons of group "Video" of adv) + if not picked then + try + click p + delay 0.3 + if (exists menu item "${labelLit}" of menu 1 of p) then + click menu item "${labelLit}" of menu 1 of p + set picked to true + else + key code 53 + end if + end try + end if + end repeat + if not picked then + click button "Cancel" of adv + error "Compression Type \\"${labelLit}\\" not found in Advanced Export Options." + end if + click button "OK" of adv + delay 0.6 + + -- filename + destination folder + set value of text field "Export As:" of sg to "${base}" + delay 0.2 + keystroke "g" using {command down, shift down} + delay 0.5 + keystroke "${dir}" + delay 0.3 + keystroke return + delay 0.6 + + click button "Export" of sg + delay 0.6 + -- overwrite confirmation, if the file already exists + try + if (count of sheets of window swin) > 0 then + click button "Replace" of sheet 1 of window swin + end if + end try + end tell +end tell`); +}