Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 25 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,21 @@ rewrites the timeline to keep only the good segments.
project.tscproj while a document is open, so the edit cycle is
`close (saves) → edit JSON → open (reloads)`. Throws on other platforms.
- **@camkit/cli** — the `camkit` binary: info, clips, sources, rebuild,
export-audio, captions, silences, transcribe, status, close, open, docs.
export-audio, export-video, captions, silences, transcribe, status, close,
open, docs.
Rebuild always backs up to `.bak` and refuses to run with a
`~project.tscproj` lock or an existing backup unless `--force`. Always
`--dry-run` first. `export-audio` flat-mixes the timeline's audio to one file
(m4a/wav/flac/…) for cleanup in Audacity/Auphonic — pure ffmpeg, honours track
mute and per-clip gain (`--raw` to bypass). `captions` injects an animated
Dynamic Caption track straight into the project from a transcript (same
backup/lock safety as rebuild).
backup/lock safety as rebuild). `export-video` renders the current timeline to
a ProRes 422 `.mov` by driving Camtasia's GUI export (Camtasia stays the
renderer — effects, transitions, Dynamic Captions and `.trec` video streams
need its engine, no ffmpeg re-encode). macOS-only; needs Accessibility
permission. **Brittle:** it drives Camtasia's export menus via UI scripting,
verified only on Camtasia 2026.1.3 — menu renames or a Welcome window can
break it. Prefer a short smoke export after Camtasia updates.
- **@camkit/mcp** — placeholder; will wrap core later.

## Prerequisites
Expand All @@ -42,57 +49,24 @@ rewrites the timeline to keep only the good segments.
- **ffmpeg** on PATH — required by `camkit export-audio`, `silences`, and
`transcribe` (`brew install ffmpeg`)
- **A transcription engine** — required by `camkit transcribe` only; see
*Transcription engines* below. `OPENAI_API_KEY` (cloud),
[Transcription engines](docs/transcription.md). `OPENAI_API_KEY` (cloud),
`REPLICATE_API_TOKEN` (hosted), or `whisper-cpp` (local,
`brew install whisper-cpp`).
- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` only;
everything else is cross-platform

## Transcription engines

`camkit transcribe` resolves an engine by precedence (highest wins): an
explicit `--engine openai|replicate|whisper-cpp` flag, then environment, then
the `auto` default. `auto` picks:

1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is
pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models
don't return the word-level timestamps the rebuild step needs.
2. **Else `REPLICATE_API_TOKEN` set → Replicate** hosted
`vaibhavs10/incredibly-fast-whisper` (word-level timestamps). Beats local
whisper-cpp whenever the token is present — use `--engine whisper-cpp` to
force local. Model version is pinned in code; refresh via Replicate's
`/v1/models/.../versions` API if predictions start failing.
3. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the
`ggml` model Camtasia downloads to
`Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized —
fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a
larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`.
4. **None → an error** telling you to set `OPENAI_API_KEY` or
`REPLICATE_API_TOKEN`, or run `brew install whisper-cpp`. camkit never
auto-installs (no silent `brew`).

camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib`
(private, code-signed, undocumented ABI) — you bring your own `whisper-cli`
runner. The tiny local model has coarser word timestamps, so cross-checking
with `camkit silences` matters even more on the local path.

## Captions in the Camtasia UI

To get higher-quality captions than Camtasia's built-in tiny model: transcribe
with camkit (OpenAI or a larger local model), then either bring the result into
Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to
inject an animated **Dynamic Caption** track straight into the project. Do
**not** swap Camtasia's bundled model file — it's redownloaded on update and
unsupported.

`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"`
writes the word-level stream onto the source and adds a styled caption track via
the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The
style comes from a Camtasia Dynamic Caption preset, resolved on demand from
Camtasia's app-support dir — list them with `camkit captions --list-presets`,
including any custom presets you've saved. Classic (non-animated) captions
aren't supported; they can't do the per-word highlight, and you can promote a
Dynamic track's styling further in Camtasia's UI.
- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` and
`export-video`; everything else is cross-platform
- **Accessibility permission** — required by `camkit export-video` only. It
drives Camtasia's export GUI via System Events, so grant your terminal under
System Settings ▸ Privacy & Security ▸ Accessibility.

## Docs

- [Transcription engines](docs/transcription.md) — engine precedence (OpenAI,
Replicate, local whisper.cpp), model overrides.
- [Captions in the Camtasia UI](docs/captions.md) — higher-quality captions,
Dynamic Caption presets, SRT import.
- [Why `export-video` drives the GUI](docs/export-video.md) — why rendering
can't be done from JSON, the export approaches tried, and why UI scripting is
the only working path.

## Use

Expand Down
17 changes: 17 additions & 0 deletions docs/captions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Captions in the Camtasia UI

To get higher-quality captions than Camtasia's built-in tiny model: transcribe
with camkit (OpenAI or a larger local model), then either bring the result into
Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to
inject an animated **Dynamic Caption** track straight into the project. Do
**not** swap Camtasia's bundled model file — it's redownloaded on update and
unsupported.

`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"`
writes the word-level stream onto the source and adds a styled caption track via
the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The
style comes from a Camtasia Dynamic Caption preset, resolved on demand from
Camtasia's app-support dir — list them with `camkit captions --list-presets`,
including any custom presets you've saved. Classic (non-animated) captions
aren't supported; they can't do the per-word highlight, and you can promote a
Dynamic track's styling further in Camtasia's UI.
58 changes: 58 additions & 0 deletions docs/export-video.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Why `export-video` drives the GUI

**This command is brittle.** It does not call a stable export API — it clicks
Camtasia's Export menus through macOS Accessibility (System Events). Verified
only on **Camtasia 2026.1.3**. After a Camtasia update, or if the Welcome window
steals focus, menu labels change, or Accessibility is missing, expect failures
that look like "Export dialog did not appear" rather than clean errors. Smoke-test
a short project after upgrades; do not treat this as a headless CI-grade pipeline.

Every other camkit command works on `project.tscproj` directly — that file
*describes* the timeline, and a description is just data we can read and
rewrite. **Rendering is different**: turning that description into a `.mov`
means compositing every track, effect, transition and Dynamic Caption,
decoding the `.trec` `tscc2` screen-recording stream (which ffmpeg can't), and
encoding ProRes. Only Camtasia's engine does that. No file you can write makes
the rendered video appear — something has to run the renderer. So `export-video`
has to invoke Camtasia, not edit JSON.

That left the question of *how* to invoke it. Options tried, in order of
preference, verified against **Camtasia 2026.1.3**:

1. **AppleScript `export` verb — the right way. Dead.** `sdef
/Applications/Camtasia.app` shows `project` only `responds-to "export"`
(`exportToFile:`) — there is no `<command name="export">` block, so the verb
declares **no parameters** (no file, no codec, no preset). Tested live:
`export (project 1 …) file "…"` → syntax error (destination has no grammar
to attach to); `tell (project 1 …) export file "…"` → `doesn't understand
"export" (-1708)`; bare `export (project 1 …)` → dispatches then dies `Can't
continue export. (-1708)`. It routes to `exportToFile:` but there's no way to
supply the file argument, and it won't run without one. Consistent with the
already-documented broken suite (`-10000` on media elements). Re-check this
each Camtasia release — if TechSmith fixes the suite, it should replace UI
scripting.

2. **A headless render/CLI binary in the app bundle.** Ruled out:
`Camtasia.app/Contents/MacOS` ships only `Camtasia` and `CamtasiaSupport.app`
— no standalone exporter. The render code is internal dylibs
(`libCSRenderLib.dylib`, `libCSEncodeLib.dylib`) driven by GUI-only Obj-C
controllers (`ExportMenuControllerProtocol`, …). No documented command-line
entry point.

3. **Re-encode the `.tscproj` ourselves with ffmpeg.** Rejected by design — see
above: `.trec` `tscc2` is undecodable and Camtasia's effects/transitions
aren't reproducible. camkit's principle is "Camtasia stays the final
renderer."

4. **UI scripting via macOS Accessibility (what shipped).** Drive the real
on-screen export UI through `osascript` → `System Events`: Export ▸ Local
File → File format *QuickTime Movie* → Options → Compression Type *Apple
ProRes 422* → set name/folder → Export. This is the **only** path that
actually produces a file today.

Consequences of (4): **not headless** and **version-fragile.** It needs
Camtasia running and frontmost on a logged-in GUI session (no `ssh`/daemon),
the controlling terminal must have **Accessibility** permission (System
Settings ▸ Privacy & Security ▸ Accessibility), and the render blocks the GUI
while it runs. Element paths and menu labels are verified only on 2026.1.3 —
re-probe after each Camtasia release.
27 changes: 27 additions & 0 deletions docs/transcription.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Transcription engines

`camkit transcribe` resolves an engine by precedence (highest wins): an
explicit `--engine openai|replicate|whisper-cpp` flag, then environment, then
the `auto` default. `auto` picks:

1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is
pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models
don't return the word-level timestamps the rebuild step needs.
2. **Else `REPLICATE_API_TOKEN` set → Replicate** hosted
`vaibhavs10/incredibly-fast-whisper` (word-level timestamps). Beats local
whisper-cpp whenever the token is present — use `--engine whisper-cpp` to
force local. Model version is pinned in code; refresh via Replicate's
`/v1/models/.../versions` API if predictions start failing.
3. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the
`ggml` model Camtasia downloads to
`Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized —
fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a
larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`.
4. **None → an error** telling you to set `OPENAI_API_KEY` or
`REPLICATE_API_TOKEN`, or run `brew install whisper-cpp`. camkit never
auto-installs (no silent `brew`).

camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib`
(private, code-signed, undocumented ABI) — you bring your own `whisper-cli`
runner. The tiny local model has coarser word timestamps, so cross-checking
with `camkit silences` matters even more on the local path.
110 changes: 108 additions & 2 deletions packages/cli/src/camkit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
* up to .bak and refuses to run with a ~project.tscproj lock or an existing
* backup unless --force.
*/
import { copyFileSync, existsSync, readFileSync, writeFileSync } from "node:fs";
import { copyFileSync, existsSync, readFileSync, statSync, writeFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import {
applyRebuild,
Expand All @@ -34,7 +34,7 @@ import {
wordsInRange,
type KeepSeg,
} from "@camkit/core";
import { camtasiaDocPaths, closeProject, openProject, projectStatus } from "@camkit/darwin";
import { camtasiaDocPaths, closeProject, exportVideo, openProject, projectStatus } from "@camkit/darwin";
import { exportAudio, runSilencedetect, transcribeRecording } from "./media.ts";
import { listPresets, resolvePreset } from "./presets.ts";
import { version } from "../package.json";
Expand Down Expand Up @@ -127,6 +127,28 @@ const HELP: Record<string, { usage: string; about: string[] }> = {
" --raw ignore mute/solo/gain — dry unity sum of every clip",
],
},
"export-video": {
usage: "camkit export-video [--project PATH] [--out FILE] [--codec prores422]",
about: [
"Render an open project's timeline to a ProRes 422 .mov by driving",
"Camtasia's GUI export (Export ▸ Local File ▸ QuickTime ▸ Apple ProRes",
"422). Camtasia stays the renderer — effects, transitions and Dynamic",
"Captions need its engine, and .trec screen recordings carry a video",
"stream ffmpeg can't decode. Run after rebuild to render the edited cut.",
"",
"macOS-only and needs Accessibility permission for your terminal (System",
"Settings ▸ Privacy & Security ▸ Accessibility). The project must already",
"be open in Camtasia. With several open, pass --project to pick which",
"window to raise (export always drives the UI of that document). Verified",
"on Camtasia 2026.1.3 only — menu renames, locale, or the Welcome window",
"can break it (brittle by design until TechSmith ships a real export",
"verb). Blocks until the output file appears and its size stops growing.",
"",
" --project PATH project that must be open (required if several are open)",
" --out FILE output path (default ./<project>.mov)",
" --codec NAME only prores422 for now (default)",
],
},
captions: {
usage:
"camkit captions [--project PATH] --from FILE.transcript.json (--preset NAME | --preset-file PATH) [--list-presets] [--src ID] [--strip-punctuation] [--strip-apostrophes] [--dry-run] [--force]",
Expand Down Expand Up @@ -244,6 +266,7 @@ function printHelp(cmd?: string): void {
sources: "list media-bin sources, placed or not",
rebuild: "rewrite timeline to kept segments (rough cut)",
"export-audio": "mix the timeline's audio to one file (m4a/wav/…)",
"export-video": "render the timeline to a ProRes 422 .mov via Camtasia",
captions: "inject an animated Dynamic Caption track from a transcript",
silences: "ffmpeg silencedetect on a recording",
transcribe: "word-level Whisper transcript of a recording",
Expand Down Expand Up @@ -371,6 +394,88 @@ async function cmdExportAudio(argv: string[]) {
await exportAudio({ segs, projectPath: path, out, durationSeconds });
}

/** Pick which open Camtasia document export-video should drive. */
function resolveOpenExportDoc(argv: string[]): { path: string; name: string } {
const open = camtasiaDocPaths();
if (!open.length) {
throw new Error(
"No project open in Camtasia. Open it first (camkit open) — export drives the open timeline.",
);
}
const explicit = flag(argv, "--project");
if (explicit) {
const tscproj = resolveProjectPath(explicit);
const name = bundleName(tscproj);
const bundleDir = resolve(dirname(tscproj));
const match = open.find(
(d) => d.name === name || resolve(d.path) === bundleDir || resolve(d.path) === resolve(tscproj),
);
if (!match) {
throw new Error(
`${name} is not open in Camtasia (open: ${open.map((d) => d.name).join(", ")}). ` +
"Open it first — export cannot target a closed project.",
);
}
return { path: match.path, name: match.name };
}
if (open.length === 1) return { path: open[0].path, name: open[0].name };
throw new Error(
`Several projects are open in Camtasia; pass --project to pick one:\n` +
open.map((d) => ` --project "${d.path}"`).join("\n"),
);
}

/** Wait until `path` exists and its size is stable (export finished writing). */
async function waitForExportFile(
path: string,
opts: { appearMs?: number; stableMs?: number; overallMs?: number } = {},
): Promise<void> {
const appearMs = opts.appearMs ?? 3 * 60 * 1000;
const stableMs = opts.stableMs ?? 2500;
const overallMs = opts.overallMs ?? 2 * 60 * 60 * 1000;
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

const appearDeadline = Date.now() + appearMs;
while (!existsSync(path)) {
if (Date.now() >= appearDeadline) {
throw new Error(
`Export file did not appear within ${appearMs / 1000}s: ${path}. ` +
"Camtasia may still be showing a dialog, or the UI path failed.",
);
}
await sleep(500);
}

const overallDeadline = Date.now() + overallMs;
let lastSize = -1;
let stableSince = Date.now();
while (true) {
if (Date.now() >= overallDeadline) {
throw new Error(`Export still writing after ${overallMs / 1000}s: ${path}`);
}
const size = statSync(path).size;
if (size > 0 && size === lastSize) {
if (Date.now() - stableSince >= stableMs) return;
} else {
lastSize = size;
stableSince = Date.now();
}
await sleep(500);
}
}

async function cmdExportVideo(argv: string[]) {
const doc = resolveOpenExportDoc(argv);
const base = doc.name.replace(/\.(cmproj|tscproj)$/, "");
const out = flag(argv, "--out") ? resolve(flag(argv, "--out")!) : resolve(`${base}.mov`);
const codec = flag(argv, "--codec") ?? "prores422";
console.log(`Starting Camtasia export of ${doc.name} → ${out} (${codec})…`);
exportVideo({ out, codec, documentName: doc.name });
console.log(`→ export started; waiting for ${out} to finish writing…`);
await waitForExportFile(out);
console.log(`✓ exported ${out}`);
}

function cmdCaptions(argv: string[]) {
if (has(argv, "--list-presets")) {
const presets = listPresets();
Expand Down Expand Up @@ -561,6 +666,7 @@ const COMMANDS: Record<string, (argv: string[]) => void | Promise<void>> = {
sources: cmdSources,
rebuild: cmdRebuild,
"export-audio": cmdExportAudio,
"export-video": cmdExportVideo,
captions: cmdCaptions,
silences: cmdSilences,
transcribe: cmdTranscribe,
Expand Down
Loading