Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 113 additions & 20 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,29 +1,122 @@
# aeon-movie-maker — environment variables.
# ────────────────────────────────────────────────────────────────────────
# aeon-movie-maker — environment variables
# ────────────────────────────────────────────────────────────────────────
# Copy to `.env` and fill in your values. movie_maker_fast.py reads these
# via os.environ at runtime.
#
# cp .env.example .env && $EDITOR .env && ./setup.sh
# ════════════════════════════════════════════════════════════════════════

# === ComfyUI server ===

# ────────────────────────────────────────────────────────────────────────
# 1. ComfyUI server endpoint (REQUIRED)
# ────────────────────────────────────────────────────────────────────────
# HTTP endpoint of your ComfyUI server. The CLI sends LTX 2.3 workflow
# JSONs to /prompt and polls /history for results.
#
# ─── LOCAL MODE — ComfyUI on this machine: ──────────────────────────────
# COMFYUI_URL=http://127.0.0.1:8188
#
# ─── REMOTE MODE — ComfyUI on a different machine: ──────────────────────
# Direct HTTP:
# COMFYUI_URL=http://<gpu-box-ip>:8188
# # Start ComfyUI with --listen 0.0.0.0 and open port 8188 in firewall.
#
# SSH tunnel (recommended):
# # In a separate terminal:
# # ssh -L 8188:127.0.0.1:8188 user@gpu-host
# COMFYUI_URL=http://127.0.0.1:8188
#
COMFYUI_URL=http://127.0.0.1:8188

# === ComfyUI install root (REQUIRED for screenplay mode) ===
# This is where the script stages intermediate seed-image frames into
# input/_movie_fast_frames/ so the ComfyUI VAE encoder can pick them up.
# Must be the actual path of your ComfyUI install on whatever host the
# ComfyUI server is running on.
# COMFYUI_ROOT=/path/to/your/ComfyUI

# === Output directory ===
# ────────────────────────────────────────────────────────────────────────
# 2. ComfyUI install root (REQUIRED for screenplay mode + I2V)
# ────────────────────────────────────────────────────────────────────────
# IMPORTANT: this script needs file-system access to ComfyUI's `input/`
# directory because it stages seed-image frames at:
# ${COMFYUI_ROOT}/input/_movie_fast_frames/<scene>/frame_<N>.png
# so the ComfyUI VAE encoder can read them as I2V conditioning.
#
# This means COMFYUI_ROOT must point at a path that the ComfyUI server
# can also read. Three valid configurations:
#
# ─── LOCAL MODE: ────────────────────────────────────────────────────────
# Just point at your local ComfyUI install:
# Linux/macOS: COMFYUI_ROOT=/home/yourname/ComfyUI
# Windows: COMFYUI_ROOT=C:/Users/yourname/Documents/ComfyUI
#
# ─── REMOTE MODE Option A — Run CLI on the REMOTE machine: ──────────────
# Set COMFYUI_ROOT to the ComfyUI path on the remote box (the same
# filesystem as the ComfyUI process). Invoke the CLI via SSH:
# ssh user@remote 'cd /path/to/aeon-movie-maker && python scripts/...'
#
# ─── REMOTE MODE Option B — Run CLI locally with a SHARED filesystem: ───
# Mount or symlink the remote ComfyUI's `input/` directory locally
# (NFS, SMB, sshfs, Tailscale Files, etc.) so the local CLI can write
# frames where the remote ComfyUI server can read them.
# COMFYUI_ROOT=/mnt/remote-comfyui # local mount point
#
# ─── DOES NOT WORK: ─────────────────────────────────────────────────────
# Local CLI + remote ComfyUI without shared filesystem (HTTP-only).
# The ComfyUI server can't read seed images from your local disk.
# For T2V (no seed image), this would work — but I2V + screenplay
# carry-forward needs disk access. Use Option A or B above for video.
#
COMFYUI_ROOT=


# ────────────────────────────────────────────────────────────────────────
# 3. Output directory (OPTIONAL)
# ────────────────────────────────────────────────────────────────────────
# Where rendered MP4s + clips_manifest.json land. Defaults to
# ${COMFYUI_ROOT}/output if unset.
#
# OUTPUT_DIR=./output

# === Binary paths (rely on PATH if unset) ===
# FFMPEG=/usr/bin/ffmpeg
# FFPROBE=/usr/bin/ffprobe

# === HuggingFace token (required for some gated LTX 2.3 / EROS variants) ===
# Get one at: https://huggingface.co/settings/tokens
# HF_TOKEN=
# ────────────────────────────────────────────────────────────────────────
# 4. Binary paths (OPTIONAL)
# ────────────────────────────────────────────────────────────────────────
# FFMPEG=/usr/local/bin/ffmpeg
# FFPROBE=/usr/local/bin/ffprobe


# ────────────────────────────────────────────────────────────────────────
# 5. HuggingFace token (OPTIONAL — for gated LTX 2.3 / EROS auto-download)
# ────────────────────────────────────────────────────────────────────────
# Some Lightricks models on HuggingFace require a token even for public
# read access. Get one (free) at:
# https://huggingface.co/settings/tokens
# Choose "Read" scope. Token starts with "hf_...".
#
# Most users don't need this — install models via ComfyUI Manager UI.
#
# HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


# === Civitai API token (required for the 4 style LoRAs sourced from Civitai) ===
# Get one at: https://civitai.com/user/account → API Keys
# Required for the optional style LoRAs (CyberPunkAI, Smooth_Tribal, Claymation,
# LTX23-GalaxyAce, StudioGhibli, ghibli_style_offset, Illustration concept Variant 3A).
# Civitai download URL pattern: https://civitai.com/api/download/models/<version_id>?token=$CIVITAI_TOKEN
# ────────────────────────────────────────────────────────────────────────
# 6. Civitai API token (OPTIONAL — only for the 7 style LoRAs)
# ────────────────────────────────────────────────────────────────────────
# Style LoRAs (cyberpunk / claymation / ghibli / galaxy / tribal /
# illustration) are hosted on Civitai, not HuggingFace, and require a
# Civitai API token to download.
#
# How to get one:
# 1. Sign in at https://civitai.com (free account)
# 2. Go to https://civitai.com/user/account
# 3. Scroll to "API Keys" → "Add API key"
# 4. Name it (e.g. "aeon-movie-maker"), copy the token
# 5. Paste below
#
# Style LoRAs are STRICTLY OPTIONAL. Plain prompts without `style: <name>`
# tags work without any of them. Don't bother getting a Civitai token
# unless you actually want stylized output.
#
# Download URL pattern (the 'ltx2/' subfolder is mandatory):
# curl -L --create-dirs \
# -H "Authorization: Bearer $CIVITAI_TOKEN" \
# -o "$COMFYUI_ROOT/models/loras/<name>.safetensors" \
# "https://civitai.com/api/download/models/<version_id>"
#
# CIVITAI_TOKEN=
2 changes: 2 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Sponsor button — points to the tip jar on the AEON-7 profile README.
custom: ['https://github.com/AEON-7/AEON-7#-support-the-work']
33 changes: 33 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,39 @@

Instructions for AI agents that operate this tool.

## Step 0 — Determine execution mode

This tool has **stricter execution requirements** than its sibling repos because LTX 2.3 I2V and screenplay carry-forward write seed-image PNGs to `${COMFYUI_ROOT}/input/_movie_fast_frames/` for the ComfyUI VAE encoder to read. **Pure-HTTP remote mode (no shared filesystem) does NOT work for I2V or screenplay** — only for `clip` subcommand with `--no-image` / T2V.

### Local mode — CLI on the same machine as ComfyUI ✓ all subcommands work

Symptoms: `COMFYUI_URL=http://127.0.0.1:8188` reachable + `COMFYUI_ROOT` is a real local dir with `models/`, `input/`, `output/`. Invoke directly:
```bash
python scripts/movie_maker_fast.py {clip|screenplay|stitch} ...
```

### Remote mode — ComfyUI on a different machine

Pick the variant based on what subcommand you need:

**Remote-A — Run CLI on the remote machine via SSH** (recommended for `screenplay` and `clip --seed-image`):
```bash
ssh ${SSH_USER}@<host> 'cd /path/to/aeon-movie-maker && python scripts/movie_maker_fast.py screenplay screenplay.json'
scp ${SSH_USER}@<host>:/path/to/output/movie_fast/<project>/finished.mp4 .
```
✓ I2V works. ✓ screenplay carry-forward works. Outputs stay remote until you `scp` them.

**Remote-B — Local CLI + shared filesystem mount** (advanced):
User must have NFS / SMB / sshfs / Tailscale Files mounting the remote ComfyUI's `input/` dir locally. `COMFYUI_ROOT` points at the local mount. Don't recommend this unless the user explicitly tells you they have it set up.

**Remote-C — Local CLI + HTTP only** (T2V `clip` only):
```bash
python scripts/movie_maker_fast.py clip --prompt "..." --output local.mp4
# No --seed-image, no screenplay — these would fail because ComfyUI can't reach local files
```

**Default**: Remote-A (SSH-invoke on the remote box) for any work involving I2V or screenplay. Remote-C is only viable for one-off T2V clips with no character continuity needed.

## When to invoke

- User asks for a "film", "movie", "cinematic video", "short film", "music video with cinematic clips"
Expand Down
133 changes: 133 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# aeon-movie-maker


[![☕ Tips](https://img.shields.io/badge/%E2%98%95_Tips-Support_the_work-ff5e5b?style=flat)](https://github.com/AEON-7/AEON-7#-support-the-work)
> Fast cinematic video generation built around LTX 2.3 22B (distilled fp8). Three subcommands: render a single clip, render a full screenplay (sequential clips with last-frame carry-forward for character/scene continuity), or stitch dialogue + music + SFX into a finished film with sidechain ducking. ~10–15× faster than WAN-based pipelines while delivering comparable cinematic quality.

Part of the **AEON Media Production** family.
Expand Down Expand Up @@ -141,6 +143,102 @@ For non-narrative work (music videos), substitute `aeon-music-maker` for the aud

`setup.sh` checks all of this and lists download commands for any missing pieces. See `references/AGENT_CINEMA_AUTOPILOT.md` for the full agent runbook.

## Configuration

All config goes through environment variables. Copy `.env.example` to `.env` and fill in your values.

### Where to run this CLI: local vs remote ComfyUI

> ⚠️ **Movie Maker has a constraint the other AEON tools don't have:** the screenplay mode + I2V (image-to-video with seed images and last-frame carry-forward) writes intermediate seed-image PNGs into `${COMFYUI_ROOT}/input/_movie_fast_frames/<scene>/` so the ComfyUI VAE encoder can read them. This means the CLI needs **filesystem-level write access to a path that the ComfyUI server can also read**. Pure-HTTP remote mode (without shared filesystem) does NOT work for I2V or screenplay mode — only for T2V single clips.

#### Mode A — Local (CLI runs on the same machine as ComfyUI) ✓ supports everything

The simplest setup. Both processes share the same filesystem.

```bash
COMFYUI_URL=http://127.0.0.1:8188
COMFYUI_ROOT=/path/to/local/ComfyUI
```

All three subcommands (`clip`, `screenplay`, `stitch`) work without restriction.

#### Mode B — Remote (ComfyUI on a different machine)

Pick the sub-option based on whether you need I2V / screenplay carry-forward:

**B1 — Run the CLI ON the remote machine** (recommended for screenplay work):
```bash
# In .env on the REMOTE box:
COMFYUI_URL=http://127.0.0.1:8188
COMFYUI_ROOT=/path/to/ComfyUI/on/remote
```
Invoke from local terminal:
```bash
ssh ${SSH_USER}@<gpu-host> 'cd /path/to/aeon-movie-maker && python scripts/movie_maker_fast.py screenplay screenplay.json'
scp ${SSH_USER}@<gpu-host>:/path/to/output/movie_fast/<project>/finished_film.mp4 .
```
Everything stays on the remote box; you pull the final cut. ✓ I2V works. ✓ screenplay carry-forward works.

**B2 — Run CLI locally + shared filesystem** (advanced):
Use NFS / SMB / sshfs / Tailscale Files / similar to mount the remote ComfyUI's `input/` directory as a local path. Then point `COMFYUI_ROOT` at the local mount point. The local CLI writes into the shared mount; the remote ComfyUI reads from its native path. ✓ everything works, but adds infrastructure complexity.

**B3 — Run CLI locally + remote HTTP only** (T2V only):
```bash
COMFYUI_URL=http://<gpu-box-ip>:8188
COMFYUI_ROOT=./local-staging # local dir; only used for output mp4 collection
```
Works for `clip` subcommand with **no `--seed-image`** (T2V mode). Does NOT work for screenplay mode or any I2V. The ComfyUI server can't reach your local files for VAE encoding.

### All environment variables

| Variable | Required? | Default | What it is |
|---|---|---|---|
| `COMFYUI_URL` | required | `http://127.0.0.1:8188` | ComfyUI HTTP endpoint. |
| `COMFYUI_ROOT` | **required for I2V/screenplay** | (none) | Path the CLI uses to stage seed-image frames into `input/_movie_fast_frames/`. **Must be readable by the ComfyUI server** — see "local vs remote" above. |
| `OUTPUT_DIR` | optional | `${COMFYUI_ROOT}/output` | Where rendered MP4s + clips_manifest.json land |
| `FFMPEG` / `FFPROBE` | optional | PATH lookup | Override binary paths if not on PATH |
| `HF_TOKEN` | optional | (none) | HuggingFace token for gated Lightricks/LTX-Video models. Get one at https://huggingface.co/settings/tokens (Read scope). Most users install via ComfyUI Manager and never need this. |
| `CIVITAI_TOKEN` | optional | (none) | Civitai API token for the 7 style LoRAs (cyberpunk / claymation / ghibli / galaxy / tribal / illustration / ghibli_offset). **Only needed if you actually use those `style:` tags.** Get one at https://civitai.com/user/account → API Keys. |

### How to know which model files you need

Run `./setup.sh`. It walks the canonical model paths under `${COMFYUI_ROOT}/models/` and reports what's missing. Easiest installation paths:

1. **ComfyUI Manager** (in-browser UI button) — most LTX 2.3 models are one-click installable
2. **`huggingface-cli download Lightricks/LTX-Video --include '*.safetensors'`** — for batch installs from the official HF repo
3. **Manual download** — visit https://huggingface.co/Lightricks/LTX-Video and grab the specific filenames `setup.sh` lists, place at the canonical paths

For Civitai LoRAs (style tags), search Civitai by filename to find each model's page, then download via the API URL pattern shown in `setup.sh`. License terms are set per-LoRA by the original Civitai uploader.

## Updating an existing install

```bash
cd /path/to/aeon-movie-maker
./sync.sh
```

The script:
1. **Detects local uncommitted changes** and offers to stash + re-apply them
2. **Shows a diff preview** of incoming commits + files-changed list
3. **Asks for confirmation** before pulling
4. **Refreshes Python deps** + re-runs `setup.sh` model check (so any new LTX 2.3 / EROS / LoRA additions are flagged)

### Flags

| Flag | What it does |
|---|---|
| `./sync.sh` | Default — interactive, shows diff |
| `./sync.sh --dry-run` (or `-n`) | Show what would change without pulling |
| `./sync.sh --yes` (or `-y`) | Non-interactive |
| `./sync.sh --no-models` | Skip the model file check (faster) |
| `./sync.sh --help` | Print usage |

### What if I customized something?

The sync script auto-stashes any uncommitted local edits before pulling, then re-applies them. `.env`, your `output/` directory, the staging frames at `${COMFYUI_ROOT}/input/_movie_fast_frames/`, and other personal files are gitignored — they're never touched by sync.

If you've added your own custom LoRA mappings to the `SCENE_LORAS` dict in `scripts/movie_maker_fast.py`, those local edits will be auto-stashed and re-applied. If they conflict with upstream changes (rare), sync stops with clear instructions for resolving.

## Project structure

```
Expand Down Expand Up @@ -172,3 +270,38 @@ MIT.
- [`aeon-music-maker`](https://github.com/AEON-7/aeon-music-maker) — music score
- [`aeon-music-video`](https://github.com/AEON-7/aeon-music-video) — audio-reactive editing
- [`comfyui-aeon-spark`](https://github.com/AEON-7/comfyui-aeon-spark) — base ComfyUI Docker stack

---

## ☕ Support the work

If this release has been useful, tips are deeply appreciated — they go directly toward more compute, more models, and more open releases.

<table align="center">
<tr>
<td align="center" width="50%">
<strong>₿ Bitcoin (BTC)</strong><br/>
<img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/btc.png" alt="BTC QR" width="200"/><br/>
<sub><code>bc1q09xmzn00q4z3c5raene0f3pzn9d9pvawfm0py4</code></sub>
</td>
<td align="center" width="50%">
<strong>Ξ Ethereum (ETH)</strong><br/>
<img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/eth.png" alt="ETH QR" width="200"/><br/>
<sub><code>0x1512667F6D61454ad531d2E45C0a5d1fd82D0500</code></sub>
</td>
</tr>
<tr>
<td align="center" width="50%">
<strong>◎ Solana (SOL)</strong><br/>
<img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/sol.png" alt="SOL QR" width="200"/><br/>
<sub><code>DgQsjHdAnT5PNLQTNpJdpLS3tYGpVcsHQCkpoiAKsw8t</code></sub>
</td>
<td align="center" width="50%">
<strong>ⓜ Monero (XMR)</strong><br/>
<img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/xmr.png" alt="XMR QR" width="200"/><br/>
<sub><code>836XrSKw4R76vNi3QPJ5Fa9ugcyvE2cWmKSPv3AhpTNNKvqP8v5ba9JRL4Vh7UnFNjDz3E2GXZDVVenu3rkZaNdUFhjAvgd</code></sub>
</td>
</tr>
</table>

> **Ethereum L2s (Base, Arbitrum, Optimism, Polygon, etc.) and EVM-compatible tokens** can be sent to the same Ethereum address.
Loading