fix: use ltx_t2v_pure.json workflow for T2V — builder produces still images#2
fix: use ltx_t2v_pure.json workflow for T2V — builder produces still images#2NotMainstream wants to merge 5 commits into
Conversation
- .github/FUNDING.yml -> Sponsor button at top of repo page - README header: ☕ Tips badge linking to AEON-7 profile tip section - README footer: full tip jar block (BTC/ETH/SOL/XMR with QR codes) QR images sourced from AEON-7/AEON-7 profile repo (single source of truth).
- .env.example: every variable now has a labeled 'how to use' section with where to get the value (HF tokens, Civitai tokens, etc.) - README.md: new 'Configuration' section explicitly walking through Mode A (local) vs Mode B (remote ComfyUI), with sub-options for direct HTTP vs SSH tunnel vs SSH-invoked CLI - AGENTS.md: 'Step 0 — Determine execution mode' lets agents figure out whether to invoke directly or wrap commands in ssh - aeon-movie-maker specifically documents the I2V/screenplay frame- staging constraint: pure-HTTP remote mode doesn't work for those, needs SSH-invoked CLI or shared-filesystem mount
sync.sh now: - Detects local uncommitted changes and offers to stash + re-apply - Shows a diff preview (commit list + files-changed) before pulling - Asks for confirmation unless --yes - Supports --dry-run, --yes, --no-models, --help - cd's to script dir so it works from any cwd - Resolves origin/main vs origin/master automatically - Reports n commits + n files changed; pulls --ff-only README adds 'Updating an existing install' section that documents: - The basic upgrade flow (cd repo && ./sync.sh) - Each flag's purpose - The fact that .env / output / __pycache__ / models are gitignored and never touched by sync For users who cloned at v1, this is the canonical upgrade path.
…images ROOT CAUSE: The builder (build_ltx_i2v_workflow) generates a fundamentally different pipeline than the working ltx_t2v_pure.json workflow. Builder pipeline (BROKEN T2V): EmptyLatentVideo → RandomNoise → KSamplerSelect → BasicScheduler → CFGGuider → SamplerCustomAdvanced → VAEDecode → Video The builder has NO I2V conditioning infrastructure. At very low denoising with no conditioning, diffusion produces a static photograph. Working ltx_t2v_pure.json pipeline: LoadImage → ImageScale → LTXVPreprocess → LTXVImgToVideoConditionOnly(bypass=True) → Sampler → LTXVLatentUpsampler → VAEDecode → Video The working workflow has full I2V infrastructure (nodes 3159, 4970) with bypass=True. When bypass=True, the node participates in the graph but with zero image influence — properly initializing temporal evolution. Changes: - Add _load_t2v_workflow() that loads ltx_t2v_pure.json and patches it - clip command now uses _load_t2v_workflow for T2V mode instead of builder - Add workflows/api/ directory with ltx_t2v_pure.json, ltx_i2v_api.json, ltx_t2v_vbvr.json (proven working workflow JSONs) - Fix duration print statement to use correct node ID per workflow type - Also includes prior fixes: _SEP=os.sep, MAX_CLIP_SECONDS=10.0, text_encoder path, quality LoRA path (from fix/model-paths-vbvr-ic-lora)
|
Thanks for the deep dive on this — the diagnosis that the builder's T2V path produces still images is plausible and worth fixing. But this PR mixes the architectural fix with several unrelated, concerning changes, and the workflow JSONs need scrubbing before they can ship in this repo. I'm going to leave it open without merging while we sort these out, and would love a focused follow-up PR on just the T2V fix once these are addressed. Blockers (must fix before merge)1. Workflow JSONs ship embedded prompts + non-zero seeds + wrong checkpoint nameAuditing
Per the repo conventions ( 2. Text encoder filename change loses abliteration silentlyThe diff renames the text encoder reference: Two problems:
This is a non-trivial behavioral change. Should be its own PR with explicit user discussion and the abliterated LoRA wired in correctly. 3. Distill LoRA path rename has the same issueThe added Concerns (please address but not strict blockers)4. PR mixes unrelated changes — please splitThis single PR contains:
Each of these is a separate decision. Mixing them makes the PR hard to evaluate. Splitting into focused PRs would let me merge the safe ones quickly and discuss the others independently. 5. T2V diagnosis could use a minimal reproThe claim "builder produces still images for T2V" is plausible — the Path forward — suggested splitIf you're up for re-splitting, this would let us merge most of it fast:
We have a sanitizer at Thanks again for the diagnosis work. T2V producing static frames if true is a real bug we want fixed properly — just want to do it without the collateral changes that would land in this same merge. |
ROOT CAUSE: The builder (build_ltx_i2v_workflow) generates a fundamentally different pipeline than the working ltx_t2v_pure.json workflow.
Builder (broken T2V)
EmptyLatentVideo → RandomNoise → KSamplerSelect → BasicScheduler → CFGGuider → SamplerCustomAdvanced → VAEDecode → Video
The builder has NO I2V conditioning infrastructure. At very low denoising with no conditioning, diffusion produces a static photograph.
Working ltx_t2v_pure.json
LoadImage → ImageScale → LTXVPreprocess → LTXVImgToVideoConditionOnly(bypass=True) → Sampler → LTXVLatentUpsampler → VAEDecode → Video
The working workflow has full I2V infrastructure (nodes 3159, 4970) with bypass=True. When bypass=True, the node participates in the graph but with zero image influence — properly initializing temporal evolution.
Changes