How a FlashDreams application is put together, and what happens when one runs.
This is the mental model the v2 packages assume. Each package documents its own surface in detail; the job here is to say how they fit together and why the seams are where they are.
flashdreams.api_v2- the protocols an application implements.flashdreams.runtime_v2- the machinery that runs one.integrations_v2- how to write an application.apps/t2v- the text-to-video layer built on all of it.
The model side has a mental model of its own, warmup, CUDA-graph capture, the autoregressive-step body, ring attention, finalize, described in the inference pipeline overview. That covers what happens inside one generation step; this covers what drives the steps.
Four layers, each of which only knows about the one below.
flowchart TB
subgraph integration [Application code under apps and integrations_v2]
App["IApplication, arguments and whatever is expensive to load"]
Session["ISession, one run"]
Model["IModelLoop, generates"]
UI["IUILoop, optional, decides what is shown"]
end
subgraph api [flashdreams.api_v2, the protocols an integration implements]
Protocols["IApplication, ISession, ILoop, IClientWindow, InputSource, OutputSink"]
end
subgraph runtime [flashdreams.runtime_v2, everything that runs one]
Entry["cli, application_registry, find the application"]
Runner["ApplicationRunner, owns the lifecycle"]
RunSession["run_session, owns the two threads"]
Meet["EventBuffer, PresentationManager, where the threads meet"]
Default["BlitModelOutputToScreenLoop, the UI loop nothing has to write"]
Types["SessionDesc, StepResult, UserInputEvents, VideoTensorLayout"]
end
subgraph windows [A client window, the runtime's rather than the application's]
Mp4["Mp4ClientWindow, over Mp4OutputSink"]
Web["WebRTCClientWindow, over WebRTCServer"]
end
App -->|creates| Session
Session -->|registers| Model
Session -.->|may register| UI
App -.->|implements| Protocols
Entry --> Runner
Runner -->|"init, create_session"| App
Runner --> RunSession
RunSession -->|"steps, through the protocols"| Model
RunSession -->|"steps, through the protocols"| UI
RunSession -->|moves input and frames through| Meet
Session -.->|registers this when given no IUILoop| Default
RunSession -->|input and output| Mp4
RunSession -->|input and output| Web
An integration knows the protocols and the runtime knows the protocols, and
neither knows the other: every arrow crossing between them is a call through
api_v2. The runtime holds the only reference to a window, and an integration
holds none.
An application is what someone writes, and it names nothing beyond the two protocols it implements — no windows, no sinks, no threads.
A session is one run of that application. It owns the state for the run and registers the loops that do the work: a model loop that generates, and optionally a UI loop that turns what was generated into what is shown.
The runtime owns everything else. It finds the application, decides which session to ask for, creates the window, starts the threads, moves frames between them, and closes it all down in the right order when the run ends.
A client window is where the run goes and where input comes from — an MP4 file, or a browser over WebRTC. Being the runtime's rather than the application's is what lets one application write a file in a benchmark and stream to a browser in a demo without knowing which is happening.
The seam that matters is between the session and the runtime. An application
describes what it would generate, the runtime honours that description, and from
then on the two communicate only through StepResult objects going one way and
input events going the other.
The same pieces again, but as calls rather than as layers: what happens in what order, and on which thread.
flowchart TB
CLI["flashdreams-run-v2, cli.entrypoint"]
Registry["application_registry.create_application"]
Runner["ApplicationRunner.run"]
Init["IApplication.init, then create_session"]
Take["ISession.init registers the loops"]
CLI --> Registry
Registry --> Runner
Runner --> Init
Init --> Take
subgraph mainThread [Main thread, run_session]
Window["IClientWindow, mp4 or webrtc"]
Buffer["EventBuffer"]
Present["PresentationManager"]
UILoop["IUILoop.step"]
end
subgraph modelThread [flashdreams-model-generation-thread]
ModelLoop["IModelLoop.step"]
end
Take --> Window
Window -->|"get_user_input_events"| Buffer
Buffer -->|"read, reader 0"| UILoop
Buffer -->|"read, reader 1"| ModelLoop
ModelLoop -->|"publish a list of StepResult"| Present
ModelLoop -->|"StepResult.metrics"| Metrics["MetricsOutputSink"]
Present -->|"advance, one frame per tick"| UILoop
UILoop -->|"write one StepResult"| Window
Starting a run is ordered so that the cheap ways to fail happen first. The command line resolves the application slug before it builds a window, so a name nothing installed matches costs nothing to discover. It validates the window arguments while it can still print usage, rather than after a checkpoint has loaded. And an application is asked what it would generate before it is asked to load anything, so a layout it was never going to accept is refused in milliseconds rather than gigabytes.
Which session gets asked for is settled once, before the run: the application says what it would generate, and the frame arguments on the command line are laid over the top. An application that describes nothing gets the arguments alone. After that the description is fixed, and both the session and the window are configured from the same copy of it.
Generation and presentation run at different rates and cannot wait for each other, so they get a thread each.
The main thread, whichever one called run_session, reads input from the
window, advances the presentation buffer, runs the UI loop, and writes the result
back to the window. It ticks at the session's UI rate, and it paces itself by
waiting on the shutdown event rather than sleeping, so a client closing is
noticed immediately rather than up to a tick later.
The model thread runs the model loop and publishes each result. It paces itself to the session's step rate. Nothing else runs there, and it is the only thread that touches model state.
This is the reason for most of the runtime's design. A window is only ever touched from the main thread, so window implementations need no locking except where their own backend delivers input from elsewhere. Model state is only ever touched from the model thread, so a model needs none either. The two places the threads do meet — input going one way, frames going the other — are the two buffers below, and they are the only synchronised objects in the system.
Loops that genuinely need to reach across send a message instead of sharing
memory: invoke_async queues an operation against the other loop's state, and
that loop runs it on its own thread before its next step.
A launcher such as torchrun starts one process per GPU. The application
initializes a tensor × context mesh and exposes it through
ISession.parallel_context, available before session.init. The default is
None, so a local session keeps its existing execution path. Mesh planning,
process groups, uneven token gathers, and sharded linear projections live in
flashdreams.core.distributed; the integration maps its model onto them.
The two-loop ownership rule applies per process. Rank zero's main thread owns
the only client window, UI presentation, and metrics sink. Every rank has one
model thread that binds its mesh CUDA device before running model hooks and
executes the same admitted steps. TP shards weights; CP shards views/tokens and
replicates each TP weight shard across its CP group. With tp_size=1, every CP
rank holds the full model weights. CUDA device selection on the main thread
does not select the device on a newly created model thread.
Worker main threads manage startup and cleanup without a client window. The
runtime creates no coordination thread. PyTorch, communication, and window
backends may have their own internal threads.
Application control and model execution have separate owners:
flowchart TB
Control["runtime_v2 application control (Gloo)<br/>StepAgreement: admission, input/reset,<br/>session replacement and shutdown"]
Inference["Integration-owned inference (normally NCCL)<br/>ParallelContext, TP/CP collectives<br/>and required output gathers"]
Publication["Rank-zero StepResult<br/>publication and presentation"]
Control -->|"Admit the same step on every rank"| Inference
Inference --> Publication
run_session owns StepAgreement; the model loop consumes a private structural
hook for admission and input. The runtime decides whether a step may begin;
the integration decides how it is sharded.
The step and lifecycle boundaries are:
| Boundary | Runtime action | Required ordering |
|---|---|---|
| Before preparation | Stop if any rank requests cancellation or reaches its step bound. | All ranks meet before preparing a step. |
| Input | Broadcast rank zero's event batch and reset generation. | Every model sees the same reset and input events. |
| Step admission | Check preparation, completion, and cancellation on every rank. | A preparation error fails the run on every rank. |
| Model execution | Execute the admitted step and its TP/CP collectives. | A later UI stop cannot make a rank skip this step. |
| Publication | Rank zero publishes complete results; workers return []. |
The integration finishes all required output collectives before returning; workers do not enter presentation backpressure. |
| Session result | After model threads stop, rank zero broadcasts continue, terminal, or replacement state on every UI tick. | Workers remain responsive through long idle UIs; replacement sessions start together. |
| Cleanup | Join the model thread, release loops and sinks, then release the control group. | No collective is introduced during error cleanup. |
A failed rank bypasses session-result polling and begins cleanup, which releases a healthy peer waiting in the next poll without making the failed rank wait. A synchronized replacement result is authoritative across the application runner's deadline check, so every rank enters the same replacement session.
This belongs to the runtime: is_finished is a local predicate, and reset
and close are local state hooks. Integrations must not add independent
collective decisions to these hooks. All ranks must enter matching model
collectives in the same order inside each admitted step.
Before rank zero returns non-empty results, the integration must gather all CP
shards needed for publication in the original view/token order. Workers return
[] only after participating in every required collective. Gather before decode
when it needs the complete latent/view sequence; decode only on rank zero when
the model permits it. TP reductions that already replicate a complete value
need no extra gather. Use build_shard() and gather_tokens() when their tensor
contract fits; the runtime cannot infer a result's axis, order, or padding.
The integration passes its configured TP degree to
init_parallel(tensor_parallel=..., head_groups=...); core derives
cp_size = world_size // tp_size. Missing or None TP retains plan_mesh()'s
TP-first GCD fallback, which may leave cp_size=1. Integrations requiring CP
choose TP explicitly and validate projection/checkpoint shapes, partitionable
views/tokens, and output order. Prefer the smallest TP that meets model
memory/performance needs and keep it within a node when possible. Tensor groups
contain consecutive ranks; context groups stride by tensor size.
invoke_async remains process-local; it does not replicate callbacks or state
across ranks. Use the replicated input events for model changes that must agree
across the mesh, and keep rank-local callbacks free of collective decisions.
The control group is created on every rank before session initialization can
fail independently. Its five-minute timeout bounds a missing participant.
An exception inside a model collective is a process failure: the runtime lets
the original error reach torchrun, which terminates the other ranks. It does
not attempt to recover an incomplete distributed step. NCCL/Gloo timeouts and
the Slurm time limit remain necessary for a lost process or a GPU kernel that
never returns; arbitrary model code cannot be made deadlock-free by a Python
thread join.
The step-agreement tests exercise these boundaries with real Gloo processes and bounded joins, including close before startup, stop after admission, reset/input replication, worker completion, unfinished post-inference UI, replacement sessions, repeat sessions, and injected failures.
Input is collected once, on the main thread, but both loops need it and they
read at different rates. EventBuffer holds a flat list of events plus a cursor
per reader, hands each reader only what it has not seen, and drops what they have
all passed.
Frames go the other way through PresentationManager, a bounded queue of
generated chunks. The UI thread takes one frame per tick, walking through the
frames within a chunk before taking another, so a step that generated twelve
frames is presented over twelve ticks rather than eleven being thrown away.
Because the queue is bounded, something has to give when the rates diverge, and which thing gives is a session's choice rather than the runtime's. A session declares whether a fast model should be held back or should drop old frames, and whether a fast UI should re-present the newest frame or wait for a new one. A run whose output has to be compared frame by frame picks the combination that keeps every frame and shows each exactly once; an interactive run picks the one that favours latency. The runtime does not decide this because the right answer depends on what the run is for.
A reset is a client asking to start over without closing the window. It arrives as an input event, and the event buffer turns it into a generation number. Every loop and the presentation manager compare that number against their own: loops reset their state and restart their step index, and frames generated before the reset are discarded rather than shown. One counter is the whole mechanism — none of those components has to know about the others.
A failure ends the run, and a loop's failure is the one reported: a loop raised on either thread outranks one the main thread raised itself, whichever happened first. A model-thread exception is handed to the session and sets the shutdown event; the main thread then joins the thread, shuts the loops down, closes every sink it opened, closes the session, closes the application, and only then raises. Failures that happen during that cleanup are logged rather than raised over the top of the failure that caused them, so a run always reports the thing that actually went wrong.
A run ends normally when the client closes the window or the UI reports itself finished after model output drains. Model completion alone does not end an unfinished interactive UI, so rank zero may keep post-inference controls alive. The default file-output UI finishes after its last frame, preserving automatic termination for non-interactive runs.
- A third fixed-rate thread for game logic.
- A result field for the number of dropped frames.
- Slowing model generation before the presentation buffer fills.