Skip to content

Commit 0522ee2

Browse files
committed
Remove account-specific publishing and publisher defaults
Remove Local Inference Lab container, APT, and announcement publication automation. Use neutral publishing examples and the NVIDIA repository homepage while retaining external dependency and checkpoint sources. Source builds contain no embedded publisher identity. Existing signed packages can use explicitly supplied publisher profiles. Qwen exports default to the local recipe namespace; explicit repository selections still win. Validation: 92 CLI/signing tests, 105 package-protection tests, and 7 Rust publisher/trust tests passed. Rebuilt the installed Heliopause extension. Staged whitespace check passed. Rust builds retain unused-code warnings. GPU serving and binary/container distribution were not tested. Design-Signed-Off-By: Codex (GPT-6); design-review=6aeb
1 parent c3d0672 commit 0522ee2

23 files changed

Lines changed: 65 additions & 1436 deletions

‎.github/workflows/changelog.yml‎

Lines changed: 0 additions & 71 deletions
This file was deleted.

‎.github/workflows/container.yml‎

Lines changed: 0 additions & 82 deletions
This file was deleted.

‎docs/cli-design.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -508,13 +508,13 @@ from `ignition export` and the diagnostic `ignition serve` commands.
508508
### Downloading published packages
509509

510510
```sh
511-
flare download local-inference-lab/MODEL --gpus 0,1,2,3
512-
flare run local-inference-lab/MODEL --gpus 0,1,2,3
513-
flare download local-inference-lab/MODEL --revision gb10
514-
flare run local-inference-lab/MODEL@gb202 --filename model.safetensors --gpus 0,1,2,3
511+
flare download ORG/MODEL --gpus 0,1,2,3
512+
flare run ORG/MODEL --gpus 0,1,2,3
513+
flare download ORG/MODEL --revision gb10
514+
flare run ORG/MODEL@gb202 --filename model.safetensors --gpus 0,1,2,3
515515
```
516516

517-
These examples use `MODEL` as a repository-name placeholder. Repositories contain
517+
These examples use `ORG/MODEL` as a repository placeholder. Repositories contain
518518
prepared Flare packages, not source Transformers checkpoints. Hardware variants
519519
are separate Hugging Face revisions named `gb10` and `gb202`. Automatic selection
520520
queries the selected GPUs' PCI device identifiers through the CUDA driver without
@@ -591,7 +591,7 @@ With the default package cache, the layout is:
591591
```text
592592
~/.cache/flare/packages/
593593
blobs/<sha256>.safetensors
594-
models--local-inference-lab--Qwen3.8-Flash-Next-FLARE/
594+
models--qwen38/
595595
snapshots/<snapshot-id>/
596596
model.safetensors
597597
<sha256>.safetensors
@@ -652,8 +652,8 @@ Configure published exports with all three values:
652652
```toml
653653
[publishing]
654654
signing_key = "/private/flare-signing-key.pem"
655-
namespace = "local-inference-lab"
656-
official_source = "https://huggingface.co/local-inference-lab/foobar"
655+
namespace = "example-publisher"
656+
official_source = "https://huggingface.co/example-publisher/MODEL"
657657
```
658658

659659
The key must be an owner-only Ed25519 PKCS#8 PEM file. Encrypted keys prompt

‎docs/container.md‎

Lines changed: 2 additions & 117 deletions
Original file line numberDiff line numberDiff line change
@@ -31,121 +31,6 @@ or select it with Docker Buildx. ARM64 also includes libibverbs and its provider
3131
because Flare enables the RoCE transport on that architecture. Multi-host RoCE
3232
deployment additionally requires the host's RDMA devices and network setup.
3333

34-
To build and publish both Linux architectures under one tag, log in to the
35-
destination registry, then run:
36-
37-
```sh
38-
./scripts/build_container.sh
39-
```
40-
41-
The script creates a `docker-container` builder if the selected name is missing,
42-
preserves existing builder configuration, and starts the builder before building.
43-
The script uses builder `flare-multi`, pushes
44-
`ghcr.io/local-inference-lab/flare:latest`, and displays the published manifest.
45-
Use `--beta` or `--release` to publish `ghcr.io/local-inference-lab/flare:beta`
46-
or `ghcr.io/local-inference-lab/flare:release`, respectively:
47-
48-
```sh
49-
./scripts/build_container.sh --beta
50-
./scripts/build_container.sh --release
51-
```
52-
53-
Use `--tag IMAGE:TAG` for a custom image reference. The three tag options are
54-
mutually exclusive. Override the builder with `--builder NAME`. Build output
55-
uses plain progress logs, including when redirected. The script resolves the
56-
repository root from its own location, so it can run from any working directory.
57-
Docker selects the matching architecture when pulling the published tag.
58-
59-
## Continuous publication
60-
61-
The [serving-container workflow](../.github/workflows/container.yml) builds Linux
62-
AMD64 and ARM64 images on pushes to `main`. It can also be started manually with
63-
GitHub Actions' **Run workflow** control on `main`. A push containing several
64-
commits builds its tip commit, rather than a separate image for every intermediate
65-
commit. There is no path filter or pull-request trigger.
66-
67-
The workflow first publishes a staging tag of the form
68-
`ghcr.io/local-inference-lab/flare:sha-COMMIT-RUN-ATTEMPT`. It checks the image
69-
index and runs `--version`, `run --help`, `agent --help`, and `orbit --help` for
70-
both architectures, without GPUs, networking, ports, or host mounts. These CPU
71-
checks validate executable startup and CLI parsing; they do not validate model
72-
serving. Only a successful build whose commit still matches remote `main` is
73-
promoted to `ghcr.io/local-inference-lab/flare:latest`. Promotion uses the tested
74-
index digest and verifies the resulting digest. A superseded build retains its
75-
staging tag. The workflow does not change `beta` or `release`.
76-
77-
Register a dedicated, trusted Linux X64 self-hosted runner for the Flare
78-
repository with label `lil-flare-builder`. An existing runner registered only
79-
for another repository cannot accept this workflow. The runner needs `gh`,
80-
`jq`, `flock`, Docker, and a system-installed Buildx plugin. Its rootless Docker
81-
daemon must be reachable at
82-
`unix:///run/lil-flashinfer-docker/docker.sock`, with the containerd image store
83-
and ARM64 emulation configured. The workflow uses builder `default` to retain
84-
that daemon's BuildKit caches; it does not install emulation or create a builder.
85-
86-
The runner user must be able to open
87-
`/var/lib/github-flashinfer/locks/cu134-sm120-build.lock`. The workflow holds this
88-
shared lock through building, smoke tests, and promotion, waiting up to six
89-
hours for other builds. GitHub Actions queues runs without canceling the active
90-
build; a remote `main` check prevents queued, superseded commits from replacing
91-
`latest`. Docker credentials live in a temporary per-run configuration directory
92-
and are removed when the build step exits.
93-
94-
The workflow grants its `GITHUB_TOKEN` repository-content read access and package
95-
write access. The existing GHCR `flare` package must also grant the Flare
96-
repository Actions write access through inherited repository permissions or
97-
the package's **Manage Actions access** settings. Public pull access and an OCI
98-
source label alone do not establish that write permission.
99-
100-
## Discord changelog
101-
102-
The [published-changes workflow](../.github/workflows/changelog.yml) checks for
103-
unannounced changes after a successful container workflow and hourly at minute
104-
17. It resolves `latest` through GHCR and requires both image architectures to
105-
identify the same source commit. A private repository token reads commit
106-
metadata; source files, patches, and commit bodies are not sent to the model.
107-
The model receives sanitized commit subjects and produces short English notes.
108-
The notes describe changes in `main`, including preparation changes that are
109-
not part of the serving container.
110-
111-
At most one announcement is sent in six hours. Commits accumulate while that
112-
interval is active, and the hourly check flushes them even without another push.
113-
State is stored outside the checkout at
114-
`/var/lib/github-flashinfer/state/flare-announcements`. Its runner-owned directory
115-
must survive job cleanup and host restarts. A separate file lock serializes
116-
updates. A successful Discord acknowledgment advances the commit cursor; a
117-
transport failure triggers message-history reconciliation before another send.
118-
119-
Configure the repository secret `FLARE_DISCORD_BOT_TOKEN` and these repository
120-
variables before enabling publication:
121-
122-
| Variable | Meaning |
123-
| --- | --- |
124-
| `FLARE_DISCORD_CHANNEL_ID` | Destination channel the bot can view and send to |
125-
| `FLARE_ANNOUNCE_BASE_SHA` | Complete source SHA covered by the preceding announcement |
126-
| `FLARE_ANNOUNCE_BASE_TIME` | That announcement's timestamp with a UTC offset |
127-
| `FLARE_CHANGELOG_MODEL_URL` | OpenAI-compatible API base URL reachable from the runner |
128-
| `FLARE_CHANGELOG_MODEL` | Model name served by that API |
129-
130-
The baseline variables initialize an absent state file. They do not reset an
131-
existing cursor. The bot does not automatically mention users or roles. Preview
132-
an explicitly selected range without sending a Discord message or changing state:
133-
134-
```sh
135-
GH_TOKEN="$(gh auth token)" python3 scripts/announce_changes.py \
136-
--repository local-inference-lab/flare \
137-
--channel-id "$FLARE_DISCORD_CHANNEL_ID" \
138-
--preview --base "$FLARE_ANNOUNCE_BASE_SHA" \
139-
--image ghcr.io/local-inference-lab/flare:latest \
140-
--model-url "$FLARE_CHANGELOG_MODEL_URL" --model "$FLARE_CHANGELOG_MODEL"
141-
```
142-
143-
If an uncertain send cannot be found in channel history, the job stops rather
144-
than risking a duplicate. An operator can retry through **Run workflow**, using
145-
the exact `pending.nonce` and `pending.head` from the state file as `retry_nonce`
146-
and `retry_head`. This rechecks the channel history and still requires six hours
147-
since the uncertain attempt; it preserves the frozen message and commit range.
148-
14934
## Run
15035

15136
This example exposes host GPU 0 and mounts a directory containing a prepared
@@ -196,11 +81,11 @@ identity through Docker's `--user` flag. For example:
19681
mkdir -p "$HOME/.cache/huggingface/hub"
19782
docker run --rm --user "$(id -u):$(id -g)" \
19883
--mount "type=bind,src=$HOME/.cache/huggingface/hub,dst=/cache" \
199-
flare:local download local-inference-lab/MODEL \
84+
flare:local download ORG/MODEL \
20085
--revision gb202 --cache-dir /cache
20186
```
20287

203-
`MODEL` is a repository-name placeholder. An explicit revision permits downloading
88+
`ORG/MODEL` is a repository placeholder. An explicit revision permits downloading
20489
without exposing GPUs. For automatic selection, expose the intended host GPUs
20590
with Docker's `--gpus` and select their container-local indices with Flare's
20691
`--gpus`. `flare run ORG/NAME` accepts the same download options, then loads the

‎docs/core-design.md‎

Lines changed: 4 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,7 @@ prepared forwards before warmup and CUDA graph capture. Model structure,
4646
numerical semantics, compiler optimizations, and kernel preparation remain
4747
connected to the Python ecosystem throughout execution construction.
4848

49-
The vLLM fork's Qwen and GLM paths maintain engine-specific model
50-
implementations. vLLM also offers a
49+
vLLM offers a
5150
[Transformers backend](https://huggingface.co/docs/transformers/main/en/transformers_as_backend),
5251
whose attention, state, and execution contracts still need to fit each model.
5352
Flare's preparation boundary lets canonical model code and PyTorch compilation
@@ -74,17 +73,6 @@ concentrated on composing those capabilities into a service; Flare does not
7473
need to establish a competing model ecosystem or develop another kernel
7574
library to become useful.
7675

77-
The [Local Inference Lab vLLM fork](https://github.com/local-inference-lab/vllm/tree/dev/karmic-kraken)
78-
already integrates this stack. Its support extends through checkpoint
79-
transfers, compressed-scale expansion, PCIe and RoCE collectives, coordinated
80-
kernel preparation, graph workspaces, decode context parallelism, recurrent
81-
prefix checkpoints, external checkpoint transfers, speculative verification,
82-
and prefill scheduling. Those concerns cross model, worker, scheduler, cache,
83-
and allocator interfaces. Maintaining a specialized serving contract is already
84-
part of supporting these models in vLLM. The cost comparison is between
85-
carrying those integrations through a general engine and maintaining a smaller
86-
runtime whose invariants directly express their requirements.
87-
8876
vLLM's device, backend, model, and distributed-execution breadth creates a
8977
large compatibility matrix. Changes to cache layouts, compilation, offload,
9078
and scheduling must coexist across those combinations. Flare limits that
@@ -156,10 +144,9 @@ Keeping a conversation outside GPU memory imposes related requirements. A
156144
resumable endpoint includes recurrent matrices, selector state, attention
157145
tails, and saved hidden state alongside its KV pages. Every part must describe
158146
the same committed token prefix. Transfer sources must remain immutable, and
159-
destinations cannot be recycled while a copy can still write to them. The
160-
vLLM fork's offload integration spans these rules, from frozen recurrent
161-
snapshots and padded selector tails to restore admission, cancellation, and
162-
cache reset. A disk transport depends on all of them.
147+
destinations cannot be recycled while a copy can still write to them. Offload
148+
integration spans frozen recurrent snapshots, padded selector tails, restore
149+
admission, cancellation, and cache reset. A disk transport depends on all of them.
163150

164151
The [tiered prefix-cache design](prefix-cache-offload.md) extends the checkpoint
165152
object Helios already owns. Stores retain published immutable page chains and

0 commit comments

Comments
 (0)