@@ -31,121 +31,6 @@ or select it with Docker Buildx. ARM64 also includes libibverbs and its provider
3131because Flare enables the RoCE transport on that architecture. Multi-host RoCE
3232deployment additionally requires the host's RDMA devices and network setup.
3333
34- To build and publish both Linux architectures under one tag, log in to the
35- destination registry, then run:
36-
37- ``` sh
38- ./scripts/build_container.sh
39- ```
40-
41- The script creates a ` docker-container ` builder if the selected name is missing,
42- preserves existing builder configuration, and starts the builder before building.
43- The script uses builder ` flare-multi ` , pushes
44- ` ghcr.io/local-inference-lab/flare:latest ` , and displays the published manifest.
45- Use ` --beta ` or ` --release ` to publish ` ghcr.io/local-inference-lab/flare:beta `
46- or ` ghcr.io/local-inference-lab/flare:release ` , respectively:
47-
48- ``` sh
49- ./scripts/build_container.sh --beta
50- ./scripts/build_container.sh --release
51- ```
52-
53- Use ` --tag IMAGE:TAG ` for a custom image reference. The three tag options are
54- mutually exclusive. Override the builder with ` --builder NAME ` . Build output
55- uses plain progress logs, including when redirected. The script resolves the
56- repository root from its own location, so it can run from any working directory.
57- Docker selects the matching architecture when pulling the published tag.
58-
59- ## Continuous publication
60-
61- The [ serving-container workflow] ( ../.github/workflows/container.yml ) builds Linux
62- AMD64 and ARM64 images on pushes to ` main ` . It can also be started manually with
63- GitHub Actions' ** Run workflow** control on ` main ` . A push containing several
64- commits builds its tip commit, rather than a separate image for every intermediate
65- commit. There is no path filter or pull-request trigger.
66-
67- The workflow first publishes a staging tag of the form
68- ` ghcr.io/local-inference-lab/flare:sha-COMMIT-RUN-ATTEMPT ` . It checks the image
69- index and runs ` --version ` , ` run --help ` , ` agent --help ` , and ` orbit --help ` for
70- both architectures, without GPUs, networking, ports, or host mounts. These CPU
71- checks validate executable startup and CLI parsing; they do not validate model
72- serving. Only a successful build whose commit still matches remote ` main ` is
73- promoted to ` ghcr.io/local-inference-lab/flare:latest ` . Promotion uses the tested
74- index digest and verifies the resulting digest. A superseded build retains its
75- staging tag. The workflow does not change ` beta ` or ` release ` .
76-
77- Register a dedicated, trusted Linux X64 self-hosted runner for the Flare
78- repository with label ` lil-flare-builder ` . An existing runner registered only
79- for another repository cannot accept this workflow. The runner needs ` gh ` ,
80- ` jq ` , ` flock ` , Docker, and a system-installed Buildx plugin. Its rootless Docker
81- daemon must be reachable at
82- ` unix:///run/lil-flashinfer-docker/docker.sock ` , with the containerd image store
83- and ARM64 emulation configured. The workflow uses builder ` default ` to retain
84- that daemon's BuildKit caches; it does not install emulation or create a builder.
85-
86- The runner user must be able to open
87- ` /var/lib/github-flashinfer/locks/cu134-sm120-build.lock ` . The workflow holds this
88- shared lock through building, smoke tests, and promotion, waiting up to six
89- hours for other builds. GitHub Actions queues runs without canceling the active
90- build; a remote ` main ` check prevents queued, superseded commits from replacing
91- ` latest ` . Docker credentials live in a temporary per-run configuration directory
92- and are removed when the build step exits.
93-
94- The workflow grants its ` GITHUB_TOKEN ` repository-content read access and package
95- write access. The existing GHCR ` flare ` package must also grant the Flare
96- repository Actions write access through inherited repository permissions or
97- the package's ** Manage Actions access** settings. Public pull access and an OCI
98- source label alone do not establish that write permission.
99-
100- ## Discord changelog
101-
102- The [ published-changes workflow] ( ../.github/workflows/changelog.yml ) checks for
103- unannounced changes after a successful container workflow and hourly at minute
104- 17 . It resolves ` latest ` through GHCR and requires both image architectures to
105- identify the same source commit. A private repository token reads commit
106- metadata; source files, patches, and commit bodies are not sent to the model.
107- The model receives sanitized commit subjects and produces short English notes.
108- The notes describe changes in ` main ` , including preparation changes that are
109- not part of the serving container.
110-
111- At most one announcement is sent in six hours. Commits accumulate while that
112- interval is active, and the hourly check flushes them even without another push.
113- State is stored outside the checkout at
114- ` /var/lib/github-flashinfer/state/flare-announcements ` . Its runner-owned directory
115- must survive job cleanup and host restarts. A separate file lock serializes
116- updates. A successful Discord acknowledgment advances the commit cursor; a
117- transport failure triggers message-history reconciliation before another send.
118-
119- Configure the repository secret ` FLARE_DISCORD_BOT_TOKEN ` and these repository
120- variables before enabling publication:
121-
122- | Variable | Meaning |
123- | --- | --- |
124- | ` FLARE_DISCORD_CHANNEL_ID ` | Destination channel the bot can view and send to |
125- | ` FLARE_ANNOUNCE_BASE_SHA ` | Complete source SHA covered by the preceding announcement |
126- | ` FLARE_ANNOUNCE_BASE_TIME ` | That announcement's timestamp with a UTC offset |
127- | ` FLARE_CHANGELOG_MODEL_URL ` | OpenAI-compatible API base URL reachable from the runner |
128- | ` FLARE_CHANGELOG_MODEL ` | Model name served by that API |
129-
130- The baseline variables initialize an absent state file. They do not reset an
131- existing cursor. The bot does not automatically mention users or roles. Preview
132- an explicitly selected range without sending a Discord message or changing state:
133-
134- ``` sh
135- GH_TOKEN=" $( gh auth token) " python3 scripts/announce_changes.py \
136- --repository local-inference-lab/flare \
137- --channel-id " $FLARE_DISCORD_CHANNEL_ID " \
138- --preview --base " $FLARE_ANNOUNCE_BASE_SHA " \
139- --image ghcr.io/local-inference-lab/flare:latest \
140- --model-url " $FLARE_CHANGELOG_MODEL_URL " --model " $FLARE_CHANGELOG_MODEL "
141- ```
142-
143- If an uncertain send cannot be found in channel history, the job stops rather
144- than risking a duplicate. An operator can retry through ** Run workflow** , using
145- the exact ` pending.nonce ` and ` pending.head ` from the state file as ` retry_nonce `
146- and ` retry_head ` . This rechecks the channel history and still requires six hours
147- since the uncertain attempt; it preserves the frozen message and commit range.
148-
14934## Run
15035
15136This example exposes host GPU 0 and mounts a directory containing a prepared
@@ -196,11 +81,11 @@ identity through Docker's `--user` flag. For example:
19681mkdir -p " $HOME /.cache/huggingface/hub"
19782docker run --rm --user " $( id -u) :$( id -g) " \
19883 --mount " type=bind,src=$HOME /.cache/huggingface/hub,dst=/cache" \
199- flare:local download local-inference-lab /MODEL \
84+ flare:local download ORG /MODEL \
20085 --revision gb202 --cache-dir /cache
20186```
20287
203- ` MODEL ` is a repository-name placeholder. An explicit revision permits downloading
88+ ` ORG/ MODEL` is a repository placeholder. An explicit revision permits downloading
20489without exposing GPUs. For automatic selection, expose the intended host GPUs
20590with Docker's ` --gpus ` and select their container-local indices with Flare's
20691` --gpus ` . ` flare run ORG/NAME ` accepts the same download options, then loads the
0 commit comments