Skip to content

Commit 568edeb

Browse files
authored
docs: distinguish mirrored corpora from curated identity records (#61)
1 parent 2b66a6d commit 568edeb

2 files changed

Lines changed: 58 additions & 8 deletions

File tree

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,14 +88,18 @@ README.md # this file
8888
design.md # full architectural design — read this before changing anything
8989
```
9090

91-
### Data branches (orphan, bot-pushed only)
91+
### Data branches (orphan)
92+
93+
Mirrored branches are bot-pushed; `other` is human-curated and pushed
94+
directly. See design.md → "Carve-out: curated USB identity records are pushed
95+
directly".
9296

9397
| Branch | What it carries | Refresh trigger |
9498
|---|---|---|
9599
| `platformio` | Slim mirror of `boards/*.json` + `platform.json` from the 37 PlatformIO `platform-*` repos (Apache-2.0). | `workflow_dispatch` (manual for v1; nightly cron later) |
96100
| `arduino` | Parsed JSON form of `boards.txt` from upstream Arduino cores (Arduino AVR, SAMD, Espressif Arduino, Adafruit nRF52, STM32duino, Silicon Labs, …). | `workflow_dispatch` |
97101
| `vendors` | Authoritative PID allocation registries: `raspberrypi/usb-pid`, `espressif/usb-pids`, our curated `vendor_names_inlined` overlay. | `workflow_dispatch` |
98-
| `other` | Heterogeneous: udev rules, esptool VID tables, gowdy.us scrapes, anything that doesn't fit a clean upstream category. | `workflow_dispatch` |
102+
| `other` | Heterogeneous: udev rules, esptool VID tables, gowdy.us scrapes, plus **hand-curated** identity records and overrides (`overrides.json`, `*_pids.json`) — anything that doesn't fit a clean upstream category. | direct push (curated) + `workflow_dispatch` |
99103

100104
Each carries a top-level `_meta.json` recording every source URL, upstream
101105
commit SHA, sync timestamp, and license — provenance is part of the data.

design.md

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -162,8 +162,12 @@ the per-source debuggability).
162162
(`raspberrypi/usb-pid`, `espressif/usb-pids`) + our curated
163163
`vendor_names_inlined` overlay. Authoritative but small.
164164
4. **`other`** — the eclectic catch-all: udev rules, esptool tables,
165-
gowdy.us scrapes, anything that doesn't fit. Lives separately so the
166-
noise from this branch doesh't muddy the cleaner three.
165+
gowdy.us scrapes, plus our **hand-curated** identity records and
166+
overrides (`overrides.json`, `*_pids.json`) — anything that doesn't fit
167+
the mirrored three. Lives separately so the noise from this branch
168+
doesn't muddy the cleaner three, and so curation has an unambiguous
169+
home. See "Carve-out: curated USB identity records are pushed directly"
170+
under Anti-goals.
167171

168172
Adding a 5th class (e.g. `zephyr` for the Zephyr `board.yml` corpus) is
169173
a single follow-up: new sync workflow, new branch, add the branch name
@@ -376,10 +380,13 @@ Each phase is a PR-reviewable change, gated by tests on `main`.
376380

377381
- **Per-board issue tracker** — issues live on `FastLED/fbuild` for now.
378382
When this repo goes public, it gets its own.
379-
- **PR-based board contributions** — data branches are bot-pushed; if a
380-
human needs to add a board manually, the right place is the upstream
381-
source (PlatformIO platform-*, Arduino core, vendor registry). We
382-
mirror, we don't curate-by-PR.
383+
- **Forking upstream *board definitions*** — if a board's JSON is wrong or
384+
missing, the right place to fix it is the upstream source (PlatformIO
385+
`platform-*`, Arduino core, vendor registry), not a local edit here. We
386+
mirror those corpora; a local fork of a mirrored record silently diverges
387+
on the next sync. This is about the mirrored layers (`platformio`,
388+
`arduino`, and the registry half of `vendors`) — it is **not** a blanket
389+
ban on human-authored data. See the curation carve-out below.
383390
- **Real-time push notifications to fbuild clients** — the existing
384391
pull / cache model is fine; push would add infrastructure
385392
(websockets, durable subscriptions) that doesn't fit this repo's
@@ -393,6 +400,45 @@ Each phase is a PR-reviewable change, gated by tests on `main`.
393400
of `force-with-lease` rewrites. The `last-good-*` tags are the
394401
stable references.
395402

403+
### Carve-out: curated USB identity records are pushed directly
404+
405+
The anti-goal above is about mirrored board definitions. **Curated USB
406+
identity records are the opposite: we own them, and they are pushed straight
407+
to the data branch.** Nothing about them is mirrored, so there is no upstream
408+
to defer to and nothing to diverge from.
409+
410+
This is already the established shape of the `other` branch — `overrides.json`
411+
(`vid_overrides` / `vidpid_overrides` / `vidpid_board_links`),
412+
`nxp_debug_probe_pids.json`, `teensy_pids.json`, `ftdi_bridge_pids.json` — plus
413+
the `vendor_names_inlined` overlay on `vendors`. These exist precisely because
414+
some identities have no upstream registry that carries them, or carry them
415+
under a name that is wrong for an embedded-board context (the Teensy /
416+
VOTI 0x16C0 re-attribution being the canonical case).
417+
418+
The procedure for a missing identity is therefore **push, not file**:
419+
420+
1. Add a flat `{vid, pid, product}` record to a topic file on `other`
421+
(`_comment` / `_skip` header record for provenance; see
422+
`nxp_debug_probe_pids.json`).
423+
2. Verify ingestion locally *before* pushing:
424+
`uv run --no-project python builders/extract_other.py --in <worktree> --out <tmp>`
425+
3. Push to the data branch; `build-site.yml` rebuilds on push, with the
426+
nightly backstop and `workflow_dispatch` as override.
427+
428+
Downstream consumers need no coordinated release: fbuild fetches the published
429+
catalogue at runtime and picks new identities up on its next cache refresh. A
430+
version bump downstream is warranted only when ingestion or resolution *logic*
431+
changes, not when data does.
432+
433+
Worked example: FTDI FT232H `0403:6014` was absent registry-wide, reported by a
434+
downstream audit as the last unresolved literal in FastLED's `ci/` tree. It went
435+
from absent to resolving downstream in minutes via exactly the three steps above
436+
— one commit, one site rebuild, no fbuild release (#60).
437+
438+
Filing an issue and waiting is the wrong reflex for a repo we control. Issues
439+
are for questions about *which* identity is correct, not for requesting that a
440+
known-correct one be added.
441+
396442
## Open questions
397443

398444
- **GitHub Pages plan availability** — does FastLED org have Team or

0 commit comments

Comments
 (0)