test: insta snapshot tripwire for persisted-storage JSON shapes#975
test: insta snapshot tripwire for persisted-storage JSON shapes#975anikettuli wants to merge 1 commit intoStremio:developmentfrom
Conversation
Adds a new src/unit_tests/snapshots.rs module that uses `insta` to freeze
the JSON serialization of every bucket stremio-core writes to persistent
storage. Each storage key listed in src/constants.rs (profile, library,
library_recent, streams, search_history, streaming_server_urls,
notifications, calendar, dismissed_events) now has a corresponding
committed `.snap` file; any schema or field-name change produces a
visible `cargo insta review` diff that cannot accidentally land.
Why this matters: storage keys are installed on end-user devices. A
silent change to a serialized field name breaks every existing install
with no compiler hint. The snapshot suite converts that class of bug
from "manual audit required" to "CI-gated diff".
Scope chosen deliberately:
- In: storage bucket types (Profile, LibraryBucket, StreamsBucket,
NotificationsBucket, SearchHistoryBucket, ServerUrlsBucket,
DismissedEventsBucket). These are the persistence contract.
- Out: Msg/Action/Event JSON - not persisted to storage; covered
implicitly by the existing serde unit tests in src/unit_tests/serde/.
- Out: deep-link URLs - already covered by the existing tests in
src/unit_tests/deep_links/ (with the new round-trip helper).
Updating a snapshot intentionally: `cargo insta review`, inspect the
diff, accept it, and pair it with a migration step in runtime/env if
the wire format is not backward-compatible.
Files:
- src/unit_tests/snapshots.rs (new, 7 tests)
- src/unit_tests/snapshots/*.snap (7 committed fixtures)
- src/unit_tests/mod.rs: register the module
- Cargo.toml: add `insta = { version = "1", features = ["json"] }` dev-dep
Verified locally (Rust 1.85.1 stable-x86_64-pc-windows-gnu):
cargo test -p stremio-core --lib -> 209 passed, 0 failed (202 + 7 new)
cargo clippy --all --no-deps -D warnings -> clean
cargo fmt --check -> clean
stremio-core-web prebuildPrebuild artifact published by the Paste one of these into Release "@stremio/stremio-core-web": "https://stremio.github.io/stremio-core/stremio-core-web/claude/snapshot-tripwire/stremio-stremio-core-web-0.56.3.tgz"Dev "@stremio/stremio-core-web": "https://stremio.github.io/stremio-core/stremio-core-web/claude/snapshot-tripwire/dev/stremio-stremio-core-web-0.56.3.tgz" |
There was a problem hiding this comment.
Pull request overview
Adds an insta-based snapshot test suite to lock down the JSON serialization “shape” of persisted storage buckets, so accidental serde/schema drift becomes a visible diff during review.
Changes:
- Added
src/unit_tests/snapshots.rswith snapshot tests forProfile::default()and several bucket defaults. - Committed
.snapfixtures undersrc/unit_tests/snapshots/. - Registered the new unit-test module and added
instaas a dev-dependency (with correspondingCargo.lockupdates).
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/unit_tests/snapshots.rs |
New insta snapshot tests + documentation for persistence-contract tripwire. |
src/unit_tests/mod.rs |
Registers the new snapshots unit-test module. |
src/unit_tests/snapshots/*.snap |
New committed JSON snapshot fixtures for each test. |
Cargo.toml |
Adds insta under [dev-dependencies]. |
Cargo.lock |
Lockfile updates including new insta deps and additional resolved changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| //! `streaming_server_urls`, `notifications`, `calendar`, | ||
| //! `dismissed_events`. |
There was a problem hiding this comment.
The module docs claim the snapshot suite covers the calendar storage key, but this file doesn't define a calendar snapshot test and there is no .snap fixture for it. Also CALENDAR_STORAGE_KEY appears to be unused outside src/constants.rs, so either add the missing coverage (if something is persisted under this key) or remove calendar from the documented coverage list to avoid misleading future readers.
| //! `streaming_server_urls`, `notifications`, `calendar`, | |
| //! `dismissed_events`. | |
| //! `streaming_server_urls`, `notifications`, `dismissed_events`. |
| /// Empty `LibraryBucket` — covers both the `library` and `library_recent` | ||
| /// storage keys (same wire shape, split by recency of access). | ||
| #[test] | ||
| fn snapshot_library_bucket_empty() { | ||
| assert_json_snapshot!(LibraryBucket::default()); | ||
| } |
There was a problem hiding this comment.
Snapshotting LibraryBucket::default() (and the other buckets) only verifies the top-level field names/container types; it won’t catch breaking JSON changes inside persisted item types (e.g., LibraryItem field rename/default-value changes) because items is empty. If the goal is a “shape tripwire” for persisted data, consider snapshotting a minimal non-empty bucket with one representative item so nested serialization changes also produce an insta diff.
| /// [`crate::constants::NOTIFICATIONS_STORAGE_KEY`]. | ||
| #[test] | ||
| fn snapshot_notifications_bucket_empty() { | ||
| assert_json_snapshot!(NotificationsBucket::default()); |
There was a problem hiding this comment.
This test uses NotificationsBucket::default(), but NotificationsBucket’s default impl is only derived under cfg(test) and sets created to the Unix epoch, while production code initializes this bucket via NotificationsBucket::new::<E>(...) (which uses E::now()). To better reflect the real persisted wire format (and avoid test-only defaults masking changes), consider constructing the bucket via the production constructor with a deterministic test Env/time, or redact the timestamp field in the snapshot.
| assert_json_snapshot!(NotificationsBucket::default()); | |
| assert_json_snapshot!( | |
| NotificationsBucket::default(), | |
| { | |
| ".created" => "[created]", | |
| } | |
| ); |
|
i think the real fix for this would be to ignore the fields that cannot be parsed etc instead of just guarding, example now users cannot downgrade core if they would like to right? the app would break adding tests here does absolutely nothing |
|
Makes sense — if the UI-level "clear data" flow is the intended backstop, a CI-only tripwire is friction without matching value. Appreciate you taking the time to explain rather than just closing. I'll keep the reasoning in mind for future contributions. |
Summary
Adds a new
src/unit_tests/snapshots.rsmodule that uses theinstacrate to freeze the serialized JSON shape of every bucket thatstremio-corewrites to persistent storage. Each storage key listed insrc/constants.rsnow has a committed.snapfixture; any schema or field-name change to one of these types produces a visiblecargo insta reviewdiff that cannot accidentally land.Why this change
Storage keys (
profile,library,library_recent,streams,search_history,streaming_server_urls,notifications,calendar,dismissed_events) are installed on end-user devices. A silent rename of a serialized field — say a typo fix on a variant, or a#[serde(rename)]added during a refactor — breaks every existing install with no compiler warning and no user-visible cue until someone tries to load their saved state.The existing coverage in
src/unit_tests/serde/tests round-trips for individual types, but has no complete tripwire for the composite bucket shapes. If someone adds a field toProfilethat defaults toserde_json::Value::Nullon deserialization but changes the serialized output, the round-trip tests still pass. The snapshot suite catches that kind of drift.Effect
instadiff in CI. Contributors can't accidentally break existing user data.cargo insta review→ inspect the diff → accept. The expected pairing with a migration step insrc/runtime/env.rsis now a visible, enforceable reviewer checkbox rather than an unwritten rule.Scope chosen deliberately
In scope (this PR): storage bucket types — the persistence contract.
Profile(default state, including the bundled official addons)LibraryBucket(empty) — covers bothlibraryandlibrary_recentStreamsBucket,SearchHistoryBucket,ServerUrlsBucket,NotificationsBucket,DismissedEventsBucket(empty)Out of scope (not this PR):
Msg/Action/EventJSON — these are dispatch-internal, not persisted. Breaking them affects live clients (where the on-the-wire contract matters for a running app session), but doesn't corrupt installed user data. The existingsrc/unit_tests/serde/suite covers this class at the type level.src/unit_tests/deep_links/*.rstests (now with a round-trip helper for the flate2-encoded player URL, see build: unblock cargo update + stremio-watched-bitfield base64 0.13 → 0.22 #974).Files changed
src/unit_tests/snapshots.rs— new, 7 tests.src/unit_tests/snapshots/*.snap— 7 committed fixtures.src/unit_tests/mod.rs— register the module.Cargo.toml— addinsta = { version = "1", features = ["json"] }to[dev-dependencies].How to review a snapshot change in a future PR
When a legitimate schema change lands, the author runs:
which shows each new/changed snapshot side-by-side with the previous committed version. The author accepts the diff and commits the updated
.snapfile in the same PR as the code change. Reviewers see the.snapdiff alongside the type change and can verify:src/runtime/env.rsfor it?Test plan
Verified locally (Rust 1.85.1
stable-x86_64-pc-windows-gnu):cargo test -p stremio-core --lib— 209 passed, 0 failed (202 original + 7 new snapshot tests)cargo clippy --all --no-deps -- -D warnings— cleancargo fmt --check— cleanDependency on other PRs
[dev-dependencies]only, so no effect on published crate graph.