docs: Add Snapshots EA product documentation#17327
Conversation
Add three new pages for the Snapshots visual regression testing feature (Early Adopter). Covers the overview and prerequisites, upload setup with CI workflow, and the review/approval workflow with project settings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
|
||
| ## Status Checks | ||
|
|
||
| After you upload snapshots from a pull request branch, a **Snapshot Testing** GitHub Check appears on the pull request. If no snapshots changed, the check passes. If any snapshots were added, removed, or modified, the check fails and requires approval. |
There was a problem hiding this comment.
Should we explain that the default is that added snapshots don't fail here?
There was a problem hiding this comment.
will have that in the settings section and maybe just a link to "if you want to change your failure settings..."
| - **Wipe** — Drag a slider across the image to compare base and current branch. | ||
| - **Onion** — Overlay both images with an adjustable opacity slider. | ||
|
|
||
| IMAGE: Sentry Snapshot comparison viewer showing a modified image with split diff |
There was a problem hiding this comment.
Can we make this a gif of using the onion mode on an ogre ? that would make my day.
There was a problem hiding this comment.
lol we'll save that for the EA blog
| ```json | ||
| { | ||
| "display_name": "Billing Page", | ||
| "group": "Settings" |
There was a problem hiding this comment.
we can also add other fields as extras here, they will become tags.
For example:
"device": "id:pixel_5",
"nightMode": true
There was a problem hiding this comment.
didn't want to put that until they're actually used downstream (my understanding is that it's not in any way atm)
| branches: [main] | ||
| pull_request: | ||
|
|
||
| concurrency: |
There was a problem hiding this comment.
Let's leave the concurrency out of this snippet.
There was a problem hiding this comment.
whoops, slipped in, 👍
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| # Use PR head SHA, not the merge commit — avoids phantom diffs |
There was a problem hiding this comment.
took from our sentry yml, not sure if we want in example workflow or not: https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml#L42
There was a problem hiding this comment.
@rbro112 I read through the commit history and looks like we'll probably want to recommend this to customers? WDYT?
| build snapshots "${{ env.SNAPSHOT_OUTPUT_DIR }}" | ||
| --app-id <your-app-id> | ||
| --project <your-sentry-project> | ||
| --head-sha "${{ github.event.pull_request.head.sha || github.sha }}" |
There was a problem hiding this comment.
head-sha, vcs-provider and head-repo-name will be auto-detected in github actions so we should rely on that.
There was a problem hiding this comment.
going off our workflow: https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml#L86
also how much do we want to write for github actions vs. general? not sure what we do for size/distro
There was a problem hiding this comment.
Well the syntax here only applies to github only i.e github.event.pull_request.head.sha. There's no way we can provide all these for all CI systems and it isn't snapshot specific. We should documet these somewhere else.
Anyways, I made a PR to update the workflow you linked to remove all the slop.
| --head-repo-name "${{ github.repository }}" | ||
| ) | ||
|
|
||
| if [[ "${{ github.event_name }}" == "pull_request" ]]; then |
There was a problem hiding this comment.
these will also all be auto detected so we should just rely on that.
| sentry-cli "${ARGS[@]}" | ||
| ``` | ||
|
|
||
| A few things to note about this workflow: |
There was a problem hiding this comment.
let's remove these 3 points. they are slop.
|
|
||
| ## How It Works | ||
|
|
||
| 1. **Generate snapshots** — Your CI job produces PNG screenshots however you like: Playwright, Storybook, simulator captures, or any tool that outputs PNGs. |
There was a problem hiding this comment.
Should we mention Paparazzi?
|
|
||
| <Include name="feature-available-for-user-group-early-adopter" /> | ||
|
|
||
| ## Status Checks |
There was a problem hiding this comment.
Should we add PR comments on this page as well?
There was a problem hiding this comment.
I'll put mention in the settings but since status check control a core workflow i think worth the callout. tbh this page is all slop rn and will get a rework
runningcode
left a comment
There was a problem hiding this comment.
Gave it another round!
| - PNG | ||
| - JPEG | ||
|
|
||
| ## Upload With CI |
There was a problem hiding this comment.
I think we should re-use the uploading with CI like we did for size analysis, or just bring it to a separate section since we are repeating ourselves everywhere https://docs.sentry.io/product/size-analysis/integrating-into-ci/
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| # Use PR head SHA, not the merge commit — avoids phantom diffs |
There was a problem hiding this comment.
@rbro112 I read through the commit history and looks like we'll probably want to recommend this to customers? WDYT?
| ## How It Works | ||
|
|
||
| 1. **Generate snapshots** — Your CI job produces screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs | ||
| 2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory to Sentry |
There was a problem hiding this comment.
Might not necessarily be sentry-cli. Could be fastlane/gradle
| 1. **Generate snapshots** — Your CI job produces screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs | ||
| 2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory to Sentry | ||
| 3. **Sentry diffs against the baseline** — Sentry compares images against corresponding base build (TODO: link to section explaining automated comparisons). With Sentry, you do not have to manage "Golden" images and everything is determined via git metadata. | ||
| 4. **Results post to your PR** — Snapshot results post to your PR\*. You can configure settings(TODO: link to settings) to control status check and comment behavior. |
There was a problem hiding this comment.
post might mean "comment" to some people. i think we should be clear that by default we only have status checks here but comments can also be added
Co-authored-by: Nelson Osacky <nelson.osacky@sentry.io>
Co-authored-by: Nelson Osacky <nelson@osacky.com>
## DESCRIBE YOUR PR Restructure the snapshots docs so CI setup lives on its own page, mirroring the pattern used by the Size Analysis docs. The current \`uploading-snapshots\` page embeds a full GitHub Actions workflow with a \`<your-snapshot-command>\` placeholder. Snapshot generation varies wildly by platform (Playwright, Paparazzi, XCUITests, Laravel Dusk…), so the single baked-in template is misleading. This PR extracts the CI guidance into a dedicated page and keeps the upload page focused on the universal bits (directory structure, JSON metadata, \`sentry-cli\` reference). - New \`docs/product/snapshots/integrating-into-ci.mdx\` with a skeleton GitHub Actions workflow that handles \`push\` to main and \`pull_request\` in one file, plus troubleshooting - Trimmed \`docs/product/snapshots/uploading-snapshots/index.mdx\` — removed the opinionated CI block, added the CI-alert include - New includes \`includes/snapshots/ci-alert.mdx\` and \`includes/snapshots/github-only.mdx\`, modeled on the size-analysis includes - Cross-links from \`index.mdx\` and \`reviewing-snapshots/index.mdx\` back to the new CI page - Left \`TODO\` links next to Android/iOS bullets for future platform-specific snapshot pages ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## DESCRIBE YOUR PR Add Android setup documentation for the Snapshots EA product. Covers enabling snapshots via the Sentry Android Gradle Plugin and wiring it up to popular snapshot tools. - Adds `docs/product/snapshots/android/index.mdx` - Documents SAGP 6.4.0+ setup with `sentry.snapshots` DSL - Documents Paparazzi (recommended), Roborazzi, and generic-tool integration - Links back to the existing uploading-snapshots CI guide Linked to EME-1031. Stacked on top of #[snapshots-ea-product-docs](https://github.com/getsentry/sentry-docs/pull/new/mtopo27/snapshots-ea-product-docs). Merge that PR first. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Claude <noreply@anthropic.com>
## DESCRIBE YOUR PR Small follow-ups on top of `mtopo27/snapshots-ea-product-docs`: - Bump the Sentry Android Gradle Plugin minimum version from 6.4.0 to 6.5.0 in the Android snapshots setup guide. - Reword "snapshot tests" / "snapshot testing" to emphasize snapshot generation, since Sentry generates snapshots rather than runs tests. Note: references to the **Snapshot Testing** GitHub Check were left as-is — those appear to name a literal status check produced by Sentry. Let me know if you'd like those renamed too. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
coolguyzone
left a comment
There was a problem hiding this comment.
Made a few suggestions, but looks good to me!
First round of edits from Alex Co-authored-by: Alex Krawiec <alexanderkrawiec@gmail.com>
## DESCRIBE YOUR PR Aligns Android Snapshots docs with the DSL restructuring from sentry-android-gradle-plugin [PR #1167](getsentry/sentry-android-gradle-plugin#1167): - Nest `theme`, `includePrivatePreviews`, and `generateTests` under the new `previews {}` block - Rename `generateSnapshotTests` → `generateTests` - Document the new `packageTrees` option ## IS YOUR CHANGE URGENT? - [ ] Urgent deadline (GA date, etc.): - [ ] Other deadline: - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
||
| ### Generate Snapshots From Compose Previews (Recommended) | ||
|
|
||
| Paparazzi and ComposePreviewScanner automatically turns every `@Preview` composable in your project into a automatically, so you don't have to maintain explicit snapshot tests. |
There was a problem hiding this comment.
| Paparazzi and ComposePreviewScanner automatically turns every `@Preview` composable in your project into a automatically, so you don't have to maintain explicit snapshot tests. | |
| Paparazzi and ComposePreviewScanner automatically turns every `@Preview` composable in your project into a snapshot automatically, so you don't have to maintain explicit snapshot tests. |
Not 100% sure if this is 'snapshot', but looks like something is missing here.
## DESCRIBE YOUR PR Add a dedicated `/cli/snapshots/` page for `sentry-cli build snapshots`, the command that underpins Snapshots uploads. Makes the command discoverable from the CLI left nav and documents behavior that wasn't previously covered. - Adds `docs/cli/snapshots.mdx` — basic usage, git metadata auto-detection (head/base SHA, VCS provider, PR number), `--selective` sharding, `--diff-threshold`, and a full flag reference. - Moves the `sentry-cli build snapshots` flag table out of `docs/product/snapshots/uploading-snapshots/index.mdx` — that page now links to the CLI reference. - Expands the one-line `sentry-cli` mention in `docs/platforms/android/snapshots/index.mdx` into a short section with an example for Android users who don't use the Sentry Android Gradle Plugin. Stacked on top of `mtopo27/snapshots-ea-product-docs`. ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Alex Krawiec <alexanderkrawiec@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
<!-- Use this checklist to make sure your PR is ready for merge. You may delete any sections you don't need. --> ## DESCRIBE YOUR PR *Tell us what you're changing and why. If your PR **resolves an issue**, please link it so it closes automatically.* ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [ ] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) ## LEGAL BOILERPLATE <!-- Sentry employees and contractors can delete or ignore this section. --> Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms. ## EXTRA RESOURCES - [Sentry Docs contributor guide](https://docs.sentry.io/contributing/)
Add three new pages for the Snapshots visual regression testing feature (Early Adopter) at
/product/snapshots/.Snapshots lets users generate PNG screenshots in CI, upload them to Sentry via
sentry-cli, and get pixel-level diffs reported as GitHub Checks on pull requests. This is the initial EA documentation covering the core workflow.Pages added:
index.mdx) — Overview, how it works (5-step flow), and prerequisitessetup/index.mdx) — Directory format, JSON sidecar metadata, and a complete GitHub Actions workflowreview-workflow/index.mdx) — Status checks, comparison viewer, approval flow, and project settingsAll pages include the EA banner. Screenshot placeholders (
IMAGE: ...) are used where real screenshots will be added later.REFS EME-1029