How to Approve Visual Changes in a Screenshot Regression Test
Review screenshot diffs, approve intentional UI changes, and update baselines safely in Playwright and Chromatic without hiding regressions.
A screenshot regression test tells you that a new capture differs from its approved baseline; it does not tell you whether the difference is a bug or an intended design change. Inspect the changed region in context, confirm the capture conditions are comparable, and approve the change only when it is intentional and correct. If it is unexpected, reject it or fix the code and rerun the test. Updating a baseline changes what future runs consider correct, so do not use an update command to make an unexplained failure disappear.
1. Confirm the screenshots are comparable
Before judging the UI, check that the test used the same capture conditions as the baseline. Differences in browser, operating system, viewport, fonts, device pixel ratio, or other rendering inputs can alter pixels even when the application code did not change. Playwright recommends generating comparisons in the same environment as the baseline. Chromatic also documents device-pixel-ratio changes as a condition that can flag screenshots as changed.
- Confirm the browser and browser version are consistent.
- Check viewport dimensions and device scale factor.
- Check operating system, installed fonts, and relevant rendering dependencies.
- Confirm the page reached the same state: data, feature flags, animations, and asynchronous content can affect the capture.
- Look for capture or test configuration changes alongside application changes.
If the environment changed, reproduce the baseline conditions or intentionally create and review a new baseline for the changed environment. Do not approve a diff until you can explain what changed.
2. Inspect the diff in context
Open the comparison view or image diff and locate each changed region. Then inspect the full rendered page or component, not just the highlighted pixels. A small diff can indicate a meaningful issue such as clipped text, a shifted control, or a missing icon; a larger diff can be an intentional redesign.
- Identify the changed component and the affected states or pages.
- Compare the new rendering with the relevant code, design change, or issue.
- Check nearby layout and content for unintended effects.
- For noisy regions such as timestamps or rotating content, determine whether the test should stabilize or deliberately exclude that content. Keep exclusions narrow so meaningful regressions remain visible.
A diff is evidence to review, not an automatic verdict. The reviewer decides whether the new appearance is correct.
3. Decide: approve, reject, or investigate
| What you find | Action |
|---|---|
| The visual change matches an intended and reviewed product change. | Approve the new appearance and update the baseline through the tool’s normal review process. |
| The change is unexpected, incorrect, or has not been explained. | Reject the change or fix the code, then rerun the screenshot test. |
| The diff may come from changed capture conditions or unstable content. | Investigate and make the capture reproducible before deciding. |
With Chromatic, accepting a change updates the story baseline; denying it marks the change as a regression and fails the build. Its quickstart instructs reviewers to go through each snapshot and approve or reject the change. See the Chromatic quickstart.
4. Approve changes with Playwright
Playwright stores visual reference screenshots with the project. After reviewing an intentional change, update those references with the snapshot update command:
npx playwright test --update-snapshotsReview the generated file changes before committing. The update command changes the expected images used by later runs; it is not itself an approval decision. Playwright recommends committing the snapshot directory to version control and reviewing the changes. Read Playwright’s visual comparisons documentation.
- Run the relevant visual test in the same environment used to generate its baseline.
- Inspect each failed comparison and establish that the new rendering is intended.
- Run
npx playwright test --update-snapshotsfor the approved change. - Inspect the repository diff to confirm only the expected reference images changed.
- Commit the reviewed snapshots with the code or design change that explains them.
- Rerun the tests and confirm the intended baseline now passes without accepting unrelated changes.
If your project uses a narrower Playwright project or test selection, run the update against the relevant tests according to your configuration, then inspect the resulting snapshot diff just as carefully.
5. Review hosted visual tests and other workflows
In hosted review tools, use the build’s review queue to inspect changed snapshots and make an explicit accept or reject decision. Chromatic documents that accepting updates its baseline and denying marks a regression that fails the build. For teams evaluating workflows, compare where baselines live, how approval is recorded, whether capture conditions can be reproduced, how reviewers collaborate, and how the tool fits the existing test runner and CI process.
Percy’s Playwright client routes Playwright screenshot assertions through Percy so visual verdicts are handled in its review UI. Integration details can change, so consult the Percy Playwright client documentation before adopting or changing setup.
6. Make approvals traceable
- Review every changed snapshot rather than approving a whole build without inspection.
- Keep baseline updates with the code or design change that caused them.
- Inspect the version-control diff for unexpected files or unrelated snapshots.
- Record the reason for a broad or difficult-to-understand visual update in the pull request or review discussion.
- Rerun the comparison after approval to verify the intended reference is now used.
Repository-managed baselines make the image changes visible in version control. Hosted review tools provide a shared review interface. Choose the process that makes it easy for your team to see what changed and who approved it.
7. Troubleshoot common screenshot approval problems
| Symptom | Likely cause | What to do |
|---|---|---|
| Many unrelated snapshots fail at once. | Browser, operating system, fonts, viewport, or device scale factor differs from the baseline environment. | Restore the baseline capture conditions first. If the environment change is intentional, review and regenerate the affected baselines as a deliberate change. |
| The same test changes between runs. | Unstable page state, asynchronous content, animation, time-dependent data, or external content. | Make test inputs and page state deterministic where possible; wait for the intended state before capture. Avoid approving a baseline that merely records a transient state. |
| The update command makes the test pass, but the change is unexplained. | The reference was overwritten before the diff was understood. | Inspect the baseline file changes and compare with the prior reference. Revert unexplained updates, investigate the cause, and approve only after review. |
| A baseline update includes snapshots unrelated to the code change. | The update run covered more tests or environments than intended, or shared rendering inputs changed. | Review the full file diff, separate expected from unexpected changes, and rerun a suitably scoped update after resolving the cause. |
| A hosted build remains unreviewed or fails after review. | One or more changed snapshots still need an explicit decision, or a change was denied. | Check the review queue for every changed snapshot. Accept only intentional changes; fix or reject regressions and rerun the build as needed. |
8. Performance, reliability, and maintenance
Approval does not make screenshot capture more reliable; consistency does. Keep the baseline-generation and comparison environments aligned, stabilize page state, and avoid unnecessary changes to capture configuration. When a rendering environment must change, expect that references may need a deliberate review.
For repository baselines, updates create files that reviewers must inspect and version. For hosted review, reviewers need to process the changed snapshots in the service’s interface. In either workflow, large bulk updates increase review effort: identify the reason for the changes, check affected snapshots individually, and keep unrelated changes out of the approval.
There is no universal cost or performance figure for these workflows established here. Compare tools against your own CI needs, review process, and repository or hosted-service requirements rather than assuming a particular speed or price.
9. Or skip the browser setup
If you need a clean screenshot of a page for a review, bug report, or reference and do not need to configure a browser capture yourself, ScreenshotNeo is a website screenshot API and MCP server. One GET request returns a screenshot or PDF. The API is not a visual regression approval system: use your test runner or review workflow to compare captures and decide whether a change is intentional.
See the ScreenshotNeo API documentation. Example request:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpimport requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
timeout=90,
)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before capture. Bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up for free and capture up to 1,000 screenshots a month with no card.
10. FAQ
Does a passing visual test prove the UI is correct?
No. It shows that the capture matches its current baseline within the configured comparison behavior. The baseline itself may need review when the UI changes.
Should I approve every pixel difference caused by a design update?
Approve only after checking the changed area and its surrounding UI for unintended effects. A planned design change can still introduce a defect.
Can I update Playwright snapshots automatically in CI?
The update command replaces reference screenshots. Treat those updates as reviewed changes: inspect the generated files and commit the approved baselines with the related code change.
What should I do if I cannot reproduce a diff locally?
Compare the CI and local capture environments and page state. Playwright recommends running in the same environment that generated the baseline for consistent screenshots.
