How to review screenshot differences in Reg-suit

To review screenshot differences in Reg-suit, open its generated HTML report, first confirm that the expected snapshot is the intended baseline, then inspect each changed, new, or deleted item and compare its affected region with the change planned for the pull request. A difference is a signal for human review; it does not by itself prove a defect or a failed test.

1. Confirm what the report is comparing

Before judging a visual change, establish which two versions Reg-suit compared. The sync-expected step fetches expected snapshots selected through the configured key-generator and publisher plugins. The compare command compares those images with the images in actualDir and generates an HTML differences report. A configured publish step can publish the result and current images to external storage. See the Reg-suit repository and configuration documentation.

1. Find the report produced by the CI run or local comparison.
2. Check the branch, commit, and expected-snapshot selection used by the run. Confirm the expected image represents the comparison point intended for this pull request.
3. Confirm that actualDir contains screenshots produced by the current code and environment.
4. Check whether an expected snapshot existed. On a first run, Reg-suit may classify an image as new because there was no prior snapshot. That is not evidence of a regression against a missing baseline. In the official demo, publishing the first snapshot makes it the expected image for a later run; see the official Reg-suit demo.

Baseline selection can depend on the configured key-generator and publisher. The project site describes a GitHub-flow setup that selects a topic branch’s parent commit as the baseline. Follow your repository’s configuration rather than assuming every report compares against the main branch or the immediately previous run. The Reg-suit project site also describes report publishing and integrations.

2. Review every item category

Reg-suit reports items as changed, new, or deleted. Review each category; additions and removals can be as meaningful as pixel changes to an existing screenshot.

Report item	What it means to investigate	Review question
Changed	An image exists in both sets but differs under the configured comparison.	Do the highlighted or differing regions correspond to the intended UI change?
New	An image appears in the current set without a matching expected image.	Was this page, route, or screenshot intentionally added? Is this a first snapshot with no baseline?
Deleted	An expected image has no corresponding current image.	Was the route or screenshot intentionally removed, renamed, or excluded? Could capture or file naming have failed?

For changed images, inspect the entire screenshot as well as the visibly affected region. A small local change may be expected, while a broad shift can point to a layout, font, viewport, or rendering difference. Check the diff against the pull request’s intended code and design change, not just against whether the page looks plausible in isolation.

3. Decide whether a difference is intended

1. Connect the image to the change. Identify the component, style, content, or behavior the author meant to alter.
2. Inspect the affected region. Look for unexpected movement, clipping, missing content, changed spacing, or an unintended color or typography change. Also verify intended additions and removals.
3. Check surrounding context. A change can affect neighboring elements or pages that were not part of the planned edit.
4. Record the disposition. Mark the difference as expected when it matches the intended change; otherwise, ask for investigation or a fix. If the baseline is wrong or absent, correct the comparison setup before drawing a conclusion.

The official demo explains the key judgment clearly: “If the snapshot testing tool detects differences, it does not mean the test is failure. It only means we need to review the differences.” Treat the report as evidence to inspect, not an automatic verdict.

4. Interpret sensitivity settings before judging small diffs

Comparison settings determine which differences appear or count as significant. Review the configuration alongside borderline or noisy results. The settings below are documented in the Reg-suit configuration documentation; check the current docs for exact plugin-specific configuration and version details.

Setting	Effect	How to reason about it
thresholdRate	Ratio of differing pixels to the whole image; documented range is 0–1.	A ratio threshold can make a small percentage of changed pixels less significant. Confirm the configured value before interpreting a small diff.
thresholdPixel	Absolute differing-pixel threshold, offered as an alternative to the ratio threshold.	Consider image dimensions: the same absolute count represents a different proportion on small and large screenshots.
matchingThreshold	Controls YUV color-distance matching from 0 to 1; smaller values make comparison more sensitive.	Check this when subtle color shifts appear or disappear from the report.
enableAntialias	Enables detection and ignoring of anti-aliased pixels.	Useful context for edge noise around text and shapes; do not assume it explains large layout or content changes.
ximgdiff	Optional richer report information; its engine describes structural information such as inserted or moved image parts.	Use the additional details to understand a change, while still checking it against the intended UI edit.

Do not tune thresholds simply to make an inconvenient report disappear. First establish whether the difference is a rendering artifact, expected design change, or real problem; then adjust comparison settings only when the team’s intended sensitivity warrants it.

5. Find and share the report in CI

When a publisher plugin is configured, the report and snapshots may be published to external storage. Optional notifier plugins can direct reviewers to results through GitHub commit status or pull-request comments, GitLab merge-request comments, Slack, or Chatwork. A notification is a pointer to the result, not a substitute for inspecting the HTML report. Check the run’s logs and configured storage destination if the report link is missing. Reg-suit’s project site summarizes the publishing and integration workflow.

6. Troubleshooting common review problems

Symptom	Likely cause	What to check or fix
Everything is marked new	No expected snapshot was available, often on the first run, or the expected-key selection does not match.	Check whether a baseline was published and whether the configured key-generator and publisher select the intended snapshot. Do not label first-run images regressions against a nonexistent baseline.
Expected pages are missing or marked deleted	The current capture set, naming, route list, or baseline selection differs from what the report expects.	Compare expected and actual item names; confirm the capture job completed and the expected snapshot corresponds to the intended commit.
A report has many unexpected changes	The comparison context or rendering inputs may differ, or a broad code change may have affected multiple pages.	Verify baseline, viewport and capture inputs available in your workflow, then inspect the affected regions and the pull request’s scope. Do not assume all changes are harmless noise.
Only tiny edge or color differences appear	Threshold, YUV matching, or antialias settings can affect sensitivity.	Review thresholdRate, thresholdPixel, matchingThreshold, and enableAntialias in the active configuration before deciding whether the diff matters.
The HTML report or image links cannot be found	The report may be local, publishing may not be configured, or the external-storage destination may be unavailable or misidentified.	Check the compare output, CI artifacts, publisher configuration, and job logs. Use notifier messages as pointers and verify the destination they reference.
A notification exists but gives no useful verdict	Notifiers surface the result; they do not decide whether a visual difference is intended.	Open the linked report and review the baseline and affected image regions.

7. Performance, reliability, and cost considerations

Reg-suit’s review stage is only as useful as the baseline and current screenshots supplied to it. Keep the capture set and baseline-selection rules understandable, and make the report accessible from the pull request or CI result so reviewers can reach the evidence. A first snapshot needs special care because it establishes an expected image rather than showing a regression from an earlier one.

When comparisons are noisy, inspect sensitivity settings and rendering context before changing thresholds. Optional ximgdiff can add report detail, but it does not replace human review. The retrieved official documentation does not establish fixed runtime, reliability, or hosted-service pricing figures, so those depend on the project’s capture, storage, CI, and hosting setup; consult the current documentation and your own environment for estimates.

Or skip the browser setup

If you need a clean screenshot for a report or a separate visual check, ScreenshotNeo is a website screenshot API and MCP server for developers. It does not replace Reg-suit’s baseline comparison or HTML report; it provides screenshot capture through one GET request. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

• Cookie and consent banners are accepted like a visitor, then 60+ known consent platforms, newsletter popups, and chat widgets are removed before the shot; each step can be turned off.
• Bot checks, CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing. Responses include X-Page-Verdict and X-Billed headers.
• An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients.
• The free plan includes 1,000 screenshots a month with no card. Paid plans start at $5 for 3,000 screenshots. Every feature is on every plan.

Create a free ScreenshotNeo account and get 1,000 screenshots a month with no card.

FAQ

Does a Reg-suit difference mean the test failed?

No. It means the compared images differ under the configured settings and need review. Whether that change is a defect depends on the intended change and comparison context.

Why can a first run show new screenshots?

There may be no earlier expected snapshot. In the official demo, the initial image is new; publishing it provides an expected snapshot for a later comparison.

Should I accept every difference that matches the pull request?

Review the full affected region and any surrounding changes. A planned component edit can create unintended effects elsewhere, so confirm the entire reported change is understood.

Does a notifier approve the visual changes?

No. It can point reviewers to the result through an integration, but a person still needs to inspect the report.