Why Does Reg-suit Report Every Screenshot as Changed?

If Reg-suit reports every screenshot as changed, first verify that it fetched the intended baseline and paired each current image with the right expected image. Then compare the capture conditions used for the baseline and current run. Change comparison tolerances only after those inputs are correct. Without your report, key-generator configuration, capture tool, and CI environment, the exact cause cannot be identified.

Reg-suit compares images in actualDir with expected images fetched by sync-expected, then creates an HTML report. Its key-generator plugin selects the expected key, and its publisher plugin retrieves the images. A wrong key, missing baseline, unexpected filenames, or incorrect directory pairing can therefore make the report misleading before pixel comparison is even considered. See the Reg-suit project documentation.

1. Check what the report says changed

Start with the report categories and the actual file pairs. “New” or missing items point toward baseline availability, filenames, or pairing; changed items have a pair to compare, so inspect those images and their diff. Use the exact labels shown by your run rather than assuming every category means the same thing.

Choose a few representative entries, including one from the beginning, middle, and end of the report.
Record each current filename and its expected counterpart, if present.
Check whether the report marks the files as new, missing, or changed.
Open the original screenshots as well as the diff. A diff alone can make a missing or unrelated baseline look like a page-wide visual change.

The official Puppeteer demo shows Reg-suit recognizing screenshots as new items. That is a useful reminder to distinguish newly discovered images from paired images that differ. See the Reg-suit Puppeteer demo.

2. Verify baseline synchronization and key selection

Confirm that sync-expected completed successfully and that it fetched the snapshot set you meant to compare against. Check the key selected by the configured key-generator plugin and confirm that it corresponds to the intended branch, commit, or workflow. Then inspect the expected-image directory and verify the names and count of files that were fetched.

Reg-suit’s standard run workflow combines synchronization, comparison, and publishing. When diagnosing a run, separate those stages in the logs if your setup permits it: a successful comparison against the wrong or empty expected set can still produce a large report. Check the installed key-generator and publisher configuration in your repository’s regconfig.json and the CI logs; do not assume a particular plugin or storage provider.

3. Check whether capture conditions changed

If the expected images are present and paired correctly, compare how the baseline and current screenshots were produced. Visual regression differences can come from a changed capture environment as well as an application change. The Storybook visual testing guidance discusses environment differences as a source of visual diffs.

Browser and capture versions: compare browser version, automation library version, and screenshot options.
Viewport and scale: check viewport dimensions, device scale factor, and whether full-page capture is enabled consistently.
Fonts and assets: confirm fonts, images, and other assets loaded successfully and were available in both runs.
Locale and time: check locale, timezone, dates, and any content that depends on current time or geographic location.
Readiness and animation: confirm screenshots are taken at the same application state, after the same required content loads, with animations handled consistently.
CI differences: compare operating system, installed fonts, environment variables, and network-dependent content between baseline generation and the current job.

Look for a shared visual pattern. A shift across every page could point to a shared capture setting or font; missing images could point to asset loading; changed text might be dynamic content. These patterns guide investigation but do not prove a cause.

4. Inspect representative diffs before changing thresholds

Open a small set of diffs and decide whether they show a meaningful application change, an environment mismatch, or rendering noise. If the report is difficult to read, Reg-suit documents optional x-img-diff-js reporting to help expose inserted or moved regions. Use the report to understand the change, not as a substitute for checking the original images.

Reg-suit documents these comparison settings:

Setting	What it controls	When to consider it
`thresholdRate`	Ratio of differing pixels allowed.	When you intentionally want to allow a limited proportion of changed pixels.
`thresholdPixel`	Absolute differing-pixel alternative to the ratio.	When an absolute pixel count fits the comparison better than a percentage.
`matchingThreshold`	Sensitivity to YUV color distance.	When small color-distance variation is the specific noise under review.
`enableAntialias`	Ignores pixels detected as antialiased.	When edge rendering differences are the identified source of noise.

Reg-suit’s configuration example uses thresholdRate: 0.05; that is an example value, not a generally correct setting. The related reg-cli documentation also describes threshold rate. A more permissive setting can hide real regressions, so make one change at a time and review the resulting diffs.

5. Update baselines only after review

If the differences are intentional, review and publish the new expected screenshots using your team’s normal baseline workflow. Before publishing, verify that the selected key is correct, the capture environment is the intended one, and representative diffs show the approved change. Refreshing every baseline just to make CI pass can encode a failed asset load, wrong environment, or accidental rendering change as the new expectation.

Troubleshooting checklist

Symptom	Likely area to inspect	Next action
Many files appear new or have no expected counterpart	Baseline sync, key selection, or naming	Confirm sync succeeded; inspect the chosen key, fetched directory, and filename pairing.
All paired images have broad layout changes	Capture environment or shared application state	Compare viewport, scale, fonts, browser versions, readiness, and CI environment.
Images or large regions are blank	Asset loading or capture timing	Check network access and whether capture waits for required content to load.
Only text edges or fine borders differ	Font or antialiasing variation	Check installed fonts and scale; consider antialias handling only after confirming the cause.
Only some pages differ	Page-specific content or state	Inspect those routes for dynamic data, time, locale, or page-specific asset failures.
Report looks unchanged after a configuration edit	Config not loaded, or different job/config path	Check the config path and CI command, then inspect logs to confirm the changed setting is used.

Performance, reliability, and cost considerations

For a report where every image changes, inspect a few representative pairs before opening every diff. If they share one obvious pattern, investigate the shared input first; if the patterns differ, sample across pages and categories. Keep baseline synchronization, capture configuration, and comparison settings visible in CI logs so future runs can be compared.

Reg-suit’s expected-image workflow depends on the configured publisher and storage being available, and on the selected key resolving to the intended snapshot set. The dossier does not establish a universal runtime, cost, or failure rate; those depend on your capture, CI, and storage setup. Review your own provider and job usage rather than assuming a particular cost profile.

Or skip the browser setup

If your task is to capture pages for visual review or to generate screenshots for a baseline workflow, ScreenshotNeo offers a screenshot API and MCP server. It returns a screenshot or PDF from one GET request. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

In Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

In Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners are accepted and removed before capture; known consent platforms, newsletter popups, and chat widgets are removed. Each step can be turned off.
Bot checks, blank pages, failed loads, timeouts, and cache hits cost nothing. Responses include X-Page-Verdict and X-Billed headers.
An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents.
The free plan includes 1,000 screenshots a month with no card. Paid plans start at $5 for 3,000 screenshots.

FAQ

Does “every screenshot changed” prove the application changed?

No. It means the comparison inputs differ according to the configured process. Check the expected snapshot and capture conditions before attributing the changes to the application.

Should I set the threshold to zero or make it very permissive?

Neither is a general fix. Choose a tolerance only after identifying the kind of difference you intend to accept, then review what the setting hides.

Can I tell the cause without the report or configuration?

No. The report categories, selected key, expected images, and capture setup are needed to narrow the cause for a particular run.