How to Report Bugs Found During Visual Regression Testing

A visual diff is a review signal, not proof of a bug. Open the changed region, compare it with the intended design or change description, and confirm the difference is unintended. Then file a ticket that says what changed, where it appears, how to reproduce it, why it matters, and links to the original comparison.

A useful visual regression ticket includes the affected page or component and snapshot label, build or commit, expected and actual appearance, reproduction steps and state, browser and viewport context, evidence, and known user impact. Percy’s workflow recommends confirming a regression before filing it and including a build or diff link, snapshot label, expected-versus-actual behavior, and browser or device context. Browser and operating-system differences can affect screenshot rendering, so record the environment and investigate mismatches before concluding that an application change is responsible.

1. Review the diff before filing a bug

Open the comparison. Find the changed page or component, snapshot or test label, build or commit, and changed region. Copy the direct build or diff link while it is available.
Compare with the intended change. Check the design, requirements, and pull request description. A changed screenshot may reflect an intentional update. If the change is intentional, follow your team’s review process to approve it and update the baseline.
Check the captured state. Confirm the route, data, user state, and actions that led to the screenshot. A comparison is useful only if the baseline and current image represent the same intended state.
Check the environment. Record browser and version, operating system or device, viewport, and relevant screenshot settings. Playwright documents that screenshot rendering can vary with host OS, browser version, settings, hardware, power source, and headless mode. Investigate an environment mismatch; do not treat it by itself as proof that the application is correct.
Decide whether it is unintended and reproducible. If the difference is unexplained, rerun or inspect it in the baseline environment when possible. Record whether it recurs and under which conditions.

Percy and Chromatic both describe reviewing visual changes to distinguish intentional updates from regressions. Do not update a baseline just to clear an unexplained difference: first establish whether the new appearance is expected.

2. Write an observable bug report

Describe what a person can see or do, rather than guessing at the cause. Be specific about position, clipping, overlap, alignment, or readability. For example: “Expected the primary action to remain fully visible below the heading; actual: the alert banner overlaps its upper edge at the mobile viewport.” This is an example of clear wording, not a report of a particular incident.

Include enough context for another person to recreate the captured state:

Location: route, page, component, snapshot label, and any relevant test name.
Starting state: account or user state, data, feature flags, and setup needed to reach the view. Avoid including secrets or personal data.
Actions: numbered steps from a known starting point, including clicks or navigation that reveal the defect.
Environment: build or commit, browser and version, OS or device, viewport, and relevant test configuration.
Expected and actual: state the intended appearance and the observed difference separately.
Impact: explain what content or task is obscured, misaligned, or difficult to use, and which viewport or users are affected if known.
Reproducibility: say whether it recurs and under what environment or state. If you have not established frequency, do not imply one.

3. Attach evidence that preserves context

Link the original visual build or diff when the comparison platform provides one. Percy identifies its build and diff as the source of truth for reviewing the comparison. If the issue tracker cannot preserve that view, attach the before-and-after screenshots or a crop that makes the changed region easy to find. Keep the full comparison available too: a crop can hide surrounding layout and state.

Label which image is expected and which is actual. If the capture tool records a snapshot name, viewport, or browser, retain that context with the evidence. A screenshot alone may show the defect but not explain the route, setup, or actions needed to reproduce it.

4. Copy-ready visual regression ticket template

Title: [Page/component] [visible defect] at [browser/device/viewport]

Environment:
- App build or commit:
- Browser and version:
- OS or device:
- Viewport:
- Relevant test or screenshot configuration:

Location and state:
- Route or component:
- Test or snapshot label:
- Setup, data, or state required:

Steps to reproduce:
1. Start from:
2. Navigate to or do:
3. Observe:

Expected:
[What the interface should show or how it should behave.]

Actual:
[What the comparison shows, including where and how it differs.]

Evidence:
- Build or visual diff link:
- Before/after screenshots or annotated crop, if needed:

Impact:
[What content or task is impaired, and affected users or viewports if known.]

Reproducibility:
[Whether it recurs and under which environment or state.]

The fields capture the useful comparison details recommended in Percy’s workflow, with reproduction and impact details to help a team triage the issue without guessing. Adapt the template to the fields in your issue tracker.

5. Triage without overstating the finding

Route the ticket to the relevant component or team if you know it. Set priority using your team’s own criteria and the impact you can observe. The cited visual testing workflows do not prescribe a universal severity scale, so avoid presenting a local priority convention as a general rule.

If the change matches an approved design update, handle it through review and baseline approval.
If the change is unintended and repeatable, file a bug with the reproduction details and comparison evidence.
If the result changes across environments, include the environment details and investigate rendering differences before assigning a cause.
If the diff does not include enough context to reproduce the state, gather the route, data, actions, and capture configuration before treating the image as a complete report.

6. Capture supporting screenshots when needed

Use the visual testing platform’s build or diff as the primary evidence when it is available. A separate screenshot can help when the issue tracker needs an attachment, the comparison view is not accessible to the assignee, or you need to show a focused region alongside the full page. Keep the original comparison link or unaltered screenshots so a reviewer can inspect the entire changed area.

For teams choosing a visual comparison workflow, useful evaluation questions include whether it fits the current test runner and CI, whether diffs are reviewed locally or in a hosted interface, what context accompanies the screenshot, which browsers and viewports are needed, and how the team will maintain and review baselines. Playwright documents native screenshot comparisons; Chromatic documents hosted snapshot review for Playwright workflows; Percy describes build and diff review. These examples do not establish a universal best tool.

Or skip the browser setup

If you need a standalone screenshot to attach to a ticket, ScreenshotNeo is a website screenshot API and MCP server. The API takes a URL and returns an image or PDF; it is useful for supporting evidence, while your visual regression system’s build or diff remains the comparison artifact for reviewing a regression. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
with open("shot.webp", "wb") as image:
    image.write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const image = Buffer.from(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', image));

Replace the example URL with the page you want to document and provide your API key. ScreenshotNeo removes supported cookie and consent banners, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and responses identify the page verdict and billing status in headers. Its MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 shots.

Performance, reliability, and cost notes

Keep evidence focused but complete. A focused crop helps reviewers find a small defect, while the original full comparison retains surrounding context. Include both when a crop could conceal the scope.
Control environmental variation. Record the rendering environment and compare under consistent conditions when investigating a mismatch. Differences in browser, OS, or screenshot settings can affect output.
Preserve links and labels. A build or diff link plus the snapshot label makes it easier to return to the relevant result than an unlabeled image attachment.
Account for screenshot service costs separately. ScreenshotNeo bills only clean shots; the listed non-billable cases and cache hits are reported in response headers. Its free plan provides 1,000 shots monthly, and paid tiers range from $5 for 3,000 shots to $249 for 1,000,000 shots. Yearly billing gives two months free. Every listed feature is on every plan. For an issue report, use a separate capture only when it adds useful evidence.

Troubleshooting visual regression reports

Symptom	Likely cause	What to do
The diff looks different on a developer’s machine than in CI.	Browser, OS, hardware, settings, or headless rendering differs.	Include the environment details and investigate using the environment that produced the baseline.
The report cannot be reproduced from the screenshot.	The ticket omits route, state, data, actions, or test configuration.	Add a known starting point, numbered steps, snapshot label, and the needed state or data.
The ticket says “layout is broken” but reviewers disagree on what changed.	The description does not identify the affected region or expected appearance.	Describe the exact visible difference and state expected and actual separately.
The attached crop appears harmless, but the page looks wrong.	The crop omits surrounding content or layout context.	Attach or link the full comparison as well as the crop.
A baseline update clears the failure, but the cause is unknown.	The changed baseline was approved without confirming whether the UI change was intentional.	Review the design or change description and determine whether the new appearance is intended before approving the baseline.
A standalone ScreenshotNeo capture shows a bot check or blank page.	The target returned a bot check or did not produce a clean page capture.	Check the response’s `X-Page-Verdict` and `X-Billed` headers, then use the original test build or diff as evidence if the page could not be captured cleanly.

FAQ

Should every visual diff become a bug?

No. Review the changed state against the intended design or change description first. File a bug for a confirmed unintended difference.

Should I update the baseline before or after filing?

Follow your team’s review process after deciding the change is intentional. Do not use a baseline update to dismiss an unexplained difference.

Is a screenshot enough to reproduce a visual defect?

Usually not by itself. Include the route or component, state, steps, environment, and a link to the original diff when available.

Does a standalone screenshot replace a visual regression diff?

No. It can support a ticket, but the build or diff is the relevant comparison record when your visual testing platform provides one.