Best Way to Test Screenshot Output Across Chrome and Firefox
Compare Chrome and Firefox screenshots against separate, reproducible baselines. Set up Playwright Test, diagnose visual diffs, and choose a hosted review workflow.
Best practice: capture the same application state in Chrome and Firefox under controlled conditions, then compare each browser’s output to its own approved baseline. Do not use a Chrome screenshot as Firefox’s expected image: browser rendering can differ. For a code-first workflow, Playwright Test can capture screenshots and compare them with stored snapshots.
1. Why Chrome and Firefox need separate baselines
The same HTML and CSS can render differently across browsers. Font rasterization, layout details, browser versions, operating systems, and headless settings can all affect pixels. Playwright warns that screenshot output can vary with the host OS, browser version and settings, hardware, power source, and headless mode. Keep each browser’s reference separate and compare Chrome only to Chrome, and Firefox only to Firefox.
A passing test means the current capture is sufficiently close to the accepted image for that same browser under the configured comparison rules. It does not prove the page looks identical across browsers. Cross-browser review still means inspecting both browser results and deciding whether the differences are acceptable.
2. Set up a reproducible Playwright workflow
The example below uses Playwright Test with two projects. It captures the same page in Chromium and Firefox, with browser-specific snapshot names. Playwright’s screenshot assertion creates the reference on the first run; review and commit those reference images, then later runs compare against them.
Install and configure
npm init -y
npm install --save-dev @playwright/test
npx playwright install chromium firefoxAdd a test script to package.json:
{
"scripts": {
"test:visual": "playwright test"
}
}Create playwright.config.ts:
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
{ name: 'chromium', use: { browserName: 'chromium' } },
{ name: 'firefox', use: { browserName: 'firefox' } },
],
use: {
baseURL: 'http://127.0.0.1:3000',
viewport: { width: 1440, height: 900 },
deviceScaleFactor: 1,
},
});If the app is not already running in CI, configure Playwright’s webServer option to start it and wait for its ready URL. Use the same command and runtime settings when generating and checking baselines.
Write a screenshot test
Create tests/homepage.visual.spec.ts. Replace the route and readiness condition with ones that match the application. Use stable data and wait for the UI state that matters before taking the screenshot.
import { test, expect } from '@playwright/test';
test('homepage matches its browser baseline', async ({ page }) => {
await page.goto('/');
await expect(page.getByRole('heading', { name: 'Welcome' })).toBeVisible();
await expect(page).toHaveScreenshot('homepage.png', {
fullPage: true,
animations: 'disabled',
maxDiffPixelRatio: 0.001,
});
});Run npm run test:visual. On the initial run, inspect the generated snapshots before accepting them. With multiple projects, Playwright includes project identity in snapshot paths so browser references remain distinct. Commit accepted snapshots with the test code. Later runs report diffs when a capture exceeds the configured tolerance.
3. Make the capture state deterministic
A visual test is only useful when the page reaches the same meaningful state on every run. Control the parts your application owns:
- Environment: pin the Playwright dependency and browser versions, and use a consistent operating system and headless mode for baseline generation and CI comparison.
- Viewport and scale: set viewport dimensions and device scale factor explicitly. Keep them the same for both browsers unless the test intentionally covers multiple viewport configurations.
- Data: use fixtures or seeded test data. Avoid relying on changing production content, current time, randomized IDs, or external services.
- Readiness: wait for a meaningful visible condition, such as a heading or loaded component, rather than relying only on an arbitrary delay.
- Motion and changing elements: disable animations where appropriate. Playwright also supports a screenshot stylesheet through
stylePathto hide or normalize volatile content such as clocks, rotating banners, or animated decorations. - Scope: use a full-page capture for page-level regressions, or capture a locator when a component is the specific subject of the test.
Example using a locator and a stylesheet to suppress a known volatile region:
await expect(page.locator('[data-testid="checkout-summary"]')).toHaveScreenshot('checkout-summary.png', {
animations: 'disabled',
stylePath: './tests/visual-stability.css',
maxDiffPixelRatio: 0.001,
});In tests/visual-stability.css, target only elements whose changes are irrelevant to the assertion, for example a live clock. Do not broadly hide application content to make failures disappear.
4. Choose comparison tolerance deliberately
Playwright screenshot assertions offer controls such as pixel thresholds and differing-pixel limits. These are useful for small rendering noise, but a loose threshold can let a real layout regression pass. Start with strict settings in the stable environment, examine actual diffs, and adjust only when you can explain the expected variation.
When a test fails, inspect the actual image, expected image, and diff together. Determine whether the cause is an intended UI change, a browser-specific behavior, an unstable test state, or an environment change. Update a baseline only after a reviewer confirms the visual change is intended. Do not refresh all baselines just to make CI green.
5. Other visual review workflows
Choose the workflow based on where your team wants baselines and how it reviews changes. Vendor documentation describes product features; it does not establish which tool has the best accuracy or value for every team.
| Tool | Documented workflow | Consider it when |
|---|---|---|
| ScreenshotNeo | Website screenshot API and MCP server. A single GET request returns an image or PDF; its capture options include browser-style controls such as viewport, device presets, waits, cookies, headers, and CSS. | You need screenshots from an API or AI agent, or want clean captures with consent banners and other known overlays removed. |
| Playwright Test | Local screenshot assertions, image comparison, and project-specific snapshots. | You want code-first regression checks and baselines stored alongside repository tests. |
| Chromatic | Hosted visual testing with documented Chrome, Firefox, Safari, and Edge support and browser-specific baselines. | Your team wants hosted visual review and uses its supported workflow. |
| BrowserStack Percy | Hosted cross-browser snapshots, browser-specific diffs, and configurable OS/browser combinations. Its documentation says each browser screenshot counts separately toward monthly usage. | You need hosted rendering across selected browser and OS combinations; account for per-browser snapshot usage. |
For Chromatic or Percy, check their current documentation and plans for the exact browser matrix, integrations, usage limits, and pricing you need. The available feature documentation does not support a universal ranking or a claim about relative detection accuracy.
6. cURL, Python, and Node.js capture examples
These calls capture a URL through ScreenshotNeo’s API. They are useful for producing a screenshot artifact; they do not replace a browser-specific baseline test unless the capture configuration and browser behavior you need are controlled for that purpose. See the ScreenshotNeo API documentation for parameters and response headers.
cURL
curl -G "https://api.screenshotneo.com/v1/shot" \
-d access_key=YOUR_API_KEY \
--data-urlencode url=https://stripe.com \
-o shot.webpPython
import requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)Node.js
const q = new URLSearchParams({
access_key: 'YOUR_API_KEY',
url: 'https://stripe.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const bytes = new Uint8Array(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', bytes));Use an environment variable or secret store for the API key in a real application; do not commit credentials. ScreenshotNeo also supports custom CSS and JavaScript, waits, viewport and device settings, element selection, full-page capture, image formats, caching, async jobs, and bulk capture. Its response includes page-verdict and billed headers, which can help distinguish successful captures from bot checks, blank pages, failed loads, and cache hits.
7. Troubleshooting visual test failures
| Symptom | Likely cause | Fix |
|---|---|---|
| Many pixels differ on every run | Changing content, animation, timing, font loading, or inconsistent environment. | Stabilize data and readiness, disable or mask only known volatile content, and align OS, browser version, viewport, scale, and headless mode. |
| Firefox fails while Chromium passes | The browsers render differently, or the Firefox-specific baseline is missing or stale. | Inspect Firefox’s own actual and expected images. Accept a separate Firefox baseline only after reviewing the intended result. |
| Snapshot path or baseline seems wrong | Project name or configuration changed, or snapshots were generated under a different project setup. | Check the configured project and snapshot path. Preserve distinct browser projects and regenerate references only as a reviewed change. |
| Test captures a loading or empty state | The test navigated but did not wait for the application’s meaningful ready condition. | Wait for a visible element or application-specific state. Avoid arbitrary sleeps when a deterministic condition is available. |
| Small diffs cause noisy failures | Comparison settings are stricter than the stable output warrants, or rendering varies across environments. | First align the environment. Then inspect diffs and make a narrowly justified threshold adjustment. |
| Hosted snapshots consume more usage than expected | Cross-browser captures can count per browser; Percy documents each browser screenshot as a separate unit. | Choose only the browser and OS combinations needed for the test and review the vendor’s current usage terms. |
| Screenshot API output shows a challenge or blank page | The target may show bot checks, return an empty page, or fail to load. | Check the page-verdict and billed response headers, verify the URL and access requirements, and retry only when the underlying condition is transient. |
8. Performance, reliability, and cost
Visual test runtime grows with the number of pages, states, browsers, and viewport combinations you capture. Prioritize important flows and components, and run broad matrices where they fit your CI budget. Reusing the same deterministic environment makes results easier to interpret and avoids spending review time on environment-only diffs.
For reliability, treat accepted snapshots as reviewed test assets: version them, keep the capture configuration stable, and make baseline changes part of code review. A browser update or OS change can affect rendering, so upgrade deliberately and review resulting diffs instead of silently replacing references.
For a hosted workflow, compare the current plan and usage model directly with the number of browser captures your suite produces. Percy’s documentation specifies that each browser screenshot counts separately. No universal price comparison or performance benchmark is established here.
9. Or skip the browser setup
ScreenshotNeo can return an image with one GET request. Use your own browser tests when you need explicit Chrome-versus-Firefox baselines; use the API when you need a clean website capture without maintaining capture infrastructure.
curl -G "https://api.screenshotneo.com/v1/shot" \
-d access_key=YOUR_API_KEY \
--data-urlencode url=https://stripe.com \
-o shot.webpScreenshotNeo removes cookie banners, popups, and chat widgets before the shot. Bot checks, blank pages, and failed loads are never billed, and response headers report the page verdict and billing status. Its MCP server gives AI agents tools to take screenshots, inspect page information, and capture PDFs. The free plan includes 1,000 screenshots each month with no card; paid plans start at $5 for 3,000 shots. See the API docs for options, then sign up free.
10. FAQ
Should I use the same image as the expected result in both browsers?
No. Maintain an accepted reference for each browser and compare each run to its matching reference.
Does a passing screenshot test mean the two browsers look identical?
No. It means each capture passed its configured comparison against that browser’s baseline.
When should I use a hosted visual testing service?
Use one when hosted review, browser coverage, or team workflows are a better fit than repository-managed Playwright snapshots. Confirm current browser support and usage terms in the vendor’s documentation.
Can an API screenshot replace cross-browser regression testing?
Not by itself. A single API capture is useful for obtaining an image, but cross-browser regression checks require captures and references for the browsers you intend to validate.
