How to Run Website Screenshot Change Checks in GitHub Actions
Set up Playwright screenshot comparisons in GitHub Actions, keep visual baselines stable, and review changes without hiding real regressions.
Use Playwright Test’s built-in toHaveScreenshot() assertion to check website screenshots in GitHub Actions. The first run creates a reference image; later runs compare new captures against it. Keep the browser and operating-system environment consistent, review initial references before committing them, and update baselines deliberately when a visual change is expected. These checks complement functional tests: a matching screenshot does not prove that the page behaves correctly.
1. Add Playwright screenshot assertions
Choose representative pages and states that matter to users: for example, a landing page, a signed-in dashboard, or a form with validation shown. Make the test site deterministic enough to capture. The example below assumes your app is available at http://127.0.0.1:3000 and that Playwright Test is installed and configured in the repository.
import { test, expect } from '@playwright/test';
test('home page matches its visual baseline', async ({ page }) => {
await page.goto('http://127.0.0.1:3000');
await expect(page).toHaveScreenshot('home.png');
});Run the test once to create the reference image, then inspect it as the expected output before keeping it. Commit reviewed reference files with the test project so CI can compare future captures against them. Playwright documents the assertion and reference-image workflow in its visual comparisons guide.
If your Playwright project already uses baseURL, prefer a relative path so the test can run against the configured site:
await page.goto('/');
await expect(page).toHaveScreenshot('home.png');For an element-focused check, pass a locator to the screenshot assertion:
await expect(page.locator('[data-testid="checkout-summary"]'))
.toHaveScreenshot('checkout-summary.png');Use stable selectors such as test IDs where available. A whole-page capture can include unrelated content, while an element capture narrows the check to a component or region.
2. Make captures repeatable
Screenshot output depends on more than the page’s CSS. Playwright warns that host operating system, browser version and settings, hardware, power source, and headless mode can affect rendering. Run the baseline and CI comparison in the same browser and operating-system environment wherever possible. See Playwright’s notes on rendering differences and its CI setup guidance.
- Use a predictable test page. Seed test data and avoid depending on changing production content.
- Wait for the state you mean to capture. Wait for a page-specific locator or application-ready signal before taking the screenshot.
- Control animation and time-dependent content. Disable or stabilize animations, clocks, rotating banners, and randomly generated values when they are not the subject of the test.
- Keep fonts and assets available. A missing font or image can alter layout and produce widespread pixel differences.
- Use the same browser setup for baseline updates and CI. A baseline generated on a different platform may produce noise even when the site is unchanged.
When tests use a local server, configure the Playwright web server so the test runner starts it consistently. Adapt the command to your project’s existing scripts:
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
baseURL: 'http://127.0.0.1:3000',
},
webServer: {
command: 'npm run start -- --host 127.0.0.1',
url: 'http://127.0.0.1:3000',
reuseExistingServer: !process.env.CI,
},
});Adjust the server command and host syntax to match your framework. If the site is deployed in a preview environment instead, pass that stable preview URL through configuration and ensure it is ready before the tests begin.
3. Run the checks in GitHub Actions
Add a workflow that checks out the repository, sets up the runtime, installs dependencies and the browser needed by your project, then runs the Playwright test command. This example assumes an npm project and a test script named test:e2e; adapt the Node version and package-manager commands to your repository.
name: Website screenshot checks
on:
pull_request:
push:
branches: [main]
jobs:
visual:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- name: Install dependencies
run: npm ci
- name: Install Playwright browser
run: npx playwright install --with-deps chromium
- name: Run screenshot checks
run: npm run test:e2ePinning action versions and choosing a runtime version are repository maintenance decisions; align them with your project’s supported setup. If your test command invokes Playwright directly, it can instead be npx playwright test. Use the browser installation command that matches the browsers declared by your Playwright project. The official Playwright CI guide covers the setup pattern.
On a pull request, a mismatch should fail the job so the visual change is visible during review. Save Playwright’s test report or failure artifacts using your repository’s artifact workflow if reviewers need to inspect them; configure artifact retention according to your team’s needs.
4. Review and update visual baselines
A reference image is part of the test’s expected output. When a design change is intentional, regenerate references using Playwright’s update option:
npx playwright test --update-snapshotsInspect the changed images, make sure the changes match the intended design, and commit the new baselines with the related code change. Do not update references just to make a failing check pass: that can encode an accidental regression as the new expectation. Playwright documents --update-snapshots and screenshot comparison options in its snapshot documentation.
5. Set comparison tolerance carefully
Playwright supports options including threshold and maxDiffPixels to control acceptable image differences. Set them only after reviewing representative diffs. A larger tolerance can reduce noise, but it can also allow a real visual regression through.
await expect(page).toHaveScreenshot('home.png', {
threshold: 0.2,
maxDiffPixels: 100,
});The values here are illustrative starting points, not universal recommendations. Choose values for the scale and stability of your own pages, and keep the compared area as focused as practical. Refer to Playwright’s documented screenshot options for current semantics and defaults.
6. Choose local comparisons or hosted review
| Approach | What it provides | Questions to consider |
|---|---|---|
| Playwright screenshot assertions | Captures and compares against reference images in the test project; references can be updated with --update-snapshots. |
How will you review and commit baseline changes? Do you want local control of the reference files? |
| Percy with Playwright | Percy documents a Playwright client workflow for uploading snapshots and an optional visual-change gate. | Do you want a hosted review workflow and a visual verdict separate from local assertions? Check current integration behavior. |
| Chromatic | Chromatic documents GitHub Actions automation and Playwright visual testing. | Does its documented workflow fit your existing component and Playwright setup? Check current service terms. |
See the official documentation for Percy’s Playwright client, its snapshot workflow, Chromatic with GitHub Actions, and Chromatic’s Playwright setup. The available research establishes these hosted workflows, not a universal accuracy, ease, or price advantage. Check the vendors’ current documentation for product details before choosing.
Or skip the browser setup
If the goal is to capture a page and inspect its visual output, ScreenshotNeo provides a website screenshot API and MCP server. Its request returns an image or PDF, while Playwright assertions above are designed to compare captures against committed reference images.
curl -G "https://api.screenshotneo.com/v1/shot" \
-d access_key=YOUR_API_KEY \
--data-urlencode url=https://stripe.com \
-o shot.webpSee the ScreenshotNeo API documentation for request options. Python equivalent:
import requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
timeout=90,
)
open("shot.webp", "wb").write(r.content)Node.js equivalent:
const q = new URLSearchParams({
access_key: 'YOUR_API_KEY',
url: 'https://stripe.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const bytes = new Uint8Array(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', bytes));- Cookie and consent banners, newsletter popups, and chat widgets are removed before capture; each cleanup step can be turned off.
- Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed. Response headers report the page verdict and billing status.
- An MCP server provides
take_screenshot,get_page_info, andcapture_pdftools for AI agents and MCP clients. - The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.
Learn about ScreenshotNeo or sign up free for 1,000 screenshots a month, no card required.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Screenshot differs across runs without a code change | Unstable content, animation, time-dependent values, fonts, or a changed rendering environment. | Stabilize the page state and assets, disable irrelevant motion, and keep browser and operating-system conditions aligned. |
| Many tests fail after moving from a local machine to CI | The host OS, browser version, settings, or headless rendering differs from the baseline environment. | Regenerate and review baselines in the CI-like environment, then keep future baseline updates and CI runs consistent. |
| Page is blank or incomplete in the capture | The app server was not ready, navigation failed, or the test captured before the target state loaded. | Check the server URL and startup command, wait for the expected page element, and inspect the Playwright error output. |
| Text wraps differently or sections shift | A web font did not load, viewport dimensions differ, or content changed. | Ensure fonts are available before capture, use a consistent viewport, and seed deterministic content. |
| Screenshot assertion reports a small pixel diff repeatedly | Minor rendering variation exceeds the configured threshold. | First identify the changing pixels and their source. If they are understood and immaterial, tune threshold or maxDiffPixels narrowly. |
| Intentional design update still fails | The committed reference image still represents the old design. | Run npx playwright test --update-snapshots, review the regenerated images, and commit the approved references. |
Performance, reliability, and cost
Visual checks add browser work to CI, and runtime depends on the number and complexity of pages, browser setup, and how tests are organized. Start with a small set of representative pages, then expand where the checks catch meaningful regressions. Reuse the project’s normal dependency and browser installation workflow, and investigate slow tests before adding broad parallelism that might make a shared test site less stable.
Reliability comes primarily from repeatable inputs and rendering conditions: predictable test data, a ready server, stable assets, and reviewed baselines. Screenshot comparisons are a visual signal, not a substitute for assertions about links, forms, accessibility, or application behavior.
The research sources do not establish current Percy or Chromatic pricing, nor a comparative cost benchmark. Review their current terms directly when evaluating hosted review. ScreenshotNeo’s stated plans are free for 1,000 shots per month with no card, then $5 for 3,000, $15 for 15,000, $39 for 60,000, $99 for 250,000, and $249 for 1,000,000; yearly billing gives two months free, and every feature is on every plan. ScreenshotNeo charges only for clean shots, with verdict and billing headers on each response. These API captures do not by themselves provide Playwright’s committed-baseline comparison workflow.
FAQ
Does a screenshot check prove the page works?
No. It detects rendered visual differences. Keep functional assertions for behavior such as navigation, form submission, and validation.
Should references be generated locally or in CI?
Use an environment that matches the one doing comparisons as closely as practical. Whatever environment you choose, review the images and use it consistently for updates and CI.
Can I test only one component instead of an entire page?
Yes. Pass a locator to toHaveScreenshot() to compare a specific element, which can reduce unrelated page changes in the diff.
When should I choose a hosted visual testing service?
Consider Percy or Chromatic when your team wants the hosted workflow their documentation describes. Compare the current integrations and service terms against your review process and existing test setup.
