How to Trigger Visual Regression Tests on Changes

Trigger visual regression tests from your CI system when a pull request is opened or updated, and on pushes to branches where you want direct or post-merge coverage. A typical job checks out the change, installs dependencies and the browser runtime, runs the visual test suite, then publishes a report or sends the result to a visual review service.

This guide uses Playwright Test and GitHub Actions. The same event choices and review decisions apply to other CI systems, but commands and artifact configuration will differ. See the Playwright CI guide for its official workflow examples.

1. Decide which changes should trigger a run

For feedback before merge, use the pull_request event. Add push when you also need runs for direct pushes or after changes land on selected branches. Filter the events to match your branch policy; the example below runs pull request checks targeting main and push checks on main.

Choose the event scope deliberately:

Pull requests: show visual changes to reviewers while the proposed change is still under review.
Pushes to protected branches: provide a post-merge run or cover direct pushes if your workflow permits them.
Both: gives pre-merge feedback and branch-level coverage, at the cost of running the suite for both events.

For pull requests from forks, check your repository’s permissions and secret handling before adding services that need credentials. The workflow below does not require a visual service token.

2. Add a GitHub Actions workflow

Save this as .github/workflows/visual-regression.yml. It assumes the repository has a committed Playwright configuration and tests, and that npm ci can install the project from its lockfile.

name: Visual regression

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  visual:
    timeout-minutes: 20
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm

      - name: Install project dependencies
        run: npm ci

      - name: Install Playwright browsers and operating-system dependencies
        run: npx playwright install --with-deps

      - name: Run visual tests
        run: npx playwright test

      - name: Upload HTML report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          if-no-files-found: ignore
          retention-days: 14

The workflow uses the standard Playwright test command and uploads the HTML report as an artifact. Playwright’s CI documentation covers browser installation, execution, and report artifacts; use its instructions to adapt the setup to your runner and project. Playwright: Continuous Integration

Make sure the project produces an HTML report

If your project does not already configure the HTML reporter, add it in playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['html', { open: 'never' }]],
});

Keep your existing configuration for projects, snapshot paths, retries, and other reporters; merge this reporter setting rather than replacing settings the suite depends on. Playwright Test needs actual screenshot assertions or visual tests in the project for this job to detect visual changes. The workflow only schedules and runs those tests.

3. Ensure the tests compare meaningful screenshots

A CI trigger does not create a visual baseline by itself. Your suite must capture pages or components and compare them against approved baseline images. With Playwright Test, a minimal visual assertion looks like this:

import { test, expect } from '@playwright/test';

test('home page matches its visual baseline', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot('home.png', {
    fullPage: true,
  });
});

This example assumes the app’s base URL is configured for the test environment and that the project has established the corresponding baseline. Review Playwright’s screenshot assertion documentation for its snapshot workflow and assertion options: Visual comparisons.

Keep the URL, test data, fonts, viewport, browser version, and rendering environment stable between baseline creation and CI runs.
Wait for the page state that matters to the test before capturing. Avoid timing assertions against transient loading or animation states.
Use a deliberate baseline update process. A changed screenshot should be reviewed before its baseline is accepted.
Prefer focused component or page checks that identify the source of a difference, alongside broader coverage where appropriate.

4. Choose between a full suite and changed-test selection

Running npx playwright test executes the configured suite. For a faster preliminary pass, Playwright also documents --only-changed, which uses the test dependency graph to select tests likely affected by a changeset:

npx playwright test --only-changed=main

Use the branch name appropriate to your comparison. Playwright describes this selection as a heuristic that can miss tests. Treat it as early feedback, then run the full suite when complete coverage matters. See Playwright test CLI filtering.

5. Decide how results affect pull requests

There are two separate decisions: how a visual difference is reported, and whether it blocks merging.

Approach	What reviewers get	When it fits
CI report artifact	A downloadable Playwright HTML report attached to the workflow run	Teams that want a simple record and can review results in CI
Visual review service	A service workflow that presents visual changes in the pull request	Teams that want baseline review and change feedback integrated into PRs
Failing CI gate	A failed status when the configured check detects a change	Teams that have established which changes should block merging and how baselines are approved

Playwright’s report artifact is one way to expose results. Chromatic documents CI and GitHub Actions automation for visual testing, including pull request feedback: Chromatic CI, Chromatic with GitHub Actions, and Chromatic Playwright setup. Percy documents routing Playwright screenshot assertions through its client and an optional reporter gate; verify the current behavior and configuration in its documentation before making it a required check: Percy Playwright client.

A detected difference is evidence for review, not automatically a defect. Decide whether your policy should fail on any unapproved difference, allow an explicit baseline approval process, or publish the result without blocking. Make that policy visible to contributors so they know how to resolve a failure.

6. Keep the browser environment reproducible

Visual tests are sensitive to environment differences. Install the project dependencies and the browser binaries expected by the tests on the CI worker. Playwright’s CI documentation also describes using a container to provide a consistent environment, which can help when screenshot comparisons vary across operating systems. Playwright CI environment guidance

Pin dependencies: use a committed lockfile and npm ci so CI installs the locked dependency set.
Install browser dependencies: use Playwright’s browser installation command for the browsers configured in the project.
Keep environments aligned: create and update baselines using the same browser and operating-system environment used by CI when possible.
Control test inputs: stabilize test data, external dependencies, locale, timezone, and app state where those affect rendering.
Use containers when useful: a consistent container image can reduce differences between runner environments; account for image maintenance and browser compatibility.

7. Common failures and fixes

Symptom	Likely cause	Fix
Playwright reports that a browser executable is missing	The browser binary was not installed on the runner, or the installed browser does not match the Playwright package.	Install browsers in the job with `npx playwright install --with-deps`, after installing project dependencies.
Tests pass locally but fail in CI with screenshot differences	The operating system, browser version, fonts, viewport, data, or rendering state differs.	Align the browser and environment, use stable data and page state, and review a new baseline only after confirming the intended rendering changed.
The report artifact is missing	The reporter did not produce files, the configured output directory differs, or the test process was cancelled before report generation.	Check the Playwright reporter configuration and output path. Preserve the upload step’s `if` condition so it can upload after a test failure.
The workflow never runs for a pull request	The target branch does not match the workflow filter, or the file is not on the branch/event path GitHub evaluates.	Check the event name and branch filters against the repository’s actual target branch.
Visual checks pass but a changed page was not covered	The affected test was not in the suite, or changed-test selection did not choose it.	Add or correct coverage and use the full suite for complete coverage; treat `--only-changed` as a heuristic.
CI is slow or times out	Browser installation, app startup, or the full visual suite exceeds the job’s time budget.	Inspect which stage consumes time, keep test setup deterministic, and use changed-test selection only as a preliminary pass. Set the job timeout to match actual suite needs.
A service integration cannot report on forked pull requests	Credentials may not be available to workflows from forks under the repository’s security settings.	Review the service’s current CI and token guidance and choose a safe workflow for external contributions; do not expose secrets to untrusted code.

8. Performance, reliability, and cost

CI runtime depends on the project’s test count, browser setup, app startup, and environment. The supplied sources do not establish a universal runtime or cost figure, so measure your own workflow. Browser installation and a full suite add work to each triggered run; using both push and pull request triggers can produce duplicate coverage for a change.

Reduce wasted runs: scope triggers to branches where the result is useful and avoid redundant events if your merge policy does not need them.
Preserve debuggability: upload reports after failed runs where possible, and retain enough artifacts for contributors to diagnose failures.
Keep the baseline trustworthy: review and approve changes intentionally rather than automatically replacing expected screenshots after every failure.
Plan service use separately: if you adopt a managed review service, check its current terms, integrations, and configuration in its official docs. The cited sources establish integration patterns, not pricing.
Balance speed with coverage: a selective pass can provide earlier feedback, but it does not prove that every affected visual behavior was tested.

Or skip the browser setup

If you need a screenshot of a page in a script or CI step without installing and managing a browser, ScreenshotNeo provides a website screenshot API and MCP server. A screenshot API call can capture a page, but it does not replace a visual regression suite’s baseline comparison and review policy.

cURL:

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js:

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

See the ScreenshotNeo API documentation for request options. Cookie banners, popups, and chat widgets are removed before the shot; bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card, and paid plans start at $5 for 3,000.

FAQ

Should visual tests run on every commit?

Run them on the events that provide useful feedback for your branch and review process. Pull requests are the key trigger for pre-merge feedback; add pushes where direct or post-merge coverage is needed.

Can screenshot capture alone detect a regression?

No. Capture produces an image. A visual regression check also needs an expected baseline, a comparison, and a way to review or gate detected differences.

Does a passing changed-test run guarantee full visual coverage?

No. Playwright documents changed-test selection as a heuristic that can miss tests. Run the full suite when complete coverage matters.

Where should I keep screenshot baselines?

Use the baseline workflow supported by your chosen test runner or visual review service, and make baseline updates reviewable alongside the change. Keep the rendering environment consistent so comparisons remain useful.