How to Set Up Screenshot Change Detection for a SaaS Documentation Site

Use Playwright Test’s toHaveScreenshot() assertion to capture representative documentation pages and compare them with reviewed reference images. The first run creates the references; commit those images after reviewing them. Later runs compare new captures with the committed baselines and fail when the difference exceeds the configured tolerance. Run the same browser and operating environment locally and in CI so rendering differences do not overwhelm real changes. Playwright visual comparisons documents the assertion, baseline workflow, and comparison options.

Visual checks catch unintended layout and styling changes. They do not replace functional tests, accessibility checks, or checks that documentation content is correct.

1. Choose a useful set of pages and states

Begin with a small set of pages that cover distinct documentation layouts and high-value interactions. For a SaaS documentation site, a practical starting set is:

The documentation landing page.
A typical article with headings, links, and code samples.
A long article with a table of contents, callouts, or tables.
Navigation in its normal desktop state and, if important, its expanded or collapsed state.
A search results page with fixed query and fixture results.
A narrow viewport to expose responsive navigation, wrapping, and overflow problems.

Prefer representative pages over a screenshot of every route. Add coverage when a layout, component, or user state is unique. Keep route data stable: use fixed test content, a controlled account, and known search results rather than production data that changes between runs.

2. Install Playwright Test

In the repository that serves the documentation, install the test runner and initialize its configuration. If the site already uses Playwright, keep its existing version and configuration instead of initializing a second setup.

npm install --save-dev @playwright/test
npx playwright install

For Linux CI, install the browser dependencies as well, using npx playwright install --with-deps. Playwright’s CI guide shows this setup in its GitHub Actions example.

3. Add a stable configuration and screenshot test

Set a predictable base URL, viewport, and browser project. The example below uses Chromium and a desktop viewport. Replace the URL and readiness condition with values that fit your site. It also adds a mobile project so the same test can catch responsive regressions.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

const baseURL = process.env.DOCS_BASE_URL ?? 'http://127.0.0.1:3000';

export default defineConfig({
  testDir: './tests/visual',
  fullyParallel: true,
  forbidOnly: Boolean(process.env.CI),
  retries: process.env.CI ? 2 : 0,
  reporter: process.env.CI ? [['html', { open: 'never' }]] : 'list',
  use: {
    baseURL,
    locale: 'en-US',
    timezoneId: 'UTC',
    colorScheme: 'light',
    reducedMotion: 'reduce',
    screenshot: 'only-on-failure',
    trace: 'retain-on-failure',
  },
  expect: {
    toHaveScreenshot: {
      animations: 'disabled',
      caret: 'hide',
      // Start strict. Add a tolerance only after removing known instability.
      maxDiffPixelRatio: 0.001,
    },
  },
  projects: [
    {
      name: 'chromium-desktop',
      use: { ...devices['Desktop Chrome'], viewport: { width: 1440, height: 1000 } },
    },
    {
      name: 'chromium-mobile',
      use: { ...devices['iPhone 13'] },
    },
  ],
});

Playwright snapshot names include browser or project context because browser and platform rendering can differ. Keep baseline generation and comparison on the same browser version, OS image, fonts, and relevant settings. The current defaults and options are documented under visual comparisons; confirm option availability against the Playwright version pinned in your lockfile.

Create a test that visits representative routes, waits for meaningful content, and captures a screenshot:

// tests/visual/docs.spec.ts
import { test, expect } from '@playwright/test';

test('documentation landing page', async ({ page }) => {
  await page.goto('/', { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
  await expect(page).toHaveScreenshot('docs-home.png', { fullPage: true });
});

test('getting started article', async ({ page }) => {
  await page.goto('/getting-started', { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page.getByRole('heading', { name: 'Getting started' })).toBeVisible();
  await expect(page).toHaveScreenshot('getting-started.png', { fullPage: true });
});

test('search results for a fixed query', async ({ page }) => {
  await page.goto('/search?q=authentication', { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page.getByRole('heading', { name: /search/i })).toBeVisible();
  await expect(page).toHaveScreenshot('search-authentication.png', { fullPage: true });
});

These are patterns, not assumptions about a particular docs framework. Match route paths and accessible names to your site. If search is client-rendered, wait for its results container or a stable result count rather than relying only on navigation completion.

Pick the right screenshot scope

await expect(page).toHaveScreenshot('article.png') captures the visible viewport.
{ fullPage: true } captures the full scrollable page. This can expose missing sections and page-length changes, but long pages cost more time to capture and compare.
await expect(page.getByRole('navigation')).toHaveScreenshot('navigation.png') can isolate a component whose full-page context is too noisy.
Use a named file for each meaningful state. Avoid ambiguous generated names when a test has multiple captures.

4. Generate, inspect, and commit the first baselines

Run the visual tests in the same environment you intend to use for CI. On the first run, Playwright writes reference screenshots and reports that a snapshot did not exist. Inspect those images at normal size and zoom in on relevant details. Confirm that the capture shows the intended route, viewport, fonts, assets, and state; then commit the snapshot directory with the test.

npx playwright test tests/visual

Do not treat first-run output as automatically approved. A baseline encodes what future runs will consider expected, so review it like a code change.

5. Make captures repeatable

Visual tests are sensitive to more than application code. Playwright notes that screenshots may vary with the host operating system, browser version, settings, hardware, power state, and headless mode. Keep the capture environment consistent and remove avoidable page variability.

Control what the page renders

Use fixed fixtures and deterministic route state. Freeze dates or replace live API responses when their content changes over time.
Wait for the content you care about, such as the article heading, navigation, code block, or search results. Avoid fixed sleeps as the only readiness check.
Disable animations and hide the caret. If a page has known volatile elements such as a rotating announcement or current-time badge, use a screenshot stylesheet to hide or stabilize them.
Be careful with fonts and external assets. Load them in the same way every run; missing fonts or images can shift page layout.
Set a consistent locale, time zone, color scheme, and viewport when these affect rendered content.
Test hover, expanded navigation, or other interaction states deliberately. Move the pointer away or reset the state before capturing a different screenshot.

For example, a screenshot stylesheet can hide a timestamp whose exact value is not under test:

/* tests/visual/screenshot.css */
.visual-test-clock,
.rotating-announcement {
  visibility: hidden !important;
}

// Add to an assertion when appropriate
await expect(page).toHaveScreenshot('article.png', {
  fullPage: true,
  stylePath: './tests/visual/screenshot.css',
});

Use this sparingly: hiding a large region can conceal a genuine regression. Playwright’s stylePath option applies a stylesheet while capturing and is intended for filtering volatile content. Its screenshot assertion also retries captures until consecutive screenshots match, which helps with transient rendering but cannot make genuinely nondeterministic page data stable. See the visual comparison options.

Thresholds and masks

Start with the default comparison behavior or a very small, explicit tolerance. Playwright offers options such as maxDiffPixels, maxDiffPixelRatio, and threshold; their defaults can also be set in configuration. A tolerance is not a fix for instability. First identify whether the changed pixels come from a real UI change, environment drift, dynamic content, or antialiasing. Increase tolerance only when the remaining variation is understood and does not hide changes that matter.

6. Review intentional changes and update baselines

When a design or content change is intentional, regenerate the references explicitly:

npx playwright test --update-snapshots

Inspect the resulting image diffs and commit the approved snapshots with the implementation change. Do not run this flag automatically in CI: that would turn unexpected regressions into new expected output without review. Playwright stores screenshots in snapshot directories associated with test files and recommends committing and reviewing them. See updating screenshots.

7. Run screenshot checks in CI

Run the suite on pull requests so reviewers can inspect visual changes before merge. Use a pinned Node version, lockfile install, and the same Playwright browser environment used to create baselines. The example below runs the suite on pushes and pull requests and retains the HTML report when available.

# .github/workflows/playwright.yml
name: Playwright visual checks

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  visual:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npm run build
      - run: npm run start -- --port 3000 &
      - run: npx playwright test tests/visual
        env:
          DOCS_BASE_URL: http://127.0.0.1:3000
      - uses: actions/upload-artifact@v5
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Adapt the start command to your framework. A more reliable pattern is to configure Playwright’s webServer option so it starts the documentation server and waits for its URL before running tests. If testing a pull-request preview instead, wait until deployment is ready and set DOCS_BASE_URL to that preview URL. Keep baseline generation on the same pinned environment as the CI comparison job.

Playwright’s CI documentation includes GitHub Actions, browser dependency installation, report artifacts, containers, and deployment-triggered testing. Its guide currently advises against browser binary caching by default because cache restoration can take about as long as downloading, while system dependencies still need installation; check the current guide when tuning your workflow.

8. Decide whether local snapshots are enough

Committed Playwright snapshots work well when your team is comfortable reviewing image diffs in pull requests and maintaining the baseline files. An optional hosted service can help when you want centralized visual review or broader browser and responsive-width capture.

Decision	Playwright snapshots	Percy with Playwright
Where review happens	In test output and repository image diffs	In Percy’s hosted build and review flow
Baseline handling	Snapshot files are stored with the project and updated in reviewed commits	Snapshots are compared with approved hosted baselines
CI result	Native screenshot assertion can fail when a comparison differs	The visual verdict is reviewed in Percy; configure a gate if unapproved changes must fail CI
Coverage	Configured Playwright browser and viewport projects	Percy documents browser and responsive-width captures
Operational work	Maintain stable environment and committed image files	Add vendor setup, token handling, review flow, and program terms to verify

Percy’s Playwright integration documentation describes capturing snapshots and routing existing toHaveScreenshot() assertions through its integration. In that drop-in flow, the local assertion passes and visual review moves to Percy; it does not mean the page had no visual changes. Configure the documented reporter gate if CI must fail on changes that remain unapproved. Verify current product behavior, access requirements, and pricing directly before adopting it; those details are not established here. Percy is optional, not required to get started with screenshot change detection.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. A single request captures a URL as PNG, JPEG, WebP, or PDF. For a baseline workflow, call it from your own job and store or compare the returned image using the visual-diff system you choose; it does not replace Playwright’s test assertions or baseline approval process. See the ScreenshotNeo API documentation.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(({ writeFile }) =>
  writeFile('shot.webp', Buffer.from(await res.arrayBuffer()))
);

In an actual project, replace the example URL with your docs URL and keep the API key in a CI secret rather than in source control. ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and the response identifies page verdict and billing status in headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The Free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots, and every feature is on every plan. Sign up for 1,000 free screenshots a month with no card.

Troubleshooting

Symptom	Likely cause	What to do
Missing snapshot on the first run	No reference image exists yet	Inspect the generated screenshot, then commit the approved snapshot directory.
Many pixels change on every run	Dynamic content, animation, incomplete readiness, or unstable data	Use fixed fixtures, wait for a visible content condition, disable animation, and hide only understood volatile regions with `stylePath`.
Snapshots differ only in CI	Different OS, browser build, fonts, headless mode, or rendering settings	Generate and compare baselines in the same pinned environment; consider using the same Playwright container image in both places.
Screenshot captures a loading or empty page	The test navigated before the page’s meaningful content was ready, or the server was not available	Wait for the site server to be ready, assert on a route-specific element, and verify the test base URL.
Mobile test shows a desktop layout	The project did not use a mobile viewport/device configuration, or the page uses a fixed-width layout	Check the project’s device settings and inspect responsive CSS at the captured viewport.
A tiny rendering variation fails the test	Antialiasing or remaining environment variation	First align environment and assets. If the remaining pixel differences are harmless and understood, tune `threshold` or a diff limit carefully.
Updating snapshots makes failures disappear	New references were accepted without reviewing whether the change was intended	Inspect every updated baseline diff and commit only expected changes.
Percy review shows changes while Playwright passes	The integration moves the visual verdict into Percy’s review flow	Review and approve the Percy build; configure the supported gate if unapproved changes should block CI.

Performance, reliability, and cost

Performance

Start with a few representative routes and one or two important viewport sizes. Full-page captures and additional browser projects increase the amount of browser work and image comparison, so add them where they cover a real regression risk. Parallel Playwright workers can reduce elapsed time, but avoid overloading the docs server or making shared test fixtures race with each other. A stable setup is usually more useful than maximizing the number of screenshots.

Reliability

Pin the Playwright dependency through the lockfile and use the matching browser installation in CI. Keep capture OS, browser, fonts, viewport, locale, and site data consistent. Use retry behavior to gather a clear failure report, not to excuse flaky output; repeated retries that pass intermittently usually point to nondeterministic content or setup. Save the HTML report and failure artifacts so reviewers can inspect the actual capture.

Cost

Local Playwright snapshots use repository storage and CI compute; the test setup itself does not require a hosted visual-review service. CI consumption grows with routes, browser projects, screenshot size, and retry count. Percy adds a hosted service and its own account, token, workflow, and program terms; verify current pricing and limits with the vendor. Choose coverage based on the cost of missing a regression, not on an arbitrary screenshot count.

FAQ

Do screenshot tests prove the docs are accessible?

No. A screenshot is a visual rendering. Test keyboard behavior, semantic structure, and accessibility separately.

Should every documentation URL get a baseline?

Usually not at first. Cover distinct templates and meaningful states, then expand when a route has unique layout or risk.

Can I use Playwright screenshots without the Playwright Test runner?

The toHaveScreenshot() assertion is part of Playwright Test. For other runners, use their screenshot and comparison mechanism or use a hosted integration that supports your setup.

When should a baseline be changed?

When the corresponding UI change is intentional and someone has reviewed the new capture. Keep the updated reference with the implementation change that explains it.