How to Run Visual Regression Tests for an Indian Government Website

Use Playwright Test’s toHaveScreenshot() assertion to capture important page states, compare them with reviewed reference images, and inspect differences in CI. Keep baselines tied to a controlled browser and operating-system environment, then test accessibility and GIGW requirements separately: a pixel comparison can reveal appearance changes, but cannot establish accessibility conformance or GIGW certification.

GIGW applies to Indian government websites and apps at central, state, district, and local levels, and its standards include WCAG 2.1. The GIGW 3.0 feature summary identifies WCAG 2.1 Level AA. See the [GIGW scope and objective](https://guidelines.india.gov.in/scope-and-objective/) and [official GIGW resources](https://guidelines.india.gov.in/). Playwright’s visual comparison guide explains baseline creation and the rendering-environment caveat: [Visual comparisons](https://playwright.dev/docs/test-snapshots).

1. Choose journeys and page states

Start with representative tasks a resident must complete, rather than attempting to snapshot every URL. Include high-value templates and states that could block a task:

• Homepage and primary navigation.
• Service or scheme detail page.
• Search results, including no-results and error states.
• Instructions and application forms, including validation errors.
• Confirmation or receipt screens, using synthetic data.
• Key language variants if the site publishes more than one language.

Make each state repeatable. Seed or stub data where practical, choose a fixed locale and timezone, and wait for meaningful content rather than an arbitrary long delay. Avoid committing screenshots containing real personal or citizen data. This journey selection is a practical testing approach; GIGW describes user-centric goals but does not prescribe a particular screenshot-test count.

2. Define a supported environment matrix

GIGW recommends testing across browsers and versions, operating systems, connection speeds, and screen resolutions. Translate that broad advice into a documented matrix based on supported environments and important service journeys. Start with the browser and operating-system combinations your team supports, plus representative desktop and mobile widths. Add slower network conditions when they affect a critical flow. See the [GIGW guidelines](https://guidelines.india.gov.in/guidelines/).

Do not treat images from different renderers as interchangeable baselines. Browser version, operating system, fonts, rendering settings, hardware, power conditions, and headless mode can affect pixels. Keep baseline creation and CI on the same pinned environment. If you intentionally test multiple Playwright projects or platforms, maintain and review separate baselines for them.

3. Install Playwright Test and add a baseline

For an existing Node.js project, install Playwright Test and its browser binaries:

npm install --save-dev @playwright/test
npx playwright install

Create playwright.config.ts. Set an explicit base URL and viewport so local runs and CI use the same target dimensions:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: {
    baseURL: process.env.BASE_URL ?? 'http://127.0.0.1:3000',
    browserName: 'chromium',
    viewport: { width: 1365, height: 900 },
    locale: 'en-IN',
    timezoneId: 'Asia/Kolkata',
    colorScheme: 'light',
    trace: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium-desktop' },
  ],
});

Then create tests/service-page.visual.spec.ts. Replace the example route and locator with stable selectors from your application:

import { test, expect } from '@playwright/test';

test('service detail page matches its visual reference', async ({ page }) => {
  await page.goto('/services/example');
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page.getByRole('heading', { name: 'Example service' })).toBeVisible();
  await expect(page).toHaveScreenshot('service-detail.png', {
    fullPage: true,
  });
});

Run it once with npx playwright test. On the first run Playwright creates a reference image; review it as an expected state before committing it. The test file’s snapshot directory should be checked into version control so a code change and its intended visual change can be reviewed together. Later runs compare the new capture to that reference.

4. Add responsive and browser coverage intentionally

Use explicit projects for materially different viewports or browser engines. Each project can produce a distinct expected image. Keep the matrix small enough that failures can be triaged, then expand it when audience or support requirements justify it.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: {
    baseURL: process.env.BASE_URL ?? 'http://127.0.0.1:3000',
    locale: 'en-IN',
    timezoneId: 'Asia/Kolkata',
  },
  projects: [
    {
      name: 'chromium-desktop',
      use: { browserName: 'chromium', viewport: { width: 1365, height: 900 } },
    },
    {
      name: 'chromium-mobile',
      use: { browserName: 'chromium', viewport: { width: 390, height: 844 }, isMobile: true, deviceScaleFactor: 1 },
    },
    {
      name: 'firefox-desktop',
      use: { browserName: 'firefox', viewport: { width: 1365, height: 900 } },
    },
  ],
});

Install the additional browsers with npx playwright install firefox (and the corresponding browser name for any other project). A mobile viewport emulates dimensions and mobile settings; it does not replace testing on actual target devices when device-specific behavior matters. GIGW’s advice includes screen resolutions and operating systems, so document the environments this automated matrix does and does not cover.

5. Control screenshot noise without hiding defects

Playwright’s screenshot assertion waits until two consecutive captures match before comparing, and screenshot assertions disable animations by default. This reduces some transient noise, but your test still needs a stable page state. Prefer fixing deterministic data, clocks, and application state over allowing a large diff.

• Wait for content: use locator assertions for the page’s real heading, form, or status element. Avoid relying only on waitForTimeout().
• Control data: use a seeded fixture or route stubbing where appropriate, especially for rotating notices and changing service records.
• Handle volatile areas cautiously: Playwright can apply a stylesheet during screenshot capture with stylePath. Hide only genuinely irrelevant variability, not regions whose layout or content is part of the user experience.
• Use thresholds sparingly: maxDiffPixels, maxDiffPixelRatio, and threshold change what differences pass. Inspect representative diffs before setting them; a permissive threshold may hide a small but important text, spacing, or contrast regression.

Example of a narrowly scoped screenshot stylesheet:

/* tests/screenshot.css */
/* Mask only a timestamp known to change independently of the UI. */
.test-generated-timestamp {
  visibility: hidden !important;
}

import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { test, expect } from '@playwright/test';

const here = path.dirname(fileURLToPath(import.meta.url));

test('service page keeps its intended layout', async ({ page }) => {
  await page.goto('/services/example');
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page).toHaveScreenshot('service-page.png', {
    fullPage: true,
    stylePath: path.join(here, 'screenshot.css'),
    maxDiffPixels: 20,
  });
});

The pixel allowance above is an example only, not a recommended universal threshold. Begin with strict comparisons, inspect actual noise, and choose a tolerance only when the team can explain why those differences are acceptable. See the [Playwright screenshot assertion API](https://playwright.dev/docs/api/class-pageassertions) for supported assertion options.

6. Run in CI and review failures

Run tests in CI using the same pinned browser and operating-system image used to generate the baselines. A typical command is:

npx playwright test

When a comparison fails, inspect the expected image, actual image, and generated diff. Determine whether the change is intended; then check whether it affects task completion, responsive behavior, or accessibility. Update a reference deliberately with npx playwright test --update-snapshots, review the changed images, and include them in the same code review as the UI change. Do not make automatic snapshot replacement the normal response to failed tests.

For useful CI diagnostics, retain Playwright traces or test artifacts on failure and make them available to reviewers. Keep image artifacts from real user data out of public build logs. Limit retries to diagnosing flaky infrastructure; a passing retry does not explain or resolve a nondeterministic rendering problem.

7. Pair visual checks with GIGW and accessibility evaluation

GIGW 3.0 includes WCAG 2.1 Level AA and criteria that cover more than appearance. Its guidance includes text alternatives, contrast, scaling, responsive reflow, and visible identification of interface components and graphical objects. For example, its contrast criterion specifies at least 4.5:1 for ordinary text and images of text, with stated exceptions; large text has a 3:1 minimum under its stated exception. Its responsive presentation criterion references a width equivalent to 320 CSS pixels, subject to the criterion’s conditions. Consult the [official criteria](https://guidelines.india.gov.in/guidelines/) for the complete wording and exceptions.

A screenshot diff cannot reliably tell whether a control has an accessible name, reading order is correct, keyboard navigation works, or a screen-reader user can finish a task. Pair the visual suite with semantic assertions, automated accessibility checks, keyboard testing, and manual evaluation. Playwright also supports ARIA snapshot assertions as a distinct way to compare accessible structure; see [Playwright ARIA snapshots](https://playwright.dev/docs/aria-snapshots). Screenshot tests are one quality check, not a substitute for accessibility evaluation or STQC/GIGW certification.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. A single request captures a URL as an image or PDF; read the [ScreenshotNeo API documentation](https://screenshotneo.com/docs/) for parameters and response details. A remote capture can help inspect a page or generate reference material, but it does not replace a controlled Playwright regression suite whose baseline and CI renderer must match.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', new Uint8Array(await res.arrayBuffer()));

Before capture, ScreenshotNeo accepts cookie or consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed; response headers say the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The free plan includes 1,000 shots per month without a card; paid plans start at $5 for 3,000 shots.

Create a free ScreenshotNeo account for 1,000 screenshots a month with no card.

Troubleshooting

Symptom	Likely cause	Fix
First run reports a missing snapshot	No reference exists yet.	Review the generated image, then commit it as the baseline if it represents the intended page.
Snapshots fail on every CI run	CI uses a different browser, OS, font set, viewport, locale, or rendering mode than baseline generation.	Pin the environment and configuration, regenerate baselines in that same environment, or keep separate project/platform baselines.
Only content below the fold differs	Lazy-loaded content, font loading, or full-page capture timing differs.	Wait for the relevant content to appear and ensure lazy content has loaded before capture; check whether full-page capture is required.
Images or fonts intermittently appear missing	Network resources are still loading or the test depends on an unavailable external service.	Wait for the meaningful image or font state, use deterministic fixtures where possible, and inspect trace/network artifacts for failed requests.
Diffs occur only around dates, banners, or counters	The application includes dynamic values or rotating content.	Fix time/data in test setup or apply a narrowly scoped screenshot stylesheet to the volatile element. Do not mask its surrounding layout.
Small but meaningful text changes pass	Diff threshold is too permissive.	Reduce or remove the configured tolerance, then inspect the image diff and use semantic text assertions for critical copy.
Snapshot names differ by project	Playwright includes project or platform identity in snapshot naming.	Keep the projects intentional and commit their reference images; do not copy a baseline across renderers without review.
Test passes locally but fails in CI	Different app data, base URL, font availability, browser binaries, or environment variables.	Compare configuration and runtime versions, use deterministic data, and run locally in the same container or pinned environment.

Performance, reliability, and maintenance

• Control suite cost: begin with a few important templates and states, then add cases where a visual defect would materially affect a citizen task. Full-page images and multiple browser projects increase capture and review work.
• Keep capture state cheap: reuse seeded test fixtures, avoid unnecessary waits, and wait for a specific visible state instead of sleeping for a fixed duration.
• Reduce false alarms: pin browser and OS versions, keep fonts consistent, and avoid changing screenshot thresholds casually. A flaky visual test consumes reviewer attention and makes genuine changes harder to spot.
• Review images like code: keep baselines in version control and require a human review for intentional updates. A snapshot is an expectation, not proof that the page is correct or compliant.
• Protect sensitive data: use synthetic content for captured states and control access to CI artifacts, especially when forms or confirmation pages are involved.
• Plan for network variability: external assets and backend state can make screenshots unreliable. Use controlled fixtures where suitable and record which real integrations remain outside the screenshot test.

FAQ

Can screenshot tests certify that a government site meets GIGW?

No. They can help catch unintended visual changes, but GIGW evaluation includes accessibility and other requirements that an image comparison cannot establish. Follow the official criteria and the applicable evaluation or certification process.

Should I make one baseline work across Chrome, Firefox, and WebKit?

No. Rendering differs across browsers and operating systems. Use separate project-specific references when testing those environments.

Should every page have a screenshot test?

Usually begin with representative templates and citizen journeys, including key success and error states. Add pages when they introduce distinct layout or service risk.

Can a visual regression test catch a contrast failure?

It may reveal a color change, but a pixel diff does not calculate or validate contrast conformance. Check contrast against the applicable criterion with dedicated evaluation.

Do I need a screenshot service to use Playwright visual assertions?

No. Playwright Test creates and compares local reference images. A screenshot API is useful for separate capture workflows, but it does not remove the need to control the browser environment for Playwright baselines.