ScreenshotNeo
Sign in Get a free key

BlogHow-to

Run Visual Tests on Vercel Preview Deployments

Run Playwright visual tests against the exact Vercel Preview deployment for a change, including protected previews, and publish reviewable results with the pull request.

By the ScreenshotNeo team4 October 202610 min read

Run visual tests only after the Vercel Preview deployment succeeds, and give Playwright the URL for that specific deployment. For evidence tied to one revision, use its commit-specific deployment URL and check out the event’s commit SHA. If Deployment Protection is enabled, configure an authorized automation bypass or other supported authentication path for CI.

This guide uses GitHub Actions and Playwright for full browser journeys, then explains baseline management, protected previews, hosted review options, reliability, costs, and common failures.

1. Understand the deployment URL and revision

Vercel creates Preview deployments for testing and collaboration before a production release. Each deployment has a unique URL. A commit URL identifies a specific deployment; a branch URL follows the branch’s latest deployment, so it can move as new commits deploy. Use the commit-specific URL when a visual result must stay associated with one revision. [Vercel Environments; Generated URLs]

Pass both the deployment URL and commit SHA into the test run. The URL determines what the browser visits; the SHA identifies the source revision to check out and record with the result. Do not assume a branch alias still points to the deployment that triggered the workflow.

2. Configure Vercel to trigger CI after deployment

For GitHub Actions, Vercel documents sending a repository_dispatch event with type vercel.deployment.success. Other CI systems can use Vercel’s deployment.succeeded webhook. Configure the Vercel integration or webhook to send the deployment target URL and commit identity needed by your workflow. Follow Vercel’s current guide for the event payload and setup details; payload fields and integration settings can vary. [Vercel: Run end-to-end tests after a Preview Deployment]

In your repository, ensure the workflow is allowed to receive repository dispatch events. Store any webhook secret or access credentials as CI secrets. Triggering from deployment success prevents tests from racing the build, but the application can still need time to become ready; the browser test should wait for a meaningful page state rather than relying only on a fixed delay.

3. Add a Playwright visual test

Install Playwright and its browser in the project, and commit the generated lockfile so CI installs the same dependency versions. The test below visits a stable route, waits for a visible landmark, and captures a screenshot assertion. Replace the route and locator with a representative state from your app. Playwright’s toHaveScreenshot compares the current capture with a stored reference snapshot. [Playwright snapshot testing]

// tests/preview.spec.ts
import { test, expect } from '@playwright/test';

test('preview home page visual state', async ({ page }) => {
  const baseURL = process.env.BASE_URL;
  if (!baseURL) throw new Error('BASE_URL is required');

  await page.goto(baseURL, { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('main')).toBeVisible();
  await expect(page).toHaveScreenshot('home.png', {
    fullPage: true,
    animations: 'disabled',
  });
});

Set a stable browser project, viewport, and device scale factor in Playwright configuration. For example:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: {
    browserName: 'chromium',
    viewport: { width: 1440, height: 900 },
    deviceScaleFactor: 1,
    locale: 'en-US',
    timezoneId: 'UTC',
    colorScheme: 'light',
    trace: 'retain-on-failure',
  },
  reporter: [['list'], ['html', { open: 'never' }]],
});

Generate the initial reference snapshots in a controlled environment and review them before committing. A baseline is required for comparison; otherwise the first run cannot tell whether a change is expected. Keep snapshot updates intentional and review the resulting image diffs instead of automatically accepting every changed capture. [Playwright visual comparisons]

4. Run the test from GitHub Actions

The workflow below illustrates the essential sequence: receive Vercel’s success event, check out the triggering commit, install dependencies and browser binaries, set the deployment URL as BASE_URL, and run Playwright. The exact dispatch payload shape depends on your Vercel integration; map its deployment URL and SHA to the two environment variables shown here.

# .github/workflows/preview-visual.yml
name: Preview visual tests
on:
  repository_dispatch:
    types: [vercel.deployment.success]

permissions:
  contents: read

jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.client_payload.git.sha }}
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - name: Run visual tests against the deployed preview
        env:
          BASE_URL: ${{ github.event.client_payload.url }}
        run: npx playwright test

Adjust client_payload.git.sha and client_payload.url to the actual payload your integration sends. Validate that the URL belongs to the successful deployment before running tests, and fail clearly if either value is missing. If your CI does not use GitHub Actions, implement the same sequence in its webhook handler: wait for success, identify the deployment URL and commit SHA, check out that revision, run the browser tests, and publish the result.

5. Run tests against protected previews

Deployment Protection can restrict access to Preview and production URLs. A CI runner that cannot authenticate will usually see a protection page or an authorization error rather than your application. Vercel’s guide instructs projects with Deployment Protection to use Protection Bypass for Automation so test environments can reach deployments. [Vercel Deployment Protection; Vercel post-deployment test guide]

  1. Enable and configure the supported automation bypass for the project.
  2. Save the bypass credential as a secret in the CI system, not in source control or test logs.
  3. Pass it to the test environment using the mechanism Vercel documents for your setup, and scope access to the necessary project and workflow.
  4. Confirm the browser reaches the application route before taking snapshots; do not treat a protection page as an app baseline.

Vercel’s supported bypass mechanism and configuration details can change, so use the current documentation for the precise header or credential procedure. Keep the preview protected and grant CI the intended automation access instead of making the deployment public just to enable screenshots.

6. Choose what to compare

Playwright snapshots

Playwright snapshots keep route journeys, viewport choices, and screenshot assertions with the test code. They fit teams that want to control the browser state and maintain references in the repository. The tradeoff is that the team must manage snapshot updates and inspect diffs in its development workflow. [Playwright snapshot testing]

Hosted visual review

Hosted services can collect captures and present diffs for review in a centralized interface. Argos documents a Playwright SDK, CI uploads, and pull request review. Its documentation says builds on pull requests can be marked orphan until a build on the default branch establishes a baseline. Chromatic documents interactive snapshots and pixel comparison for Playwright. Review each service’s current plan, limits, retention, and terms before adopting it. [Argos Playwright quickstart; Argos Vercel Preview workflow; Chromatic Playwright documentation]

Compare options by route and journey coverage, commit-pinned URL handling, baseline setup, diff approval, browser and operating system consistency, protected-preview access, artifact retention, CI effort, and service cost. The sources describe workflows; they do not establish a controlled vendor performance comparison.

7. Make visual results reproducible

  • Keep browser version, operating system, fonts, viewport, device scale factor, locale, and timezone consistent between baseline and candidate runs.
  • Use deterministic data and a known account state. Avoid depending on live third-party content when it can change independently.
  • Wait for the interface state that matters, such as a heading or loaded component. Disable animations where appropriate and mask timestamps, rotating content, or other volatile regions.
  • Choose full-page screenshots for page layout, or element screenshots for a focused component. Keep the capture dimensions and scroll behavior consistent.
  • Retain the deployment URL, commit SHA, browser and test versions, logs, and screenshot artifacts with the CI result so a failure can be tied back to the same build.

These are engineering practices inferred from how screenshot comparison works; they are not claims of a benchmark or controlled test. Playwright also recommends consistent environments for visual comparisons. [Playwright CI guidance; Playwright visual comparisons]

8. Publish results where reviewers can act

Make the visual job a required or clearly visible pull request check according to your team’s review policy. Upload the HTML report and failure artifacts, or configure the chosen hosted review integration to show the diff in the pull request. Include the deployment URL and commit SHA in the job summary. A green test status without accessible captures is harder to review; a capture without its revision and target URL is harder to diagnose.

9. Troubleshooting

Symptom Likely cause Fix
Workflow never starts The deployment success event was not delivered, the dispatch type differs, or the repository is not configured to accept it. Check Vercel’s integration or webhook delivery log, confirm the event type and repository, and verify the GitHub Actions trigger matches the configured dispatch.
Checkout uses the wrong revision The workflow uses the default branch or a moving branch alias rather than the deployment event’s SHA. Check out the event commit SHA and record it with the run. Map the actual event payload field rather than assuming the example field names apply.
Navigation fails or times out The preview may not be ready, the URL may be wrong, a redirect may be unexpected, or Deployment Protection may block the runner. Confirm the deployment succeeded and the URL belongs to it. Inspect redirects and response details, configure authorized automation access, then wait for an application landmark before capture.
Screenshot shows a Vercel login or protection page The runner has not been granted the supported access path. Configure Protection Bypass for Automation or the applicable supported authentication method; keep credentials in CI secrets and verify they reach the browser request.
Every run reports a missing snapshot No reference baseline exists for this test, platform, or project configuration. Create and review an initial snapshot in the intended environment, then commit it or establish the hosted service’s default-branch baseline.
Snapshots fail intermittently Dynamic content, animations, fonts, data, or environment differences alter pixels or layout. Stabilize test data and browser settings, wait for the intended state, disable animation, and mask genuinely volatile regions.
Large unrelated diffs appear Viewport, device scale factor, browser version, operating system, font availability, or page content changed. Compare run metadata with the baseline environment and restore consistent settings before accepting an update.
Hosted pull request build is marked orphan The service has no baseline build from the default branch yet. Run the default branch first to establish the baseline, following the service’s documented workflow.
Workflow gets a 404 or tests the wrong page The event URL was mapped incorrectly, the route is not deployed, or a branch alias moved. Print the non-secret target URL in the job summary, verify the deployed route, and use the deployment’s commit-specific URL for revision-pinned evidence.

10. Performance, reliability, and cost

Browser journeys take longer and use more CI resources than a simple URL capture because they launch a browser, navigate, establish state, and render the selected pages. Keep the suite focused on high-value routes and states; run broad journeys when they provide useful coverage. Parallelize only when the CI capacity and test data support independent runs. Cache dependency downloads where your CI supports it, and install a consistent browser build.

Reliability depends on deployment readiness, access, deterministic data, and matching capture environments. A navigation failure is not necessarily a visual regression. Preserve logs and metadata so you can distinguish an unavailable deployment from a pixel mismatch.

Costs can include CI runner time, storage for artifacts or baselines, and any hosted visual review service. The research sources do not provide comparable current prices or limits, so check vendor terms directly before budgeting. Playwright snapshots do not require a hosted visual review service, though they still use CI and repository storage.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. For a visual check that needs a page capture rather than a full interactive journey, make one request to capture the deployment URL. See the ScreenshotNeo API documentation for options and setup.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://your-preview-url.vercel.app -o shot.webp
import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://your-preview-url.vercel.app"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://your-preview-url.vercel.app',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

ScreenshotNeo removes cookie banners, popups, and chat widgets before the shot. Bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots, and 1,000 screenshots per month are free with no card; paid plans start at $5 for 3,000. These captures do not replace Playwright when your test needs to interact with the application or compare a specific logged-in journey.

Sign up free for 1,000 screenshots a month, with no card required.

FAQ

Can I test a preview before the deployment is ready?

Wait for the successful deployment event, then run the suite. Starting earlier can turn a deployment race into a misleading navigation failure.

Should I use the branch URL or commit URL?

Use the commit URL when the result must stay pinned to one deployment. A branch URL is useful for collaboration but follows the newest branch deployment.

Do I need a hosted visual testing service?

No. Playwright can compare committed snapshots directly. A hosted service is an option when centralized diff review and pull request presentation fit your team’s workflow.

Can a screenshot API replace these browser tests?

It can capture a page image, but a URL capture alone does not exercise application interactions or prove a user journey. Use browser tests when those behaviors are part of the requirement.

Keep reading

7 min read

Full-Page Capture Chrome Plugin

Capture an entire webpage in Chrome with extensions, understand their limits, and automate clean full-page screenshots with ScreenshotNeo.

7 min read

Facebook Open Graph Debugger

Use Facebook’s Open Graph debugger to refresh stale link previews, verify metadata, and fix incorrect titles, descriptions, and images.