How to Set Up Argos CI with Playwright for Visual Regression Testing

To set up Argos with Playwright: connect your repository to Argos, install its Playwright integration, register the Argos reporter in playwright.config.ts, and call argosScreenshot(page, "stable-name") in the tests you want compared. In CI, set the Argos token as a secret, install the Playwright browser, and run playwright test. Argos receives the screenshots for hosted comparison and pull request review; Playwright still runs the browser tests.

This guide uses GitHub Actions. Argos’s package and onboarding steps can change, so check its current Playwright visual regression guide and GitHub Actions tutorial if your installed package versions differ.

1. Connect your repository and install the packages

Install the Argos GitHub App and grant access to the repository. This enables Argos to report visual results on pull requests.
In the project root, install the Playwright integration and CLI:

npm install --save-dev @argos-ci/playwright @argos-ci/cli

If Playwright is not set up in the project yet, initialize it and follow its prompts:

npm init playwright@latest

Commit the package manifest and lockfile so local development and CI use the same dependency versions. Use the Argos project onboarding flow to obtain the project token; add it to your CI provider’s secret store as ARGOS_TOKEN. Never commit the token in source code or workflow YAML.

2. Register the Argos reporter

In playwright.config.ts, include the Argos reporter. This configuration keeps Playwright’s list reporter locally and adds Argos reporting in CI:

import { defineConfig } from "@playwright/test";

export default defineConfig({
  reporter: process.env.CI
    ? [["list"], ["@argos-ci/playwright/reporter"]]
    : [["list"]],
});

If your project already uses another reporter, preserve it in the array. Playwright accepts multiple reporters, so you can keep the HTML or JUnit output your team already relies on. The exact reporter options are package-version-sensitive; consult the Argos setup guide when configuring options such as conditional upload.

3. Capture the pages and states that matter

Add a Playwright test that navigates to the running application and calls argosScreenshot with a stable, descriptive name:

// tests/homepage.spec.ts
import { argosScreenshot } from "@argos-ci/playwright";
import { expect, test } from "@playwright/test";

test("homepage visual state", async ({ page }) => {
  await page.goto("http://127.0.0.1:3000/");
  await expect(page.getByRole("heading", { name: "Welcome" })).toBeVisible();
  await argosScreenshot(page, "homepage");
});

Replace the URL and heading assertion with your application’s route and a meaningful readiness check. Add screenshots for distinct states that users depend on, such as a signed-in dashboard or a validation error. Keep names stable across runs: changing a screenshot name can create a new comparison identity instead of comparing against the previous capture.

Argos’s helper is intended to improve capture stability by waiting for fonts, images, and network activity to settle and managing visual noise such as carets and scrollbars. It cannot make changing application data deterministic. Use fixed test data and a repeatable login or fixture setup when the page includes timestamps, random content, rotating banners, or user-specific data.

4. Run the tests in GitHub Actions

Store ARGOS_TOKEN under the repository’s GitHub Actions secrets. Then create .github/workflows/visual-tests.yml:

name: Visual tests

on:
  pull_request:
  push:
    branches: [main]

jobs:
  playwright:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browser and system dependencies
        run: npx playwright install --with-deps chromium

      - name: Run Playwright tests
        run: npx playwright test
        env:
          ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}

Use the Node version your application supports. The workflow installs Chromium only; if your tests use Firefox or WebKit, install those browsers too. Ensure the application is available before the tests navigate to it. For a typical app, add a start step and configure Playwright’s webServer option so the server starts and becomes ready automatically.

For example, add this to the existing Playwright configuration when npm run start serves the app on port 3000:

export default defineConfig({
  webServer: {
    command: "npm run start",
    url: "http://127.0.0.1:3000",
    reuseExistingServer: !process.env.CI,
    timeout: 120_000,
  },
  reporter: process.env.CI
    ? [["list"], ["@argos-ci/playwright/reporter"]]
    : [["list"]],
});

Merge this into your existing configuration rather than replacing other settings. For pull requests from forks, GitHub does not provide repository secrets to the workflow by default. Keep the secret protected; do not work around that restriction by exposing the token to untrusted code.

5. Review visual changes on the pull request

Push the workflow and open a pull request. The job runs the Playwright tests and the reporter uploads screenshots when configured. Review the Argos result linked from the pull request check: determine whether each difference is an intended design change or an unintended regression, then update the application or accept the expected change through the review workflow.

Keep functional assertions alongside visual captures. A screenshot comparison shows rendered differences; an assertion such as “the heading is visible” checks behavior that a pixel diff alone may not explain.

Configuration and workflow choices

Reporter behavior

Use the reporter array to retain existing Playwright outputs. The basic example enables Argos reporting only in CI. If your installed Argos version documents an uploadToArgos option, you can register the reporter in both environments and set that option based on process.env.CI. Verify the option against the version you installed instead of assuming every release accepts the same configuration.

What to capture

Give each meaningful page state a stable name.
Wait for app-specific content to be ready before capture; navigation completion alone may not mean client-rendered content is ready.
Prefer fixed fixtures over live or randomized data.
Capture a few representative states rather than every route at every possible viewport. Add coverage where visual regressions would affect users.

Argos or Playwright’s built-in screenshot assertions?

Playwright’s expect(page).toHaveScreenshot() stores reference images in the repository and compares later captures against them. That can suit a small suite whose team wants baselines managed in Git. Argos uses Playwright to capture the page, then provides hosted screenshot comparison and review. This can reduce the need to maintain baseline image changes in repository history. Choose based on who owns baselines, how the team wants to review diffs, and how consistently CI can reproduce rendering.

For committed Playwright baselines, keep the operating system, browser, fonts, and Playwright version consistent. The Argos CI guide recommends a pinned official Playwright Docker image as one way to keep the rendering environment consistent. The same environment consistency matters when diagnosing Argos captures, even though its screenshots are reviewed through a hosted workflow.

Performance, reliability, and cost

Keep CI time predictable

Install only the browser engines the suite uses; the example installs Chromium alone.
Use npm ci and cache npm dependencies through the setup action.
Playwright browser binaries can be cached using a key based on operating system and Playwright version. Argos documents this as an optimization, not a requirement, in its Playwright speed guide.
Do not add repeated fixed delays as a general readiness strategy. Wait for a selector, application state, or other condition that reflects the page being ready.

Reduce flaky image differences

Pin the Playwright dependency through the lockfile and use the same browser and operating system in CI over time. Fonts, anti-aliasing, animation, scrollbars, asynchronous images, and dynamic content can all alter pixels. Prefer fixing the source of changing output—such as using deterministic fixtures or waiting for fonts—before accepting broad diff tolerances or updating expected images.

Plan service usage and secrets

The workflow requires an Argos project token and sends captured screenshots to Argos for comparison. Review the current Argos plan and account settings for applicable limits and pricing; the setup sources cited here do not establish a price for your project. Keep screenshots free of secrets and sensitive personal data, and use test accounts with controlled content.

Troubleshooting

Symptom	Likely cause	Fix
No Argos result appears	The reporter is missing, the reporter is not enabled in CI, or the token is unavailable.	Check the reporter configuration, confirm the workflow passes `ARGOS_TOKEN` from the secret store, and inspect the CI log for upload errors.
Authentication or upload failure	The token is missing, invalid, or belongs to a different project configuration.	Recheck the Argos project token and secret name. Do not print the token in logs; update the stored secret if it was rotated.
Playwright cannot find the browser	The workflow did not install the browser matching the locked Playwright version.	Run `npx playwright install --with-deps chromium` after `npm ci`. When the Playwright version changes, reinstall or invalidate a browser cache keyed to the previous version.
Navigation times out or the page is blank	The app server did not start, the URL or port is wrong, or the test ran before the server was ready.	Configure `webServer`, verify its readiness URL, and use the same host and port in the test.
Large diffs appear on every run	Rendering differs across environments, or the page contains dynamic data, animation, caret, font, or image changes.	Pin the Playwright environment, stabilize test data, wait for the relevant UI state, and inspect the changed region before accepting it.
Local and CI captures differ	The local operating system, browser build, fonts, or device scale differs from CI.	Reproduce in the same pinned container or diagnose the diff in the CI capture environment. Avoid treating a local image as the only reference for a differently rendered CI system.
Pull request from a fork cannot upload	Repository secrets are not provided to untrusted fork workflows.	Keep secret protection enabled. Use a trusted workflow arrangement appropriate to your repository rather than exposing the token to fork code.
Reporter option is rejected	The configuration was copied from a different Argos package version.	Check the installed package version and use the matching Argos documentation. Keep the minimal reporter configuration until the basic upload works.

Or skip the browser setup

If you need a page screenshot outside a Playwright visual regression suite, ScreenshotNeo can return an image or PDF with one GET request. It does not replace Argos’s role in comparing test captures across pull requests; it is a direct screenshot API for capturing pages without configuring a browser runner. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

ScreenshotNeo removes cookie banners, popups, and chat widgets before the shot. Bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The Free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

FAQ

Does Argos replace Playwright?

No. Playwright runs the browser tests and captures page states; Argos receives screenshots for hosted comparison and review.

Should every test take an Argos screenshot?

No. Capture the stable UI states that provide useful visual coverage. Keep behavior-focused tests separate when a screenshot adds no review value.

Can I keep the Playwright HTML report?

Yes. Playwright supports multiple reporters. Retain the reporters your team uses alongside the Argos reporter.

Can I use this setup with a CI provider other than GitHub Actions?

Yes. The essential pieces are installing dependencies and browsers, providing the Argos token securely, and running the Playwright tests with the configured reporter.