How to Run Argos CI Visual Tests in Docker
Run Argos visual checks with Playwright in a version-pinned Docker container, configure CI secrets and screenshots, and troubleshoot common rendering and upload failures.
Run Playwright and Argos in Docker by pinning the Microsoft Playwright image to the same version as your project’s Playwright package, installing dependencies from the lockfile, passing ARGOS_TOKEN through your CI secret store, enabling the Argos reporter, and capturing named states with argosScreenshot. Docker standardizes the browser and operating-system environment; deterministic test data and careful secret handling are still necessary.
This guide uses GitHub Actions as a concrete example. The same ingredients apply to other CI providers, whose container and secret syntax may differ. Check the Playwright Docker documentation and Argos documentation when adapting the version and configuration to your project.
1. Align Playwright and Docker versions
The official Playwright image includes browser binaries and their system dependencies. It does not include your project’s Playwright package, so your CI job must install the project dependencies too. The image and installed Playwright package should match: if they do not, Playwright may be unable to find the expected browser executable.
Use the Playwright version already declared by your project’s lockfile. For example, if the project uses Playwright 1.63.0, the matching image can be pinned as mcr.microsoft.com/playwright:v1.63.0-noble. The available tags change; the version shown here is a point-in-time example. Check the official docs for current tags and select an OS suffix such as noble or jammy only when you have a reason to choose that base image.
- Check the version of
@playwright/testinpackage.jsonand the resolved version in your lockfile. - Use the same version in the Docker image tag.
- Commit and install from the lockfile in CI. For npm, use
npm ci.
Pinning makes the browser environment more reproducible. Update the package and image together, then review any visual changes caused by the browser or operating-system update.
2. Install the Argos Playwright integration
Add the Argos Playwright package as a development dependency and commit the updated manifest and lockfile:
npm install --save-dev @argos-ci/playwrightYour project should already have @playwright/test installed. If it does not, add Playwright using its setup guide and commit the resulting lockfile before running the workflow.
3. Configure the Argos reporter
Add the reporter to playwright.config.ts. This example uses Playwright’s dot reporter in CI and its list reporter locally. It enables Argos uploads only when the CI environment variable is set.
import { defineConfig } from "@playwright/test";
export default defineConfig({
reporter: [
process.env.CI ? ["dot"] : ["list"],
["@argos-ci/playwright/reporter", { uploadToArgos: !!process.env.CI }],
],
});Make sure your CI provider sets CI, or set it explicitly in the job. Keep ARGOS_TOKEN out of source files. Add it to your CI provider’s secret store, then expose it to the test process as an environment variable. The reporter can also accept a token in its options, but a committed token would expose credentials; use the secret environment variable instead.
4. Capture named screenshots in a Playwright test
Navigate to the state you intend to compare, make the necessary interactions, and call argosScreenshot with a stable name:
import { argosScreenshot } from "@argos-ci/playwright";
import { test } from "@playwright/test";
test("homepage visual", async ({ page }) => {
await page.goto("http://127.0.0.1:3000/");
await argosScreenshot(page, "homepage");
});Use names that remain stable across runs and identify the page or state. For example, name separate screenshots homepage, pricing-monthly, and account-signed-in. The helper waits for fonts, images, and network idle, and hides carets and scrollbars before capture. It cannot make changing application data deterministic: seed test data, control animations or clocks where needed, and wait for the meaningful state before capturing.
Keep functional checks as ordinary Playwright assertions. A visual capture tells you what the page looked like; it does not replace checks for navigation, form behavior, or content correctness.
5. Run the app and tests in a container in CI
The following GitHub Actions example assumes the project has a start script that serves the app at port 3000, and a committed npm lockfile. Adjust the server command and readiness URL to your application. The job-level container keeps the app and browser tests in the same environment.
name: visual-tests
on:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
container:
# Match this tag to the Playwright package version in your lockfile.
image: mcr.microsoft.com/playwright:v1.63.0-noble
options: --init --ipc=host
steps:
- uses: actions/checkout@v4
- name: Install locked dependencies
run: npm ci
- name: Start the application
run: |
npm run start > /tmp/app.log 2>&1 &
for attempt in $(seq 1 60); do
if curl --fail --silent http://127.0.0.1:3000/ > /dev/null; then
exit 0
fi
sleep 1
done
cat /tmp/app.log
exit 1
- name: Run Playwright and Argos checks
run: npx playwright test
env:
CI: "true"
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}Add ARGOS_TOKEN to the repository or environment secrets in GitHub before the job runs. Pull requests from forks generally do not receive repository secrets; if a workflow runs without the token, uploads may not happen. Choose a workflow policy that protects the secret and matches whether untrusted contributions need to run visual tests.
Playwright recommends Docker’s --init option to help avoid zombie processes and --ipc=host for Chromium, which can otherwise run out of shared memory and crash. GitHub’s job container accepts Docker options through options; for other CI providers, use the corresponding container configuration.
When tests target a preview deployment
If the app is already deployed for the pull request, point tests at its preview URL rather than starting a local server in the job. Set a base URL in Playwright configuration and provide the deployment URL through CI:
import { defineConfig } from "@playwright/test";
export default defineConfig({
use: {
baseURL: process.env.BASE_URL ?? "http://127.0.0.1:3000",
},
reporter: [
process.env.CI ? ["dot"] : ["list"],
["@argos-ci/playwright/reporter", { uploadToArgos: !!process.env.CI }],
],
});test("homepage visual", async ({ page }) => {
await page.goto("/");
await argosScreenshot(page, "homepage");
});Configure the deployment step to export BASE_URL for the test step. The preview must be reachable from inside the container, and any authentication or test data setup must work in that environment.
6. Decide how to manage visual baselines
Native Playwright screenshot assertions and Argos both capture browser output, but they manage comparison and review differently. Native Playwright stores baselines in the repository; Argos uploads captures for hosted comparison and pull-request review.
| Workflow | Baseline and review | Useful when |
|---|---|---|
| Native Playwright screenshots | Screenshot files live in Git. Update with npx playwright test --update-snapshots in a controlled environment and inspect the changed files. |
A small suite where repository-managed files and direct local review are sufficient. |
| Playwright with Argos | Captures are uploaded for hosted comparison and pull-request review. | Your team wants centralized review rather than maintaining screenshot baselines in the repository. |
With either workflow, inspect visual changes before accepting them. A baseline update can conceal a regression if it is approved without review. For native snapshots, generate and update baselines in the same Docker environment used by CI to avoid operating-system and browser rendering differences.
7. Troubleshoot common failures
| Symptom | Likely cause | Fix |
|---|---|---|
| Playwright cannot find or launch the browser executable | The Playwright package and Docker image use different versions, or the project dependencies were not installed. | Align the image tag with the locked Playwright version and run the package manager’s lockfile install, such as npm ci. |
| Argos has no screenshots for a build | The reporter is missing or not enabled, ARGOS_TOKEN is unavailable, the test never reaches the capture call, or uploads are disabled. |
Check the reporter configuration, secret availability and workflow logs. Confirm the test reaches argosScreenshot and that uploadToArgos is enabled for CI. |
| Pull request reports a token or upload error | The secret is unset, misnamed, invalid, or unavailable to that event’s workflow. | Verify the secret name is exactly ARGOS_TOKEN and that the workflow has access to it. Do not print the token in logs or commit it to the repository. |
| Chromium crashes or reports memory pressure | The container’s default shared-memory allocation may be too small. | Use --ipc=host where supported by the CI runner, as Playwright recommends for Chromium. |
| Container leaves zombie processes or does not exit cleanly | The main process is running as PID 1 without init-style process handling. | Enable Docker’s --init option where the runner supports it. |
| Screenshots differ between laptop and CI | Operating system, browser version, fonts, or rendering behavior differ. | Generate native baselines in the same pinned Docker environment as CI. Keep the image and package versions aligned, and review diffs after upgrades. |
| Only some captures are flaky | Dynamic data, animation, delayed content, or incomplete application setup changes the page between runs. | Seed or stabilize data, wait for the target state, and hide or mask genuinely variable content where appropriate. Argos waits for fonts, images, and network idle, but the test must still control its inputs. |
| Test cannot reach the app from the container | The server is not running, is bound to an inaccessible interface, or the test uses the wrong hostname. | Start and health-check the server in the job, bind it so the container can reach it, and use the correct preview or service URL. When a container accesses a service on the host, localhost refers to the container itself; use the runner’s documented host-gateway or service-network setup. |
| Browser sandbox errors or uncertainty about running untrusted pages | The official image runs as root by default, which disables Chromium’s sandbox. | Playwright says root may be acceptable for trusted end-to-end tests. For untrusted browsing, use a separate user and appropriate seccomp configuration; do not treat the test image as a general-purpose untrusted browsing environment. |
Performance, reliability, and cost considerations
Performance
Docker adds image setup and dependency installation to the job, so use your CI provider’s supported dependency or image caching where appropriate. Avoid reinstalling unrelated dependencies for every run, but do not skip lockfile-based installation. Chromium may need more shared memory than the default container allocation provides; --ipc=host is Playwright’s recommended Docker setting for Chromium when available.
Reliability
Pin the image, lock the JavaScript dependencies, and run the same browser environment for visual captures and native baseline updates. Keep the test data and page state stable. Treat image or Playwright upgrades as changes that may affect rendering, and review resulting diffs deliberately.
Security
Keep Argos credentials in CI secrets and expose them only to steps that need them. Pull-request workflows have different trust boundaries, especially for contributions from forks. Playwright describes its Docker image as intended for testing and development, not for visiting untrusted websites; it recommends a separate user and suitable seccomp configuration for untrusted browsing.
Cost
The Docker image is a runtime environment; CI execution time and Argos service terms are separate cost considerations. This guide does not rely on current Argos pricing, which can change. Check the service’s current plan details for your expected volume before choosing a hosted workflow.
Or skip the browser setup
For a one-off website screenshot, ScreenshotNeo provides a screenshot API and MCP server. It is different from this Playwright-plus-Argos workflow: it captures a URL through one API request rather than running your application’s Playwright test suite. Use Playwright and Argos when you need app-specific test state, assertions, and pull-request visual review.
One-call screenshot example (see the ScreenshotNeo API documentation for options):
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpScreenshotNeo removes cookie banners, popups, and chat widgets before the shot; bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card, and paid plans start at $5 for 3,000 screenshots. Sign up for 1,000 free screenshots a month, with no card required.
FAQ
Does the Playwright Docker image include Playwright Test?
No. It includes browser binaries and operating-system dependencies. Install your project’s Playwright package and other dependencies in the job.
Can the Argos screenshot helper replace functional assertions?
No. Use Playwright assertions for behavior and use named Argos captures to review visual changes.
Should I commit an Argos token in the Playwright config?
No. Store it in your CI secret store and make it available to the workflow as ARGOS_TOKEN.
Do I need Docker if I use Argos?
This guide uses Docker to make browser and operating-system rendering more consistent. Argos handles hosted screenshot comparison and review; it does not remove the need to choose a stable test environment.
