How to Run BackstopJS Tests in GitHub Actions

Run BackstopJS in GitHub Actions by preparing a reachable copy of your app, installing the version pinned in your project lockfile, capturing a test set against reviewed reference screenshots, and preserving the visual report and JUnit output for review. The core lifecycle is backstop init, backstop test, and backstop approve. In CI, run tests on pull requests; update and approve references only after a person reviews the visual changes.

BackstopJS captures configured scenarios and compares them with a reference set. Its project describes the tool as automating visual regression testing by comparing screenshots over time. See the BackstopJS project for its commands and configuration details.

1. Add BackstopJS to the project

Install BackstopJS as a development dependency, then commit both the package manifest and the package manager lockfile. Keeping the version in the project makes local and CI runs reproducible. The example below uses npm; equivalent scripts can be added with another package manager.

npm install --save-dev backstopjs
npx backstop init

Initialization creates backstop.json at the project root by default. Add npm scripts so developers and CI invoke the same local executable:

{
  "scripts": {
    "visual:test": "backstop test",
    "visual:approve": "backstop approve"
  }
}

Adjust an existing package.json rather than replacing its scripts. Use npm run visual:test locally before wiring the command into CI.

2. Configure scenarios and viewports

BackstopJS needs scenarios describing the pages to capture and viewports describing the browser sizes. A minimal configuration looks like this:

{
  "id": "webapp",
  "viewports": [
    { "label": "desktop", "width": 1365, "height": 900 },
    { "label": "mobile", "width": 390, "height": 844 }
  ],
  "scenarios": [
    {
      "label": "Home",
      "url": "http://127.0.0.1:3000/"
    },
    {
      "label": "Pricing",
      "url": "http://127.0.0.1:3000/pricing"
    }
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  "report": ["browser", "CI"],
  "engine": "puppeteer"
}

This illustrates the key fields, not every supported setting. Keep the scenario URLs pointed at the app instance accessible from the test process. If a route depends on authentication, seeded data, feature flags, or a particular locale, provide the same conditions on every run. Add scenarios for the states that matter, such as a menu-open state or a logged-in page, rather than assuming the homepage represents the whole site.

For all supported configuration fields and documented defaults, check the BackstopJS documentation in the project repository. Defaults can change; explicitly configure paths or behavior that your workflow depends on.

3. Establish and review reference screenshots

Reference screenshots are the accepted visual baseline. Capture them in a controlled environment, inspect the generated report, and approve the set only when the differences are intentional:

npm run visual:test
npm run visual:approve

The first test may fail because no references exist yet; follow the report and project instructions to establish the initial reference collection. Thereafter, keep approval as an intentional maintenance action. Do not have every pull request silently promote its test images: a regression could then become the new expected output before anyone reviews it.

When a design change is expected, update references on a deliberate branch or commit, include the changed screenshots or report for review, and approve the set after confirming the differences. Keep the baseline change traceable to the code change it represents.

4. Make the app reachable from the job

BackstopJS can only capture pages that its browser process can reach. Start the app and prepare required data before running the test command. The precise GitHub Actions steps depend on your app’s build and startup commands; the BackstopJS project does not prescribe a universal GitHub Actions service or startup recipe.

Use a deterministic test database or fixture data. Avoid tests that depend on production data or third-party services changing during a run.
Wait for the app’s ready condition before starting BackstopJS. A fixed short sleep can race on slower jobs; use a readiness check appropriate to your app.
Use a URL valid from the browser’s execution environment. A host loopback address inside a container refers to that container, not necessarily the host machine.
Keep test credentials and secrets in the repository’s supported secret mechanism, and avoid printing them in logs.

5. Add the GitHub Actions job

GitHub Actions workflow syntax and action versions can change. The following is a workflow outline, not a claim about current action versions or a copy-paste verified action recipe. Before committing a workflow, check the current official GitHub Actions documentation for runner images, checkout and setup actions, artifact upload syntax, and the permissions required by your repository.

# .github/workflows/visual-regression.yml
name: Visual regression

on:
  pull_request:
  push:
    branches: [main]

jobs:
  backstop:
    runs-on: ubuntu-latest
    steps:
      - name: Check out source
        uses: actions/checkout@<verified-version>

      - name: Set up Node.js
        uses: actions/setup-node@<verified-version>
        with:
          node-version: <project-node-version>
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Build and start the app
        run: |
          npm run build
          npm run start:test > app.log 2>&1 &

      - name: Wait until the app is ready
        run: <project-specific-readiness-check>

      - name: Run BackstopJS
        run: npm run visual:test

      - name: Retain visual report and CI report
        if: always()
        uses: actions/upload-artifact@<verified-version>
        with:
          name: backstop-report
          path: |
            backstop_data/html_report/
            backstop_data/ci_report/
            app.log

Replace each placeholder with values verified for your repository and the current official GitHub documentation. Define start:test and the readiness check for your application. If the test step fails, the unconditional artifact step should still retain whatever report files were produced; confirm the artifact step’s current syntax and behavior before relying on it. Reviewers can then inspect the HTML report and compare images when a job fails.

BackstopJS documents JUnit XML CI reporting, with a default output under test/ci_report/xunit.xml. If you use that default, update the artifact path accordingly; alternatively configure the CI report path as shown above and verify the produced location. Retaining an XML file as an artifact makes it downloadable; publishing it into GitHub’s test-results interface requires a separate, currently supported integration and should be configured from that integration’s official documentation.

6. Choose runner-native or Docker rendering

Approach	When it fits	Things to account for
Runner-native	You want a simpler job and can keep the runner, browser dependencies, and rendering environment stable enough for your project.	Browser and operating system differences can affect pixels. Pin the Node version and dependencies; investigate any browser setup your chosen BackstopJS version requires.
Docker	You want a more controlled rendering environment. BackstopJS documents `--docker` as a way to reduce rendering differences across environments.	Docker must be available, the container must reach the app, generated files need usable ownership, and piped CI output should not use Docker’s `-t` option.

To run through the documented Docker option, try:

npx backstop test --docker

Docker can reduce cross-environment differences, but it does not guarantee pixel-identical output in every environment. If the app runs on the host while BackstopJS runs in a container, localhost inside that container may not point to the host app. The BackstopJS project gives host.docker.internal for Mac and Windows examples; verify the appropriate host access method for your runner. The project also advises configuring the container user to match the host user and group where appropriate to avoid file ownership problems.

BackstopJS documents a Docker image listing, but do not assume an old image listing is a maintained or suitable version. Verify the image and pin an appropriate version before using it in CI.

7. Keep reports useful to reviewers

A failing job is most useful when a reviewer can see which scenario changed and inspect the before-and-after images. Preserve the HTML report and generated test screenshots as artifacts, including on failure. Preserve the JUnit XML too if your team consumes test reports. Ensure the artifact paths match the actual configured paths; a successful upload step cannot retain files that were written elsewhere.

Reviewers should distinguish expected changes from regressions. If a change is approved, update the references through the explicit approval workflow and include that update in a reviewable commit. If it is not approved, fix the page or test setup and rerun the comparison against the existing baseline.

8. Troubleshooting

Symptom	Likely cause	Fix
Navigation or connection errors	The app has not started, is listening on another port, or is unreachable from the browser/container.	Check the app log and configured scenario URL. Add a readiness check, and verify network reachability from the environment running the browser.
Every screenshot differs from the baseline	The browser, operating system, fonts, viewport, data, or app state changed.	Stabilize the rendering environment and test data. Compare the report before updating references; approve only intentional changes.
First run reports missing references	No approved reference set exists for the configured scenarios.	Capture in a controlled environment, review the output, then create and commit the initial references using the documented approval process.
Docker cannot load a host app URL	`localhost` points at the container itself.	Use a host address reachable from the container; BackstopJS documentation mentions `host.docker.internal` in Mac/Windows examples. Confirm the runner-specific networking setup.
Permission denied or root-owned files	The container wrote output with a different user or group than the checkout owner.	Configure the container user and group to match the host where appropriate, as the project recommends, or correct ownership before later workflow steps need the files.
CI output is malformed or TTY-related errors occur	Docker is being run with a TTY option in a piped environment.	Remove Docker’s `-t` option in CI output pipelines.
JUnit file is missing from artifacts	The report path differs from the configured or documented output path, or the test failed before report generation.	Check the BackstopJS configuration and logs, then point artifact collection at the actual output. The documented default is `test/ci_report/xunit.xml`.
Artifacts are empty after a failed test	The artifact step did not run after failure, or its path does not match the report output.	Use the current GitHub-supported always-run condition for collection, verify the action syntax, and check the job’s files and paths.

9. Performance, reliability, and cost

Visual test runtime grows with the number of scenarios and viewports, plus the time each page needs to load and render. Start with critical routes and the viewport sizes that affect users; add coverage where it catches meaningful layout risks. Keep test data and page state deterministic, and avoid unnecessary animations or time-dependent content where the app allows a stable test mode.

For reliability, pin the dependency through the lockfile, choose a consistent rendering environment, wait for app readiness, and inspect reports before accepting a baseline update. Docker is an option to reduce environment differences, with networking and file ownership tradeoffs. GitHub Actions runner and artifact costs depend on your repository setup and current GitHub terms; this workflow does not imply a particular cost or runtime.

10. Or skip the browser setup

For a one-off screenshot or a capture outside your CI browser environment, ScreenshotNeo provides a website screenshot API and MCP server. The API accepts one GET request with a URL and can return PNG, JPEG, WebP, or PDF. See the ScreenshotNeo site and API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo accepts cookie and consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each of those steps can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and the response identifies the page verdict and billing status in headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 screenshots. ScreenshotNeo is useful for obtaining a clean capture without setting up a browser in your own job; it does not replace BackstopJS’s reference comparison and approval workflow.

Create a free ScreenshotNeo account for 1,000 screenshots a month with no card.

FAQ

Should a pull request automatically approve its screenshots?

No. Review the visual differences first, then promote the intended result into the reference set through the approval workflow.

Can JUnit XML replace the BackstopJS visual report?

They serve different review needs. JUnit XML is structured CI test output; the visual report helps people inspect screenshot differences.

Does Docker make screenshot comparisons identical everywhere?

No. The BackstopJS project presents Docker as a way to reduce environment differences, not a guarantee of identical rendering.