How to Run Reg-suit Screenshot Tests on GitLab CI

Reg-suit compares image files and creates a visual difference report; it does not render your application or take screenshots. In GitLab CI, first run your existing browser, Storybook, or other capture command and write its images to the directory configured as core.actualDir. Then run npx reg-suit run after configuring a key generator and a publisher for expected snapshots.

This guide uses a project-local Reg-suit install. Treat the CI YAML as a starting shape: browser dependencies, runner image, Git history, artifacts, credentials, and storage settings depend on your project.

1. Install and configure Reg-suit

Install Reg-suit and initialize its configuration in the repository. The project documentation describes init, prepare, and run; setup plugins according to how you want to select baselines and store snapshots.

npm install --save-dev reg-suit
npx reg-suit init

Commit the resulting regconfig.json and the project dependency lockfile. The initialization flow can help configure plugins. The exact plugin choices are project decisions:

Key generator: determines which expected snapshot key or commit is used. The Git-hash plugin walks the Git branch graph; a simple key generator can be suitable when the project manages baseline keys another way.
Publisher: retrieves expected images and publishes current images and reports. Reg-suit lists S3 and GCS publisher plugins. Storage access and retention are your responsibility.
Notifier: optional. A notifier can make results visible in GitLab merge requests; it is separate from the core comparison.

Reg-suit’s README documents the CLI and configuration. The project overview describes the workflow.

2. Generate screenshots into actualDir

Use the screenshot command your application already relies on. Configure it to write the images Reg-suit should compare into the directory named by core.actualDir. The capture step must finish before the Reg-suit command starts.

// Example package.json scripts; replace with your project's real commands.
{
  "scripts": {
    "build": "your-build-command",
    "screenshots": "your-existing-screenshot-command"
  }
}

Keep the capture environment stable across baseline and candidate runs: use the same viewport, browser version, fonts, locale, data fixtures, and animation policy where possible. Otherwise the diff may reflect environment variation instead of a product change. This is a workflow recommendation; the specific controls depend on your capture tool.

Verify that expected files exist at the exact configured path and that corresponding filenames are produced for the baseline and current run. Missing images or inconsistent names can make comparison incomplete or misleading.

3. Configure comparison behavior

The Reg-suit README documents these relevant settings:

Setting	Purpose	Documented default
`core.actualDir`	Directory containing current screenshot files	Required directory; set it to your capture output
`thresholdRate`	Ratio of changed pixels to all pixels tolerated	`0`
`thresholdPixel`	Absolute changed-pixel threshold	`0`
`enableAntialias`	Controls antialias handling	`false`
`matchingThreshold`	Comparison matching threshold	See the project configuration documentation
Comparison concurrency	Number of comparisons run concurrently	`4`

Use a threshold that fits your app’s visual tolerance. There is no universally correct value: a threshold can reduce noise, but a generous threshold can also hide a real regression. Review the generated diff before changing it, then document why the chosen tolerance is acceptable.

4. Add a GitLab CI job

Run dependency installation, build, screenshot generation, and Reg-suit in that order. The upstream GitLab example checks out $CI_COMMIT_REF_NAME before invoking Reg-suit because the Git-hash key generator needs branch context. Confirm that your job actually has the branch and commit history needed by the selected key generator. GitLab checkout depth, detached checkouts, fork pipelines, and merge-request pipelines can differ.

visual-regression:
  stage: test
  script:
    - npm ci
    - npm run build
    - npm run screenshots
    - git checkout "$CI_COMMIT_REF_NAME" || git checkout -b "$CI_COMMIT_REF_NAME"
    - npx reg-suit run

This is an ordering example, not a tested universal pipeline. Adapt it to the runner and screenshot system. A browser-capable image, system packages, services, cache, artifacts, or fetch configuration may be needed. The project’s sample includes a git pull after checkout; do not copy that blindly. Validate credentials, branch availability, and shallow history for your pipeline before relying on it.

The run command is documented as the combined sync-expected, compare, and publish -n sequence: it gets expected snapshots through the configured key generator and publisher, compares them with actualDir, publishes actual images and a report, and invokes installed notifier plugins.

5. Make baseline selection meaningful

A baseline is useful only if its selection matches your review process. With the Git-hash plugin, Reg-suit selects a comparison commit by walking the Git branch graph. The available history and branch graph therefore matter. Make sure CI has fetched enough history and the relevant branch references. Do not assume that a merge-request pipeline automatically selects the target branch: the project overview’s automatic parent-commit description is specifically about GitHub flow, not a guarantee of GitLab merge-request detection.

Choose the baseline semantics your team wants: for example, compare against a prior commit identified by the configured key generator.
Check that the chosen key generator can see the required commits and refs in the GitLab job.
Confirm the publisher can retrieve the expected snapshots for that key.
Inspect the report for a known intentional change and verify that it compares the images you expect.

Git-hash and simple key generators serve different baseline workflows. Pick based on how your project identifies and updates expected images; neither is a universal choice.

6. Add optional GitLab merge-request comments

To post results in a merge request, install and prepare the GitLab notifier plugin:

npm install --save-dev reg-notify-gitlab-plugin
npx reg-suit prepare -p notify-gitlab

The plugin requires a GitLab API token. Its documentation says it can detect gitlabUrl and projectId from GitLab CI predefined environment values, so project ID can be omitted in that context. Keep the token in masked and appropriately protected CI configuration, and check GitLab’s current token permission requirements for your project. The plugin README does not establish a universally authoritative minimum scope.

The plugin can put its output in a merge-request note, description, or discussion; the documented default is a note. Configure the destination that fits your review workflow. A comment is optional and does not replace the comparison or report.

See the GitLab notifier documentation for its setup options.

7. Choose snapshot storage and access

Publisher plugins provide the expected-image retrieval and publication step. The project lists S3 and GCS publishers. Choose storage based on your team’s existing access, retention, and review needs.

For the S3 plugin, CI must be authorized to access the bucket. Its documentation lists bucket name, ACL, server-side encryption, custom domain, path prefix, and SDK options. It lists object read, write, and delete actions plus bucket listing among the IAM actions. The README documents public-read as the default ACL; review the access settings and choose deliberately rather than assuming public access is needed.

See the S3 publisher documentation for configuration details. Configure credentials as CI variables or through the runner’s supported identity mechanism, and grant only the access the project requires.

8. Inspect reports and tune the workflow

Review the generated report when a job changes. Confirm which baseline was fetched, whether every intended image was found, and whether differences are expected. If the report is too noisy, investigate unstable inputs and rendering conditions before raising thresholds. If comparisons take longer as the suite grows, review image count, image dimensions, publisher transfer, and configured comparison concurrency; the documented default concurrency is four.

Keep reports or relevant screenshot artifacts available to reviewers if your GitLab job needs them after completion. Artifact retention and access settings are GitLab project choices, not Reg-suit guarantees.

Common errors and fixes

Symptom	Likely cause	What to check
No current images or an empty comparison	Capture step did not run, failed, or wrote elsewhere	Run the capture command locally; verify files and match the output path to `core.actualDir`.
Expected images cannot be found	Publisher access or key selection is wrong	Check the selected key, publisher configuration, credentials, and whether the baseline was previously published.
Git-hash plugin cannot find a base	Required branch refs or commit history are missing	Inspect checkout state, fetch depth, and branch availability; ensure the job has the graph required by the configured key generator.
Checkout fails for `CI_COMMIT_REF_NAME`	Ref is unavailable in that pipeline context, or checkout is detached/shallow	Inspect GitLab’s actual ref variables and fetched refs. Adjust fetch and checkout logic for the pipeline type instead of assuming the sample applies unchanged.
Browser capture fails in CI	Runner image lacks a browser or its system dependencies	Use a runner environment supported by your capture tool and install its documented dependencies.
MR note is missing	Notifier is not installed/configured, token is absent or lacks permission, or pipeline is not an MR context	Check plugin setup, masked token availability, GitLab project variables, and the notifier destination.
S3 publish or fetch is denied	CI identity lacks required bucket or object access	Review bucket policy, IAM actions, credentials, and chosen ACL; do not make the bucket public as a workaround without an explicit access decision.
Many unexpected pixel diffs	Rendering inputs vary or threshold is too strict for accepted variation	Stabilize browser, fonts, viewport, data, locale, and animation behavior; review diffs before adjusting threshold.
Real changes are not flagged	Threshold may be too permissive or compared images are not the intended files	Verify filenames and baseline, then lower tolerance if the diff review shows meaningful changes are being suppressed.

Performance, reliability, and cost

Reg-suit’s work depends on the number and size of images, comparison concurrency, Git history lookup, and publisher transfer. Keep captures focused on meaningful states, avoid regenerating unrelated images, and use CI caching only where it does not reuse stale build or screenshot output. The documented comparison concurrency default is four; tune it only after observing the job and runner constraints.

Reliability depends on deterministic capture and a retrievable baseline. Pin or otherwise control the browser environment, ensure the screenshot command exits unsuccessfully when capture fails, and preserve the Git refs and publisher permissions the baseline flow requires. The dossier provides no benchmark or universal runtime estimate.

Reg-suit is an open-source project workflow in the cited material; actual costs come from your CI minutes, runner resources, and any storage or services you choose. The sources do not provide a general cost estimate. Keep storage access private where appropriate and factor artifact retention into your own platform settings.

Or skip the browser setup

Reg-suit still compares image files, so a capture step is needed before comparison. If you need a screenshot API for that step, ScreenshotNeo returns a screenshot or PDF from one GET request and offers an MCP server for AI agents.

With ScreenshotNeo, cookie banners, newsletter popups, and chat widgets are removed before the shot. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed. The API response includes page-verdict and billing headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for MCP clients such as Claude and Cursor.

Example cURL request (see the ScreenshotNeo API documentation for options):

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer())));

ScreenshotNeo includes 1,000 screenshots a month free with no card; paid plans start at $5 for 3,000 shots. Every feature is on every plan. Use the response image as an input to your project’s comparison workflow where that fits your needs. Sign up for the free plan and get 1,000 screenshots a month with no card.

FAQ

Does Reg-suit take screenshots?

No. It compares image files. Your application’s browser, Storybook, or another capture process must create them first.

Does `reg-suit run` automatically detect a GitLab merge-request target branch?

Do not assume so. Baseline selection depends on the configured key generator and the refs and history available in CI.

Do I need the GitLab notifier plugin?

No. It is optional; install it when you want Reg-suit results posted into a merge-request note, description, or discussion.

Can I use Reg-suit without S3?

Yes. S3 is one publisher option; the project also lists GCS, and publisher choice depends on your snapshot storage workflow.