How to debug missing screenshots in an Argos CI build
Trace missing Argos CI screenshots through capture, reporter setup, authentication, file uploads, and baseline matching with a step-by-step Playwright guide.
When screenshots are missing from an Argos CI build, first find out which stage failed: the test may not have captured a named screenshot, the Argos reporter may have skipped upload, authentication may be unavailable, the uploader may be looking in the wrong place, or the screenshot may not match a baseline. Check the CI workspace and test output before changing configuration. The example below is specific to Playwright; other integrations have their own setup.
1. Confirm the test captured the screenshot
Start with the test command and its exit status. Check test filters, skipped tests, and failures, then inspect the CI workspace for the expected file. If the file is absent, troubleshoot capture before upload.
For the Argos Playwright integration, a named visual snapshot uses argosScreenshot(page, "name"). The quickstart writes these screenshots to ./screenshots by default. Playwright’s screenshot: "only-on-failure" setting is useful for debugging failed tests, but it is separate from the named Argos snapshot call; enabling it alone does not create an Argos visual snapshot.
import { test } from "@playwright/test";
import { argosScreenshot } from "@argos-ci/playwright";
test("home page visual snapshot", async ({ page }) => {
await page.goto("https://example.com");
await argosScreenshot(page, "home");
});Confirm the test that contains the call actually ran in CI. If your project overrides the Argos screenshot output directory, use that actual path in the later upload checks.
2. Check the Playwright reporter and CI upload condition
In the Playwright quickstart, the reporter is added to the reporter list and configured to upload when CI is truthy. Local snapshots can therefore exist while CI upload is disabled if the process does not have the expected environment variable.
import { defineConfig } from "@playwright/test";
import { createArgosReporterOptions } from "@argos-ci/playwright/reporter";
export default defineConfig({
reporter: [
["list"],
[
"@argos-ci/playwright/reporter",
createArgosReporterOptions({ uploadToArgos: !!process.env.CI }),
],
],
});Keep any existing reporters in the list. Check the effective configuration used by the CI test command, including project-specific overrides. The documented condition checks CI; do not assume an unrelated runner variable enables upload. For Cypress, Puppeteer, WebdriverIO, Storybook, Vitest, or another integration, follow that package’s current configuration rather than copying Playwright settings. The official Argos JavaScript SDK lists these integrations and packages. Argos JavaScript SDK repository.
3. Verify authentication in the job that runs tests
The Playwright GitHub Actions example passes ARGOS_TOKEN to the test step. Argos identifies this as the project token found in Settings → General → Token. Check that the credential is available to the exact job and step that runs the tests, and inspect your repository’s secret-access rules for forked pull requests or other restricted contexts. Never print the token into logs.
name: visual-tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npx playwright test
env:
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}This is an illustrative workflow based on the Argos quickstart sequence; adapt action versions and project commands to your repository. Argos also documents OIDC or tokenless authentication for GitHub Actions. Use its current setup instructions if choosing that route instead of inventing a provider configuration. Argos Playwright Quickstart.
4. Check the upload directory, working directory, and file pattern
If screenshots exist in CI but not in Argos, compare the actual file paths and extensions against the uploader configuration. A direct upload using @argos-ci/core can point to a root directory and glob such as ./screenshots and **/*.png:
import { upload } from "@argos-ci/core";
await upload({
root: "./screenshots",
files: ["**/*.png"],
});Run the upload from the expected working directory. Confirm the files are PNGs if the glob ends in .png; change the pattern when your capture format differs. This example uses the SDK’s default process.env.ARGOS_TOKEN behavior. If you manage upload separately from the Playwright reporter, make sure you have not accidentally configured both paths in a way that obscures which one is responsible for upload. See Argos core SDK documentation.
The Playwright helper’s default screenshot directory and the core SDK’s upload root are examples, not guarantees about your repository. Inspect the CI workspace itself rather than relying only on local output.
5. Decide whether the screenshot is unmatched or missing
A build can contain a screenshot that does not appear as a comparison against the baseline. Argos reports an unmatched screenshot as added. Check whether the screenshot’s name changed, whether the default branch has a completed build, and whether the build page maps the expected baseline.
Argos requires a build on the default branch before pull request builds have a baseline; without one, pull request builds are marked orphan. A newly named screenshot may also have no same-name baseline. When using fallback baselines, provide names in priority order: try the variant-specific name first, then the fallback snapshot name. After a baseline exists for the variant, its own name can be used.
await argosScreenshot(page, "home-dark", {
baseName: ["home-dark", "home"],
});Use the option shape documented for your installed Argos Playwright package version. For SDK-free metadata uploads, Argos documents transient.baseName; those names include file extensions and can include a Playwright project prefix such as chromium/home.png. Check the build page’s mapping when the file is present but comparison behavior is unexpected. Argos fallback baselines.
6. Use the evidence to locate the broken stage
| What you observe | Likely stage | Next check |
|---|---|---|
| No screenshot file in the CI workspace | Test capture | Test selection, test status, and named argosScreenshot call |
| File exists, but no Argos upload/build entry | Reporter, upload condition, credentials, or file selection | Reporter list, CI, token availability, working directory, and glob |
| Build exists, screenshot is marked added | Baseline matching | Snapshot name, default-branch baseline, and fallback names |
| Some files upload, others do not | Path or file pattern | Extensions, nested paths, and glob coverage |
Before making changes, collect a compact diagnostic record: commit and branch, CI provider and job, test framework and SDK version, exact test command and exit status, whether screenshot files exist and their paths/extensions, reporter and upload settings, whether a token is available without revealing its value, upload logs, and the Argos build link or status. This checklist follows the documented capture, upload, and baseline stages; it is a practical debugging aid.
7. Troubleshooting common causes
| Symptom | Cause to check | Fix |
|---|---|---|
| No output file | The relevant test did not run, failed before capture, or lacks the named Argos helper call. | Check filters and test output; add or restore the named capture call in the test path that should produce the image. |
| Files appear locally but not in CI | The CI test command uses different configuration or environment settings. | Inspect the config selected by CI and verify the reporter is enabled for that process. |
| Local capture works but CI build has no upload | The reporter is absent or uploadToArgos evaluates false because CI is missing. |
Add the reporter and verify the expected environment variable in the test step. |
| Authentication or upload error | ARGOS_TOKEN is missing from the executing job, unavailable in that pull request context, or invalid. |
Check secret scope and project token configuration without logging the credential; for GitHub Actions, consult Argos’s OIDC/tokenless documentation if appropriate. |
| Upload finishes but no expected images appear | Wrong working directory, root path, extension, or glob. | Compare configured paths and pattern to files in the CI workspace; account for nested directories and actual file formats. |
| Screenshot shown as added, with no comparison | No same-name baseline exists, or the default-branch baseline is missing. | Run the default branch, verify snapshot names, and configure fallback baseline names where needed. |
Manual upload instructions refer to argos-cli |
The standalone CLI repository is marked deprecated. | Use the current Argos JavaScript SDK documentation and the integration package appropriate to the project. |
If the files exist and the reporter, credentials, path, and glob check out but the build still omits them, the next useful evidence is the framework-specific SDK version and complete upload error/log output. The title alone does not identify which stage failed.
8. Improve reliability and keep CI costs predictable
- Make the capture path explicit. Keep named snapshots in the test that owns them, and use a stable snapshot name unless you intentionally need a separate baseline.
- Validate the environment in the job. Confirm the same process that invokes the tests receives the reporter configuration and authentication method. Avoid printing secret values.
- Align capture and upload settings. If you customize output paths, update the uploader root and glob together. A mismatch can look like successful tests with missing visual results.
- Establish the baseline before relying on PR comparisons. Ensure the default branch has completed a build, then use fallback names intentionally for variants.
- Separate useful diagnostics from noisy logs. Record test status, file inventory, upload outcome, and build link so a rerun can be compared without exposing credentials.
These checks target wasted CI reruns caused by a broken pipeline configuration. The supplied Argos documentation does not establish timing or cost benchmarks, so treat performance and CI spend as project-specific; measure your own test, upload, and rerun time.
Or skip the browser setup
ScreenshotNeo is a website screenshot API and MCP server from ScreenshotNeo. It does not replace Argos’s visual baseline workflow, but it can capture a page without requiring you to set up a browser in your own code. See the ScreenshotNeo API documentation.
curl -G "https://api.screenshotneo.com/v1/shot" \
-d access_key=YOUR_API_KEY \
--data-urlencode url=https://example.com \
-o shot.webpimport requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
timeout=90,
)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({
access_key: 'YOUR_API_KEY',
url: 'https://example.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer())));Cookie and consent banners, newsletter popups, and chat widgets are removed before the shot. Bot checks, blank pages, and failed loads are never billed. An MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up free for ScreenshotNeo.
FAQ
Does Playwright’s failure screenshot option create an Argos snapshot?
No. The Argos Playwright example uses the named argosScreenshot helper for visual snapshots. Failure-only screenshots serve a separate debugging purpose.
Why does an uploaded screenshot show as added?
It may not have a matching baseline name. Check the default-branch build and configure fallback names for variants when appropriate.
Can I apply this exact reporter configuration to Cypress or Vitest?
No. The code shown is for Playwright. Use the current documentation for the SDK package that matches your framework.
What information should I share when asking for help?
Share the framework and SDK version, CI job and command, file paths and extensions, reporter and uploader configuration, redacted upload errors, and Argos build status. Do not share secret values.
