How to Run Parallel Lighthouse Tests in Your CI Pipeline

To run Lighthouse tests in parallel in CI, split your URLs into groups and run Lighthouse CI (LHCI) in separate CI jobs or matrix workers. Treat CI job fan-out as the parallel layer. Within each worker, use lhci autorun or the collection, assertion, and upload commands. LHCI’s numberOfRuns repeats measurements for each URL; it is not a documented control for parallel workers. Keep collection settings and run counts consistent across shards so their results remain comparable.

This approach can reduce elapsed time when your CI provider runs jobs concurrently and has enough runner capacity. It also adds artifact and upload decisions: separate jobs do not automatically share LHCI’s local .lighthouseci/ reports. The official documentation describes LHCI’s collection and report flow, but does not prescribe one universal matrix-sharding or cross-job merge recipe. Adapt the examples below to your CI provider and decide explicitly how a build’s reports will be represented.

1. Understand the two kinds of repetition

LHCI collection accepts multiple URLs, and runs Lighthouse three times per URL by default. The numberOfRuns option changes how many measurements LHCI takes for each URL. It does not tell LHCI to start that many workers. The configuration also supports passing a URL multiple times to lhci collect. These are collection options, not a documented general local worker-concurrency setting. See the official LHCI configuration documentation.

Mechanism	What it repeats	When to use it
CI matrix/jobs	Independent groups of pages	To distribute work across available CI runners
numberOfRuns	Measurements of each URL	To reduce the effect of natural run-to-run variability
Multiple --url options	URLs collected by one LHCI worker	To audit a group together without CI fan-out

Parallel jobs and repeated measurements solve different problems. Fan-out may shorten wall-clock time, but it does not itself make scores more stable. Repeated runs improve the basis for evaluating variable measurements, but take longer. For thresholds and comparisons, use consistent repeated runs and aggregate results rather than treating a single score as definitive.

2. Choose how to start LHCI

Use autorun for the standard workflow

lhci autorun is the simplest integration point. It runs the applicable collection, assertion, and upload steps using your project configuration and defaults. It is a good fit when each matrix worker can use the same configuration with a different assigned URL group.

Use individual commands for custom control

LHCI’s typical command flow is healthcheck, collect, assert, and upload. Collection writes report files, assertions evaluate them, and upload sends reports and assertion results to a configured target. The local handoff directory is normally .lighthouseci/. Use the individual commands when a job needs explicit health checks, staged execution, or different handling for reports. See the official LHCI architecture documentation.

3. Make the site available to the audit job

Each worker must be able to reach the version of the site it is auditing. For a static build, configure staticDistDir. For a custom server, configure startServerCommand and the audit URLs. LHCI’s getting-started guide covers both approaches and CI setup examples: Getting started with Lighthouse CI.

Install LHCI as a project dependency or invoke the CLI through your package manager. Pin a version compatible with your project and update it deliberately; documentation examples may refer to older versions. The following example assumes the CLI is installed in the project and available as lhci through an npm script or package runner.

Example LHCI configuration

This minimal configuration collects three runs per URL and sets representative assertions. Adjust the URLs, server setup, and assertion thresholds to suit the project. If the build uses static assets, set staticDistDir to the built directory; with a custom server, configure its start command and the URLs served by it instead.

{
  "ci": {
    "collect": {
      "url": ["http://localhost:8080/"],
      "numberOfRuns": 3
    },
    "assert": {
      "assertions": {
        "categories:performance": ["warn", {"minScore": 0.8}],
        "categories:accessibility": ["error", {"minScore": 0.9}]
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

Temporary public report storage means anyone with the report URL can access it. Choose a storage target appropriate for your report privacy. The example thresholds are illustrative configuration values, not Lighthouse recommendations or guarantees.

4. Split URLs across CI workers

Divide pages into groups with a reason behind each assignment: for example, core product pages, commerce flows, and help pages. Assign each worker only its own URLs. Keep the groups similar enough that no single worker becomes a long-running tail, while preserving a simple mapping from reports to the pages and build they represent.

Provider-neutral workflow shape:

jobs:
  lighthouse:
    strategy:
      matrix:
        page_group: [core, commerce, help]
    steps:
      - checkout
      - install dependencies
      - build application
      - start or configure the built site
      - select URLs assigned to matrix.page_group
      - run LHCI on those URLs
      - retain or upload the worker reports

This is pseudocode, not a ready-to-run workflow for a specific CI provider. Matrix syntax, artifact retention, runner concurrency, and report-upload behavior differ by provider. In particular, confirm whether each worker’s upload should represent part of one build or a separate build result. The sources document LHCI’s local report handoff, not a universal cross-job aggregation process.

Keep worker settings aligned

• Use the same LHCI version, browser mode, run count, form factor, throttling, and assertions on every shard.
• Use the same build revision and equivalent server configuration.
• Keep authentication and storage state consistent for pages that require login.
• Match runner hardware class and, where possible, geography for comparisons across workers.
• Preserve each worker’s reports as artifacts or upload them to a configured target that suits your privacy and build-reporting needs.

If one shard uses a five-run median and another uses one run, comparing the resulting scores as though they were equivalent can mislead. Likewise, a shard auditing a different build, device form factor, or login state cannot provide a clean comparison.

5. Configure runs and assertions responsibly

numberOfRuns defaults to three per URL. You can raise it when stable thresholds matter more than collection time. LHCI documents this autorun example:

lhci autorun --collect.numberOfRuns=5

When passing child-command options through autorun, use the equals form. A greater run count increases audit work. Decide whether every page needs repeated runs or whether a smaller, representative set should receive the more expensive stability treatment.

Lighthouse’s variability guidance recommends repeated runs and aggregate measures such as median, percentile, or minimum/maximum when choosing thresholds. It states: “The median Lighthouse score of 5 runs is twice as stable as 1 run.” That is guidance about the cited median score, not a promise that every metric, page, or environment will behave identically. See Lighthouse’s variability documentation.

Assertions can warn during rollout or fail a build for conditions that are stable and important enough to enforce. Start with useful reports and assertions, inspect normal variation, and then set thresholds based on the measures you care about. If a score fluctuates near a hard boundary, a single measurement may create noisy failures. More runs and aggregate measures can help, but controlling the environment and the page’s nondeterminism matters too.

6. Reduce variance across pages and builds

Lighthouse scores can change with both the page and its environment. The LHCI troubleshooting guide highlights hardware, form factor, throttling, storage clearing, authentication state, browser mode, and geography as factors to align when comparing results. Use comparable settings on all workers and across the builds you compare.

Reduce nondeterministic page behavior where practical: random timeouts, A/B tests, and variable third-party integrations can make repeated measurements disagree. Assert direct facts where possible before relying on higher-level outcomes. For thresholds enforced in CI, use reliable, dedicated hardware. The LHCI troubleshooting guide’s guidance mentions machines with at least two cores and 4 GB RAM as a recommendation for variance work, not as a universal guarantee of stable results. See LHCI troubleshooting guidance.

7. Handle reports across job boundaries

In the standard local sequence, collect writes reports under .lighthouseci/; subsequent assertion and upload steps use that local output. Separate CI jobs have separate filesystems unless you explicitly transfer artifacts or otherwise share output. Plan this boundary before enabling the matrix.

1. Give every worker a clear page-group identity in logs and artifact names.
2. Save the worker’s .lighthouseci/ output if a downstream job or developer needs the raw reports.
3. Choose whether each worker uploads its own results or a later job handles the build’s report presentation.
4. Check the destination’s privacy and retention behavior before uploading reports.
5. Verify that your chosen arrangement presents the intended build coherently; do not assume separate worker uploads merge into one build automatically.

For a single-job setup, let autorun manage the normal configured flow or invoke collection, assertion, and upload in sequence. For multi-job setup, CI artifacts are the explicit bridge for local reports, while the exact downstream aggregation design depends on the provider and reporting target.

8. Performance, reliability, and cost

CI fan-out can reduce elapsed time when the provider has spare concurrent runners. It can also consume more runner capacity and make reports harder to consolidate. If the provider queues matrix jobs or limits concurrency, extra shards may not improve wall-clock time. There is no universal speedup figure: page weight, browser work, runner capacity, and queueing all affect duration.

Repeated runs increase collection work in proportion to the number of measurements. Three runs per URL is the LHCI default; more runs trade time and compute for a better sample of natural variability. Start with the least expensive setup that answers the team’s question, then increase runs for thresholds that must be dependable.

Lighthouse CI is software installed with npm, so its cost in a pipeline depends on your CI provider’s runner and artifact policies; the cited Lighthouse documentation does not establish a universal price. Dedicated hardware may improve consistency, but it has a resource cost. Consider how often audits run, how many URLs each build covers, how many runs each URL needs, and whether reports must be retained.

9. Troubleshooting

Symptom	Likely cause	What to do
Scores jump between runs	Variable page behavior, third-party content, A/B tests, or different environments	Repeat measurements, use aggregate values, align worker settings, and reduce nondeterministic behavior.
One matrix job cannot reach the site	The server is not started in that job, or the URL is only reachable from another job’s network	Build and start/configure the site within each worker, or provide a reachable preview URL; check the configured URL and server command.
Assertions fail intermittently	A threshold sits within normal variance or the assertion relies on a noisy outcome	Inspect reports over repeated runs, select a stable aggregate or direct assertion, and set thresholds based on observed variation.
Reports are missing in a later job	.lighthouseci/ exists only in the collecting job’s filesystem	Retain and download the directory as a CI artifact, or upload reports in the worker according to the chosen design.
Reports expose sensitive information	The configured temporary public target makes reports accessible to anyone with the URL	Use a storage target suitable for private reports and avoid publishing sensitive report links.
GitHub Actions cannot find an ancestor commit	The checkout lacks enough history or the base branch is unavailable	Fetch the base branch and increase checkout depth; LHCI troubleshooting suggests fetch-depth: 20 as a starting point, which may need adjustment.
More matrix jobs do not make the build finish sooner	Jobs are queued, concurrency is capped, or the split creates an uneven slow shard	Check runner availability and queue time, then rebalance URL groups. Avoid assuming matrix fan-out guarantees a speedup.

10. Or skip the browser setup

If your pipeline also needs clean page screenshots for visual review, issue tracking, or documentation, ScreenshotNeo provides a one-request screenshot API and an MCP server for AI agents. It accepts the cookie or consent banner like a visitor, then removes 60+ known consent platforms, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers identify the page verdict and billing status. AI agents can use its MCP tools, including take_screenshot, get_page_info, and capture_pdf. It is a screenshot service, not a replacement for Lighthouse performance audits or LHCI assertions.

Example cURL request:

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Equivalent Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Equivalent Node.js:

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await require('node:fs/promises').writeFile('shot.webp', Buffer.from(await res.arrayBuffer()));

See the ScreenshotNeo API documentation for request options. Cookie banners, popups, and chat widgets are removed before the shot; bot checks, blank pages, and failed loads are never billed; an MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card, and paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month, with no card.

FAQ

Does more numberOfRuns mean more tests run at once?

No. It repeats Lighthouse measurements for each URL. Use CI jobs or matrix workers for the parallel layer.

Should every URL have five runs?

Not necessarily. Choose run counts based on how much stability your decision needs and the time available. LHCI defaults to three; Lighthouse describes the five-run median as twice as stable as a single run.

Can I use PageSpeed Insights collection in LHCI?

The LHCI configuration documentation says that the PageSpeed Insights method can audit only sites publicly available over the internet and ignores other collection options. Use local Node collection when you need to audit localhost or a private preview environment.

Can I treat each shard’s upload as one combined build?

Do not assume so. The documented architecture explains local collection and upload handoff, but does not define one universal cross-job merge pattern. Confirm how your selected storage and CI artifacts should represent the build.