ScreenshotNeo

BlogEngineering

How to Run 100 UI Tests in 20 Seconds

Learn what Applitools’ 21-second Storybook demo actually measured, how parallel visual testing works, and how to apply the approach without confusing it with end-to-end testing.

By the ScreenshotNeo team4 October 20267 min read

Short answer: Applitools reported processing 100 visual test runs in 21 seconds in a specific Storybook demonstration. The setup covered one Storybook story, 50 unique viewport sizes, and Firefox and Google Chrome. It collected DOM snapshots locally, uploaded them, then rendered those snapshots in parallel for visual analysis. The title rounds the reported time to 20 seconds; it is not a general promise that 100 arbitrary UI or end-to-end tests can finish in that time.

This distinction matters. In the demonstration, the work distributed across browsers was snapshot rendering and visual comparison. A conventional end-to-end test may need to launch a browser, navigate, authenticate, perform actions, wait for application state, and assert behavior for each workflow. Parallelizing that work can help, but it does not make its duration equivalent to a snapshot-rendering batch.

What the 20-second claim actually means

Applitools’ 2019 article reports 21 seconds for its 100-run Storybook visual-testing batch. The configured runs represented a single Storybook story viewed at 50 viewport sizes across two browsers: Firefox and Google Chrome. Applitools described the measured workflow as including SDK work and server-side processing, including snapshot upload, rendering, baseline storage, and reporting. This was a vendor demonstration, not an independent benchmark or a controlled comparison of test frameworks. Applitools’ original demonstration

The same article gives a separate four-second figure for rendering after upload in a smaller example with two browsers and two viewports. Keep that result separate from the 21-second batch: they describe different configurations and stages.

Question What the demonstration says
What was tested? A Storybook visual-testing run for one story, configured across many browser and viewport combinations.
How many configurations? 100 runs, using 50 unique viewport sizes and two browsers: Firefox and Google Chrome.
What took 21 seconds? The vendor describes the SDK/server workflow, including snapshot upload and cloud processing through results.
Was it independently reproduced? The cited material is Applitools’ own demonstration; this research did not find independent replication.

How parallel visual rendering works

The workflow separates snapshot collection from browser rendering. The client-side Eyes Storybook SDK reads the stories, runs them through the Storybook engine, collects DOM snapshots, and uploads them. The cloud service then uses the requested browser and viewport configurations to render those snapshots in parallel, capture images, and send them through visual analysis. Applitools’ 2021 workflow explanation

  1. Define the story and configurations. Choose the Storybook content and the browser and viewport combinations you want to inspect.
  2. Collect snapshots. The SDK runs the story locally and sends its DOM snapshot data to the service.
  3. Render in parallel. The service renders snapshots across the requested configurations rather than replaying the full interaction sequence independently in every browser.
  4. Compare and report. Rendered images are checked against visual baselines, and results are returned for review.

Parallelism reduces the serial rendering time when the service has capacity to process configurations concurrently. It does not remove local snapshot generation, upload time, visual analysis, or the cost of maintaining useful baselines. Nor does a DOM snapshot prove that a real user can complete a workflow: visual coverage and behavioral coverage answer different questions.

How to apply this approach to a project

For a Storybook component library, the useful lesson is to multiply visual coverage across configurations while keeping test execution and rendering as separate stages. The exact SDK commands and configuration may change by version; the cited demonstration dates from 2019 and the workflow explanation from 2021. Consult the vendor’s current documentation before copying implementation-specific setup.

  1. Choose representative stories. Cover important component states such as default, disabled, validation error, long content, and loading. A single story across many sizes demonstrates rendering scale, but it is not broad component coverage.
  2. Select configurations deliberately. Include the browsers and viewport sizes that matter to your users. Avoid a large matrix of near-duplicate dimensions unless each adds meaningful coverage.
  3. Keep story rendering deterministic. Fix dates, random values, animation states, and network-dependent data so baseline changes represent UI changes rather than incidental variation.
  4. Run local checks and upload snapshots. Make sure stories render successfully before interpreting cloud results. Track collection and upload separately from remote rendering if your tooling exposes those timings.
  5. Review diffs before accepting baselines. A baseline update can hide a regression if approved without checking what changed.
  6. Keep end-to-end tests for behavior. Use browser interaction tests for navigation, forms, authentication, and important user journeys. Use visual snapshots to add appearance coverage across configurations.

Plan the matrix

If you have S stories, V viewport sizes, and B browsers, the nominal number of render configurations is S × V × B. This is a count of visual configurations, not necessarily a count of independent user workflows. Start with a compact set of representative viewports and expand where layout behavior changes materially, such as breakpoints, dense tables, or long localized strings.

What parallel visual testing does not replace

Visual snapshot workflow End-to-end interaction workflow
Renders captured story state in selected browser and viewport configurations. Drives a browser through navigation, actions, and application states.
Useful for finding unintended layout, typography, and styling changes. Useful for verifying behavior such as sign-in, checkout, routing, and form submission.
Can render collected DOM snapshots remotely in parallel, as described by Applitools. Usually incurs browser startup, app setup, network, and interaction time per worker or scenario.
Depends on stable stories and reviewed visual baselines. Depends on reliable fixtures, selectors, waits, and assertions.

Do not claim that turning on parallelism alone makes any suite run in 20 seconds. Runtime depends on what is executed, setup and upload overhead, concurrency, application stability, and the configurations being covered. The source material provides no neutral head-to-head benchmark against other systems.

Reliability, runtime, and cost considerations

  • Measure stages independently. Record story startup and snapshot generation, transfer time, remote rendering, analysis, and queue or reporting time where available. A fast rendering stage can still sit behind slow setup or uploads.
  • Limit noisy inputs. Disable or control animation and time-dependent content, stabilize external data, and use consistent fonts and assets to reduce misleading differences.
  • Use concurrency thoughtfully. More configurations increase coverage and may require more service capacity or plan allowance. The dossier does not provide current pricing or capacity limits, so verify those details with the vendor.
  • Budget for baseline review. A large visual suite can produce many diffs. Make changes easy to triage by grouping results by story and configuration and setting clear ownership for approvals.
  • Retain behavioral coverage. Snapshot rendering does not exercise every browser event or backend integration. Keep a smaller set of end-to-end checks for critical workflows.

Troubleshooting

Symptom Likely cause What to do
A story fails before snapshots are uploaded. Storybook startup, story code, or local SDK setup failed. Run the story locally, inspect the SDK and Storybook logs, and resolve rendering errors before investigating cloud rendering.
Snapshots render differently between runs. Dynamic content, animation, unstable fonts, or uncontrolled data changed. Freeze dates and random inputs, disable motion, load deterministic fixtures, and ensure assets are available consistently.
Some browser or viewport results are missing. The requested configuration may not have completed, or snapshot upload may have been incomplete. Check run status and configuration selection, then retry the missing configurations after confirming upload succeeded.
The full batch takes much longer than the reported 21 seconds. The demo’s setup differs from yours; local collection, upload, queueing, suite size, or analysis may dominate. Time each stage separately and compare equivalent configuration counts. Treat the published number as a setup-specific vendor result.
A visual test passes but a user flow is broken. The snapshot checked appearance, not the full interactive behavior or backend effect. Add or retain an end-to-end assertion for the relevant action and outcome.
Many diffs appear after a harmless change. A shared style, font, browser rendering, or baseline environment changed. Inspect representative diffs first, identify the common cause, and approve a baseline update only after confirming expected appearance.

Or skip the browser setup

If the goal is to capture a page screenshot for documentation, review, or an AI workflow rather than build a visual regression suite, ScreenshotNeo offers a one-request website screenshot API. It accepts a URL and returns PNG, JPEG, WebP, or PDF. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
with open("shot.webp", "wb") as f:
    f.write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await require('node:fs/promises').writeFile('shot.webp', Buffer.from(await res.arrayBuffer()));
  • Cookie and consent banners are accepted or removed before capture, and known newsletter popups and chat widgets are removed; each step can be turned off.
  • Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing. Response headers report the page verdict and billing status.
  • An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients.
  • The Free plan includes 1,000 screenshots per month with no card. Paid plans start at $5 for 3,000 screenshots; every feature is available on every plan.

Create a free ScreenshotNeo account and get 1,000 screenshots a month with no card.

FAQ

Can I run any 100 UI tests in 20 seconds?

No. The reported figure concerns a specific Storybook visual-testing demonstration, not 100 arbitrary interactive end-to-end scenarios.

Why use 50 viewport sizes with two browsers?

That was the configuration described in the demonstration. Your matrix should reflect meaningful layout and browser coverage for your application.

Does parallel rendering eliminate upload time?

No. The described workflow collects and uploads DOM snapshots before the cloud rendering and analysis stages.

Is the 21-second result independently verified?

The cited evidence is Applitools’ own demonstration. The research used for this article did not establish an independent reproduction.