How to Set Up Headless Browser Automation for Website Screenshots

Headless browser automation lets a script open a website without a visible browser window, wait for the page to become ready, and save a screenshot. For most projects, start with Playwright: it supports Chromium, Firefox, WebKit, and branded Chrome or Edge channels, and it manages browser installation. Use Puppeteer when your JavaScript project already depends on its Chrome-focused API. Use Chrome Headless from the command line for a one-off URL capture.

The dependable sequence is always the same: install the browser runtime, launch it headlessly, set a known viewport, navigate, wait for a deterministic ready state, capture the viewport, an element, or the full page, then close the browser.

1. Choose the right headless tool

Tool	Best fit	Strengths	Trade-offs
Playwright	New automation and cross-browser work	Chromium, Firefox, WebKit, managed browser installation, selector and network controls	Requires browser binaries and Linux dependencies in CI
Puppeteer	JavaScript projects already using Puppeteer or Chrome	Familiar JavaScript API, navigation waits, element screenshots	Primarily a Chrome-oriented workflow; cross-browser support is narrower than Playwright
Chrome Headless	Simple shell captures and smoke checks	No application code required; easy to run from a script	Interaction, authentication, retries, and selector waits require extra tooling

Playwright runs browsers headlessly by default. Its screenshot API supports viewport, element, and full-page captures in PNG, JPEG, and WebP, with CSS-pixel or device-pixel scaling. See the Playwright screenshot documentation and browser installation guide.

2. Set up Playwright

Install the package and Chromium

npm init -y
npm install playwright
npx playwright install --with-deps chromium

The --with-deps option installs Linux packages needed by the browser. In a headless-only CI image, Playwright also documents an --only-shell installation option. Pin your package and browser versions in the build image so a browser update does not silently change your output.

Minimal runnable screenshot script

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch({ headless: true });
  try {
    const context = await browser.newContext({
      viewport: { width: 1440, height: 900 },
      deviceScaleFactor: 1,
    });
    const page = await context.newPage();
    await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });
    await page.screenshot({ path: 'screenshot.png', fullPage: true });
  } finally {
    await browser.close();
  }
})();

domcontentloaded only means the initial HTML has been parsed. Replace it with a readiness condition that matches the page you are capturing.

3. Wait for the page state that matters

Waiting is the largest source of unreliable screenshots. A fixed sleep can be useful as a last resort, but a selector, application signal, or known network condition is more deterministic.

Wait for a selector

await page.goto('https://example.com/dashboard', { waitUntil: 'domcontentloaded' });
await page.locator('[data-testid="dashboard-ready"]').waitFor({ state: 'visible' });
await page.screenshot({ path: 'dashboard.png', fullPage: true });

Wait for network quietness

await page.goto('https://example.com', { waitUntil: 'networkidle' });

Network idle is appropriate when the page fetches its content during navigation and then becomes quiet. It is not proof that a dashboard, animation, ad slot, or lazy image is visually complete. Puppeteer’s official example uses waitUntil: 'networkidle2' as a practical starting point.

Wait for fonts, images, and application state

await page.evaluate(async () => {
  if (document.fonts) await document.fonts.ready;
  const images = Array.from(document.images);
  await Promise.all(images.map((img) => {
    if (img.complete) return Promise.resolve();
    return new Promise((resolve) => {
      img.addEventListener('load', resolve, { once: true });
      img.addEventListener('error', resolve, { once: true });
    });
  }));
});
await page.screenshot({ path: 'stable.png', fullPage: true });

For visual baselines, disable animations and use fixed test data. If a page continuously polls, opens a WebSocket, or streams content, define an application-specific “ready” marker instead of waiting forever.

4. Capture the viewport, an element, or the full page

Viewport screenshot

await page.screenshot({ path: 'viewport.webp', type: 'webp', quality: 85 });

A viewport capture is the visible area at the configured width and height.

Element screenshot

const chart = page.locator('#revenue-chart');
await chart.screenshot({ path: 'revenue-chart.png' });

Element capture is useful for cards, charts, receipts, and components. Wait for the element to be visible and ensure its content has finished rendering first.

Full-page screenshot

await page.screenshot({ path: 'full-page.png', fullPage: true });

Full-page mode captures the scrollable document. Very tall pages can use substantial memory and may expose lazy-loading bugs. Scroll through the page or trigger the application’s lazy-load behavior before capturing when necessary.

Format and scale

await page.screenshot({
  path: 'retina.png',
  type: 'png',
  scale: 'device',
});

await page.screenshot({
  path: 'photo.jpg',
  type: 'jpeg',
  quality: 80,
});

• PNG is lossless and is the safest choice for UI review and pixel comparisons.
• JPEG is smaller for photographic pages; it is lossy.
• WebP is a compact option when the consuming system supports it.
• scale: 'css' keeps output dimensions in CSS pixels; scale: 'device' uses device pixels.

5. Control rendering conditions

Set every condition that can affect pixels. Playwright documents that rendering varies with the host OS, browser version, settings, hardware, power source, and headless mode. Keep these values stable for visual comparisons.

const context = await browser.newContext({
  viewport: { width: 1440, height: 900 },
  deviceScaleFactor: 1,
  colorScheme: 'light',
  locale: 'en-US',
  timezoneId: 'UTC',
  userAgent: 'ScreenshotBot/1.0',
});

Also keep fonts, operating-system images, browser builds, test data, and time-dependent content consistent. Store reference images with versioned code and update them deliberately when a UI change is intentional.

Freeze or disable motion

await page.addStyleTag({
  content: `*, *::before, *::after {
    animation: none !important;
    transition: none !important;
    caret-color: transparent !important;
  }`,
});

Use this for deterministic test images. Do not apply it when the purpose of the capture is to document an animation.

6. Authentication, headers, cookies, and interaction

Create a browser context with the state your page requires. Avoid putting credentials directly in source files.

const context = await browser.newContext({
  extraHTTPHeaders: { Authorization: `Bearer ${process.env.API_TOKEN}` },
  storageState: process.env.STORAGE_STATE_PATH || undefined,
});
const page = await context.newPage();
await page.goto('https://example.com/account', { waitUntil: 'domcontentloaded' });
await page.locator('button:has-text("Show details")').click();
await page.screenshot({ path: 'account.png', fullPage: true });

For cookie-based sessions, load a saved Playwright storage state or add cookies with context.addCookies(). For login flows, navigate to the sign-in page, fill fields, submit, and wait for a post-login selector before taking the screenshot.

7. Chrome Headless from the command line

Chrome’s command-line path is enough for a basic URL capture:

chrome --headless --screenshot --window-size=412,892 https://developer.chrome.com/

The --screenshot flag writes screenshot.png in the current directory. Set a maximum wait with --timeout=5000:

chrome --headless --screenshot --window-size=1440,900 --timeout=5000 https://example.com/

Use the CLI for smoke checks and simple captures. Switch to Playwright or Puppeteer when you need selectors, interaction, authentication, retries, multiple output names, or cross-browser runs.

8. Puppeteer alternative

npm install puppeteer

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({ headless: true });
  try {
    const page = await browser.newPage();
    await page.setViewport({ width: 1440, height: 900 });
    await page.goto('https://news.ycombinator.com', {
      waitUntil: 'networkidle2',
    });
    await page.screenshot({ path: 'hn.png', fullPage: true });
  } finally {
    await browser.close();
  }
})();

Puppeteer also supports an element screenshot:

const card = await page.$('.athing');
if (!card) throw new Error('News item not found');
await card.screenshot({ path: 'first-item.png' });

9. Make the capture reliable in CI

1. Pin Playwright or Puppeteer and record the browser version.
2. Install browser binaries and Linux dependencies in the image used by the job.
3. Set viewport, device scale, locale, timezone, color scheme, and user agent explicitly.
4. Use a deterministic readiness signal instead of an arbitrary delay whenever possible.
5. Disable motion and stabilize test data for visual comparisons.
6. Give each job a unique output path so parallel workers do not overwrite files.
7. Close the browser in a finally block so failed jobs do not leave processes behind.
8. Retry transient navigation failures, but do not hide persistent selector or authentication errors with unlimited retries.

A simple retry wrapper:

async function captureWithRetry(capture, attempts = 3) {
  let lastError;
  for (let i = 1; i <= attempts; i++) {
    try { return await capture(); }
    catch (error) {
      lastError = error;
      if (i < attempts) await new Promise(r => setTimeout(r, i * 1000));
    }
  }
  throw lastError;
}

10. Troubleshooting

Symptom	Likely cause	Fix
Browser executable not found	Playwright package is installed but browser binaries are missing	Run npx playwright install --with-deps chromium in the same image that runs the script.
Linux launch fails with missing shared libraries	CI image lacks browser dependencies	Use Playwright’s --with-deps installer or add the documented OS packages to the image.
Screenshot is blank	Navigation failed, a bot check blocked the page, or capture happened before rendering	Inspect the response and page content, wait for a known selector, and handle authentication or bot protection explicitly.
Text or layout differs between machines	Different OS, fonts, browser build, scale, locale, timezone, or headless mode	Pin the environment and set rendering conditions explicitly.
Lazy images are missing	Images load only after scrolling or intersection events	Scroll through the document or trigger the page’s lazy-load mechanism before the full-page capture.
Capture hangs on network idle	Analytics, polling, or WebSockets keep network activity alive	Wait for an application selector or signal instead of network idle.
Element screenshot is clipped or fails	Selector matches nothing, the element is hidden, or its size is zero	Wait for visibility, assert the locator count, and verify computed dimensions.
Fonts shift after capture	Web fonts have not finished loading	Await document.fonts.ready and ensure the font files are reachable.
Full-page capture uses too much memory	Very tall page or large images	Capture sections or an element, reduce scale, optimize page resources, or use a service designed for large captures.
Login works locally but not in CI	Missing environment secrets, cookies, proxy settings, or user agent differences	Load secrets through CI variables, save storage state securely, and log status codes without exposing credentials.

11. Performance, reliability, and cost

Launching a new browser for every URL is simple but slower. Reuse one browser process and create isolated contexts when capturing many pages. Reuse contexts only when their cookies and storage are intentionally shared. Limit concurrency to the CPU and memory available; excessive parallel pages usually reduce throughput and increase timeouts.

Use viewport captures when you do not need the entire document. Full-page and device-scale screenshots require more rendering and memory. Block unnecessary third-party resources only when doing so does not change the page you intend to document. Cache stable assets in CI where appropriate, but invalidate the cache when browser or font versions change.

For visual testing, compare images produced by the same environment. A pixel difference can reflect a browser or operating-system update rather than an application change. Keep baselines versioned and review updates intentionally.

Self-hosted automation costs compute, browser maintenance, and engineering time. A hosted screenshot API can remove browser installation and queue management; evaluate pricing by successful captures, retries, image format, full-page behavior, and whether failed pages are billed.

12. Or skip the browser setup

ScreenshotNeo provides a website screenshot API and MCP server. Send one GET request to capture a PNG, JPEG, WebP, or PDF. The API handles browser setup and supports full-page capture with lazy images loaded, CSS-selector element capture, dark mode, 12 device presets or custom viewports, retina scale, PDF paper size and margins, custom CSS and JavaScript, clicks, selector or delay waits, network-idle waits, ad and tracker blocking, custom headers, cookies, user agents, Authorization, timezone, geolocation, transparent backgrounds, resizing, caching with a chosen TTL, signed links, asynchronous jobs with signed webhooks, bulk capture of up to 100 URLs per call, a usage API, and an OpenAPI specification. Common screenshot API parameter names also work when switching.

Cookie and consent banners are accepted and removed before capture, along with more than 60 known consent platforms, newsletter popups, and chat widgets; each cleanup step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers identify the page verdict and billing result.

See the ScreenshotNeo API documentation for all options.

cURL

curl -G 'https://api.screenshotneo.com/v1/shot' -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests
r = requests.get('https://api.screenshotneo.com/v1/shot', params={'access_key': 'YOUR_API_KEY', 'url': 'https://stripe.com'}, timeout=90)
r.raise_for_status()
open('shot.webp', 'wb').write(r.content)

Node.js

const fs = require('node:fs/promises');
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot failed: ${res.status}`);
await fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer()));

An MCP server exposes take_screenshot, get_page_info, and capture_pdf to Claude, Cursor, and other MCP clients. The Free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000 shots. Create a free ScreenshotNeo account.

13. Practical checklist

• Choose Playwright for a new cross-browser workflow, Puppeteer for an existing Puppeteer codebase, or Chrome Headless for a simple shell capture.
• Install and pin the browser runtime in the same environment that runs the job.
• Set viewport, scale, locale, timezone, color scheme, and user agent.
• Navigate and wait for a deterministic ready condition.
• Choose viewport, element, or full-page capture and the appropriate image format.
• Freeze motion and stabilize data for visual comparisons.
• Close browsers reliably and bound retries and concurrency.
• Record the environment when a screenshot is used as a baseline.

14. FAQ

Is headless mode faster than a visible browser?

It usually avoids the work of displaying a window, but total time still depends on navigation, JavaScript, fonts, images, and your machine. Measure your own workflow and keep the environment fixed for comparisons.

Can I take a screenshot without installing Playwright?

Yes. Chrome Headless can capture a URL from the shell if Chrome is already installed. A hosted API such as ScreenshotNeo removes browser installation from your application.

Which format should I store for visual regression tests?

Use PNG when exact, lossless pixels matter. Use JPEG or WebP when storage and transfer size matter and compression artifacts are acceptable.

Why does a full-page screenshot differ from what I see while scrolling?

Full-page capture may use a special layout path and can interact differently with sticky elements, lazy loading, and viewport-based scripts. Test the target site and capture mode together.