Headless Website Testing Framework

Short answer: choose Playwright as the default headless website testing framework when you need Chromium, Firefox and WebKit coverage, automatic waiting, isolated tests and traceable CI failures. Use Cypress for in-browser feedback and component tests, Puppeteer for focused Chrome or Firefox automation, and Selenium when your organisation already runs WebDriver infrastructure.

What headless website testing means

Headless mode runs a real browser engine without opening a visible window. Assertions, selectors, waits and test isolation still determine test quality; headless is only the execution mode. Cypress CLI runs headlessly by default, and Puppeteer supports headless, headful and shell modes for navigation, screenshots, PDFs, UI tests and performance work.

Framework comparison

Framework	Best fit	Browser coverage	Distinctive capabilities
Playwright	Cross-browser end-to-end suites	Chromium, Firefox, WebKit	Auto-waiting, web-first assertions, fixtures, isolation, parallel workers, tracing and reporters
Cypress	In-browser debugging and component tests	Chrome-family and Firefox; WebKit experimental	Runs in the application run loop with direct access to `window`, `document` and DOM elements
Puppeteer	Programmable browser tasks, screenshots, PDFs and performance scripts	Chrome and Firefox through CDP and WebDriver BiDi	High-level JavaScript API with a small operational surface
Selenium	Existing WebDriver estates and multi-language grids	Depends on installed drivers and grid	Mature WebDriver APIs and language bindings

These are fit-based choices, not speed rankings. See the official Playwright browser guide, Cypress browser documentation, Puppeteer guide and Selenium WebDriver documentation.

Decision checklist

Need Chromium, Firefox and WebKit from one API: Playwright.
Need interactive browser-side debugging or component tests: Cypress.
Need a script that navigates, prints a PDF or captures a page: Puppeteer.
Need Java, Python, C#, Ruby or an established Selenium Grid: Selenium.
Need real Safari or physical devices: plan hosted device coverage; desktop headless engines cannot replace that validation.

Build a Playwright headless suite

Install and scaffold

npm init playwright@latest
npx playwright install --with-deps

Write a complete test

import { test, expect } from '@playwright/test';

test('checkout is usable', async ({ page }) => {
  await page.goto('https://example.com/checkout', { waitUntil: 'domcontentloaded' });
  await expect(page.getByRole('heading', { name: 'Checkout' })).toBeVisible();
  await page.getByLabel('Email').fill('qa@example.com');
  await page.getByRole('button', { name: 'Continue' }).click();
  await expect(page).toHaveURL(/\/payment/);
});

Run in CI

npx playwright test
npx playwright test --project=chromium --workers=2
npx playwright show-report

Pin the Playwright package and browser revision in your lockfile. Install only the projects your pipeline needs. Keep each test independent before increasing workers. Save traces, screenshots, console logs and videos on failure so a CI failure can be diagnosed without a local rerun.

Configuration

import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  reporter: [['html'], ['junit', { outputFile: 'results.xml' }]],
  use: {
    baseURL: 'https://example.com',
    headless: true,
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
    actionTimeout: 10000,
    navigationTimeout: 30000
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } }
  ]
});

Selectors, waits and isolation

Prefer role, label and test-id locators over CSS tied to layout. Playwright auto-waits for actionability and web-first assertions; do not add arbitrary sleeps to hide races. Wait for a meaningful URL, role or response. Create fresh browser contexts per test, seed data through an API, and clean up records in a fixture. Avoid shared accounts when workers run in parallel.

Cypress, Puppeteer and Selenium examples

Cypress

npx cypress run --browser chrome

describe('home', () => {
  it('shows navigation', () => {
    cy.visit('https://example.com');
    cy.get('nav').should('be.visible');
  });
});

Cypress runs in the same run loop as the app and exposes browser-side objects, which makes interactive debugging and component testing useful. Validate WebKit requirements before standardising because its WebKit support is experimental.

Puppeteer

import puppeteer from 'puppeteer';
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com', { waitUntil: 'networkidle2' });
await page.screenshot({ path: 'home.png', fullPage: true });
await browser.close();

Selenium with Python

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless=new')
options.add_argument('--no-sandbox')
driver = webdriver.Chrome(options=options)
try:
    driver.get('https://example.com')
    assert 'Example' in driver.title
finally:
    driver.quit()

CI reliability and scaling

Pin framework, browser and OS images; upgrades can change rendering and selectors.
Use deterministic test data and isolated contexts.
Capture traces and browser logs on failure; retain artifacts for failed retries.
Set explicit navigation and assertion timeouts based on your application.
Use retries only to collect evidence. Fix flaky selectors, shared state and external dependencies.
For hosted browsers or real devices, verify current browser versions, concurrency, data residency, test-minute limits and pricing with the provider.

Performance and cost

Reuse a browser process while creating a new context per test. Run independent projects in parallel only after isolation is proven. Cache browser binaries in CI and avoid recording video for every passing test. Headless execution removes display overhead; it does not remove network, application or test setup work. Framework licenses do not include hosted browsers, CI minutes or device-cloud charges, so budget those separately.

Troubleshooting

Symptom	Likely cause	Fix
Browser fails to launch in CI	Missing system libraries or browser binaries	Run the framework install command with dependencies, use its maintained CI container, and pin the image.
Timeout waiting for a button	Unstable selector, wrong frame, or app never reached ready state	Use a role or test-id, inspect frames, wait for a domain signal, and attach a trace.
Works headed but not headless	Viewport, timing, font or GPU difference	Set viewport and timezone explicitly, wait on assertions, and compare traces.
Flakes only with workers	Shared account, database rows or port	Generate per-test data and context; reserve unique ports.
WebKit or Safari mismatch	Desktop WebKit is not every Safari or device combination	Run WebKit coverage and add a hosted real-device Safari pass.
CAPTCHA or consent blocks the flow	Anti-bot or cookie tooling changes the page	Use a test bypass supplied by the application owner; do not attempt to defeat a production CAPTCHA.

Or skip the browser setup

If your deliverable is a page image or PDF rather than an assertion, ScreenshotNeo provides one GET request. It supports full-page and element capture, device presets, dark mode, waits, custom CSS and JavaScript, headers, cookies and PDF options. Its MCP server exposes take_screenshot, get_page_info and capture_pdf to Claude, Cursor and other MCP clients.

Cookie banners, newsletter popups and chat widgets are removed before the shot. Bot checks, blank pages and failed loads are never billed, and response headers report the page verdict and billing status. 1,000 screenshots each month are free with no card; paid plans start at $5 for 3,000.

cURL

curl -G 'https://api.screenshotneo.com/v1/shot' -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests
r = requests.get('https://api.screenshotneo.com/v1/shot', params={'access_key': 'YOUR_API_KEY', 'url': 'https://stripe.com'}, timeout=90)
r.raise_for_status()
open('shot.webp', 'wb').write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
require('fs').writeFileSync('shot.webp', Buffer.from(await res.arrayBuffer()));

See the ScreenshotNeo API documentation for option names, async jobs, signed webhooks, bulk capture, usage and OpenAPI details. Create a free ScreenshotNeo account to get 1,000 screenshots a month with no card.

FAQ

Is headless testing the same as end-to-end testing?

No. Headless describes browser visibility; end-to-end describes the workflow scope and assertions.

Should every test run on three engines?

Run the full matrix for compatibility-critical journeys. Keep a fast Chromium subset for pull requests and schedule broader coverage when runtime requires it.

Can screenshots replace assertions?

No. Visual snapshots detect rendering changes, while semantic assertions verify behaviour. Use both where appropriate.

When is Selenium still the right choice?

When your team already has WebDriver bindings, grids, reporting and operational knowledge that make migration costly.