Cross-Browser Testing: What to Test Beyond Browsers

Cross-browser testing should cover the combinations of browser, operating system, device, screen, hardware, input method, assistive technology, and network that matter to your users. Start with audience data and product requirements, define a support matrix, automate repeatable workflows, and use physical devices for high-risk interactions or hardware-dependent behavior. Testing every possible combination is impractical; prioritize the combinations with the highest user impact. MDN’s introduction to cross-browser testing and its testing strategies guide explain this audience-led approach.

1. Define what “works” means

Before choosing browsers, write down the user journeys and supported environments your release must handle. A browser name alone does not specify the rendering engine version, operating system, device constraints, or input method.

• Journeys: list the important tasks, such as signing in, completing a purchase, editing content, or installing a web app.
• Support boundary: agree on browser families and versions, operating systems, device classes, and accessibility expectations. Include contractual or policy requirements.
• Evidence: use analytics, support reports, customer commitments, and product priorities to choose combinations. Revisit the matrix when audience or product needs change.
• Risk: identify changes that touch rendering, media, authentication, payments, touch gestures, offline behavior, or OS integration. Expand coverage for these changes.

“All browsers” is not a useful release criterion. A documented, audience-informed boundary makes failures actionable and coverage maintainable.

2. Build a test matrix beyond browser names

Use the following dimensions to find differences a browser-only list can miss. You do not need every row crossed with every other row; select combinations according to your support boundary and risk.

Dimension	What to vary or verify	Why it matters
Browser and build	Supported browser families and versions; branded Chrome or Edge when required	Engines, codecs, policies, extensions, and release timing can differ.
Operating system	Supported desktop and mobile OS versions	Font rendering, media, permissions, keyboard behavior, and integrations can be OS-dependent.
Device class and hardware	Desktop, tablet, phone; lower-powered hardware where relevant	CPU, memory, screen dimensions, and available capabilities affect behavior and performance.
Viewport and screen	Meaningful widths and heights, orientation, zoom, scrolling, overflow, contrast	A layout may break at a height or zoom level even when its width breakpoint looks correct.
Input	Keyboard, mouse or trackpad, touch, and other supported controls	Hover-only controls and gesture assumptions can make a core journey inaccessible.
Assistive technology	Keyboard-only navigation and screen-reader use on supported browser/OS combinations	Browser accessibility and communication with assistive technology shape the user experience. See the W3C UAAG overview.
Network	Typical, slow, high-latency, unreliable, and offline states where promised	Bandwidth, latency, and transfer costs vary. Installed web apps should provide intentional behavior during poor connectivity.
Runtime performance	Load, interaction response, animation, memory-sensitive flows	Usability depends on runtime and device capabilities as well as visual correctness.
App installation and OS integration	Install, launch, offline fallback, screen adaptation, permissions, expected system integration	These are part of the product when it offers PWA or browser-to-OS features.

These dimensions are consistent with the categories in the W3C device-independent testing guidance, including screen, memory, CPU, network, and input constraints. The W3C note dates to 2009, so use it for these stable categories rather than as a source for current browser support recommendations.

3. Test viewport, screen, orientation, and zoom

Choose breakpoints from your content and layout, then test around them rather than checking only a few named device presets. Include at least one narrow phone-sized viewport, a larger phone or tablet layout when relevant, and a desktop layout. Test widths immediately below and above important breakpoints, plus short and tall viewports.

• Resize through breakpoints and check for clipped text, overlapping controls, horizontal scrolling, and unexpected blank space.
• Rotate supported devices and check that dialogs, forms, media, and fixed-position controls remain usable.
• Zoom in and verify content reflows and controls remain reachable. Do not assume a fixed viewport screenshot represents zoom behavior.
• Scroll long pages, open menus and dialogs, and verify sticky or fixed elements do not obscure content or keyboard focus.
• Check text contrast and content under the screen settings your product supports.

A screenshot can reveal visual regressions at a chosen viewport, but it cannot establish touch usability, keyboard behavior, screen-reader output, or performance on a physical device. Treat visual comparison as one check within the matrix.

4. Test input methods and accessibility

Run important workflows without a mouse. Tab through controls in a sensible order, activate them with expected keys, confirm focus is visible, and make sure no step traps keyboard users. For touch, check target reachability, gestures, scrolling conflicts, and whether controls that normally appear on hover remain available.

Use a screen reader to navigate landmarks, headings, forms, error messages, and dynamic updates in the browser/OS combinations you support. Verify the accessible name and state of controls, not only their visual appearance. A quick keyboard-only and screen-reader pass is a useful baseline; it does not replace a fuller accessibility evaluation.

5. Test networks, offline states, and performance

Repeat critical journeys on a representative slow or unreliable connection. Observe whether the page communicates loading and failure, whether user input is preserved, and whether retries are possible. If the product promises offline use, disconnect the network and test the actual promised workflow plus the fallback for unsupported actions. MDN’s PWA best practices recommend useful behavior on unreliable networks and an intentional offline experience for installed apps.

Measure load, response to input, animation smoothness, and resource timing in context. MDN’s web performance guidance gives general examples: 1 second for loading, 50 milliseconds for idling, 16.7 milliseconds for animation, and 50–200 milliseconds for responding to user input. Treat these as context, not universal release thresholds. Establish targets for your page, device class, network, and user journey.

6. Choose real devices, emulators, and automation

Approach	Useful for	Limit
Local browser installs	Fast feedback on primary development environments	Coverage is limited to available machines and builds.
Emulators and virtual machines	Adding OS and device profiles; reproducing some environment-specific issues	They approximate device behavior and may not reproduce physical hardware or the full user experience.
Physical phones, tablets, and computers	Touch, lower-powered hardware, OS integration, and final checks of key journeys	You are limited to devices you can access.
Hosted browser or device services	Teams that need broader environments without maintaining an internal device collection	Coverage and whether devices are real or simulated vary; check the service’s current details and terms.

MDN identifies physical devices as the most accurate option for behavior and overall experience, with emulators and virtual machines useful when physical access is limited. Use automation for repeatable functional checks and quick breadth; reserve physical checks for high-risk or hardware-sensitive paths.

7. Automate repeatable checks with Playwright

This runnable Node.js example exercises the same smoke test in Playwright’s Chromium, Firefox, and WebKit projects. It checks a locally running app’s title and a primary link; replace the URL and assertions with your own critical journey. Playwright’s browser builds are not always identical to branded releases. Its browser documentation explains when branded Chrome or Edge may matter, including codecs, enterprise policies, or extensions.

npm init -y
npm install --save-dev @playwright/test
npx playwright install

Create playwright.config.js:

const { defineConfig, devices } = require('@playwright/test');

module.exports = defineConfig({
  testDir: './tests',
  fullyParallel: true,
  reporter: 'list',
  use: {
    baseURL: 'http://127.0.0.1:3000',
    trace: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
    { name: 'mobile-chromium', use: { ...devices['Pixel 7'] } },
    { name: 'mobile-webkit', use: { ...devices['iPhone 13'] } },
  ],
});

Create tests/smoke.spec.js:

const { test, expect } = require('@playwright/test');

test('home page exposes the primary journey', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveTitle(/your app/i);
  await expect(page.getByRole('link', { name: /sign in/i })).toBeVisible();
});

Start your app at http://127.0.0.1:3000, then run:

npx playwright test

The device descriptors approximate viewport, user agent, and related settings; this is not a physical-phone test. Add assertions for important flows, use stable accessible locators, and keep screenshots or traces on failure to diagnose regressions. For branded binaries, configure the appropriate Playwright channel for Chrome or Edge and install that browser separately as described in the docs. Pin dependency versions and browser installation in CI so changes to the test environment are intentional.

8. Capture visual evidence for review

For a visual check, capture the same page at a fixed viewport in each chosen environment and compare the output. Keep the capture conditions consistent: URL, viewport, device scale, page state, and wait condition. Dynamic timestamps, randomized content, animations, ads, and cookie banners can create differences unrelated to your change. Mask or stabilize dynamic regions when your visual testing setup supports it, and inspect meaningful differences rather than treating every changed pixel as a defect.

For manual capture, open the target page in each browser and use its developer tools or screenshot capability at the same viewport. This produces evidence of rendered appearance, not a complete compatibility result: it does not test the whole journey, assistive technology, real device performance, or offline behavior.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server for developers. A single request returns a PNG, JPEG, WebP, or PDF. Its capture options include viewport and device presets, full-page capture, element selection, waits, and custom CSS or JavaScript. See the ScreenshotNeo API documentation for options and parameter details.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners are accepted and removed before the shot, along with known newsletter popups and chat widgets; each cleanup step can be turned off. Bot checks, blank pages, timeouts, and failed loads are never billed, and cache hits cost nothing; response headers indicate the page verdict and billing status. An MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf. The Free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. ScreenshotNeo can help collect consistent visual evidence, while real-device, interaction, accessibility, and network checks still belong in your test plan. Start with 1,000 free screenshots a month, no card required.

9. Make failures reproducible

For each bug, record the browser and version, operating system, device, viewport, orientation, input method, network state, steps, expected result, and observed result. Attach a screenshot or trace where useful, and include whether the issue occurs on a physical device or only in an emulator. This information helps separate layout defects from browser behavior, network failures, or device constraints.

10. Troubleshooting common cross-browser test failures

Symptom	Likely cause	What to do
A test passes in Playwright but fails in installed Chrome or Edge	The bundled Chromium build differs from the branded browser or its policies, codecs, or extensions.	Reproduce with the branded channel when that environment is in scope; record the browser build and relevant policies.
A layout breaks only at one width or orientation	A breakpoint assumption, fixed dimension, overflow, or short viewport was missed.	Test around the failing width and height, rotate if relevant, and inspect computed styles and scrolling.
A click or hover test fails on touch	The interaction depends on hover or assumes a mouse pointer.	Test with touch input and provide an equivalent visible, keyboard-operable control.
A screenshot comparison has many unexplained differences	Fonts, animations, timestamps, ads, asynchronous content, viewport, or device scale differ.	Match capture conditions, wait for the relevant content, disable or stabilize animation and dynamic regions where appropriate, then review the remaining changes.
A page looks correct but the journey fails on a slow network	Race conditions, unhandled timeouts, missing loading/error states, or lost form state.	Repeat under latency and unreliable connectivity; make loading, retry, and failure behavior explicit and preserve input where possible.
A screen-reader user cannot find or understand a control	Missing or incorrect accessible name, role, state, structure, or live update.	Inspect the accessibility tree and test the actual flow with keyboard and screen reader on a supported combination.
A mobile emulation result disagrees with a phone	Emulation does not reproduce all hardware, OS integration, or physical interaction behavior.	Reproduce on a physical device for touch, performance, permissions, and OS-specific issues.
A PWA shows a generic browser error offline	The app has no intentional offline fallback or the installable experience was not tested disconnected.	Test installed and offline states and provide the fallback promised by the product.

11. Keep coverage reliable and affordable

• Automate the stable core: run short, deterministic smoke and critical-path tests on every change; run broader combinations on a schedule or for higher-risk changes.
• Control the matrix: avoid multiplying every browser, OS, viewport, locale, and network state. Select representative combinations and add targeted cases when risk warrants them.
• Reduce flaky tests: wait for meaningful application state instead of arbitrary delays, use stable accessible locators, and isolate test data.
• Use parallelism deliberately: parallel projects shorten elapsed time but consume more CI resources. Keep a small fast gate and move expanded coverage to a separate job if it slows feedback.
• Budget physical-device access: reserve devices for high-risk paths and failures that emulation cannot resolve; use emulators and VMs for broader routine reach.
• Track maintenance: browser updates, OS releases, dependencies, and changing audience analytics require periodic matrix review. Do not infer current market share from a fixed list.
• Control screenshot costs: if using a screenshot service, check its current plan and billing rules. ScreenshotNeo charges only for clean shots; bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing. Its stated plans are Free 1,000/month, Starter $5 for 3,000, Growth $15 for 15,000, Pro $39 for 60,000, Scale $99 for 250,000, and Business $249 for 1,000,000; yearly billing gives two months free, and every feature is on every plan.

12. A release checklist

• The supported browser, OS, device, and accessibility boundary is written down and based on audience or product requirements.
• Critical journeys pass in the baseline browser matrix.
• Responsive layouts pass at meaningful widths, heights, orientations, and zoom levels.
• Keyboard-only and screen-reader checks cover core journeys; touch paths are checked where relevant.
• Slow, unreliable, and offline states are tested when the product makes promises about them.
• Performance goals are defined for the target experience and measurement conditions.
• Physical-device checks cover high-risk hardware-sensitive workflows.
• Failures include enough environment and reproduction detail for another developer to investigate.

FAQ

What should I test besides browsers?

Operating systems, device classes, screen size and orientation, hardware limits, input methods, assistive technology, network conditions, runtime performance, and app installation or OS integration when relevant.

How do I test a website across devices?

Use analytics and product requirements to choose a manageable matrix, automate repeatable browser workflows, use emulators or VMs to extend OS coverage, and verify important touch or hardware-sensitive journeys on physical devices.

Do I need to test on real phones?

For key mobile journeys, yes when practical—especially for touch, lower-powered hardware, permissions, and OS integration. Emulation broadens coverage but does not replace physical-device fidelity.

Is a screenshot enough to prove cross-browser compatibility?

No. It checks rendered appearance for one capture state. It does not prove interactions, keyboard or screen-reader access, network behavior, performance, or real-device behavior.