How to Set a Timeout for Website Capture Requests

Direct answer: In Playwright, set a per-navigation timeout in milliseconds on page.goto():

await page.goto('https://example.com', {
  timeout: 30_000,
  waitUntil: 'domcontentloaded'
});

Use a page or browser-context default when the same policy applies to many captures. Configure action timeouts separately for clicks and other interactions, and set an outer job or test timeout so one capture cannot occupy a worker forever. If you use Playwright’s direct HTTP request API, configure its request timeout independently. A browser navigation timeout does not automatically control a separate HTTP request or a hosted screenshot service.

This guide shows how to choose the timeout scope, select a readiness condition, handle slow and failed pages, and operate captures reliably. The examples use Playwright because it exposes each timeout layer directly. The values are examples, not a universal performance target.

1. Understand the timeout layers

A “website capture request” can contain several operations. Identify the operation that is actually timing out before changing a number.

Layer	What it limits	Typical setting
Navigation	Loading a URL in a page	`page.goto(url, { timeout })` or `navigationTimeout`
Action	Clicks, fills, checks, and other page actions	`actionTimeout` or an action’s `timeout`
Readiness wait	When navigation is considered complete	`waitUntil: 'commit' \| 'domcontentloaded' \| 'load' \| 'networkidle'`
Direct HTTP request	Playwright API requests made without a browser page	`APIRequestContext` request `timeout`
Whole job	Total time for navigation, waits, actions, rendering, and storage	Your worker, queue, or test timeout

Changing one layer does not change the others. For example, increasing page.goto() to 90 seconds will not prevent a click from failing after a 10-second action timeout, and it will not extend a queue worker that is killed after 60 seconds.

2. Set a timeout for one capture

Use a per-call timeout when one destination needs a different policy from the rest of your workload.

A capture timeout covers a specific operation, while a readiness signal decides when the page is ready.

import { chromium } from 'playwright';

const browser = await chromium.launch();
const page = await browser.newPage();

try {
  await page.goto('https://example.com', {
    timeout: 30_000,
    waitUntil: 'domcontentloaded'
  });

  await page.screenshot({ path: 'example.png', fullPage: true });
} finally {
  await browser.close();
}

The timeout is expressed in milliseconds. A value of 30_000 means 30 seconds. Playwright’s Page API documents commit, domcontentloaded, load, and networkidle as navigation conditions. The timeout bounds the wait for that condition; it does not guarantee that every image, font, or application request has finished.

Use domcontentloaded when the document structure is enough for your capture. Use load when you need the browser’s load event. Treat networkidle carefully: Playwright labels it discouraged for tests and recommends web assertions to assess readiness instead. Long polling, analytics, WebSockets, and ads can keep a page from becoming idle.

3. Configure defaults for repeated captures

If a worker captures many pages with the same policy, set the default on the browser context or page. A per-call value can still override it.

import { chromium } from 'playwright';

const browser = await chromium.launch();
const context = await browser.newContext();
context.setDefaultNavigationTimeout(30_000);
context.setDefaultTimeout(10_000); // actions and other default operations

const page = await context.newPage();
await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });
await page.screenshot({ path: 'example.png' });

await browser.close();

Keep navigation and action defaults separate. A slow site may require more time to navigate while its cookie-consent button should still fail quickly if it is missing. Set the defaults at the narrowest scope that matches your workload: page for one flow, context for a group of pages, or project configuration for a test suite.

4. Configure Playwright Test

In Playwright Test, navigation and action defaults are independent configuration options. The official timeout documentation shows this pattern:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  use: {
    actionTimeout: 10_000,
    navigationTimeout: 30_000,
  },
});

The overall test timeout is another limit. Make it longer than the sum of navigation, readiness waits, actions, screenshot work, and cleanup. If the test timeout is shorter than navigationTimeout, the test can be terminated before the navigation limit is reached.

5. Choose readiness instead of blindly increasing the timeout

A larger timeout only gives a page more time to reach the selected condition. It does not tell Playwright what “ready for a screenshot” means. Define readiness explicitly for applications that render after navigation.

await page.goto(targetUrl, {
  timeout: 45_000,
  waitUntil: 'domcontentloaded'
});

await page.locator('[data-capture-ready="true"]').waitFor({
  state: 'visible',
  timeout: 15_000
});

await page.screenshot({ path: 'ready.png', fullPage: true });

Other useful signals include a heading becoming visible, a loading spinner disappearing, or a known API response completing. Keep these waits finite. If the application never emits the signal, the capture should fail with a useful error rather than hang.

For a fixed rendering delay, use a bounded wait only when the page has no better readiness signal:

await page.waitForTimeout(2_000);

A delay is not proof that network activity or animations have finished, so prefer a selector or assertion when possible.

6. Direct HTTP requests need their own timeout

Some capture pipelines fetch HTML or an API endpoint through Playwright’s APIRequestContext instead of navigating a browser page. Configure that request layer directly. Page navigation settings do not inherit into it.

import { request } from 'playwright';

const api = await request.newContext({
  timeout: 20_000,
});

try {
  const response = await api.get('https://example.com/data.json');
  if (!response.ok()) {
    throw new Error(`HTTP ${response.status()}`);
  }
  const data = await response.json();
  console.log(data);
} finally {
  await api.dispose();
}

Use the request API timeout for the HTTP operation and an outer deadline for the complete capture job. A successful API response still may not mean that a browser-rendered page is ready.

7. Timeout behavior and the value zero

Playwright documents 0 as disabling the relevant timeout. That can be useful for a deliberately unbounded internal operation, but it can also leave a worker stuck on a page that never completes. In production, use a finite limit that matches your queue and user-facing latency budget. Record the URL, operation, configured limit, and elapsed time whenever a timeout occurs.

8. Complete capture example with retries and an outer deadline

Retries should be limited and should not multiply an already excessive timeout. This example gives each attempt a 30-second navigation limit and caps the whole operation at 75 seconds.

import { chromium } from 'playwright';

async function capture(url, outputPath) {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  const job = (async () => {
    let lastError;
    for (let attempt = 1; attempt <= 2; attempt++) {
      try {
        await page.goto(url, {
          timeout: 30_000,
          waitUntil: 'domcontentloaded'
        });
        await page.locator('body').waitFor({ state: 'visible', timeout: 10_000 });
        await page.screenshot({ path: outputPath, fullPage: true });
        return;
      } catch (error) {
        lastError = error;
        if (attempt < 2) await page.waitForTimeout(1_000);
      }
    }
    throw lastError;
  })();

  try {
    await Promise.race([
      job,
      new Promise((_, reject) =>
        setTimeout(() => reject(new Error('capture job deadline exceeded')), 75_000)
      )
    ]);
  } finally {
    await browser.close();
  }
}

capture('https://example.com', 'example.png').catch(console.error);

Only retry errors that are plausibly transient, such as connection resets or upstream 5xx responses. Repeating an invalid URL, a blocked bot check, or a selector that never exists will waste capacity.

9. cURL, Python, and Node.js client timeouts

If your capture endpoint is an HTTP service, its client timeout is separate from any browser timeout used inside the service. With cURL, use a maximum transfer time:

curl --max-time 90 -G "https://your-capture-service.example/shot" \
  --data-urlencode "url=https://example.com" \
  -o shot.png

In Python with Requests, pass a finite timeout. A tuple separates connection and read limits:

import requests

response = requests.get(
    'https://your-capture-service.example/shot',
    params={'url': 'https://example.com'},
    timeout=(10, 90),
)
response.raise_for_status()
with open('shot.png', 'wb') as output:
    output.write(response.content)

In Node.js, use an abort signal with fetch:

const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 90_000);
try {
  const response = await fetch('https://your-capture-service.example/shot?url=https%3A%2F%2Fexample.com', {
    signal: controller.signal,
  });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  await Bun.write('shot.png', await response.arrayBuffer());
} finally {
  clearTimeout(timer);
}

Align the client timeout with the provider’s documented request or job limit. A client that gives up early cannot receive a result even if the capture is still running.

10. Or skip the browser setup

ScreenshotNeo provides a website screenshot API and MCP server. One GET request returns a PNG, JPEG, WebP, or PDF, so you can set the timeout in your HTTP client while ScreenshotNeo handles the browser capture. See the ScreenshotNeo API documentation for request options.

Hosted capture services can remove obstructive overlays before producing the image.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo removes cookie and consent banners, newsletter popups, and chat widgets before the shot. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed; response headers identify the page verdict and whether the request was billed. Its MCP server includes take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The Free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 shots.

Create a free ScreenshotNeo account and start with the monthly free allocation.

11. Troubleshooting timeout errors

Symptom	Likely cause	Fix
`page.goto: Timeout ... exceeded`	Navigation did not reach the selected `waitUntil` condition.	Inspect the URL and logs; choose an appropriate readiness condition, then raise the finite navigation timeout if the destination is legitimately slow.
Navigation succeeds but screenshot is blank	The app renders after navigation or requires a user action.	Wait for a meaningful selector, hide a loading layer, or perform the required click before capture.
Timeout occurs while clicking consent	The banner is absent, hidden, inside a frame, or uses a different selector.	Use a short action timeout, check visibility, handle optional banners, and inspect frames.
`networkidle` never arrives	Long polling, analytics, ads, or WebSockets keep requests active.	Use `domcontentloaded` or `load` and wait for a page-specific ready selector.
Direct request times out while browser navigation works	The request API has its own timeout or the endpoint behaves differently without a browser.	Configure `APIRequestContext` or HTTP-client timeout separately and compare response headers and redirects.
Works locally, fails in CI	Different DNS, proxy, CPU, browser install, or network policy.	Log elapsed phases, verify browser dependencies and proxy settings, and keep an outer job deadline.
Retries increase load without improving results	The failure is deterministic, such as authentication, robots policy, or an invalid selector.	Classify errors before retrying; retry only transient failures with a small attempt limit.

12. Performance, reliability, and cost decisions

Measure phases: record DNS/connect time where available, navigation duration, readiness wait, actions, screenshot encoding, and upload time. This shows which timeout needs adjustment.
Keep budgets hierarchical: the whole job must outlast its child operations, while still being short enough to release stuck workers.
Avoid unlimited waits: 0 disables a timeout, but finite limits protect queue capacity and user latency.
Use readiness signals: a selector tied to rendered content is usually more deterministic than waiting for every network request.
Control concurrency: too many simultaneous browsers can increase CPU and memory pressure, creating timeouts that do not occur at low concurrency.
Cache intentionally: if the page can be reused, cache the resulting image or use a capture service’s cache policy. With ScreenshotNeo, cache hits are identified and are not billed.
Account for failed work: your own browser infrastructure still consumes worker time on failures. ScreenshotNeo bills only clean shots and does not bill bot checks, blank pages, timeouts, failed loads, or cache hits.

13. Timeout configuration checklist

Name the operation: navigation, action, direct request, readiness wait, or whole job.
Set a finite per-operation timeout in milliseconds.
Choose a readiness condition that matches the page, rather than automatically selecting networkidle.
Set a longer outer deadline for the complete capture.
Log the URL, operation, timeout, elapsed time, and error category.
Retry only transient failures, with a bounded attempt count.
Test slow pages, redirects, authentication, optional consent banners, and pages that never finish loading.
Review concurrency and memory when timeouts appear only under load.

FAQ

What is a reasonable website capture timeout?

There is no universal number. Start with a finite value such as the 30-second Playwright example, then adjust from your own latency and failure records. Keep the complete job deadline longer than the individual navigation and readiness limits.

Does increasing `page.goto` timeout fix a click timeout?

No. Navigation and action timeouts are separate. Set the timeout on the click or configure the action default.

Should I always use `networkidle` for screenshots?

No. Playwright labels networkidle discouraged for tests. A page-specific selector or assertion is usually a clearer readiness signal.

What happens when timeout is zero?

For the documented timeout options, 0 disables that timeout. Use it only when an unbounded operation is intentional and another outer limit exists.

Can a hosted screenshot API use my Playwright timeout?

No. Your Playwright settings control your browser process. A hosted service has its own limits; configure your HTTP client according to that service’s documentation.

How can AI agents take screenshots without managing Playwright?

ScreenshotNeo includes an MCP server with take_screenshot, get_page_info, and capture_pdf, usable from Claude, Cursor, and other MCP clients.