ScreenshotNeo

BlogHow-to

How to Set a Timeout for Each URL in a Playwright Bulk Screenshot Script

Set a separate navigation timeout on each Playwright page.goto() call, handle failures per URL, and choose a wait condition that fits the page.

By the ScreenshotNeo team4 October 20268 min read

Set the timeout on each page.goto() call inside your URL loop. Playwright measures this value in milliseconds, so timeout: 15_000 gives that navigation a 15-second limit. Catch errors inside the loop so one slow or unreachable URL does not stop the rest of the batch.

const timeoutMs = 15_000;

for (const url of urls) {
  try {
    await page.goto(url, {
      timeout: timeoutMs,
      waitUntil: 'domcontentloaded',
    });
    await page.screenshot({ path: screenshotPathFor(url) });
  } catch (error) {
    console.error(`Failed for ${url}:`, error);
  }
}

Use a fresh page for each URL if the batch needs isolated page state, or reuse a page if sequential navigation and shared state are acceptable. The timeout above bounds the navigation wait for the chosen waitUntil condition. It does not automatically set a deadline for the screenshot, the full script, or later processing.

1. Build a resilient bulk screenshot loop

This complete example launches Chromium, visits a list of URLs sequentially, applies a per-navigation timeout, saves successful screenshots, records failures, and closes the browser even if the batch encounters errors. It uses domcontentloaded so the navigation does not wait for every dependent resource to finish loading; choose another condition if your capture requires it.

const { chromium } = require('playwright');
const path = require('node:path');

const urls = [
  'https://example.com/',
  'https://playwright.dev/',
];
const timeoutMs = 15_000;

function screenshotPathFor(url, index) {
  const hostname = new URL(url).hostname.replace(/[^a-z0-9.-]/gi, '_');
  return path.resolve(`shot-${index}-${hostname}.png`);
}

(async () => {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  const failures = [];

  try {
    for (const [index, url] of urls.entries()) {
      try {
        const response = await page.goto(url, {
          timeout: timeoutMs,
          waitUntil: 'domcontentloaded',
        });

        if (response && response.status() >= 400) {
          console.warn(`${url} returned HTTP ${response.status()}`);
        }

        await page.screenshot({ path: screenshotPathFor(url, index), fullPage: true });
        console.log(`Saved screenshot for ${url}`);
      } catch (error) {
        failures.push({ url, message: error.message });
        console.error(`Failed for ${url}: ${error.message}`);
      }
    }
  } finally {
    await browser.close();
  }

  if (failures.length) {
    console.error('URLs that failed:', failures);
    process.exitCode = 1;
  }
})();

Install Playwright and its browser before running the script: npm install playwright followed by npx playwright install chromium. Save the code as a JavaScript file and run it with Node.js. The loop continues after a navigation or screenshot error, then returns a nonzero exit code if any URL failed, which is useful in scheduled or CI jobs.

2. Choose a timeout and navigation completion condition

The timeout only has meaning together with the event Playwright is waiting for. The Page API supports these waitUntil conditions:

Condition When navigation is considered complete Use when
commit The response has been received and the document started loading. You need the earliest navigation milestone and will handle readiness separately.
domcontentloaded The document’s DOM content has been loaded. You want a practical starting point for pages whose main content appears early.
load The page’s load event fires. This is the documented default. The screenshot depends on resources that finish by the load event.
networkidle Network activity reaches an idle state. Use cautiously; Playwright discourages this condition for testing, and pages with ongoing requests may not become idle.

For pages that render important content after navigation, wait for a meaningful selector before capturing rather than assuming navigation completion means visual readiness. For example, after goto, use await page.locator('main article').waitFor({ state: 'visible', timeout: 5_000 }) with a selector that actually identifies the content on your target sites. That selector wait has its own timeout; it is not included in the navigation timeout.

A fixed timeout is simple when all targets have similar response characteristics. If URLs vary, compute a budget for each one, perhaps from known site groups or job metadata:

for (const item of jobs) {
  const timeoutMs = item.priority === 'critical' ? 30_000 : 12_000;
  try {
    await page.goto(item.url, {
      timeout: timeoutMs,
      waitUntil: 'domcontentloaded',
    });
    await page.screenshot({ path: item.outputPath });
  } catch (error) {
    console.error(`Capture failed (${timeoutMs} ms) for ${item.url}:`, error.message);
  }
}

Per-call values make the policy visible at the point of navigation and allow each URL to get its own limit.

3. Understand timeout scope and precedence

Use the narrowest setting that matches the failure budget you intend to control. Playwright provides per-call settings as well as page and browser-context defaults. A navigation-specific default takes precedence over the general page default, and page-level settings take precedence over context-level settings.

Setting Scope When it fits
page.goto(url, { timeout }) One navigation call Recommended for URL-specific batch limits or different budgets per item.
page.setDefaultNavigationTimeout(ms) Navigation methods on one page, including goto, reload, and URL waits Set one navigation policy for a page when individual calls do not need custom limits.
page.setDefaultTimeout(ms) Page methods that accept a timeout Set a general default for actions and waits. A navigation-specific page default takes priority.
browserContext.setDefaultNavigationTimeout(ms) Navigation methods for pages in the context Apply a navigation default across pages in a context.
browserContext.setDefaultTimeout(ms) Methods that accept a timeout for pages in the context Apply a general context default; page-level settings take priority.
Playwright Test timeout The test’s total execution budget Bound a test, not an individual navigation in a standalone batch script.

A per-call setting is easiest to reason about in a loop. For example, a context default can still be useful for a shared baseline, with a larger or smaller timeout supplied to exceptional URLs. See the [Playwright Page API](https://playwright.dev/docs/api/class-page), [BrowserContext API](https://playwright.dev/docs/api/class-browsercontext), and [Playwright Test timeout guide](https://playwright.dev/docs/test-timeouts) for the documented APIs and scope details.

4. Bound the whole job separately

A navigation timeout does not limit browser launch, screenshot writing, application processing, or total batch duration. If the entire job must finish within a wall-clock budget, implement that deadline at the job or orchestration layer as well. Decide what should happen to a running browser operation when the deadline expires, and ensure the browser is closed during cleanup.

Likewise, the example’s try/catch lets later URLs proceed after an individual error, but it does not retry. If retries make sense for your sources, keep them bounded, record each attempt, and avoid retrying permanent failures indefinitely. Ensure output paths are unique per URL and attempt if retaining failed-run evidence matters.

5. Troubleshoot common failures

Symptom Likely cause Fix
Timeout ... exceeded from page.goto() The navigation did not reach the selected waitUntil state before its limit. Check URL reachability and the chosen state. Use a longer per-URL value for known slow sites, or a less demanding state such as domcontentloaded and then wait for the required content separately.
The page loads but the screenshot is missing expected content Navigation completed before client-rendered or delayed content became visible. Wait for a stable, meaningful locator or a deliberate delay only when needed; give that wait its own timeout.
networkidle frequently times out Analytics, streaming, polling, or other ongoing requests may prevent network idleness. Prefer a content readiness condition or another navigation state. Playwright discourages using networkidle as a general testing readiness signal.
One failure prevents later URLs from running The error handler is outside the loop, or the error is rethrown before the next iteration. Catch errors around each URL’s navigation and capture operations; summarize failures after the loop.
The batch exceeds its expected total runtime The per-navigation timeout is being mistaken for an end-to-end deadline, or each URL also has waits and screenshot work. Budget navigation, readiness waits, capture, and processing separately; apply a job-level deadline if needed.
A successful navigation returns an HTTP error page Navigation completion is not the same as an HTTP success status. Inspect the response returned by goto and decide whether to save the error page, flag it, or skip it.

6. Keep batch captures efficient and reliable

  • Set a realistic per-site budget. A short limit reduces time spent stuck on one URL but can classify slow, valid pages as failures. Use known site behavior to choose values.
  • Choose the earliest sufficient wait condition. Waiting for full load or network idle can add time when your screenshot only needs the initial DOM. Conversely, capturing too early creates incomplete output.
  • Use bounded concurrency when scaling. Sequential capture is easy to debug and limits browser load. Parallel pages can raise throughput, but consume more memory and may trigger site rate limits. Limit concurrency and isolate each worker’s output.
  • Record URL, duration, outcome, and error. These fields help distinguish unreachable targets, slow navigations, readiness failures, and screenshot write problems.
  • Keep browser cleanup in a finally block. A batch that fails partway through should still release its browser process.

Timeouts are a tradeoff between allowing slow pages enough time and preserving a predictable batch duration. There is no universal best value, and the official API material cited here provides no benchmark for selecting one. Start with a documented operational budget for your workload and adjust from your own failure records.

7. Or skip the browser setup

For a one-call screenshot endpoint, [ScreenshotNeo](https://screenshotneo.com) accepts a URL and returns a PNG, JPEG, WebP, or PDF. Read the [ScreenshotNeo API docs](https://screenshotneo.com/docs/) for request parameters and response details.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo accepts cookie and consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 screenshots.

Get 1,000 screenshots a month free, with no card.

8. Frequently asked questions

Is the timeout value in seconds?

No. Playwright timeout values are milliseconds. For example, 15 seconds is 15_000.

Does a timed-out navigation cancel the rest of my loop?

The exception rejects that navigation call. If you catch it inside the loop, the next iteration can run.

Should I use networkidle before every screenshot?

No. It can be unsuitable for pages with ongoing requests, and Playwright discourages it as a general testing readiness check. Select a navigation state and content-ready condition that match what the screenshot needs.

Does increasing page.goto‘s timeout bound screenshot capture too?

No. It is a navigation timeout. Give screenshot work and the overall job separate limits if they need their own bounds.