How to Fix Pyppeteer’s “networkidle0” Not Waiting for the Page to Load

Short answer: Pyppeteer’s networkidle0 condition means that there are zero active network connections for at least 500 milliseconds. It measures network quiet, not whether the application has rendered the data your script needs. Pages with analytics, polling, WebSockets, advertisements, lazy requests, or long-running connections may never satisfy it. Start navigation with domcontentloaded or load, then wait for a stable selector or an application-specific JavaScript condition.

The complete API definitions are in the Pyppeteer 0.0.25 API reference. The examples below use Python because Pyppeteer is a Python library.

What networkidle0 actually waits for

Pyppeteer documents two network-idle thresholds:

Condition	Required network state	Best use
networkidle0	No active connections for at least 500 ms	Pages known to stop all requests
networkidle2	No more than two active connections for at least 500 ms	Pages with a small amount of persistent traffic

These are navigation lifecycle thresholds. They do not assert that a target element exists, that a table has rows, or that an application’s asynchronous state is complete. A page can be visually ready while requests continue, or become network-idle before JavaScript inserts the content you need.

Choose a readiness signal that matches your task

Your requirement	Recommended wait	Why
HTML has been parsed	waitUntil: 'domcontentloaded'	Usually completes before images and third-party resources.
Load event has fired	waitUntil: 'load'	Waits for the browser’s load event.
A component exists or is visible	waitForSelector()	Directly represents the content you will use.
Application data is populated	waitForFunction()	Waits for a truthy condition in the page.
Navigation follows a click	Concurrent waitForNavigation() and click	Prevents an event-order race.
All requests really must stop	networkidle0	Use only when persistent connections are not expected.

Do not change the timeout until you have selected the right condition. A longer timeout helps when the correct condition eventually occurs slowly; it cannot make a condition occur when the page keeps a connection open forever.

A reliable baseline pattern

import asyncio
from pyppeteer import launch


async def main():
    browser = await launch()
    page = await browser.newPage()

    await page.goto(
        'https://example.com/dashboard',
        {
            'waitUntil': 'domcontentloaded',
            'timeout': 30000,
        },
    )

    await page.waitForSelector(
        '#dashboard-content',
        {
            'visible': True,
            'timeout': 10000,
        },
    )

    html = await page.content()
    print(html[:200])
    await browser.close()


asyncio.get_event_loop().run_until_complete(main())

Replace #dashboard-content with a stable element that proves the page is ready for your operation. The selector should represent the result you need, not a generic wrapper that appears before its contents are populated.

Wait for application state with waitForFunction

Use waitForFunction() when readiness is expressed as data or state rather than an element.

await page.goto(
    'https://example.com/reports',
    {'waitUntil': 'domcontentloaded', 'timeout': 30000},
)

await page.waitForFunction(
    """() => {
        return window.reportData &&
               Array.isArray(window.reportData.rows) &&
               window.reportData.rows.length > 0;
    }""",
    {'timeout': 15000},
)

rows = await page.evaluate(
    """() => window.reportData.rows"""
)

The function must eventually return a truthy value. Keep the condition specific enough to avoid reading an empty placeholder or stale state.

Use networkidle0 only when it is the right condition

This pattern is valid for a page that is documented or observed to stop all requests:

await page.goto(
    'https://example.com/static-report',
    {
        'waitUntil': 'networkidle0',
        'timeout': 30000,
    },
)

It is a poor fit for pages that poll an API, maintain analytics traffic, stream updates, open a WebSocket, load ads continuously, or retry failed resources. For those pages, use a content signal and, if necessary, a short deliberate delay after the signal for animations or secondary rendering.

Wait for a click that triggers navigation

Arm the navigation wait before clicking. Pyppeteer documents this concurrent pattern; starting the wait after the click can miss a fast navigation event.

import asyncio

await asyncio.gather(
    page.waitForNavigation({
        'waitUntil': 'domcontentloaded',
        'timeout': 30000,
    }),
    page.click('a.next'),
)

await page.waitForSelector('#results', {'visible': True, 'timeout': 10000})

If the click changes content through client-side rendering without a URL navigation, use waitForSelector() or waitForFunction() instead of waitForNavigation().

Configure navigation and wait timeouts

Pyppeteer’s documented default navigation timeout is 30 seconds. Set it for one operation:

await page.goto(
    url,
    {'waitUntil': 'domcontentloaded', 'timeout': 60000},
)

Or set the default for the page:

page.setDefaultNavigationTimeout(60000)

Selector and function waits also accept a timeout:

await page.waitForSelector(
    '.product-card',
    {'visible': True, 'timeout': 15000},
)

await page.waitForFunction(
    '() => document.querySelectorAll(".product-card").length >= 20',
    {'timeout': 15000},
)

timeout: 0 disables the timeout. Use that only when an external cancellation mechanism exists; otherwise a condition that never becomes true can leave a worker waiting indefinitely.

Diagnose a networkidle0 timeout

1. Capture the complete exception. A navigation timeout, SSL error, invalid URL, and main-resource failure have different fixes.
2. Log the final URL. Redirects may send the browser to a login page, consent page, or error page.
3. Identify the required state. Decide whether you need parsed HTML, the load event, a visible element, or populated data.
4. Check for persistent activity. Polling, WebSockets, analytics, ad requests, and retries can keep the active-connection count above zero.
5. Replace the lifecycle wait. Try domcontentloaded followed by a selector or function condition.
6. Increase the timeout only after that. Use a larger value when the chosen condition is valid but slow.
7. Record versions. Include your Pyppeteer version, Chromium revision, Python version, URL, minimal code, and full traceback when reproducing the issue.

Common errors and fixes

Symptom	Likely cause	Fix
Navigation Timeout Exceeded	The selected lifecycle condition never occurs within the timeout.	Use a selector or function condition; increase the timeout only if that condition is expected to finish.
waitForSelector times out	The selector is wrong, the element is inside an iframe, or the page shows an error state.	Inspect the final HTML and URL, verify the selector, and target the correct frame.
Page appears blank	JavaScript failed, a bot check blocked the page, or the capture happened before rendering.	Check console and response errors, wait for a meaningful element, and handle the page’s access requirements.
Click followed by navigation wait hangs	The navigation promise was started after the click, or the click only updates client-side state.	Start both operations with asyncio.gather(), or wait for the updated selector instead.
Increasing timeout changes nothing	The page continuously makes requests or the target condition is incorrect.	Stop using network-idle as the readiness test and express the required state directly.
Script waits forever with timeout disabled	timeout: 0 permits an unreachable condition to wait indefinitely.	Restore a finite timeout and add cancellation or retry handling.

Frames, lazy content, and delayed rendering

Content inside an iframe

page.waitForSelector() searches the main frame. If the required content is in an iframe, locate the frame and wait there:

frame = next(
    frame for frame in page.frames
    if 'widget.example.com' in frame.url
)
await frame.waitForSelector('.ready', {'visible': True, 'timeout': 10000})
text = await frame.JJeval('.ready', '(el) => el.textContent')

Frame URLs and structure vary by site, so inspect page.frames before hard-coding a selection rule.

Lazy-loaded images

Network idle does not guarantee that an image below the fold has been requested. Scroll to the area you need, then wait for an image or application marker:

await page.evaluate('window.scrollTo(0, document.body.scrollHeight)')
await page.waitForSelector('img[data-loaded="true"]', {'timeout': 10000})

Animations and transitions

An element can exist before its final pixels are stable. After waiting for the element, use a short bounded delay only when the page’s animation requires it. A delay alone is less reliable than a selector or state condition.

Reliability patterns for production jobs

• Use a finite navigation timeout and a separate finite content timeout.
• Retry transient navigation failures with a small limit and record each attempt.
• Make readiness selectors stable; avoid generated class names when a semantic attribute is available.
• Verify the result after waiting. For example, reject an empty table rather than treating its wrapper as success.
• Close the browser in a finally block so failed jobs do not leak Chromium processes.
• Persist the URL, final URL, wait strategy, timeout, exception, and browser revision for debugging.
• Do not disable timeouts globally unless the worker has a separate hard deadline.

browser = await launch()
page = await browser.newPage()
try:
    await page.goto(url, {'waitUntil': 'domcontentloaded', 'timeout': 30000})
    await page.waitForSelector('#content', {'visible': True, 'timeout': 10000})
    result = await page.JJeval('#content', '(el) => el.textContent.trim()')
    if not result:
        raise RuntimeError('content marker was present but empty')
finally:
    await browser.close()

Performance and cost considerations

networkidle0 can add latency because it waits for a quiet window after every request has finished. On pages with ongoing traffic, it adds latency until timeout and then fails. A selector or function wait often finishes sooner because it stops when the required state is available.

Use domcontentloaded when you do not need images or the load event. Avoid waiting for the entire document when your task needs one component. Keep selectors specific and timeouts bounded so failed pages release workers quickly.

For high-volume screenshot work, running and maintaining Chromium adds startup time, memory use, browser downloads, and failure modes. A hosted screenshot endpoint can move that browser setup out of your application.

Or skip the browser setup

ScreenshotNeo provides a website screenshot API and MCP server. A single GET request returns a PNG, JPEG, WebP, or PDF. Before capture, it accepts cookie and consent banners and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and the response identifies the result with X-Page-Verdict and X-Billed headers.

See the ScreenshotNeo API documentation for all options, including full-page capture, element selectors, device presets, custom viewports, dark mode, retina scale, PDF settings, custom CSS and JavaScript, click actions, wait conditions, request blocking, headers, cookies, user agents, authorization, timezone, geolocation, transparent backgrounds, resizing, caching, signed links, asynchronous jobs, bulk capture, usage data, and the OpenAPI specification.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot failed: ${res.status}`);
const buffer = Buffer.from(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', buffer));

ScreenshotNeo includes an MCP server with take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. Plans include 1,000 screenshots per month free with no card; paid plans start at $5 for 3,000 shots. Create a free ScreenshotNeo account.

FAQ

Is networkidle2 always better than networkidle0?

No. It tolerates up to two active connections, but it still measures network activity rather than application readiness. Choose it only when that threshold matches the page behavior.

Should I always use domcontentloaded?

No. It is a useful starting milestone, but you should add the selector or function condition that proves your task is ready.

Can I wait for both networkidle0 and a selector?

You can, but the stricter network condition can delay or block a job unnecessarily. Prefer the smallest set of conditions that represents the output you need.

Why does a page look complete in a normal browser but fail in Pyppeteer?

The normal browser may keep running while scripts retry, may already have cached data, or may handle consent and bot checks interactively. Log the final URL, browser errors, and the exact readiness condition to find the difference.

Where are the official method definitions?

Use the Pyppeteer API reference for goto(), waitForNavigation(), waitForSelector(), waitForFunction(), and timeout options. The development source also documents the 500 ms lifecycle thresholds.