How to Wait for a CAPTCHA to Load in Pyppeteer

Use an explicit wait instead of a fixed sleep. For a CAPTCHA or challenge element whose selector you know, call page.waitForSelector() with a finite timeout. If readiness depends on several page conditions, use page.waitForFunction(). These waits tell you that the UI state appeared; they do not solve or bypass a CAPTCHA.

Basic selector wait

Pyppeteer resolves waitForSelector when a matching element appears. Set visible: True when the element must also be rendered visibly, and choose a timeout that fits the page you control.

import asyncio
from pyppeteer import launch

async def main():
    browser = await launch(headless=True)
    page = await browser.newPage()

    try:
        await page.goto("https://example.com", {"waitUntil": "domcontentloaded"})
        await page.waitForSelector(
            "YOUR_PAGE_SPECIFIC_SELECTOR",
            {
                "visible": True,
                "timeout": 30000,
            },
        )
        print("The challenge element is visible")
    except Exception as exc:
        print(f"The challenge did not appear: {exc}")
    finally:
        await browser.close()

asyncio.get_event_loop().run_until_complete(main())

The Pyppeteer API reference documents a 30,000 millisecond default for selector and function waits. Supplying the timeout explicitly makes the behavior clear and lets your application classify a timeout as a page-specific failure.

Choose the right wait

What you know	Use	Why
A stable element selector	`waitForSelector`	Resolves when that element appears; can require visibility.
A state involving multiple checks	`waitForFunction`	Runs a page-side predicate until it returns a truthy value.
An action should reload or navigate	`waitForNavigation`	Waits for navigation, not for an asynchronously rendered challenge.
The challenge is inside an iframe	Frame-level `waitForSelector`	The element belongs to the frame’s document, not the top page.

Wait for a page-specific condition

Use waitForFunction when one selector is insufficient. The predicate must describe an observable state on the authorized page, such as a challenge container being present and visible.

await page.waitForFunction(
    """() => {
        const el = document.querySelector('YOUR_PAGE_SPECIFIC_SELECTOR');
        if (!el) return false;
        const style = window.getComputedStyle(el);
        const box = el.getBoundingClientRect();
        return style.display !== 'none'
            && style.visibility !== 'hidden'
            && box.width > 0
            && box.height > 0;
    }""",
    {"timeout": 30000, "polling": "mutation"},
)

Polling can be configured for the condition you need. Keep the predicate specific to the page you are automating; there is no universal CAPTCHA-loaded event or selector.

Handle challenge iframes

Many challenge widgets render their controls in an iframe. First inspect the frames, identify the frame belonging to the authorized page, then wait inside that frame.

frames = page.frames
for frame in frames:
    print(frame.url)

challenge_frame = next(
    (frame for frame in frames if "authorized-provider.example" in frame.url),
    None,
)

if challenge_frame is None:
    raise RuntimeError("Challenge frame was not found")

await challenge_frame.waitForSelector(
    "YOUR_FRAME_SELECTOR",
    {"visible": True, "timeout": 30000},
)

Frame URLs and selectors vary by provider and integration. Do not assume a selector copied from another site will work on your page.

Complete reusable helper

import asyncio
from pyppeteer import launch
from pyppeteer.errors import TimeoutError as PyppeteerTimeoutError

async def wait_for_challenge(page, selector, timeout_ms=30000):
    try:
        await page.waitForSelector(
            selector,
            {"visible": True, "timeout": timeout_ms},
        )
        return {"ready": True, "method": "selector"}
    except PyppeteerTimeoutError:
        return {
            "ready": False,
            "method": "selector",
            "reason": "timeout",
        }

async def main():
    browser = await launch(headless=True)
    page = await browser.newPage()
    try:
        await page.goto(
            "https://example.com",
            {"waitUntil": "domcontentloaded", "timeout": 30000},
        )
        result = await wait_for_challenge(
            page,
            "YOUR_PAGE_SPECIFIC_SELECTOR",
            timeout_ms=30000,
        )
        print(result)
    finally:
        await browser.close()

asyncio.get_event_loop().run_until_complete(main())

Why fixed sleeps are unreliable

await asyncio.sleep(5) only guesses how long a page might take. A fast response wastes time; a slow script still has not rendered when the sleep ends. A DOM wait ties the next step to an observable condition and gives you a bounded failure path.

Use waitForNavigation only when the preceding action is expected to navigate or reload. A challenge inserted by JavaScript after the initial response needs a selector or function wait instead.

Timeouts, errors, and fixes

Error or symptom	Likely cause	Fix
`TimeoutError`	The selector never appeared, appeared after the deadline, or was rendered in another frame.	Verify the selector on the authorized page, inspect `page.frames`, and choose a finite timeout appropriate for that page.
Element exists but wait never resolves with `visible: True`	The node is hidden by `display: none` or `visibility: hidden`.	Wait for the visible state or use a page-specific condition that checks the actual readiness state.
Selector works in DevTools but not Pyppeteer	The element is inside an iframe or a shadow DOM.	Switch to the matching frame; for shadow DOM, expose a page-specific condition from the host page.
`waitFor` behaves unexpectedly	Its string argument can be inferred as either JavaScript or a selector.	Call `waitForSelector` or `waitForFunction` explicitly.
Navigation wait hangs	The action did not navigate, or navigation completed before the wait started.	Use a navigation wait only around an action that really reloads; otherwise wait for the rendered element.
Browser closes before the check	The coroutine raised before cleanup.	Put the workflow in `try/finally` and always close the browser.
Different results between runs	Network latency, provider state, or page scripts vary.	Use explicit readiness conditions, bounded retries where allowed, logging, and captured diagnostics.

Reliability checklist

Use a selector verified for the exact authorized page and integration.
Set a finite timeout and catch the timeout exception.
Check frames before concluding that the widget did not load.
Log the URL, frame URLs, elapsed time, and failure reason.
Take a diagnostic screenshot or save HTML only when your data-handling rules permit it.
Keep waiting separate from any later, authorized human verification step.
Do not treat the presence of a challenge as proof that it has been solved.

Performance and cost considerations

Selector and condition waits return as soon as their condition is true, so they usually avoid the unnecessary delay of a conservative fixed sleep. A finite timeout also bounds the time a worker can spend on a page that never becomes ready. Choose the smallest timeout that covers normal latency for the page, then record timeout rates so you can adjust based on your own workload.

Pyppeteer runs a browser process, so concurrency consumes CPU and memory. Reuse a browser where appropriate, create isolated pages for jobs, and close pages and browsers in cleanup paths. The consulted Pyppeteer API reference is for version 0.0.25; check the version installed in your project before relying on exact defaults or method signatures.

Or skip the browser setup

If your goal is a clean screenshot rather than browser automation, ScreenshotNeo provides a single request for a PNG, JPEG, WebP, or PDF. Its capture pipeline accepts cookie and consent banners before the shot, removes more than 60 known consent platforms plus newsletter popups and chat widgets, and lets you turn each step off. Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed; the response identifies the result with X-Page-Verdict and X-Billed headers. An MCP server also exposes take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients.

See the ScreenshotNeo API documentation for the full option list.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://example.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://example.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`HTTP ${res.status}`);
const image = Buffer.from(await res.arrayBuffer());
require('fs').writeFileSync('shot.webp', image);

There is a free plan with 1,000 screenshots per month and no card. Paid plans start at $5 for 3,000 screenshots. Create a free ScreenshotNeo account.

FAQ

What selector should I use for a CAPTCHA?

There is no universal selector. Inspect the authorized page and choose a provider- and integration-specific element or condition.

Does waiting for the element solve the CAPTCHA?

No. It only observes that the page reached the state you defined. Solving or bypassing a challenge is outside this technique.

Should I increase the timeout indefinitely?

No. Keep a finite deadline, classify timeouts, and investigate whether the selector, frame, network state, or page integration is wrong.

When should I use `waitForFunction`?

Use it when readiness requires a predicate such as visibility, dimensions, or multiple DOM conditions rather than one matching element.

Why does a selector wait work on the page but not in my script?

The element may be inside an iframe, may be hidden, or may be created after your timeout. Inspect frames and verify the actual rendered state.