How to Access a Specific Network Response as JSON With Puppeteer

To access a specific network response as JSON with Puppeteer, register page.waitForResponse() before triggering the request, await the matching response, then call response.json(). Matching a URL and status narrows the response; validate the parsed data separately because a matching endpoint does not guarantee the payload is the object your code expects.

const responsePromise = page.waitForResponse(
  response =>
    response.url().includes('/api/data') &&
    response.status() === 200
);

await page.click('button');
const response = await responsePromise;
const data = await response.json();

This pattern works when a page action, such as clicking a button, triggers the endpoint you need. For a stable, unique endpoint, you can match the exact URL instead. Puppeteer documents waitForResponse() as returning a promise for the matched response; its current reference is for Puppeteer 25.12.0. See the Page.waitForResponse API and HTTPResponse.json API.

1. Install Puppeteer and set up a page

In a new Node.js project, install Puppeteer. The package downloads a compatible browser by default. If your environment supplies a browser separately, consult Puppeteer’s configuration guide for browser and executable-path settings.

npm install puppeteer

Save this complete example as response-json.js. Replace the sample page URL, button selector, and endpoint path with values from the site you are automating.

const puppeteer = require('puppeteer');

async function main() {
  const browser = await puppeteer.launch({ headless: true });
  try {
    const page = await browser.newPage();
    await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });

    // Start watching before the click can trigger the request.
    const responsePromise = page.waitForResponse(
      response => {
        const request = response.request();
        return response.url().includes('/api/data') &&
          request.method() === 'GET' &&
          response.status() === 200;
      },
      { timeout: 15000 }
    );

    await page.click('button[data-load]');
    const response = await responsePromise;
    const data = await response.json();

    if (!data || typeof data !== 'object') {
      throw new Error('Expected the endpoint to return a JSON object');
    }
    console.log(JSON.stringify(data, null, 2));
  } finally {
    await browser.close();
  }
}

main().catch(error => {
  console.error(error);
  process.exitCode = 1;
});

The finally block closes the browser whether the wait succeeds or an error occurs. The example uses a bounded 15-second response wait so an absent or mismatched request does not wait indefinitely.

2. Register the response wait before the action

The order is essential. Calling waitForResponse() creates a promise that monitors upcoming responses. If you click first and only then call it, a fast request may already have completed and the wait can miss it.

Create the response wait first, trigger the request, then parse the matched body.

Navigate to the page and satisfy any prerequisites for the action.
Create the response promise with the narrowest useful matcher.
Perform the click, form submission, or other action that starts the request.
Await the response promise and parse the body.

It is fine to await the action before awaiting the response promise, provided the promise was created first. If the click itself waits for a page condition that depends on the response, avoid serially waiting for that condition before consuming the response; start both waits first and coordinate them with Promise.all() when appropriate.

const responsePromise = page.waitForResponse(
  response => response.url() === 'https://example.com/api/items'
);
const clickPromise = page.click('#load-items');

const [response] = await Promise.all([responsePromise, clickPromise]);
const items = await response.json();

Use this coordination only when the click operation and response wait are independent. A selector error from page.click() should be handled as an action failure, not mistaken for a response timeout.

3. Choose a reliable matcher

Exact URL

If the endpoint URL is known and unique, pass it as a string:

Narrow the match by endpoint and request properties before validating the returned data.

const responsePromise = page.waitForResponse(
  'https://example.com/api/resource'
);
await page.click('#load');
const response = await responsePromise;
const body = await response.json();

An exact URL can be too strict when the request includes changing query parameters. In that case, parse the URL and compare its origin, path, and relevant parameters.

Predicate

A predicate gives control over response properties. You can check URL, status, and request method. Puppeteer’s API reference documents both string and predicate matching and includes an example that checks URL and status.

const responsePromise = page.waitForResponse(response => {
  const url = new URL(response.url());
  return url.origin === 'https://example.com' &&
    url.pathname === '/api/search' &&
    url.searchParams.get('type') === 'products' &&
    response.request().method() === 'GET' &&
    response.status() === 200;
});

Prefer exact path checks to broad substring checks when other endpoints could contain the same text. If the application sends several similar calls, distinguish them with query parameters or request method. A status check helps avoid treating an error response as a successful result, but some APIs legitimately return application data with non-200 statuses; align the condition with that endpoint’s contract.

Asynchronous predicates

The predicate may be asynchronous. That can help when the matching decision depends on response text, but reading a body inside the predicate may make selection more complex and can obscure errors. Usually select by URL and request properties first, then parse and validate the body after the wait resolves.

4. Parse and validate the JSON

await response.json() parses the response body and returns its JSON value. The value can be an object, array, string, number, boolean, or null; do not assume it is always an object with the fields you need.

const data = await response.json();

if (!data || typeof data !== 'object' || Array.isArray(data)) {
  throw new Error('Expected a JSON object');
}
if (!Array.isArray(data.results)) {
  throw new Error('Expected a results array');
}

console.log(`Received ${data.results.length} results`);

A successful match only establishes that the response met your predicate. It does not establish that the server returned valid JSON or that the payload conforms to your application’s schema. Handle parse failures and validate required fields before using values in later steps.

If you need to inspect a response body for debugging, use the response methods documented by Puppeteer and avoid assuming every response contains readable JSON. A server can return HTML, an empty body, or an error payload. The endpoint contract determines what a valid result looks like.

5. Configure timeouts and cancellation

Puppeteer documents a 30-second default timeout for waitForResponse(). Pass a timeout in the options object to tune it for the expected request. The API also permits timeout: 0 to disable the wait timeout; use that only when your surrounding code has another reliable cancellation mechanism.

const responsePromise = page.waitForResponse(
  response => response.url().includes('/api/report'),
  { timeout: 10000 }
);

You can also change the page’s default timeout with page.setDefaultTimeout(milliseconds), which affects supported page waits. The method accepts an AbortSignal in its options, allowing the caller to cancel a wait as part of a larger operation.

const controller = new AbortController();
const responsePromise = page.waitForResponse(
  response => response.url().includes('/api/report'),
  { timeout: 20000, signal: controller.signal }
);

// Cancel if the surrounding task is abandoned:
// controller.abort();

Choose a timeout long enough for normal server and page behavior, but keep it bounded so a broken trigger or incorrect matcher becomes an actionable error. A larger timeout does not fix a matcher that can never succeed.

6. When to use a response event listener

For one response that gates the next step, waitForResponse() is usually the clearest option. For ongoing monitoring of many responses, Puppeteer’s Page supports the response event. The page is an EventEmitter; listeners can be removed with page.off(). See the PageEvent reference.

function onResponse(response) {
  if (response.url().includes('/api/data')) {
    console.log('Observed:', response.url(), response.status());
  }
}

page.on('response', onResponse);
try {
  await page.click('#load');
  // Continue monitoring while this listener is needed.
} finally {
  page.off('response', onResponse);
}

Registering a listener does not return a response to the await at the registration line. If a later step needs a captured response, store it explicitly or create a promise that resolves from the listener, and remove the listener when finished. Clean up listeners to prevent duplicate handling if the same page is reused.

7. cURL, Python, and Node.js alternatives

Puppeteer is useful when the request depends on browser behavior such as a click, page state, or browser-managed session. If you already know the endpoint and can make the request directly, a command-line or HTTP client may be simpler. Those clients do not reproduce a browser action automatically: you must supply the appropriate URL, headers, cookies, and request data yourself.

cURL

curl --fail-with-body --silent --show-error \
  'https://example.com/api/data' \
  -H 'Accept: application/json' \
  -o response.json

node -e "const fs=require('fs'); const x=JSON.parse(fs.readFileSync('response.json','utf8')); console.log(x);"

For an endpoint requiring authentication, add the documented authorization header, for example -H 'Authorization: Bearer YOUR_TOKEN'. Do not put real credentials in shared shell history or source control.

Python

import requests

response = requests.get(
    "https://example.com/api/data",
    headers={"Accept": "application/json"},
    timeout=20,
)
response.raise_for_status()
data = response.json()
print(data)

Node.js fetch

const response = await fetch('https://example.com/api/data', {
  headers: { Accept: 'application/json' },
  signal: AbortSignal.timeout(20000),
});

if (!response.ok) {
  throw new Error(`Request failed: HTTP ${response.status}`);
}
const data = await response.json();
console.log(data);

These examples issue a direct HTTP request. Use the Puppeteer approach when the browser must perform the interaction or when the page’s session context matters. Do not copy browser cookies into a script unless you are authorized to use that account and the site permits the automation.

8. Troubleshooting common failures

Symptom	Likely cause	Fix
Response wait times out	The action did not trigger the request, the wait began too late, or the predicate is too narrow.	Create the wait before the action; confirm the selector and inspect observed response URLs; relax only the matcher condition that is incorrect.
The wrong response matches	A path substring also matches another endpoint, or multiple calls share a path.	Match the origin, exact pathname, relevant query parameter, method, and status.
`response.json()` rejects	The body is not valid JSON, is empty, or is an error document.	Check status and endpoint contract; inspect the response content type or body using supported response methods and handle non-JSON results explicitly.
Parsed value lacks expected fields	The endpoint returned a valid but different payload, such as an error object or empty result.	Validate the shape and required fields; adjust the endpoint or application state that selects the result.
Click fails before the wait resolves	The selector is missing, covered, or the page is not ready for interaction.	Confirm the selector and page state; wait for the relevant element or use the application’s actual interaction flow.
Wait hangs too long	The timeout was disabled or set too high.	Use a finite timeout and, for multi-step work, an AbortSignal tied to task cancellation.
Repeated logs or handlers	A response event listener was registered repeatedly and not removed.	Keep a named handler reference and call `page.off()` during cleanup.

For diagnosis, temporarily log response URLs and statuses from a response listener, then remove that instrumentation after identifying the endpoint. Avoid logging sensitive query parameters, headers, or response content in production.

9. Reliability, performance, and cost considerations

Waiting for a specific response is generally more targeted than waiting for an arbitrary delay: it lets the next step proceed when the endpoint of interest responds. Reliability comes from registering early, narrowing the match correctly, setting a realistic timeout, and validating data. The page can still fail before the request, the server can return an error, and the selected payload can change; handle each as a distinct failure.

Keep the matcher inexpensive. URL, status, and method checks are sufficient for most workflows. A broad listener that inspects every response body adds work and makes the selection logic harder to reason about. If you need to monitor many endpoints, filter early and retain only the data required by the task.

Browser automation has setup and runtime costs: a browser must launch, navigate, and execute the interaction. If the endpoint can be called directly, an HTTP client may use fewer resources. If the task is to capture a rendered page rather than extract its underlying API payload, use a screenshot-oriented tool.

10. Or skip the browser setup

If your goal is a screenshot rather than access to an API response body, ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. One GET request returns a PNG, JPEG, WebP, or PDF. See the ScreenshotNeo API documentation for request options and setup.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

ScreenshotNeo accepts cookie and consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, with response headers indicating the page verdict and billing status. Its MCP server gives Claude, Cursor, and other MCP clients the tools take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.

import requests

r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

11. Frequently asked questions

Does `waitForResponse()` return the parsed object?

No. It resolves with a Puppeteer response object. Call and await response.json() to parse a JSON body.

Can I wait for a response without clicking?

Yes. Start the wait before whichever action or page behavior causes the request. The triggering event can be navigation, a form submission, or another browser interaction.

Should I match status 200?

Match the statuses your endpoint considers successful. A strict 200 condition is useful for many APIs, but the endpoint’s contract determines which status codes carry expected data.

Can I use this for a screenshot?

This pattern extracts a network response body. For a rendered screenshot, use a browser screenshot workflow or a screenshot API such as ScreenshotNeo.