How to Log Network Requests with Puppeteer
Log outgoing request URLs, response URLs and statuses, and failed requests with Puppeteer page events. Learn how to filter logs and interpret redirects and HTTP errors.
To log outgoing request and response URLs with Puppeteer, attach listeners to the page’s request and response events before navigating. Add requestfailed to record requests that fail before receiving a response. For HTTP status errors such as 404 or 503, inspect the response status: those are responses, not necessarily request failures.
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
page.on('request', request => {
console.log('→', request.method(), request.url());
});
page.on('response', response => {
console.log('←', response.status(), response.url());
});
page.on('requestfailed', request => {
const failure = request.failure();
console.error('×', request.url(), failure?.errorText ?? 'failure text unavailable');
});
await page.goto('https://example.com');
} finally {
await browser.close();
}This uses Puppeteer’s passive page events; request interception is not required for logging. The listeners must be registered before page.goto() so they can observe the navigation request and early subresources. See the Puppeteer network logging guide and the ScreenshotNeo API documentation.
1. What each network event means
| Event | When it fires | What to log or infer |
|---|---|---|
request |
When the page issues a request. | Outgoing URL, method, resource type, and other request details. |
response |
When a response is received. | Response URL and HTTP status. A 404 or 503 is still an HTTP response. |
requestfinished |
When the response body has downloaded and the request completes. | Request lifecycle completion; it does not by itself mean the HTTP status was successful. |
requestfailed |
When a request fails instead of completing normally. | Browser-level failure information, if available. The failure details may be absent. |
A 404, 500, or 503 normally produces a response and then requestfinished, not requestfailed. Treat transport or browser failures and HTTP error statuses as separate conditions. The HTTPRequest API reference, failure() reference, and PageEvent reference describe these lifecycle details.
2. Log the complete request lifecycle
For diagnostics, include completion events as well as the basic outgoing and incoming URL logs. A request may produce a response event before its body has finished downloading.
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
page.on('request', request => {
console.log('REQUEST', request.method(), request.resourceType(), request.url());
});
page.on('response', response => {
console.log('RESPONSE', response.status(), response.url());
});
page.on('requestfinished', request => {
console.log('FINISHED', request.url());
});
page.on('requestfailed', request => {
const failure = request.failure();
console.error('FAILED', request.url(), failure?.errorText ?? 'no failure detail');
});
await page.goto('https://example.com');
} finally {
await browser.close();
}Keep the failure() null check: Puppeteer documents that its result can be nullable, and failure text may not be available in every case.
3. Filter requests and responses
Pages can issue many requests. Filter in the callback to focus on a hostname, resource type, or method. The request object exposes properties such as URL, method, headers, post data, and resource type.
const targetHost = 'api.example.com';
page.on('request', request => {
const url = new URL(request.url());
if (url.hostname !== targetHost) return;
console.log(request.method(), request.url());
});
page.on('response', response => {
const url = new URL(response.url());
if (url.hostname !== targetHost) return;
console.log(response.status(), response.url());
});To filter by resource type, check request.resourceType(), for example if (request.resourceType() !== 'document') return;. To filter by method, use request.method(). Decide whether subdomains should match before using a suffix check: a plain string suffix can also match an unintended hostname.
Network logs can contain sensitive data. Avoid logging authorization headers, cookies, or request bodies into shared output unless you have a clear need and suitable access controls. Prefer recording only the URL, method, status, and failure detail required to diagnose the issue.
4. Redirects and URL interpretation
A redirect is represented as a request to one URL that receives a redirect response, followed by a new request to the destination. Expect multiple request entries in the log. Do not assume the original request’s URL is silently replaced by the final URL.
When diagnosing a redirect chain, log every request URL and every response status. This lets you see intermediate redirects and distinguish them from a destination request that returns an error.
5. Logging versus request interception
| Approach | Use it for | Operational effect |
|---|---|---|
Page events (request, response, and lifecycle events) |
Observing URLs, statuses, and request outcomes. | Passive logging; the normal choice for inspection. |
| Request interception | Continuing, aborting, or fulfilling requests to change traffic behavior. | More complexity: requests can stall until resolved, and handlers must account for requests already handled. |
Do not enable interception solely to print URLs. Puppeteer’s network interception guide explains the interception flow and the need to resolve intercepted requests. Use the setRequestInterception API only when you intend to alter or control traffic.
6. Remove listeners when logging is done
Use named callbacks when you need to stop logging without discarding the page. Puppeteer pages are event emitters, and listeners can be removed with off.
function logRequest(request) {
console.log(request.url());
}
page.on('request', logRequest);
// Later, when this logging scope ends:
page.off('request', logRequest);This is useful in long-running scripts that reuse a page or attach temporary diagnostics. If the page is dedicated to one navigation, closing it or the browser also ends that page’s event lifetime.
7. Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| The initial document request is missing. | Listeners were attached after navigation began. | Register listeners before calling page.goto(). |
| A 404 or 503 appears as a response, not a failure. | HTTP error statuses are received responses; they are not automatically browser-level request failures. | Log response.status() and classify status codes separately from requestfailed. |
request.failure() has no useful message. |
The failure object or its error text can be unavailable. | Use optional chaining and a fallback message; preserve the URL as the key diagnostic detail. |
| Only one URL appears for a redirected page. | The script may be filtering logs or not printing all response and request events. | Log every request and response before filtering, then inspect each redirect response and its follow-up request. |
| Navigation times out even though requests are logged. | The page’s chosen navigation wait condition may not be reached, or ongoing activity may keep the page busy. | Choose a navigation condition appropriate to the page, handle navigation timeouts explicitly, and retain the network events already observed for diagnosis. |
| Requests hang after enabling interception. | An intercepted request was not continued, aborted, or fulfilled, or another handler already resolved it. | For logging only, disable interception. If modifying traffic, resolve every request and check whether an interception resolution was already handled. |
| Logs expose tokens or private data. | Headers, cookies, or post bodies were recorded too broadly. | Remove sensitive fields, restrict log access, and retain only the diagnostic data you need. |
8. Performance and reliability considerations
- Keep event handlers quick. Avoid slow synchronous work in callbacks. If writing to a remote log sink, buffer records and send them outside the event callback where practical.
- Control log volume. Filter by hostname, method, or resource type, and avoid dumping full headers or bodies by default.
- Register early and scope listeners. Attach before navigation and remove temporary handlers when finished so repeated runs do not duplicate log lines.
- Keep failure and status accounting separate. Count HTTP statuses from responses and browser-level failures from
requestfailed; these represent different outcomes. - Do not overstate coverage. These page events provide page-level request observation. They do not establish exhaustive capture of every possible browser, worker, or service-worker network operation.
- Cost depends on your browser workload. Puppeteer logging itself is local instrumentation, but browser runtime, infrastructure, and any log storage or transfer have costs determined by where and how you run them. The cited documentation provides no universal performance benchmark.
9. Or skip the browser setup
If your goal is a clean screenshot rather than inspecting request logs, ScreenshotNeo is a website screenshot API and MCP server for developers. It takes a URL in one GET request and returns a PNG, JPEG, WebP, or PDF. This does not replace Puppeteer network logging; it removes the need to set up a browser when you only need the captured page.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webpimport requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
timeout=90,
)
open("shot.webp", "wb").write(r.content)const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);See the ScreenshotNeo API documentation for setup and options. Cookie and consent banners are accepted or removed, along with known newsletter popups and chat widgets, before capture; each of those steps can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers indicate the page verdict and whether the request was billed. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents using Claude, Cursor, or another MCP client. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.
Sign up free for 1,000 screenshots a month with no card.
10. FAQ
Can I log outgoing request URLs and response URLs?
Yes. Print request.url() in the request handler and response.url() in the response handler.
Do I need to enable request interception to log traffic?
No. Page events are the passive logging mechanism. Interception is for controlling requests.
Does requestfailed mean the server returned a 500?
Not necessarily. A 500 is an HTTP response and should be detected through the response status. requestfailed indicates a request failure in the browser lifecycle.
Will these events capture every network operation in every browser context?
The cited page event guidance supports page-level request observation. It does not promise exhaustive coverage of workers or service workers, so do not treat this logger as a complete browser-wide network archive.
