What Is a Fetch API? When Your Agent Needs One Instead of a Browser

The Fetch API lets JavaScript send HTTP requests and read the responses. It does not render a web page or run the page’s JavaScript. Use Fetch when your agent knows the endpoint and can work with the returned data; use a browser when it needs rendered content, interaction, or a visual result.

For example, Fetch is a good fit for calling a JSON API, submitting a form, downloading a file, or retrieving HTML that is already present in the server response. A browser automation tool is needed when content appears only after client-side scripts run, or when the task involves clicking, typing, scrolling, inspecting layout, or using browser-managed session state.

1. What is the Fetch API?

The WHATWG Fetch Standard defines requests, responses, and the process that connects them. It also defines the JavaScript fetch() API, a relatively low-level way to perform network requests. MDN describes Fetch as an interface for retrieving resources and a more flexible replacement for XMLHttpRequest. WHATWG Fetch Standard · MDN Fetch API

Fetch is an API built into modern browser JavaScript and available as a global in supported server runtimes such as Node.js. You do not need to install a third-party HTTP client to use the basic interface. In both environments, the usual shape is: create a request with fetch(url, options), await a Response, check its status and headers, then read its body.

A Response is not a rendered page. It contains status information, headers, and a body stream. If the body is HTML, Fetch gives you the HTML bytes or text; it does not execute scripts in that HTML, build the page’s live DOM, or calculate its visual layout.

2. Decide whether an agent needs Fetch or a browser

Task	Use	Reason
Call a known JSON endpoint	Fetch	The useful result is structured response data.
Submit a webhook or download a file	Fetch	The task is an HTTP exchange and does not need page rendering.
Read static HTML or text returned by a server	Fetch	The needed content is already in the response body.
Read data inserted by page JavaScript after load	Browser, or find the underlying authorized API	Fetch does not execute the page’s scripts.
Click, type, scroll, inspect layout, or capture a visual	Browser	These tasks need a page, DOM, or rendering context.
Respond to browser permission prompts or use browser session behavior	Browser	Fetch does not provide browser UI or browser-managed interaction.

This is an architectural rule of thumb: Fetch is a request/response interface; a browser loads and renders a page. A browser generally adds startup and rendering work, while direct Fetch often has less overhead. Actual cost and reliability depend on the target, network, runtime, and workload, so measure the system you plan to operate.

When the answer is not obvious

If Fetch returns HTML but Chrome shows more information, inspect the response body and the page’s network activity in a browser’s developer tools. The visible data may arrive from a later API call, be inserted by JavaScript, require a session cookie, or depend on an interaction. If an authorized API provides the data, calling that endpoint directly can be simpler than controlling a browser. If the content depends on rendering or interaction, use browser automation such as Playwright or Puppeteer.

3. A complete Node.js Fetch example

Node.js includes a stable, browser-compatible global Fetch implementation. The following example calls an API, times out, checks HTTP status, validates the response type, and parses JSON defensively. Save it as fetch-json.mjs and run it with a supported Node.js version:

const url = 'https://api.github.com/repos/nodejs/node';
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 10_000);

try {
  const response = await fetch(url, {
    headers: {
      accept: 'application/vnd.github+json',
      'user-agent': 'example-fetch-agent'
    },
    signal: controller.signal
  });

  // Fetch resolves for HTTP errors such as 404; check the status explicitly.
  if (!response.ok) {
    const detail = await response.text();
    throw new Error(`HTTP ${response.status}: ${detail.slice(0, 500)}`);
  }

  const contentType = response.headers.get('content-type') || '';
  if (!contentType.includes('application/json')) {
    throw new Error(`Expected JSON, received ${contentType || 'no content-type'}`);
  }

  const data = await response.json();
  console.log({ full_name: data.full_name, stars: data.stargazers_count });
} catch (error) {
  if (error.name === 'AbortError') {
    console.error('Request timed out');
  } else {
    console.error('Request failed:', error.message);
  }
  process.exitCode = 1;
} finally {
  clearTimeout(timeout);
}

The example sends a public request and needs no token. For APIs that require authentication, add an explicit authorization header, for example authorization: `Bearer ${token}`, with the token read from a secret store or environment variable. Do not put secrets in source control, prompts, or request logs. Node’s Fetch API is documented as a browser-compatible implementation. Node.js Fetch documentation

Response-body handling

Choose a body reader that matches the response: response.json() for JSON, response.text() for text or HTML, response.blob() in browser code for binary data, or response.arrayBuffer() for bytes. A response body is a stream and normally can be consumed only once. If you need both the parsed result and the raw body, clone the response before consuming it or read the bytes once and parse those bytes yourself.

For large responses, avoid reading the whole body into memory without a limit. Where the runtime supports it, read the stream incrementally, count bytes, and stop when your application’s maximum is reached. Validate the content type and the data shape; servers can return an HTML error page where your code expected JSON.

4. cURL, Python, and browser JavaScript examples

cURL

cURL is useful for checking the HTTP exchange outside your agent. This command prints response headers and body, follows redirects, and fails on HTTP error statuses:

curl --fail-with-body --location --show-error \
  --header 'Accept: application/json' \
  'https://api.github.com/repos/nodejs/node'

To send a JSON request body, set the content type and pass JSON with --data. Keep credentials out of shell history when possible; use a protected environment or secret manager in automated environments.

Python

Python does not expose JavaScript’s Fetch API as a built-in interface. The equivalent common pattern uses the requests package. Install it with python -m pip install requests, then run:

import requests

url = "https://api.github.com/repos/nodejs/node"
try:
    response = requests.get(
        url,
        headers={"Accept": "application/vnd.github+json"},
        timeout=(3.05, 10),
    )
    response.raise_for_status()
    if "application/json" not in response.headers.get("content-type", ""):
        raise ValueError("Expected a JSON response")
    data = response.json()
    print({"full_name": data["full_name"], "stars": data["stargazers_count"]})
except requests.Timeout:
    print("The request timed out")
except requests.HTTPError as error:
    print("The server returned an HTTP error:", error)
except (requests.RequestException, ValueError) as error:
    print("Could not process the response:", error)

Browser JavaScript

In a web page, the same basic request looks familiar, but the browser applies origin security rules:

async function loadProfile() {
  const response = await fetch('/api/profile', {
    headers: { accept: 'application/json' },
    signal: AbortSignal.timeout(8000)
  });
  if (!response.ok) {
    throw new Error(`Profile request failed: HTTP ${response.status}`);
  }
  return response.json();
}

AbortSignal.timeout() is a convenient option in runtimes that support it. An AbortController is the more widely adaptable way to create a timeout or cancel a request when an agent task is no longer needed.

5. Fetch options agents should understand

The second argument to Fetch is a request options object. The useful options depend on the runtime and task; this table covers the ones agent authors commonly need.

Option	Use	Notes
method	Choose GET, POST, PUT, PATCH, DELETE, or another supported method.	GET is the default. Match the endpoint’s documented method.
headers	Set Accept, Content-Type, Authorization, or application-specific headers.	Do not forward sensitive headers to an untrusted URL.
body	Send request data, often with POST or PUT.	For JSON, use JSON.stringify(value) and set Content-Type: application/json.
signal	Cancel work or impose a timeout through an AbortSignal.	Aborting rejects the Fetch promise; handle that separately from HTTP status.
credentials	Control browser handling of cookies and related credentials.	Browser default is same-origin; cross-origin credentials require server agreement.
mode	Control browser cross-origin request mode.	Browser modes include cors, same-origin, and no-cors. no-cors gives an opaque response your script cannot inspect.
redirect	Control how redirects are handled where supported.	Check the final URL and do not assume a redirect preserves the response you expected.
cache	Influence browser HTTP cache behavior.	This is a browser cache control, not a general guarantee of upstream caching.

For a JSON POST request:

const response = await fetch('https://api.example.test/items', {
  method: 'POST',
  headers: {
    accept: 'application/json',
    'content-type': 'application/json',
    authorization: `Bearer ${process.env.API_TOKEN}`
  },
  body: JSON.stringify({ name: 'sample' }),
  signal: AbortSignal.timeout(10_000)
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const result = await response.json();

Replace the example host and token variable with values from the service’s documentation and your runtime’s secret configuration. Never treat arbitrary URLs supplied to an agent as safe destinations for credentials: validate the destination and scope each credential to the intended service.

6. CORS, credentials, and the Node.js difference

Browser Fetch is subject to the same-origin policy and CORS. A cross-origin request may require an OPTIONS preflight before the browser sends the actual request. The server must return appropriate CORS headers for the browser to expose the response to page JavaScript. Credentialed browser requests need additional server agreement, including an explicit allowed origin and Access-Control-Allow-Credentials; cookie SameSite rules still apply. MDN: Cross-Origin Resource Sharing · WHATWG CORS protocol

Setting mode: 'no-cors' does not solve a CORS problem when the agent needs to inspect the response. The result is opaque: JavaScript cannot read its headers or body. The practical fixes are to call an endpoint on your own origin, configure the server to permit the origin, or make the request from a trusted server-side agent when that is allowed by the service.

Node.js Fetch runs in a server runtime rather than a browser page, so it does not apply the browser’s CORS response-sharing step to its own request. This is not authorization to ignore the service’s access controls. The server can still require a key, deny the request, rate-limit it, or return different data according to identity and headers. Treat browser CORS behavior and server authentication as separate concerns.

Browser credentials can include cookies, TLS client certificates, and authorization headers. For an agent, explicit short-lived authorization tokens are usually easier to reason about than copying a browser’s cookies. Forward cookies only when the job intentionally represents an authenticated session, and avoid logging them.

7. Why Fetch can miss what you see in the browser

Fetch retrieves a response; it does not wait for a client-side application to construct the content you see on screen. A server may initially return a small HTML shell, while scripts later request data and update the DOM. Fetching that shell alone cannot reproduce those later steps. Other causes include session-dependent responses, content revealed after a click, or data assembled from multiple endpoints.

1. Request the page and inspect its status, content type, and response body.
2. If it is an application shell, use the browser’s network panel to identify the data request, if one exists and you are authorized to call it.
3. Call that API directly when it is stable, documented, and appropriate for your use.
4. Use browser automation when the page’s rendered state, JavaScript execution, interaction, or visual layout is part of the task.

For a task whose output is a screenshot, a browser renderer is necessary somewhere in the workflow: plain Fetch does not create pixels from HTML. ScreenshotNeo provides a website screenshot API and MCP server for that case. Its API accepts a URL and returns an image or PDF; details and supported options are in the ScreenshotNeo API documentation.

8. Or skip the browser setup

If the agent needs an image or PDF rather than response data, you can make a single request instead of managing a browser installation. This cURL example saves a WebP screenshot of a page:

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Equivalent Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Equivalent Node.js:

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: HTTP ${res.status}`);
await Bun.write('shot.webp', res);

Replace YOUR_API_KEY with your key. In Node.js, to save the response without Bun, use Node’s file system API:

import { createWriteStream } from 'node:fs';
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';

await pipeline(Readable.fromWeb(res.body), createWriteStream('shot.webp'));

Cookie banners, popups, and chat widgets are removed before the shot. Bot checks, blank pages, and failed loads are never billed. An MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Learn about ScreenshotNeo.

Sign up for 1,000 free screenshots a month—no card required.

9. Performance, reliability, and cost

Fetch usually has less setup and runtime work than launching a browser because it does not create a rendering context. That can make it a sensible choice for many small API calls, but it is not a universal speed or cost guarantee. Network latency, response size, server limits, retries, and runtime hosting all affect the result. Benchmark representative requests before choosing an architecture.

For reliability, set a timeout, cancel work that is no longer useful, check HTTP status, validate content type, and handle invalid or unexpected bodies. Retry only failures that are plausibly transient, such as selected network failures or rate limits when the server provides a retry policy. Avoid blindly retrying non-idempotent operations such as a payment or write request; a retry may perform the action twice. Use an idempotency key when the service supports one.

Set concurrency limits and respect service rate limits. If the agent processes untrusted URLs, restrict destinations and redirect behavior to reduce the chance of sending requests to internal services or leaking credentials. Fetch does not provide a universal response-size cap, so enforce one in your application where needed. For browser work, account for browser startup, page rendering, and resource loading; use it only when those capabilities are needed.

For screenshot workflows, the cost model is different from a generic HTTP request. ScreenshotNeo states that only clean shots are billed: bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing. Responses include X-Page-Verdict and X-Billed headers so the caller can distinguish outcomes. Its listed monthly plans are Free: 1,000; Starter: $5 for 3,000; Growth: $15 for 15,000; Pro: $39 for 60,000; Scale: $99 for 250,000; and Business: $249 for 1,000,000. Yearly billing gives two months free, and every feature is available on every plan. See the docs for request options and response details.

10. Troubleshooting Fetch in an agent

Symptom	Likely cause	Fix
Promise resolves but the task reports an error	Fetch resolves for HTTP statuses such as 404 and 504.	Check response.ok or response.status before parsing.
Browser reports a CORS error	The server did not permit the page origin, or a preflight failed.	Configure server CORS, use a same-origin endpoint, or make an authorized server-side request. Do not expect no-cors to expose the body.
response.json() throws	The response is empty, malformed, or actually HTML/plain text.	Inspect status and content type; read a bounded text sample for diagnosis.
Fetch times out or is aborted	The network or server was slow, or the task cancelled its signal.	Set a task-appropriate timeout, handle AbortError, and retry only when safe.
Fetch returns less content than the browser	Scripts, cookies, or interaction produce the visible state after the initial response.	Call an authorized data endpoint or use browser automation for rendered state.
API returns 401 or 403	Missing, invalid, expired, or insufficient credentials.	Check the service’s auth scheme and token scope; do not copy credentials to unrelated hosts.
API returns 429	Rate limit or quota reached.	Honor Retry-After when supplied, reduce concurrency, and use backoff within a bounded retry budget.
Binary download appears corrupted	Code parsed bytes as text or saved an error response as the file.	Check status and content type, then save the response as bytes.

11. Frequently asked questions

Is Fetch an API or a library?

It is a standardized web API exposed as fetch() in JavaScript runtimes, rather than a library you must install in modern browsers or supported Node.js versions.

Can an AI agent use Fetch instead of a browser?

Yes, when the task is a direct HTTP request and the response contains what the agent needs. Use a browser when the task depends on rendered or interactive page state.

Does Node.js Fetch obey CORS?

Node.js is not a browser page, so its Fetch does not use browser CORS response sharing. The target server’s authentication, authorization, and rate limits still apply.

Why does Fetch return a response for 404?

HTTP error status codes are still HTTP responses. The promise rejects for request-level failures, not merely because the server returned an error status; inspect ok or status.

Do I need Playwright or Puppeteer for a screenshot?

Use browser automation if you need custom browser interaction or control. For a URL-to-image or PDF capture without managing that browser setup, ScreenshotNeo offers a screenshot API and an MCP server for agent clients.