ScreenshotNeo
Sign in Get a free key

BlogEngineering

Building a Fetch API for Browser-Based Web Retrieval

Build a reliable browser fetch wrapper with HTTP errors, CORS, credentials, cancellation, streaming, caching, and diagnostics.

By the ScreenshotNeo team1 October 20268 min read

fetch() already is the browser’s HTTP API. The useful engineering task is to wrap it with consistent HTTP error handling, selectable parsing, cancellation, streaming, credentials, and cache policy.

A wrapper should accept a URL or Request, forward RequestInit, check response.ok, and return either a parsed result or a structured application error. Network failures and aborts reject the promise; HTTP 404 or 500 responses normally do not.

1. A production-ready fetch wrapper

export class HttpError extends Error {
  constructor(message, details) {
    super(message);
    this.name = 'HttpError';
    Object.assign(this, details);
  }
}

async function readErrorBody(response, limit = 4096) {
  try {
    const text = await response.text();
    return text.slice(0, limit);
  } catch {
    return '';
  }
}

export async function request(resource, options = {}) {
  const {
    parse = 'auto',
    timeout,
    signal: callerSignal,
    ...fetchOptions
  } = options;

  const controller = new AbortController();
  let timer;
  const onAbort = () => controller.abort(callerSignal.reason);

  if (callerSignal) {
    if (callerSignal.aborted) controller.abort(callerSignal.reason);
    else callerSignal.addEventListener('abort', onAbort, { once: true });
  }
  if (timeout != null) timer = setTimeout(() => controller.abort(), timeout);

  try {
    const response = await fetch(resource, {
      ...fetchOptions,
      signal: controller.signal
    });

    if (!response.ok) {
      throw new HttpError(`HTTP ${response.status}`, {
        status: response.status,
        statusText: response.statusText,
        headers: response.headers,
        body: await readErrorBody(response)
      });
    }

    if (parse === 'response') return response;
    if (parse === 'stream') return response.body;
    if (parse === 'blob') return response.blob();
    if (parse === 'text') return response.text();
    if (parse === 'json') return response.json();

    const type = response.headers.get('content-type') || '';
    if (type.includes('application/json')) return response.json();
    if (type.startsWith('text/')) return response.text();
    return response.blob();
  } catch (error) {
    if (error.name === 'AbortError') {
      throw new Error('Request was cancelled or timed out', { cause: error });
    }
    throw error;
  } finally {
    if (timer) clearTimeout(timer);
    callerSignal?.removeEventListener('abort', onAbort);
  }
}

Use it with an explicit parser when the response type is known:

const profile = await request('/api/profile', {
  parse: 'json',
  headers: { Accept: 'application/json' },
  cache: 'no-store'
});

const text = await request('/feed.txt', { parse: 'text' });
const file = await request('/archive.zip', { parse: 'blob' });

The wrapper preserves the caller’s RequestInit options, including method, headers, body, mode, credentials, cache, redirect, referrerPolicy, and integrity. Browser and worker contexts provide the global fetch function. See MDN’s Fetch API reference.

2. Why fetch does not throw for a 404

A fulfilled fetch promise means that a Response was received. It does not mean the status is successful. Always check response.ok or inspect response.status before parsing a success payload.

const response = await fetch('/missing');
if (!response.ok) {
  const detail = await response.text();
  throw new Error(`${response.status} ${response.statusText}: ${detail.slice(0, 500)}`);
}

ok is true for status codes from 200 through 299. Redirect handling, authentication challenges, rate limits, and server errors still require application decisions. Do not log complete error bodies when they may contain tokens or personal data.

3. Request options that matter

Option Use it for Details
method HTTP verb Use GET for retrieval and set a body only for methods that accept one.
headers Negotiation and auth Setting some headers can make a cross-origin request non-simple and trigger preflight.
body Payloads Use JSON.stringify and Content-Type: application/json for JSON.
mode Cross-origin policy cors is the normal cross-origin mode. no-cors returns an opaque response that script cannot read.
credentials Cookies and client certificates same-origin is the default; use include only with deliberate server and CSRF protection.
cache HTTP cache behavior Choose default, no-store, reload, no-cache, force-cache, or only-if-cached deliberately.
signal Cancellation Pass an AbortSignal from the caller or component lifecycle.
redirect Redirect policy Use manual or error when redirects must be visible or forbidden.
referrerPolicy Referrer privacy Limit the URL information sent in the Referer header.

4. CORS: what the browser permits

CORS is enforced by the browser and cannot be bypassed by a wrapper. For a simple cross-origin request, the browser may send the request but exposes the response to script only when the server returns the correct Access-Control-Allow-Origin. A request with a non-simple method or header usually receives a preflight request first.

const response = await fetch('https://api.example.com/items', {
  mode: 'cors',
  headers: { Accept: 'application/json' }
});

The API must answer with an origin that matches your page, plus permitted methods and headers when preflighted. mode: 'no-cors' is not a solution for application data: the result is opaque, with status 0 and no readable body or headers. Read MDN’s CORS guide.

Credentialed requests

await fetch('https://api.example.com/account', {
  credentials: 'include',
  headers: { Accept: 'application/json' }
});

Cross-origin credentials require the server to return a specific Access-Control-Allow-Origin value and Access-Control-Allow-Credentials: true. The wildcard origin cannot be used for credentialed responses. Cookies are also constrained by their SameSite attributes. Treat cross-origin credentials as a CSRF decision, not merely a networking setting.

5. Cancellation and timeouts

Use AbortController when a user navigates away, a component unmounts, or a request exceeds its deadline.

const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 8000);

try {
  const response = await fetch('/large-report', { signal: controller.signal });
  if (!response.ok) throw new Error(`HTTP ${response.status}`);
  const report = await response.json();
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('Cancelled');
  } else {
    throw error;
  }
} finally {
  clearTimeout(timeoutId);
}

If cancellation occurs after headers arrive, a later body read can still raise AbortError. Do not assume that receiving headers means the body is complete.

6. Streaming large responses

Convenience readers such as json() and text() buffer the complete body. For large downloads or progressive text, read response.body incrementally.

const response = await fetch('/events.log');
if (!response.ok) throw new Error(`HTTP ${response.status}`);
if (!response.body) throw new Error('ReadableStream is unavailable');

const reader = response.body.getReader();
const decoder = new TextDecoder();
let received = '';

try {
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    received += decoder.decode(value, { stream: true });
    renderChunk(received);
  }
  received += decoder.decode();
} finally {
  reader.releaseLock();
}

Streaming lowers peak memory and can reduce time to first usable data, but your parser must handle records split across chunks. A stream is single-use; clone the response before consuming it twice.

7. JSON requests and responses

const response = await request('/api/items', {
  method: 'POST',
  parse: 'json',
  headers: {
    Accept: 'application/json',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ name: 'sample' })
});

Check the status before parsing. A successful response may still have an empty body or an unexpected content type, so production code should handle parse failures separately from HTTP failures.

8. Caching and service workers

Expose cache policy to callers instead of silently choosing one. no-store avoids storing a response, no-cache revalidates, force-cache prefers a cached response, and only-if-cached is useful only with the appropriate same-origin constraints.

const fresh = await request('/api/status', {
  parse: 'json',
  cache: 'no-store'
});

const cached = await request('/api/catalog', {
  parse: 'json',
  cache: 'force-cache'
});

A service worker can add application-level caching, but define invalidation and freshness rules explicitly. The browser HTTP cache and service-worker cache are separate layers.

9. Retries, reliability, and performance

  • Retry only transient failures such as selected 408, 429, and 5xx responses, and honor Retry-After when present.
  • Retry idempotent operations by default. A repeated POST can create duplicate state unless the API supports idempotency keys.
  • Use exponential backoff with jitter and a maximum attempt count.
  • Set a timeout and cancel obsolete requests to avoid wasted bandwidth and stale UI updates.
  • Send narrow Accept headers and parse only the representation you need.
  • Stream large bodies instead of buffering them, and apply backpressure when processing is slower than the network.
  • Record status, duration, request ID, and bounded error details without recording authorization headers or sensitive payloads.
async function getWithRetry(url, options = {}, attempts = 3) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await request(url, options);
    } catch (error) {
      const retryable = [408, 429, 500, 502, 503, 504].includes(error.status);
      if (!retryable || attempt >= attempts - 1) throw error;
      const delay = Math.min(2000, 200 * 2 ** attempt) + Math.random() * 100;
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
}

10. Complete command-line and server examples

cURL

curl -i --fail-with-body \
  -H 'Accept: application/json' \
  'https://api.example.com/items'

Python

import requests

response = requests.get(
    'https://api.example.com/items',
    headers={'Accept': 'application/json'},
    timeout=30,
)
response.raise_for_status()
items = response.json()

Node.js

const response = await fetch('https://api.example.com/items', {
  headers: { Accept: 'application/json' },
  signal: AbortSignal.timeout(30_000)
});

if (!response.ok) {
  throw new Error(`HTTP ${response.status}`);
}
const items = await response.json();

11. Troubleshooting checklist

Symptom Cause Fix
404 does not enter catch HTTP errors fulfill the promise. Check response.ok or status.
CORS error in DevTools Server CORS headers do not authorize the page. Configure the server origin, methods, and headers; do not use no-cors for data.
Preflight fails OPTIONS response is missing or rejects a requested method/header. Handle OPTIONS and return matching Access-Control-Allow-* headers.
Cookies are missing Credentials default to same-origin, or cookie SameSite blocks them. Use credentials: 'include' only when server CORS and cookie policy allow it.
Status 0 and unreadable body Opaque no-cors response. Make the server support CORS or proxy the request from your own backend.
Request hangs No deadline was supplied. Pass an AbortSignal and implement a timeout.
JSON parse error Empty body, HTML error page, or wrong content type. Check status and Content-Type; retain a bounded diagnostic body.
Memory spikes text() or json() buffers a large response. Consume response.body as a stream.
Stale data appears Browser or service-worker cache serves an old response. Choose cache explicitly and define invalidation.

12. Or skip the browser setup

For website screenshots, you can call ScreenshotNeo instead of maintaining browser automation:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

See the ScreenshotNeo API docs for the options. Cookie banners, popups, and chat widgets are removed before the shot. Bot checks, blank pages, and failed loads are never billed. An MCP server lets AI agents take screenshots, and 1,000 screenshots a month are free with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

13. FAQ

Is fetch available in Web Workers?

Yes. Fetch is available in Window and Worker contexts, so the same wrapper can run outside the main UI thread.

Should every wrapper return JSON?

No. Return or expose json(), text(), blob(), and body so callers can select the representation they need.

Can client-side JavaScript bypass CORS?

No. CORS is a browser security policy enforced using the server’s response headers. Use a server-side request or configure the API.

When should a request be cancelled?

Cancel when the user leaves the screen, a component is disposed, a newer request supersedes it, or a timeout expires.

Do retries make every request reliable?

No. Retries help with transient failures only. Bound attempts, use backoff, and avoid repeating non-idempotent operations without an idempotency strategy.

Keep reading