ScreenshotNeo

BlogHow-to

How to Capture Screenshots for a List of URLs with ScreenshotOne

Capture many URLs with ScreenshotOne’s bulk API. Learn how to batch requests, handle lazy execution, store results, and pace reliable workflows.

By the ScreenshotNeo team4 October 202610 min read

Use ScreenshotOne’s POST https://api.screenshotone.com/bulk endpoint to capture a list of URLs. Put shared capture settings in options and one object per URL in requests. Bulk results are lazy by default: each capture runs when its returned screenshot URL is downloaded. Set execute: true when you want ScreenshotOne to run the captures before returning the response.

This guide covers the request shape, cURL, Python, and Node.js examples, lazy versus eager execution, storage, and reliable batching. For account-specific settings and options, see ScreenshotOne’s bulk screenshots documentation.

1. Get an access key and choose your workflow

  1. Sign up or sign in to ScreenshotOne, open the access page, and create or copy an access key. Keys are scoped to an organization; see the API key guide.
  2. Use HTTPS for every API call. The key and any supplied authorization headers or cookies are sensitive data; HTTP does not encrypt them in transit.
  3. Decide whether each capture should run when its result URL is fetched (lazy) or before the bulk response is returned (execute: true).
  4. Decide whether returned screenshot URLs are sufficient or whether you need to store output in an S3-compatible destination.

ScreenshotOne supports a key in a query string, POST JSON body, or X-Access-Key header. The examples below use the header to keep the key out of the URL. Avoid putting credentials in URLs that may be logged or shared.

2. Build a bulk request

A bulk request is a JSON object with an optional shared options object and a requests array. Put common settings in options; put a URL and any per-item overrides on each request object. The documented bulk examples also support HTML or Markdown input. Check the current options reference for the settings you need.

{
  "options": {
    "format": "png",
    "full_page": true
  },
  "requests": [
    { "url": "https://example.com/" },
    { "url": "https://example.org/", "options": { "format": "jpeg" } },
    { "url": "https://example.net/" }
  ]
}

In this example, the shared format and full-page setting apply to every request unless an item overrides a setting. The second item requests JPEG output. Use actual sites you are authorized to capture; a URL that redirects, requires authentication, blocks automated access, or fails to load may not produce the expected image.

cURL

Save the request body as bulk.json, set your key in the shell environment, and run:

export SCREENSHOTONE_ACCESS_KEY='YOUR_ACCESS_KEY'
curl --fail-with-body --silent --show-error \
  -X POST 'https://api.screenshotone.com/bulk' \
  -H "X-Access-Key: $SCREENSHOTONE_ACCESS_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary @bulk.json

The response contains screenshot URLs for the requested items. Save the response and fetch those URLs to obtain the image bytes when using the default lazy behavior. Use your shell’s secret-management facilities rather than committing a real key into source control.

Python

Install the HTTP client with python -m pip install requests, then save and run this script. It submits three URLs, reads the returned JSON, and downloads each result URL to a numbered file.

import os
from pathlib import Path
import requests

api_key = os.environ["SCREENSHOTONE_ACCESS_KEY"]
payload = {
    "options": {"format": "png", "full_page": True},
    "requests": [
        {"url": "https://example.com/"},
        {"url": "https://example.org/"},
        {"url": "https://example.net/"},
    ],
}

response = requests.post(
    "https://api.screenshotone.com/bulk",
    headers={"X-Access-Key": api_key},
    json=payload,
    timeout=120,
)
response.raise_for_status()
data = response.json()

# The bulk response provides screenshot URLs. Download each URL to execute
# its lazy capture and retrieve its image bytes.
# Inspect the returned response shape for your account/API version.
items = data.get("data", data.get("requests", data))
if not isinstance(items, list):
    raise ValueError(f"Unexpected bulk response shape: {data!r}")

Path("screenshots").mkdir(exist_ok=True)
for index, item in enumerate(items, start=1):
    image_url = item.get("url") or item.get("screenshot_url")
    if not image_url:
        raise ValueError(f"No screenshot URL in item {index}: {item!r}")
    image = requests.get(image_url, timeout=120)
    image.raise_for_status()
    Path("screenshots", f"{index:03}.png").write_bytes(image.content)
    print(f"Saved screenshots/{index:03}.png")

Response field names can depend on the API response shape. Inspect the JSON returned by your account and adjust the extraction of screenshot URLs accordingly; do not silently treat an unexpected response as a successful batch.

Node.js

This script uses the built-in fetch available in current Node.js releases. Set SCREENSHOTONE_ACCESS_KEY in the environment and run it as an ES module. It downloads the lazy screenshot URLs returned by the bulk call.

import { mkdir, writeFile } from 'node:fs/promises';

const apiKey = process.env.SCREENSHOTONE_ACCESS_KEY;
if (!apiKey) throw new Error('Set SCREENSHOTONE_ACCESS_KEY first');

const payload = {
  options: { format: 'png', full_page: true },
  requests: [
    { url: 'https://example.com/' },
    { url: 'https://example.org/' },
    { url: 'https://example.net/' },
  ],
};

const response = await fetch('https://api.screenshotone.com/bulk', {
  method: 'POST',
  headers: {
    'X-Access-Key': apiKey,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(payload),
  signal: AbortSignal.timeout(120_000),
});
if (!response.ok) {
  throw new Error(`Bulk request failed (${response.status}): ${await response.text()}`);
}
const data = await response.json();
const items = data.data ?? data.requests ?? data;
if (!Array.isArray(items)) throw new Error(`Unexpected response: ${JSON.stringify(data)}`);

await mkdir('screenshots', { recursive: true });
for (let index = 0; index < items.length; index++) {
  const item = items[index];
  const imageUrl = item.url ?? item.screenshot_url;
  if (!imageUrl) throw new Error(`No screenshot URL in item ${index + 1}`);
  const image = await fetch(imageUrl, { signal: AbortSignal.timeout(120_000) });
  if (!image.ok) throw new Error(`Image ${index + 1} failed (${image.status})`);
  const bytes = Buffer.from(await image.arrayBuffer());
  const filename = `screenshots/${String(index + 1).padStart(3, '0')}.png`;
  await writeFile(filename, bytes);
  console.log(`Saved ${filename}`);
}

As with the Python example, check the returned JSON shape against the response you receive and map the documented screenshot URL field. A successful POST alone does not mean the lazy image URLs have been fetched.

3. Choose lazy or eager execution

Mode When capture runs Use it when Watch for
Lazy (default) When a returned screenshot URL is downloaded You want to defer work until results are consumed Fetch every result URL and handle failures during that fetch; submitting the bulk request does not itself retrieve the images.
execute: true ScreenshotOne executes each request before returning You want completed capture work initiated as part of the bulk call Allow enough time for all jobs. A large or slow batch can take longer than a typical single HTTP request.

To request eager execution, add "execute": true to the bulk request in the location documented for that endpoint. For example:

{
  "execute": true,
  "options": { "format": "png" },
  "requests": [
    { "url": "https://example.com/" },
    { "url": "https://example.org/" }
  ]
}

Confirm the exact response behavior in the bulk endpoint documentation. Whichever mode you use, handle per-item failures: one inaccessible or unusually slow page should not cause your application to mark every URL complete.

4. Handle long lists with batches and a queue

For a short list, one bulk request can be convenient. For a long list, split the work into manageable batches and track each URL through a queue. ScreenshotOne’s multi-URL guide recommends batching, retrying failures, and checking the usage endpoint’s live concurrency values before starting more work.

  1. Keep a record for each input URL and its state: pending, submitted, completed, or failed. Retain the original URL and any returned screenshot URL or storage path.
  2. Before starting more captures, read concurrency.remaining and concurrency.reset from the usage endpoint. These describe how many screenshot requests can be started in the current one-minute bucket; they do not count renderers actively working.
  3. Start no more requests than the live remaining value allows, then wait for the reported reset when the bucket is exhausted. Do not assume a fixed limit applies to every account.
  4. Retry transient failures with bounded exponential backoff and a maximum attempt count. Keep permanent failures visible for review instead of retrying forever.
  5. Make retries safe in your own job tracking: record which URLs have produced usable output so that a worker restart does not lose completed work.

An in-memory queue can be enough for a single process where losing queued work on restart is acceptable. For work that must survive restarts or run across multiple workers, the guide suggests a durable queue such as Redis/BullMQ or SQS. Proxies may help when they are needed for the target workload; they are not a substitute for respecting the target site’s access controls.

Optimization for repeated URLs

If you need screenshots of the same URL with different parameters, the bulk API documents an optimization mode intended for that pattern. It is not guaranteed: a site may reload anyway, and some option changes require a reload. Test it with the actual pages and option combinations before relying on it for throughput or cost planning.

5. Store output when you need durable files

By default, the basic bulk workflow returns screenshot URLs. If you need outputs placed in object storage, the bulk documentation shows a stored-output configuration using store: true, response_type: "empty", and an S3-compatible destination. Set a distinct storage_path for each output; the options reference says this path is required when storing.

The storage endpoint can be left empty for Amazon S3 or configured for a compatible service such as Cloudflare R2. Use the provider’s own endpoint and credentials configuration as described in ScreenshotOne’s documentation. Do not assume the storage example is required for ordinary captures: use returned screenshot URLs if that is all your application needs.

6. Request size, security, and operating cost

  • Request size: ScreenshotOne documents a maximum POST body size of 100 MiB. For larger HTML or Markdown input, host the content and pass its URL rather than embedding it in the request. See Getting Started.
  • Credential safety: Send requests over HTTPS. Keep access keys, cookies, and authorization headers out of source control, logs, and shared URLs where possible. Rotate a key if it has been exposed.
  • Latency: Lazy mode defers render work until each screenshot URL is downloaded. Eager execution adds capture work to the bulk call, so allow a timeout appropriate to the batch and site response times.
  • Reliability: Track outcomes per URL, use bounded retries, and consult the live usage bucket. Durable queues help when jobs must survive process restarts or be shared by workers.
  • Cost: The research documentation does not establish a price or universal concurrency allowance. Check your account’s current plan and usage endpoint before estimating a large run. Avoid assuming optimization will reduce work for every site.

7. Troubleshooting

Symptom Likely cause What to do
401 or access denied The key is missing, invalid, or sent in an unsupported place. Verify the key in the organization’s access page and send it using the documented header, body, or query option. Keep the request on HTTPS.
Bulk POST succeeds but no image file appears Bulk screenshots are lazy by default, and the returned screenshot URLs have not been fetched. Download each returned URL, or request eager execution with execute: true and allow enough time for work to complete.
One or more URLs fail A target may be unavailable, slow, protected, redirect unexpectedly, or require credentials. Inspect that item’s error, verify the URL can be reached with the required access, and retry only transient failures with backoff.
Request times out The batch has many captures or one or more pages take a long time to render. Use smaller batches, allow a longer client timeout, or use lazy execution so capture work happens as results are consumed.
Rate or concurrency limit reached More starts were attempted than the account’s current one-minute bucket permits. Read concurrency.remaining and concurrency.reset from usage; pause until reset rather than hard-coding a universal limit.
Request body rejected The POST body exceeds the documented 100 MiB maximum. Reduce the payload. Host large HTML or Markdown input and pass its URL instead.
Stored output is missing or rejected Storage is enabled without the required path or with an incorrect S3-compatible destination configuration. Set a unique storage_path for each output and verify the endpoint and storage settings from the documentation.
Optimization does not reduce work The site reloads between requests or the changed options require a reload. Test optimization on representative URLs and treat it as workload-dependent.

8. Or skip the browser setup

If you need a screenshot API without building and maintaining the capture workflow, ScreenshotNeo is a website screenshot API and MCP server for developers. One GET request takes a URL and returns a PNG, JPEG, WebP, or PDF. See the ScreenshotNeo API documentation.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo accepts cookie and consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots. Every feature is on every plan.

Sign up free for 1,000 screenshots a month, with no card required.

9. Frequently asked questions

Can each URL use different capture settings?

Yes. Put common settings in shared options and request-specific overrides on the relevant item. Consult the bulk documentation for the supported request shape.

Does the bulk endpoint accept HTML or Markdown?

The documented examples include HTML and Markdown input as well as URL captures. For large content, host it and pass its URL because POST bodies are limited to 100 MiB.

Does the usage concurrency value tell me how many browsers are rendering now?

No. The guide describes it as the number of request starts available in the current one-minute bucket, not the count of active renderers.

Do I need object storage?

No. Screenshot URLs are the default output path. Configure S3-compatible storage when you need files placed in a persistent destination.

Sources