How to Take Bulk Screenshots of URLs Using AWS Lambda

For bulk URL screenshots, split the work into one task per URL: accept and validate a batch, create a job record, enqueue URL tasks, let bounded-concurrency Lambda workers capture pages with headless Chromium, write images to S3, and save a status for each URL. Return a job ID to the caller and let it poll for results. This makes retries, partial failures, and progress manageable.

A single Lambda invocation can process a small batch sequentially, but it must finish within the function timeout. Standard Lambda functions have a maximum timeout of 900 seconds; synchronous and asynchronous invocation payloads are limited to 6 MB and 1 MB respectively. Do not put an unbounded list of URLs in one event. Lambda timeout configuration · Invoke API payload limits

Choose the batch architecture

Use a queue or fan-out workflow for production. A front-door function validates and records the job, then emits one message per URL. Workers process messages independently. Store screenshots in S3 and job state in a database or another durable store. AWS’s Puppeteer screenshot example demonstrates the fan-out and S3 pattern; its older runtime and package versions should not be copied without current validation.

Pattern	Use it when	Tradeoff
One invocation, sequential URLs	Prototype or known-small batch	Simple, but one timeout can lose the whole invocation’s progress unless results are persisted incrementally.
Queue with one URL per message	Most production batches	Independent retries and concurrency controls; requires queue, job state, and result aggregation.
Fan-out workflow	Orchestration and explicit workflow state matter	More orchestration configuration, but clearer control of branches and completion.

For HTTP submission, Lambda Function URLs suit simple endpoints and prototypes. Choose API Gateway when you need richer authentication, traffic controls, throttling, custom domains, or request and response transformation. AWS comparison of Function URLs and API Gateway

Define capture behavior and URL policy

Before building workers, make capture semantics explicit. Decide whether each result is a viewport or full-page image, the viewport dimensions and device scale factor, the navigation and selector wait conditions, whether redirects are allowed, and how authenticated targets are handled. These choices affect runtime, memory, output size, and security.

• Validate that each input parses as an absolute HTTP or HTTPS URL; reject other schemes.
• Set a maximum URL count and maximum URL length per submitted job.
• If users can submit arbitrary targets, defend against server-side request forgery: block loopback, private, link-local, and cloud metadata addresses; re-check after DNS resolution and redirects; and restrict worker egress where feasible.
• Do not accept arbitrary browser flags, JavaScript, headers, or cookies from untrusted callers without an explicit policy.
• Use an idempotency key for each job/URL pair and a deterministic object key, for example jobs/{jobId}/{urlHash}.png.

These URL safeguards are implementation guidance: a screenshot worker fetches caller-selected network destinations, so its network access needs deliberate limits.

Build a Puppeteer Lambda worker

The following worker handles one URL per invocation, captures a viewport, uploads the PNG to S3, and returns a per-URL result. Use a pinned Puppeteer and Chromium combination validated in the same Lambda container environment you deploy. The example uses a container image so the browser and native dependencies can be packaged together.

// package.json
{
  "name": "screenshot-worker",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.800.0",
    "puppeteer": "^24.0.0"
  }
}

// index.mjs
import puppeteer from 'puppeteer';
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { createHash } from 'node:crypto';

const s3 = new S3Client({});
const bucket = process.env.BUCKET;
const maxNavigationMs = Number(process.env.NAVIGATION_TIMEOUT_MS || 45000);

function checkedUrl(value) {
  const u = new URL(value);
  if (!['http:', 'https:'].includes(u.protocol)) throw new Error('URL_SCHEME');
  if (u.username || u.password) throw new Error('URL_CREDENTIALS_NOT_ALLOWED');
  return u;
}

export const handler = async (event) => {
  const { jobId, url, viewport = { width: 1365, height: 900 } } = event;
  if (!jobId || typeof url !== 'string') throw new Error('INVALID_TASK');
  const target = checkedUrl(url);
  const hash = createHash('sha256').update(target.href).digest('hex').slice(0, 24);
  let browser;
  try {
    browser = await puppeteer.launch({
      headless: true,
      args: ['--no-sandbox', '--disable-setuid-sandbox']
    });
    const page = await browser.newPage();
    await page.setViewport({ width: viewport.width, height: viewport.height,
      deviceScaleFactor: viewport.deviceScaleFactor || 1 });
    await page.goto(target.href, { waitUntil: 'networkidle2', timeout: maxNavigationMs });
    const image = await page.screenshot({ type: 'png' });
    const key = `jobs/${jobId}/${hash}.png`;
    await s3.send(new PutObjectCommand({
      Bucket: bucket, Key: key, Body: image, ContentType: 'image/png'
    }));
    return { jobId, url: target.href, status: 'succeeded', key,
      capturedAt: new Date().toISOString(), viewport };
  } catch (error) {
    return { jobId, url, status: 'failed',
      errorCategory: error.name === 'TimeoutError' ? 'navigation_timeout' : 'capture_or_upload_error',
      message: String(error.message || error).slice(0, 300) };
  } finally {
    if (browser) await browser.close();
  }
};

The example gives each URL a stable key, so a retry overwrites the same logical output instead of creating a new object each time. In a production deployment, write the returned status to your job store with a conditional update or idempotency record. Avoid logging secrets embedded in URLs; the example rejects URL credentials, but query strings can also contain tokens.

networkidle2 is a practical default, not a universal readiness signal. Analytics, long polling, and streaming pages may never become idle. For those sites, use domcontentloaded or load followed by a specific selector wait or bounded delay. Set a finite navigation timeout in all cases.

Package Chromium and configure Lambda

Example container setup:

# Dockerfile
FROM public.ecr.aws/lambda/nodejs:22
WORKDIR ${LAMBDA_TASK_ROOT}
COPY package.json package-lock.json ./
RUN npm ci
COPY index.mjs ./
CMD ["index.handler"]

Build and deploy for the Lambda architecture you selected, then invoke it with a single URL task. Validate browser launch and rendering in the deployed runtime; a package that works on a developer laptop may rely on a different shared library or architecture.

docker build --platform linux/amd64 -t screenshot-worker .
# Push the image to an Amazon ECR repository, then create or update the Lambda
# function from that image. Configure BUCKET and grant the execution role
# s3:PutObject on the intended bucket/prefix.

Lambda supports ZIP packages and container images. The actual browser binary, native libraries, runtime, architecture, and package versions must be compatible with the chosen deployment. Set memory and timeout based on measured representative pages, not the defaults. Lambda memory ranges from 128 MB to 10,240 MB, and configurable /tmp storage ranges from 512 MB to 10,240 MB. CPU allocation rises with memory. Lambda memory configuration · Ephemeral storage configuration

Store only temporary browser files in /tmp; it is not durable job storage. Put screenshot objects in S3 and job records in a separate persistent store. Keep the worker role narrow: allow writes only to the expected bucket prefix and grant no unrelated permissions.

Submit a batch and track per-URL results

A submit endpoint should validate the batch, create a job record such as {jobId, total, succeeded, failed, pending}, enqueue each URL, and immediately return 202 Accepted with the job ID. A status endpoint can return aggregate progress plus each URL’s outcome and object key. Avoid returning a presigned S3 link unless the caller is authorized to see that image.

For very small batches only, a sequential Lambda handler can use the same worker logic inline. Persist each result as it completes. Do not keep a large batch in an HTTP request while waiting for all browser renders.

For a synchronous Lambda invocation, the payload limit is 6 MB; asynchronous invocation supports 1 MB. Store large input lists in S3 and pass a pointer or enqueue individual items. Invoke API limits and invocation behavior

Queue processing rules

• Make queue visibility timeout longer than the worker’s maximum expected processing interval, with enough room for retries.
• Set bounded concurrency so a burst does not overwhelm account limits or destination sites.
• Record a result for every URL, including failures. A batch is complete when every item has a terminal status, not only when one worker succeeds.
• Configure a dead-letter queue or equivalent failure destination for tasks that exhaust retries.
• Use exponential backoff with jitter for transient errors; do not repeatedly retry invalid URLs or permanent authorization failures.
• Assume an asynchronous event can be delivered more than once. AWS queues async Lambda events before processing, and retries can occur. Make writes idempotent and guard status transitions. Asynchronous Lambda invocation

Call the HTTP submission endpoint

These examples assume you have deployed a submit endpoint that accepts {"urls":[...]} and returns a job ID. Replace the example endpoint with your Function URL or API Gateway URL. The endpoint should enqueue work rather than capture the whole batch in the request handler.

cURL

curl -X POST 'https://YOUR_ENDPOINT.example/jobs' \
  -H 'content-type: application/json' \
  -d '{"urls":["https://example.com","https://www.wikipedia.org"]}'

Python

import requests

endpoint = "https://YOUR_ENDPOINT.example/jobs"
response = requests.post(endpoint, json={
    "urls": ["https://example.com", "https://www.wikipedia.org"]
}, timeout=30)
response.raise_for_status()
print(response.json())  # Expect a job ID and status URL

Node.js

const response = await fetch('https://YOUR_ENDPOINT.example/jobs', {
  method: 'POST',
  headers: { 'content-type': 'application/json' },
  body: JSON.stringify({ urls: ['https://example.com', 'https://www.wikipedia.org'] })
});
if (!response.ok) throw new Error(`Submit failed: ${response.status}`);
console.log(await response.json());

Or skip the browser setup

If the goal is screenshots rather than operating Chromium workers, ScreenshotNeo is a website screenshot API and MCP server. Its one-call API captures a URL and returns an image or PDF; see the API documentation.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before capture. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers identify the page verdict and billing outcome. Its MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. For bulk workloads, submit URLs with bounded client-side concurrency and handle each response independently.

Create a free account for 1,000 screenshots a month, with no card required.

Control concurrency, runtime, and cost

Browser startup, page load, image encoding, and S3 upload all consume time and memory. Start with one URL per worker and tune concurrency using real target pages. A slow page ties up a worker; a large full-page image can consume substantially more memory than a viewport capture. Avoid a fixed throughput estimate: page behavior and asset weight vary.

• Timeout: leave margin under the 900-second standard maximum. A worker should have explicit navigation and selector timeouts that are shorter than its Lambda timeout.
• Memory and CPU: benchmark a representative mix of simple, heavy, and slow pages. Inspect duration and maximum memory, then adjust. More memory also provides more CPU.
• Temporary storage: increase /tmp only when the browser or temporary artifacts need it. Clean up files if the environment is reused.
• Concurrency: set a reserved or event-source concurrency limit where appropriate; apply queue backpressure and respect destination-site capacity.
• Cost: account for Lambda duration and memory, queue/orchestration, S3 storage and requests, logs, and any network egress. Estimate from measured invocations and your region’s current AWS pricing; there is no universal cost per screenshot.
• Reuse: initialize reusable AWS SDK clients outside the handler. Always close the browser in a finally block.

Measure cold and warm starts separately where useful. Keep screenshots at the smallest dimensions and format that meet the use case, and set S3 lifecycle retention when old images do not need to remain indefinitely.

Reliability and observability checklist

• [ ] Every URL has a stable task ID and idempotency key.
• [ ] Each URL records normalized URL, status, object key, capture time, viewport, and a short error category.
• [ ] Retries use backoff and stop for permanent errors.
• [ ] A dead-letter or failure destination is monitored.
• [ ] Job status can distinguish queued, running, partial, complete, and failed.
• [ ] Logs include job/task IDs, duration, and error class, but omit credentials and sensitive query values.
• [ ] Alarms or dashboards cover errors, duration, throttles, timeouts, memory pressure, and queue age.
• [ ] Representative target pages are rechecked after browser or runtime updates.

Troubleshooting

Symptom	Likely cause	Fix
Task timed out	Navigation hangs, network idle never occurs, or the batch is too large.	Use one URL per worker, set explicit page timeouts, choose a more suitable wait condition, and increase Lambda timeout only within the 900-second cap.
Chromium fails to launch	Missing native libraries, incompatible architecture, or browser/runtime mismatch.	Pin a tested deployment combination and validate it in the Lambda container environment; inspect startup logs.
Out of memory or browser crash	Large pages, full-page captures, many open pages, or low memory.	Keep one page per worker, close browser resources, raise memory, and test full-page captures separately.
Blank or incomplete screenshot	Capture happened before client rendering or lazy content finished.	Wait for a meaningful selector or bounded delay; use a full-page capture strategy that scrolls to trigger lazy images when required.
Some URLs are missing after retry	Only batch-level status was persisted, or repeated delivery created ambiguous outcomes.	Track each URL separately, use idempotency keys and deterministic S3 keys, and inspect the failure queue.
S3 AccessDenied	Worker role lacks permission for the exact bucket or prefix, or bucket policy denies the write.	Grant only the required s3:PutObject path and check the bucket policy, encryption settings, and region.
Throttles or queue backlog	Worker concurrency exceeds account capacity or target-site tolerance.	Reduce concurrency, add backpressure, monitor queue age, and request quota changes only after measuring demand.
Async invocation appears successful but no image exists	An accepted async event only means it was queued; processing can later fail or retry.	Track completion in durable job state and configure a failure destination. Do not equate enqueue success with capture success.
Submit request rejected for size	The URL list exceeded the invocation payload limit or front-door limit.	Store input in S3 and pass a pointer, or send smaller batches and fan out individual URL tasks.

Frequently asked questions

Can Puppeteer run on AWS Lambda?

Yes. Package a compatible Chromium build and its required dependencies with the function, then validate that exact image, runtime, and architecture in Lambda. AWS’s published screenshot example uses Puppeteer and a container image; its older versions are a pattern, not a current version recommendation.

Where should Lambda screenshots be stored?

Use S3 for durable image objects. Use /tmp only for temporary browser files, and store job state separately so it survives worker retries and environment replacement.

Should one URL be one Lambda invocation?

For production bulk work, generally yes: one task per URL gives clearer retry behavior, bounded concurrency, and partial success. A small, bounded batch may be processed sequentially when its worst-case runtime is known and progress is persisted.

How can callers know when the batch is done?

Return a job ID immediately, then expose a status endpoint or completion event that reports each URL’s terminal status and output key.

Sources

• AWS Lambda timeout configuration
• AWS Lambda Invoke API
• AWS asynchronous invocation
• Function URLs versus API Gateway
• AWS Lambda best practices
• Lambda memory configuration
• Lambda ephemeral storage