ScreenshotNeo
Sign in Get a free key

BlogAI agents

How to Get an AI Agent to Save Website Screenshots to Amazon S3

Capture a page with Playwright and upload it to Amazon S3 using trusted AWS credentials or a short-lived presigned URL.

By the ScreenshotNeo team4 October 202612 min read

To save a website screenshot to Amazon S3, capture the page with a browser automation tool such as Playwright, then upload the resulting image bytes to S3. If your agent runs in a trusted server-side environment, use the AWS SDK with a narrowly scoped execution role. If it runs in a browser or another environment that should not receive AWS credentials, have a trusted backend create a short-lived presigned PUT URL and let the agent upload the image to that URL.

The capture and upload are separate steps: Playwright produces the image; S3 stores it as an object. The examples below use Python, Playwright, and boto3. They show both upload patterns, plus a minimal backend endpoint for issuing a presigned URL.

1. Choose where the S3 upload runs

Pattern Good fit Credential exposure What you need
AWS SDK upload A trusted agent worker or backend AWS credentials stay in the trusted runtime An execution role or other AWS credential provider with permission to write to the intended bucket and prefix
Presigned PUT A browser, client, or isolated agent that should not receive AWS credentials The uploader receives a temporary bearer URL for one signed operation A trusted signing endpoint, short expiry, unique object key, and exact matching request method and headers

In either case, choose the bucket and object key on the trusted side. Do not let an untrusted client choose arbitrary buckets or unrestricted keys. A key such as screenshots/<run-id>/<timestamp>.png makes captures easy to group and avoids accidental replacement. Uploading to an existing key replaces that object.

2. Install the Python dependencies

python -m pip install playwright boto3 requests
python -m playwright install chromium

The agent process also needs AWS credentials configured through the standard AWS credential chain if you use the SDK method. In production, prefer a workload or instance role over long-lived access keys. The role should be limited to the required S3 operation and bucket/prefix; the exact policy depends on your bucket, encryption, and organization controls.

3. Capture a website and upload it with the AWS SDK

This complete script takes a full-page PNG and uploads the bytes directly from memory. Set TARGET_URL, S3_BUCKET, and optionally AWS_REGION in the environment before running it.

import os
import time
from urllib.parse import urlparse

import boto3
from playwright.sync_api import sync_playwright

TARGET_URL = os.environ.get("TARGET_URL", "https://example.com")
S3_BUCKET = os.environ["S3_BUCKET"]
AWS_REGION = os.environ.get("AWS_REGION", "us-east-1")

parsed = urlparse(TARGET_URL)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
    raise ValueError("TARGET_URL must be an absolute http or https URL")

# Use a run-specific key so a later capture does not replace this one.
run_id = os.environ.get("RUN_ID", str(int(time.time())))
object_key = f"screenshots/{run_id}/page.png"

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page(viewport={"width": 1440, "height": 900}, device_scale_factor=1)
    response = page.goto(TARGET_URL, wait_until="domcontentloaded", timeout=45000)

    if response is not None and response.status >= 400:
        raise RuntimeError(f"Page returned HTTP {response.status}")

    # Prefer a meaningful selector or an application-specific readiness condition
    # when the page renders important content after initial navigation.
    page.locator("body").wait_for(state="visible", timeout=15000)
    image_bytes = page.screenshot(full_page=True, type="png", animations="disabled")
    browser.close()

s3 = boto3.client("s3", region_name=AWS_REGION)
s3.put_object(
    Bucket=S3_BUCKET,
    Key=object_key,
    Body=image_bytes,
    ContentType="image/png",
)
print(f"Uploaded s3://{S3_BUCKET}/{object_key}")

Playwright documents navigation followed by page.screenshot({ path: 'screenshot.png' }); its page screenshot API also supports full-page and element captures. See the Playwright screenshot guide and Page screenshot API.

Use an element screenshot instead

To capture one component rather than the entire page, replace the screenshot call with a locator screenshot. Make the selector specific enough to identify one visible element.

image_bytes = page.locator("main article").screenshot(type="png")

4. Create a presigned upload URL on a trusted backend

A presigned URL lets the agent upload without getting AWS credentials. AWS describes presigned URLs as a way to grant time-limited access to S3 objects without changing the bucket policy. Anyone holding the URL can use its delegated capability until it expires, so treat it like a secret. See AWS’s presigned URL documentation.

This small Flask endpoint chooses the bucket and key on the server and returns a URL for a single PUT. Run it only behind your normal authentication and authorization layer, and validate which capture the caller is allowed to request.

import os
import secrets
from flask import Flask, jsonify
import boto3

app = Flask(__name__)
s3 = boto3.client("s3", region_name=os.environ.get("AWS_REGION", "us-east-1"))
BUCKET = os.environ["S3_BUCKET"]

@app.post("/api/screenshot-upload-url")
def create_upload_url():
    # In a real application, authenticate the caller and associate the key
    # with an authorized user or job before signing it.
    object_key = f"screenshots/{secrets.token_urlsafe(18)}.png"
    content_type = "image/png"
    upload_url = s3.generate_presigned_url(
        ClientMethod="put_object",
        Params={"Bucket": BUCKET, "Key": object_key, "ContentType": content_type},
        ExpiresIn=300,
        HttpMethod="PUT",
    )
    return jsonify({
        "upload_url": upload_url,
        "object_key": object_key,
        "content_type": content_type,
    })

if __name__ == "__main__":
    app.run(host="127.0.0.1", port=8000)

The signing principal needs permission for the intended upload. The configured expiry is an upper bound: if the signer uses temporary credentials, the URL can stop working when those credentials expire. AWS’s uploading objects with presigned URLs guide shows the PUT pattern and notes that the content type used when signing must match the upload request.

5. Capture and PUT the bytes to the presigned URL

This agent-side example asks the backend for a URL, captures the page, and uploads the PNG. The agent sees a temporary URL but never receives AWS credentials.

import os
import requests
from urllib.parse import urlparse
from playwright.sync_api import sync_playwright

TARGET_URL = os.environ.get("TARGET_URL", "https://example.com")
SIGNING_ENDPOINT = os.environ.get(
    "SIGNING_ENDPOINT", "http://127.0.0.1:8000/api/screenshot-upload-url"
)

parsed = urlparse(TARGET_URL)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
    raise ValueError("TARGET_URL must be an absolute http or https URL")

sign_response = requests.post(SIGNING_ENDPOINT, timeout=15)
sign_response.raise_for_status()
signed = sign_response.json()

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page(viewport={"width": 1440, "height": 900})
    response = page.goto(TARGET_URL, wait_until="domcontentloaded", timeout=45000)
    if response is not None and response.status >= 400:
        raise RuntimeError(f"Page returned HTTP {response.status}")
    page.locator("body").wait_for(state="visible", timeout=15000)
    image_bytes = page.screenshot(full_page=True, type="png", animations="disabled")
    browser.close()

put_response = requests.put(
    signed["upload_url"],
    data=image_bytes,
    headers={"Content-Type": signed["content_type"]},
    timeout=60,
)
put_response.raise_for_status()
print(f"Uploaded object key: {signed['object_key']}")

Use the exact HTTP method and signed headers the backend used. Do not append query parameters, alter the URL, or change the content type. For an upload initiated by browser JavaScript, the bucket also needs a CORS rule allowing the application’s origin, the PUT method, and the request headers. CORS applies to browser-enforced cross-origin requests; a server-side agent using requests or an AWS SDK does not need browser CORS. See Amazon S3 CORS configuration.

6. Verify the saved object and retain useful metadata

A successful PUT response means the request was accepted; if your workflow needs a separate confirmation, check the object with a HEAD request or the SDK. Keep operational metadata in your job database or logs: target URL, capture timestamp, object key, content type, page response status, and upload result. Avoid logging AWS credentials or full presigned URLs.

head = s3.head_object(Bucket=S3_BUCKET, Key=object_key)
print({
    "content_type": head.get("ContentType"),
    "size_bytes": head.get("ContentLength"),
    "last_modified": str(head.get("LastModified")),
})

For stronger integrity checks, use a checksum supported by the S3 upload path and verify it according to your application’s needs. AWS documents checksum use with SigV4 presigned uploads in its presigned URL guide.

7. Configure capture behavior for the page

  • Wait for the right state: domcontentloaded is a fast starting point, but it does not guarantee that client-rendered data or images are ready. Wait for a page-specific selector, a known application condition, or a deliberate delay when required. Avoid relying on network idle for pages that keep connections open.
  • Full page or viewport: full_page=True captures the full scrollable page; omit it for the current viewport. Very tall pages can produce large images and take longer to render and upload.
  • Element or page: use locator(selector).screenshot() for a specific component. Ensure it exists and is visible first.
  • Format: PNG preserves lossless detail and works well for text-heavy pages. JPEG can reduce size for photographic pages; WebP can be useful when every consumer supports it. Match the S3 ContentType to the actual encoding.
  • Viewport and scale: set the viewport and device scale factor explicitly if captures need consistent dimensions. Higher device scale factors increase pixel dimensions and object size.
  • Authentication: use a dedicated browser context and approved test account or session state. Do not store credentials or sensitive page contents in screenshot object names or logs.
  • Navigation failures: handle redirects, access-denied pages, bot checks, and application-specific error states according to the agent’s task. A browser reaching a page does not prove that the intended content loaded.

8. Security, reliability, performance, and cost

Security

  • Keep SDK credentials in the trusted runtime and scope the role to the required bucket and prefix.
  • Presigned URLs are bearer credentials. Return them only to an authorized caller, use short expiries, avoid logging them, and do not treat them as permanent object URLs.
  • Use unique keys for retained history. Reusing a key replaces the existing object.
  • Apply bucket access, encryption, retention, and deletion rules appropriate to the captured page. Screenshots can contain account details or personal information.
  • Restrict which URLs the agent may navigate to. Browser automation that accepts arbitrary destinations can reach internal services or other resources accessible from its network; use appropriate network controls.

Reliability

  • Separate capture errors from upload errors so you can retry the failed stage without blindly recapturing or replacing an existing object.
  • Use a unique job or run identifier in the object key. If retries must be idempotent, have the trusted backend reserve the key for the job and define whether a retry may overwrite it.
  • Retry transient network failures with bounded exponential backoff and jitter. Do not repeatedly retry authorization failures, expired URLs, invalid requests, or a page that consistently returns an error.
  • For presigned uploads, request a fresh URL after expiry. The effective lifetime may be shorter than the configured expiry when temporary signing credentials expire.
  • Set timeouts for navigation, signing, and upload independently. Record the stage and error category with the job result.

Performance and cost

The main cost drivers are browser runtime, screenshot pixel dimensions, transfer size, S3 storage, and requests to S3. Full-page and high-scale captures use more memory and produce larger objects than viewport captures. Choose the smallest image format and dimensions that preserve the detail your downstream task needs. If a workflow captures the same page repeatedly, consider whether deduplication or a retention policy is appropriate; do not assume a cache is in use unless you implement one.

There is no fixed capture time or cost that applies to every page: page complexity, browser resources, output size, AWS region, storage class, retention, and request volume all matter. Estimate these from your actual workload and current AWS pricing. Keep screenshot retention bounded where the use case permits.

9. Troubleshooting

Symptom Likely cause Fix
AccessDenied from S3 The role or signing principal lacks the required permission, or a bucket policy denies the operation. Check the principal, bucket and key scope, encryption requirements, and bucket policy. Grant only the needed upload permission.
Presigned PUT returns SignatureDoesNotMatch The method, URL, content type, or another signed request detail differs from what was signed. Use the exact URL and HTTP method returned by the signer and send the same signed headers, including Content-Type.
Presigned URL returns an expired or invalid request error The URL expired, the signing credentials expired, or the request was modified. Request a fresh URL, shorten delays between signing and upload, and check the signer’s credential lifetime and bucket region.
Upload works in Python but fails in a web page The browser blocks the cross-origin request because the bucket CORS policy does not allow the origin, method, or headers. Configure S3 CORS for the exact web origin, PUT, and required headers. CORS does not grant S3 authorization.
The saved file has the wrong type or will not open The screenshot encoding and S3 Content-Type do not match, or the bytes were altered. Set the screenshot type and matching content type; upload the raw screenshot bytes without text encoding.
Screenshot is blank or missing content Capture ran before client rendering, a selector was wrong, navigation hit an interstitial, or content is below the viewport and the capture was not full page. Wait for a meaningful application selector, inspect the final URL and response, and set full_page=True when needed.
Navigation times out The page is slow, holds network connections open, or the selected wait condition is too strict. Use a realistic timeout and wait for the specific content needed instead of requiring every connection to become idle.
Old screenshot disappears after retry The retry uploaded to the same object key, replacing the prior object. Generate a unique key per run or explicitly use versioning and a deliberate overwrite policy.
Object is missing after an apparently successful workflow The client reported before upload completion, an exception was swallowed, or the workflow checked the wrong key or region. Raise on non-success HTTP responses, record the exact bucket/key, and verify with head_object or an authorized S3 listing.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. One GET request returns an image or PDF. Its clean-capture options accept cookie and consent banners and remove more than 60 known consent platforms, newsletter popups, and chat widgets before the shot; each step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server offers take_screenshot, get_page_info, and capture_pdf for AI agents and MCP clients.

Capture a page with cURL:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Or use Python:

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Or use Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

See the ScreenshotNeo API documentation for the request options, response headers, and output formats. Save the returned bytes to S3 with the same SDK or presigned upload flow described above. The Free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots. Create a free ScreenshotNeo account.

FAQ

Can an AI agent upload screenshots to S3 without AWS credentials?

Yes. A trusted backend can generate a short-lived presigned PUT URL for a chosen bucket and key. The agent uploads to that URL and never receives AWS credentials.

Does uploading from a server need S3 CORS?

No. CORS controls cross-origin requests made by browsers. Server-side SDK or HTTP uploads do not rely on browser CORS.

Can I save screenshots from a managed browser session?

Yes. A managed browser can expose a browser automation interface compatible with Playwright; the resulting screenshot bytes can then follow either S3 upload pattern. AWS documents AgentCore Browser for standard browser automation and an AWS reference architecture that stores screenshots and session transcripts in S3.

Should every capture be full page?

No. Use a viewport or element capture when that contains the information the agent needs; it can reduce image dimensions and transfer size.

Keep reading