ScreenshotNeo
Sign in Get a free key

BlogAI agents

How to Capture Website Screenshots with an AI Agent from a CSV URL List

Use an AI agent to build or guide a repeatable CSV-to-screenshot workflow, with Python and Playwright handling each URL, file, and result.

By the ScreenshotNeo team4 October 202612 min read

A reliable way to capture website screenshots from a CSV with an AI agent is to let the agent help create or operate the workflow while a script explicitly reads each CSV row, visits its URL, saves the screenshot, and records the outcome. The CSV is input data; it does not become a batch job automatically just because an AI agent is involved.

This guide uses Python’s standard-library csv.DictReader and Playwright’s Python API. It validates the URL column, uses stable row-based filenames, captures either the viewport or full page, and writes a results manifest so failures remain attached to their source rows. Python documents DictReader and opening CSV files with newline=''; Playwright documents page, full-page, and element screenshots. Python CSV documentation · Playwright screenshot documentation

1. Prepare the CSV and decide what to capture

Use a header row with a column named url, followed by one URL per record:

url
https://example.com/
https://www.python.org/
https://playwright.dev/python/docs/screenshots

The code below expects that exact header name. If your file uses another name, set URL_COLUMN accordingly. Keep the original CSV unchanged so you can compare every output and status with the input record.

Choice Use it when Playwright setting
Viewport screenshot You want the visible browser window at a consistent size. full_page=False
Full-page screenshot You need the scrollable page below the fold. full_page=True
Element screenshot You only need one component, such as a chart or product panel. Locate the element and call its screenshot() method.
Image file You want a straightforward archive. Pass a path to page.screenshot().
Screenshot bytes You want to process or transfer the image in memory. Call page.screenshot() without a path and handle the returned bytes.

For repeatable comparisons, keep the browser engine, viewport, device scale factor, color scheme, and capture scope consistent. The page itself can still change between runs because site content, personalization, network resources, and time-dependent elements may vary.

2. Install Python and Playwright

Create a project directory, make a virtual environment, install Playwright, and install its Chromium browser. Playwright’s Python setup and screenshot references are the source for the browser API used here.

python -m venv .venv
# macOS or Linux:
source .venv/bin/activate
# Windows PowerShell:
# .venv\Scripts\Activate.ps1
python -m pip install playwright
python -m playwright install chromium

Save the following as capture_csv.py in the same directory as your input CSV. Run it with the CSV path, output directory, and optional capture mode:

python capture_csv.py urls.csv screenshots --full-page

3. Run a CSV-to-screenshot script

This script uses one browser page at a time, checks for missing or malformed URL values, captures valid rows, and writes results.csv with a row number, original URL, output filename, status, and error detail. A failed row does not change the filenames assigned to later rows.

import argparse
import csv
import re
import sys
from pathlib import Path
from urllib.parse import urlsplit

from playwright.sync_api import sync_playwright

URL_COLUMN = "url"
DEFAULT_TIMEOUT_MS = 30_000


def valid_http_url(value):
    """Return True for an absolute HTTP(S) URL with a hostname."""
    try:
        parsed = urlsplit(value)
        return parsed.scheme in {"http", "https"} and bool(parsed.hostname)
    except ValueError:
        return False


def safe_host(value):
    """Create a readable hostname component for a filename."""
    try:
        host = urlsplit(value).hostname or "unknown-host"
    except ValueError:
        host = "unknown-host"
    host = re.sub(r"[^A-Za-z0-9.-]+", "-", host).strip(".-")
    return host or "unknown-host"


def main():
    parser = argparse.ArgumentParser(
        description="Capture one Playwright screenshot per URL in a CSV."
    )
    parser.add_argument("csv_file", type=Path)
    parser.add_argument("output_dir", type=Path)
    parser.add_argument(
        "--full-page", action="store_true",
        help="Capture the full scrollable page instead of just the viewport.",
    )
    parser.add_argument(
        "--timeout-ms", type=int, default=DEFAULT_TIMEOUT_MS,
        help="Navigation timeout in milliseconds (default: 30000).",
    )
    parser.add_argument(
        "--url-column", default=URL_COLUMN,
        help="CSV header containing URLs (default: url).",
    )
    args = parser.parse_args()

    if args.timeout_ms <= 0:
        parser.error("--timeout-ms must be greater than zero")
    if not args.csv_file.is_file():
        parser.error(f"CSV file does not exist: {args.csv_file}")

    args.output_dir.mkdir(parents=True, exist_ok=True)
    manifest_path = args.output_dir / "results.csv"
    results = []

    # newline='' lets the csv module handle newline conventions correctly.
    with args.csv_file.open("r", encoding="utf-8-sig", newline="") as source:
        reader = csv.DictReader(source)
        if not reader.fieldnames or args.url_column not in reader.fieldnames:
            parser.error(
                f"CSV must have a header named {args.url_column!r}; "
                f"found {reader.fieldnames!r}"
            )

        with sync_playwright() as playwright:
            browser = playwright.chromium.launch(headless=True)
            page = browser.new_page(
                viewport={"width": 1440, "height": 1000},
                device_scale_factor=1,
                color_scheme="light",
            )
            page.set_default_navigation_timeout(args.timeout_ms)

            for row_number, row in enumerate(reader, start=1):
                raw_url = (row.get(args.url_column) or "").strip()
                filename = f"{row_number:05d}-{safe_host(raw_url)}.png"
                output_path = args.output_dir / filename
                result = {
                    "row": row_number,
                    "url": raw_url,
                    "file": filename,
                    "status": "error",
                    "error": "",
                }

                if not raw_url:
                    result["status"] = "skipped"
                    result["error"] = "URL cell is blank"
                elif not valid_http_url(raw_url):
                    result["status"] = "skipped"
                    result["error"] = "Expected an absolute http or https URL"
                else:
                    try:
                        # domcontentloaded avoids waiting for every third-party
                        # image, ad, or analytics request to finish.
                        page.goto(raw_url, wait_until="domcontentloaded")
                        page.screenshot(
                            path=str(output_path),
                            full_page=args.full_page,
                            type="png",
                            animations="disabled",
                        )
                        result["status"] = "ok"
                    except Exception as exc:
                        result["error"] = f"{type(exc).__name__}: {exc}"
                        # A partial output is not reported as a successful shot.
                        output_path.unlink(missing_ok=True)

                results.append(result)
                print(f"row {row_number}: {result['status']} {raw_url}")

            browser.close()

    with manifest_path.open("w", encoding="utf-8", newline="") as destination:
        fields = ["row", "url", "file", "status", "error"]
        writer = csv.DictWriter(destination, fieldnames=fields)
        writer.writeheader()
        writer.writerows(results)

    succeeded = sum(item["status"] == "ok" for item in results)
    skipped = sum(item["status"] == "skipped" for item in results)
    failed = sum(item["status"] == "error" for item in results)
    print(
        f"Finished: {succeeded} captured, {skipped} skipped, {failed} failed. "
        f"Manifest: {manifest_path}"
    )
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main())

The script captures PNGs at a 1440 × 1000 CSS-pixel viewport, with a device scale factor of 1 and light color scheme. Change those context settings to suit the job. It waits for the DOM content event rather than full network idle: many pages keep analytics, ads, or other requests open. If a page renders important content after navigation, add a site-appropriate wait as described below.

4. Let an AI agent help without hiding the workflow

An AI coding agent can draft the script, adapt it to a CSV schema, explain failures, or help inspect a sample. Keep the row parsing, URL validation, filenames, and result manifest in ordinary code so each screenshot remains traceable to its input.

For example, give the agent this task:

Update capture_csv.py to read the URL column named "website" from my CSV.
Keep the row number in every output filename, preserve one result per input
row in results.csv, and report blank or invalid URLs as skipped. Do not add
parallel browser workers. Explain the code changes and any assumptions.

Playwright also documents a CLI intended for coding-agent browser workflows. Its documented commands can open pages and capture screenshots, but the cited CLI material does not describe a built-in CSV batch importer; CSV parsing still needs an explicit script step. See Playwright’s coding-agent CLI documentation.

For an interactive check, you can ask an agent to open a representative page with the CLI and inspect the rendered result, then use the Python script for the CSV loop. Keep credentials and private URLs out of prompts unless the agent environment is approved to handle them.

5. Adapt capture behavior for real pages

Viewport or full page

Use the default command for a viewport screenshot. Add --full-page to capture the scrollable page:

python capture_csv.py urls.csv screenshots --full-page

Full-page screenshots can be very tall and consume more memory than viewport captures. Pages with sticky headers, infinite scroll, or content loaded only after scrolling may need a site-specific capture strategy; full-page mode alone does not guarantee that every lazy resource has loaded.

Wait for a page-specific element

If a site renders its useful content after the initial DOM event, wait for a known selector after page.goto():

page.goto(raw_url, wait_until="domcontentloaded")
page.locator("main article").wait_for(state="visible", timeout=10_000)
page.screenshot(path=str(output_path), full_page=args.full_page)

Use a selector that applies to the sites in the list, or map selectors by site. A missing selector will time out and should be recorded as an error for that row. For pages with no stable selector, a short fixed delay can be used, but it adds waiting to every affected capture and does not prove that the page is complete.

Capture one element

To capture only a component, replace the page screenshot call with a locator screenshot. Handle missing or hidden elements as per-row failures:

target = page.locator("#pricing-table")
target.wait_for(state="visible", timeout=10_000)
target.screenshot(path=str(output_path), type="png")

Playwright supports element screenshots and screenshot bytes as well as page screenshots; see the screenshot API guide.

Capture bytes for another processing step

If the next step uploads or transforms each image, capture bytes instead of writing a file directly:

image_bytes = page.screenshot(full_page=args.full_page, type="png")
# Pass image_bytes to your own processing or storage code.

Choose output format and viewport

The example writes PNG, which is lossless and suitable for visual inspection. Playwright also supports JPEG with a quality setting. Use the screenshot API options documented for your installed Playwright version when selecting format or quality. To change the viewport, edit the browser.new_page(viewport=...) dimensions; keep them fixed across a comparison run.

6. Reliability, performance, and cost

  • Keep row identity: the zero-padded input row number makes names unique even when hosts repeat. The manifest retains the original URL and per-row result.
  • Start with a sample: run a copy with a few representative URLs first. Check viewport, full-page behavior, wait conditions, and the output files before processing the complete list.
  • Expect site variability: redirects, bot checks, consent dialogs, authentication, changing content, and transient network failures can affect a capture. The script records exceptions but cannot guarantee a successful or visually identical result.
  • Choose waits deliberately: domcontentloaded avoids waiting for every network request. Add a selector wait when the page has a meaningful readiness signal. A longer timeout can help slow sites but increases the time spent on each stalled row.
  • Keep concurrency conservative: this example uses one page sequentially, which is easier to debug and avoids adding load to target sites. If you later add workers, account for browser memory, site rate limits, and orderly manifest writes. No throughput or success-rate guarantee is implied.
  • Local execution costs: the script has no per-screenshot API charge, but it uses your machine’s CPU, memory, storage, and network. Very tall pages and large lists require more resources.
  • Re-run failed rows intentionally: use the manifest to identify errors and rerun only those records after addressing their cause. Avoid silently overwriting a prior run if you need an audit trail; use a separate output directory per run.

7. Troubleshooting

Symptom Likely cause Fix
Header error says the URL column is missing The CSV header differs from url, has a typo, or includes unexpected whitespace. Correct the header or pass --url-column website with the actual name. Inspect the first row of the CSV.
Many rows are skipped as invalid Cells may omit https://, contain spaces, or be blank. Normalize the source data to absolute HTTP(S) URLs and fill or remove blank records. The script deliberately skips non-HTTP schemes.
Executable doesn't exist or browser launch fails The Playwright package is installed but its browser binary is not. Run python -m playwright install chromium in the same environment.
Navigation timeout The site is slow, unreachable, or keeps navigation from completing. Check the URL in a browser, increase --timeout-ms if appropriate, and consider a different wait condition or a site-specific readiness selector.
Screenshot is blank or misses content Content may render after the chosen navigation event, require scrolling, or be blocked by a bot check. Inspect the page manually, wait for a stable content selector, or scroll to trigger lazy loading where needed. Record bot checks as an outcome rather than treating a blank image as valid content.
Full-page capture is unexpectedly huge or fails The page may have a long or effectively unbounded document, such as an infinite feed. Use a viewport capture, or implement a bounded scroll-and-capture strategy appropriate to the target. Review memory use and output dimensions.
Output files overwrite or are hard to match A naming scheme based only on hostname or URL path may collide. Retain the row number in each filename and use the manifest’s row-to-URL mapping.
Some sites show different layouts Responsive breakpoints, color schemes, personalization, or browser state differ. Set a consistent viewport and color scheme, and use an appropriate browser context per capture when site state matters.

8. Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server for developers. One GET request captures a URL as an image; the API also supports PDF. Its cookie and consent handling accepts the banner like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture. Each of those steps can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed; response headers identify the page verdict and billing status.

For a URL from your CSV, substitute that row’s URL and your API key. See the ScreenshotNeo API documentation for options and response details.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const image = Buffer.from(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', image));

Replace https://stripe.com with the URL in each CSV row and choose a distinct output filename. ScreenshotNeo also provides an MCP server with take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. Other options include full-page capture, CSS selector capture, device presets or custom viewports, custom CSS and JavaScript, waits, request blocking, custom headers and cookies, caching, bulk capture for up to 100 URLs per call, and async jobs with signed webhooks. Every feature is available on every plan.

The free plan includes 1,000 shots each month with no card. Paid plans start at $5 for 3,000 shots; yearly billing gives two months free. See ScreenshotNeo for the service and the documentation for API configuration. Sign up for 1,000 free screenshots a month, with no card.

9. FAQ

Does an AI agent automatically import and process my CSV?

Not based on the cited Playwright CLI documentation. Parse the CSV explicitly in a script or configure an agent with a tool that does so.

Can I keep one screenshot for every input record, including invalid URLs?

The example creates an image only for valid URLs it captures. It still creates a manifest row for blanks and malformed values, marked as skipped, so the record is accounted for.

Will repeated runs produce pixel-identical images?

Not necessarily. A consistent browser and viewport help, but live page content, personalization, third-party resources, and rendering conditions can change.

Can I use this for pages behind a login?

Only if you deliberately configure an authenticated browser context and handle the credentials securely. The example uses a fresh unauthenticated page and does not implement login.

What should I do with pages that use infinite scroll?

Choose a bounded capture requirement, such as a viewport or a fixed number of scrolls. A full-page screenshot does not define how much of an effectively endless feed you want.

Keep reading