How to Capture Screenshots for a List of URLs with VisualScraper

Short answer: Prepare and validate your URL list, decide what each screenshot must include, test a small sample, capture the full batch, and review results URL by URL. The research available for this guide did not identify authoritative documentation for a product named VisualScraper, so its exact controls, limits, and export formats cannot be verified here. The workflow below is tool-neutral; the local example uses the separately documented shot-scraper, not VisualScraper.

1. Confirm the tool and define the output

Before following product-specific instructions, confirm that “VisualScraper” is the exact product name and locate its official documentation. Do not assume another tool’s menus or options apply. For any batch screenshot tool, write down the output contract first:

Which input URLs are in scope, and should redirects be followed?
Should each result show the initial viewport or the full page?
What viewport dimensions and image format are required?
Are any pages behind a login or affected by consent prompts or personalization?
How will each output file map back to its source URL?
What should happen when a URL fails: continue, stop, or retry?

Keep the original URL list unchanged. If you normalize schemes or remove whitespace, retain the original value alongside the cleaned URL so you can audit redirects and input changes.

2. Prepare a durable URL list

Use one URL per row or line. Check that each value has a valid scheme such as https://, remove accidental whitespace, and flag malformed entries before the capture run. Identify duplicates deliberately: remove them if one image per unique destination is enough, or preserve them if each input row represents a separate record.

For repeat jobs, add a stable ID to each row. A minimal CSV can look like this:

id,url
home,https://example.com/
pricing,https://example.com/pricing
help,https://example.com/help

Do not use the raw URL as a filename. URLs can contain query strings, slashes, or characters that are awkward in paths, and two different URLs can collapse to the same sanitized name. Prefer a stable ID or a sequence number plus a short label, and save a manifest connecting each image to its original and final URL.

3. Choose capture settings and page state

Decide what must be visible before starting. A viewport capture is smaller and consistent in dimensions; a full-page capture includes content below the fold but can be much taller and may trigger lazy-loaded images as the page scrolls. If the tool supports them, record the viewport, format, device scale, full-page setting, wait condition, and any authentication configuration with the run.

Page state can change the result. Redirects may land on a different URL; login sessions may expire; cookie banners, newsletter overlays, and chat widgets may cover content; personalization can vary by session; animations and delayed rendering can produce inconsistent frames. For repeatable work, use the same capture settings and authentication context, and record the final URL and capture time where available.

4. Pilot a few representative pages

Before sending a large batch, capture a small sample that includes a fast page, a page with delayed content, a likely redirect, and any page that requires authentication. Inspect the files for the following:

Each output exists and opens.
Dimensions and format match the intended output.
The capture shows the expected page state rather than a consent overlay, login page, or error.
Filenames are unique and map to the corresponding inputs.
Slow or failed pages are reported in a way you can retry.

Fix the settings or input data revealed by the pilot before running the full list. This catches naming collisions and page-state problems while the batch is still small.

5. Run a local batch with shot-scraper

If you want a documented local CLI workflow, shot-scraper accepts a YAML configuration with one screenshot entry per URL and runs the batch with shot-scraper multi. Install it using the instructions in its official documentation, then create a YAML file such as:

- url: https://example.com/
  output: screenshots/001-home.png
- url: https://example.com/pricing
  output: screenshots/002-pricing.png
- url: https://example.com/help
  output: screenshots/003-help.png

Run the configured captures with:

shot-scraper multi urls.yml

This produces one image per configured URL. Add the viewport and other settings using options supported by the installed shot-scraper version; consult its multiple-screenshot documentation for the exact YAML schema. This example demonstrates shot-scraper only. It does not establish VisualScraper syntax, batch limits, or output behavior.

6. Keep a result manifest and retry failures

Record at least the input ID, original URL, cleaned URL, output path, final URL if known, capture timestamp, settings, status, and error text. A manifest lets you identify missing files, distinguish a failed capture from a skipped duplicate, and retry only the affected URLs. Batch behavior varies by tool, so check whether it stops on errors, retries automatically, or emits a per-URL report before relying on it for unattended runs.

For large or recurring batches, divide the list into manageable groups. Save results as each group completes, keep the input list and settings with the output, and make retries idempotent by writing to deterministic paths. Avoid rerunning successful URLs unless you intend to refresh their screenshots.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. Its single-request API accepts a URL and returns a PNG, JPEG, WebP, or PDF. The API accepts one URL per request, so a URL list can be processed by calling it once for each row; use your own queue and record each response against its input ID. See the API documentation for parameters and response details.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

In a batch script, substitute each row’s URL and output path, check the response status, and store the response verdict and billing headers with the manifest. ScreenshotNeo accepts cookie or consent banners and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and responses include X-Page-Verdict and X-Billed headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients. The Free plan includes 1,000 shots per month without a card; paid plans start at $5 for 3,000 shots.

Performance, reliability, and cost

Batch duration depends on page load time, wait conditions, full-page length, and how many captures the tool runs concurrently. Concurrency can reduce elapsed time, but too many simultaneous browser sessions or requests can strain the machine or trigger remote-site throttling. Start with a small pilot, then increase batch size and concurrency while checking failures and resource use.

Local capture gives you control over where browser work runs, but you must maintain the runtime, browser dependencies, authentication state, and output storage. A hosted API avoids local browser setup, while introducing per-request configuration and service pricing to evaluate. For ScreenshotNeo, the stated plans are Free: 1,000 shots/month; Starter: $5 for 3,000; Growth: $15 for 15,000; Pro: $39 for 60,000; Scale: $99 for 250,000; Business: $249 for 1,000,000. Yearly billing gives two months free, and every feature is on every plan. Do not infer another tool’s price or limits from these figures.

Troubleshooting

Symptom	Likely cause	What to do
No image is produced for one URL	Malformed URL, network failure, or a page load error	Check the scheme and address in a browser, retain the error in the manifest, and retry that URL separately.
The image shows a login page	The page needs authentication or a session expired	Confirm the page is accessible in the capture context and configure credentials or session state only through supported tool options.
The screenshot is mostly a banner or popup	Consent or promotional overlay covers the page	Use a documented consent-handling or overlay option if available, or capture after the expected interaction. Verify the result in the pilot.
Images or text are missing	Capture happened before delayed or lazy content loaded	Use a suitable wait condition or delay if supported, and test the affected page; excessively long waits slow the entire batch.
Two URLs overwrite one image	Filename generation produced a collision	Use stable unique IDs or sequence numbers and check output paths before running.
Some pages redirect unexpectedly	Canonical redirects, regional routing, or access checks	Record both source and final URL, then decide whether the destination is expected before retrying.
The batch stops after an error	The tool may be configured to fail fast	Check its documented error behavior. If supported, continue while collecting per-URL failures for a targeted retry.
Repeat captures differ	Personalization, animation, changing content, or different session state	Keep viewport, time, authentication, and wait settings consistent; disable animation only if the tool documents such a control.

FAQ

Does this guide confirm how VisualScraper works?

No. The research did not establish authoritative VisualScraper documentation. Confirm the exact product and consult its official help before relying on product-specific controls or limits.

Can I capture private pages?

Only if the selected tool supports the required authentication method and you are authorized to access those pages. Keep credentials out of shared URL lists and logs.

Should every URL be captured again on a retry?

Usually, retry only failed or incomplete entries. A stable manifest and deterministic output paths make that practical.

Can ScreenshotNeo take a whole list in one request?

The documented API call shown here captures a URL per request. Use a loop or job queue for a list; ScreenshotNeo also supports bulk capture of up to 100 URLs per call.