ScreenshotNeo

BlogHow-to

How to Screenshot Multiple Website URLs Concurrently in Go

Capture many website URLs in Go with chromedp using a bounded worker pool, per-page contexts, reliable cleanup, and clear error reporting.

By the ScreenshotNeo team4 October 20269 min read

Use chromedp with a bounded worker pool: give each screenshot job its own browser target and chromedp context, then capture and save that page independently. This lets you process a URL list concurrently without letting two jobs navigate or capture the same tab. There is no universal safe number of Chrome tabs; make the worker limit configurable and measure it with your actual pages and machine.

The example below captures full-page PNGs. It uses one browser process, a separate chromedp target per job, per-URL timeouts, collision-resistant output names, and per-URL errors so one bad page does not discard successful results. The chromedp context and screenshot APIs are documented in the package reference and official examples.

1. Set up the Go project

Install Go and a Chrome or Chromium browser available to chromedp. Create a module and add the dependency:

mkdir go-shots
cd go-shots
go mod init example.com/go-shots
go get github.com/chromedp/chromedp

Save the following as main.go. It reads URLs from command-line arguments and writes screenshots into an output directory.

2. Complete concurrent screenshot program

package main

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "errors"
    "flag"
    "fmt"
    "log"
    "net/url"
    "os"
    "path/filepath"
    "strings"
    "sync"
    "time"

    "github.com/chromedp/chromedp"
)

type result struct {
    input string
    file  string
    err   error
}

func outputName(rawURL string) string {
    parsed, err := url.Parse(rawURL)
    host := "page"
    if err == nil && parsed.Hostname() != "" {
        host = strings.ToLower(parsed.Hostname())
        host = strings.ReplaceAll(host, ":", "_")
    }
    sum := sha256.Sum256([]byte(rawURL))
    return fmt.Sprintf("%s-%s.png", host, hex.EncodeToString(sum[:6]))
}

func capture(ctx context.Context, browserCtx context.Context, rawURL, outDir string) result {
    // A child chromedp context creates a new target (tab) in the shared browser.
    tabCtx, cancel := chromedp.NewContext(browserCtx)
    defer cancel()

    jobCtx, stop := context.WithTimeout(tabCtx, 60*time.Second)
    defer stop()

    var image []byte
    if err := chromedp.Run(jobCtx,
        chromedp.Navigate(rawURL),
        chromedp.FullScreenshot(&image, 100),
    ); err != nil {
        return result{input: rawURL, err: err}
    }

    name := outputName(rawURL)
    path := filepath.Join(outDir, name)
    if err := os.WriteFile(path, image, 0o644); err != nil {
        return result{input: rawURL, err: err}
    }
    return result{input: rawURL, file: path}
}

func main() {
    workers := flag.Int("workers", 3, "maximum simultaneous page captures")
    outDir := flag.String("out", "screenshots", "directory for PNG files")
    flag.Parse()
    urls := flag.Args()
    if len(urls) == 0 {
        log.Fatal("usage: go run . [-workers N] [-out DIR] URL [URL ...]")
    }
    if *workers < 1 {
        log.Fatal("-workers must be at least 1")
    }
    for _, raw := range urls {
        parsed, err := url.ParseRequestURI(raw)
        if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") || parsed.Host == "" {
            log.Fatalf("invalid HTTP(S) URL %q", raw)
        }
    }
    if err := os.MkdirAll(*outDir, 0o755); err != nil {
        log.Fatalf("create output directory: %v", err)
    }

    // Create one browser process for the batch. Each job creates its own target.
    allocCtx, cancelAlloc := chromedp.NewContext(context.Background())
    defer cancelAlloc()
    // Run once to start Chrome before the workers create child targets.
    if err := chromedp.Run(allocCtx); err != nil {
        log.Fatalf("start Chrome: %v", err)
    }

    jobs := make(chan string)
    results := make(chan result)
    var wg sync.WaitGroup
    for i := 0; i < *workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for rawURL := range jobs {
                results <- capture(context.Background(), allocCtx, rawURL, *outDir)
            }
        }()
    }
    go func() {
        for _, raw := range urls {
            jobs <- raw
        }
        close(jobs)
        wg.Wait()
        close(results)
    }()

    var failures []error
    for r := range results {
        if r.err != nil {
            log.Printf("FAIL %s: %v", r.input, r.err)
            failures = append(failures, fmt.Errorf("%s: %w", r.input, r.err))
            continue
        }
        log.Printf("OK   %s -> %s", r.input, r.file)
    }
    if len(failures) > 0 {
        log.Printf("%d of %d captures failed", len(failures), len(urls))
        // Keep successful output and return a failing process status for automation.
        log.Fatal(errors.Join(failures...))
    }
}

The context.Context parameter to capture is included to make cancellation policy easy to extend; this simple command-line version uses a background parent context and a 60-second deadline per URL. Adjust the deadline for the pages you capture. FullScreenshot with quality 100 produces PNG according to chromedp’s API documentation. It captures the whole document and can override device emulation settings.

3. Run it

go run . -workers 3 -out shots https://example.com https://go.dev https://stripe.com

Each result is named from the host plus a short hash of the complete URL. The hash prevents paths and query strings from producing colliding filenames. If the same URL appears more than once, both jobs target the same output name; deduplicate inputs or add a unique job index if repeated captures should be retained separately.

4. Choose the capture type

Need chromedp action Considerations
Visible viewport chromedp.CaptureScreenshot(&image) Captures the current browser viewport after navigation.
Entire page chromedp.FullScreenshot(&image, 100) Quality 100 selects PNG; other quality values select JPEG. Full-page capture overrides device emulation settings.
One element chromedp.Screenshot(selector, &image, chromedp.NodeVisible) Useful for a component; the selector must match an element that can be captured.

For viewport capture, replace the full-page action with:

chromedp.CaptureScreenshot(&image)

For an element capture, use a selector appropriate to the target page:

chromedp.Screenshot("main article", &image, chromedp.NodeVisible)

5. Wait for the page you actually need

chromedp.Navigate waits for page load, but that does not mean every site’s asynchronous content, client-side rendering, or lazy-loaded images are ready. For dynamic sites, wait for an application-specific element or state before taking the screenshot. For example, add a visible-element wait between navigation and capture:

chromedp.Navigate(rawURL),
chromedp.WaitVisible("main article", chromedp.ByQuery),
chromedp.FullScreenshot(&image, 100),

Choose a selector that indicates the content of interest is present. Waiting for a generic page element can still capture an intermediate state if the site fills it asynchronously. If the page has a known readiness signal, use that rather than an arbitrary fixed sleep.

6. Tune concurrency without guessing

The worker pool caps active capture jobs. Start with a small value such as 2 or 3, then raise it gradually while measuring completion time, memory, CPU, browser stability, and failure rate on representative URLs. These are starting points for measurement, not universal optimal values. The reviewed chromedp documentation does not define a safe tab count or throughput guarantee.

  • Lower the worker count when Chrome becomes unstable, the host runs short on memory, or target sites throttle requests.
  • Use a higher count only when measurements show spare capacity and acceptable error rates.
  • For very large URL lists, keep the bounded queue pattern; do not create one goroutine and browser task per URL all at once.
  • If pages need shared cookies or session state, explicitly design how state is initialized and isolated. Independent targets avoid shared navigation state, but do not assume application-specific session behavior without checking it.
  • Separate browser processes can provide stronger failure isolation, at the cost of additional startup and resource use. Choose based on the failure containment and operating cost required by the workload; the sources provide no numeric performance comparison.

7. Reliability, cleanup, and batch behavior

The example lets successful screenshots survive failures elsewhere in the batch and exits nonzero if any URL fails, which is useful for automation. If the whole batch should stop on its first failure, create a cancellable parent context, cancel it on the first error, and pass it into each job’s timeout context. If partial results matter, keep the current per-URL reporting behavior.

Each job cancels its chromedp target context and deadline. The root browser context is canceled when main returns. Browser connection loss or Chrome process termination can cancel chromedp contexts; report those as browser-level failures and decide whether to retry. Limit retries to transient failures, use a small retry count with backoff, and avoid retrying permanent page errors indefinitely. Ensure the process is shut down cleanly in your deployment environment; chromedp documents context cancellation and Linux child-process cleanup behavior.

For production use, consider adding structured logs, a maximum batch size, graceful shutdown on OS signals, and atomic file writes (write to a temporary file then rename) if downstream consumers may read outputs during capture. These are operational choices rather than chromedp guarantees.

8. Common problems

Symptom Likely cause What to do
ErrInvalidContext A browser action is running with a context that was not created by chromedp.NewContext. Create a chromedp context for the browser or target and pass it to chromedp.Run, chromedp.Do, or chromedp.Call.
Context canceled or deadline exceeded The per-page deadline expired, Chrome closed, or its connection was lost. Check the URL and browser process, adjust the deadline for the page, and inspect whether cancellation is happening across multiple jobs. Retry only transient failures.
Blank or incomplete screenshot The initial load completed before client-side content or lazy images were ready. Wait for a page-specific selector or readiness condition, then capture. Confirm that the selected capture mode matches the desired viewport or full document.
Some URLs fail only at higher worker settings The workload may exceed available local resources or trigger target-site throttling. Reduce -workers, then increase gradually while tracking resource use and errors. No fixed tab count works for every workload.
Two outputs overwrite each other The output naming rule produced the same path, commonly because duplicate URLs were submitted. Deduplicate the input or add a unique sequence number to each job’s filename.
Missing or unexpected device emulation FullScreenshot overrides device emulation settings. Use viewport capture when emulation is essential, or configure the capture flow with the full-page behavior in mind.

9. Cost and performance notes

With self-hosted chromedp, there is no per-screenshot API charge in this implementation, but you pay for the machine, browser operation, and engineering time needed to run and maintain the workers. Increasing concurrency can shorten a batch only while the machine and destination sites can handle the extra work. Measure the complete workload, including browser startup, page readiness, screenshot encoding, and file writes. This article makes no throughput or memory benchmark claim.

10. Or skip the browser setup

If you want the screenshots without managing Chrome and its worker lifecycle, ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. Its one-call API accepts a URL and returns an image or PDF; see the API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp
import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);
  • Cookie and consent banners, newsletter popups, and chat widgets are removed before the shot; each cleanup step can be turned off.
  • Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed; responses identify the page verdict and billing status in headers.
  • An MCP server lets AI agents using Claude, Cursor, or another MCP client take screenshots.
  • The free plan includes 1,000 screenshots a month with no card. Paid plans start at $5 for 3,000 screenshots; every feature is available on every plan.

Sign up for 1,000 free screenshots a month, with no card.

FAQ

How many Chrome tabs can I screenshot at once?

There is no universal number. Set a worker limit and benchmark it on the sites, browser version, and machine you will use.

Does navigation guarantee that a page is ready for a screenshot?

It waits for page load, but asynchronous content may appear later. Wait for a page-specific readiness signal when the screenshot depends on it.

Can I capture just the visible screen instead of the whole page?

Yes. Use CaptureScreenshot for the viewport, FullScreenshot for the full document, or Screenshot for a selected element.

Can I use multiple pages in one browser context?

Playwright documents multiple pages within a browser context. That is a different tooling model; this chromedp example gives each job its own target context. The cited sources do not establish a performance winner between these approaches.