Error Handling for Screenshot APIs in Ruby

Reliable screenshot handling starts by separating four cases: your request is invalid, your credentials or quota are wrong, the target site failed, or the screenshot provider failed. Parse the provider’s structured JSON error, retain both its error code and HTTP status, and retry only documented transient failures with bounded exponential backoff.

Successful screenshot responses are binary image or PDF data. Error responses are normally JSON. Your Ruby client should therefore check the HTTP status and content type before parsing, set explicit connection and read timeouts, keep API keys outside source code, and log enough context to diagnose failures without exposing secrets.

Recommended error-handling flow

1. Build and validate the request locally before sending it.
2. Send credentials over HTTPS and use explicit Ruby timeouts.
3. For a 2xx response, verify the content type and save the binary body.
4. For a 4xx response, parse the structured error and fix the request, credentials, permissions, quota, or target access. Do not blindly retry.
5. For a 5xx response, retry only when the provider documents the condition as transient.
6. When the error says the target returned an HTTP status, inspect that target status separately from the provider status.
7. Use a maximum attempt count, exponential backoff, and jitter.
8. Record the provider code, provider HTTP status, target status when available, request identifier, and elapsed time in structured logs.

Ruby implementation with structured errors

The wrapper below uses Ruby’s standard library. It preserves the provider error code and details while exposing a consistent exception to the rest of your application.

require 'json'
require 'net/http'
require 'uri'

class ScreenshotApiError < StandardError
  attr_reader :status, :code, :details, :target_status

  def initialize(status:, code:, message:, details: {}, target_status: nil)
    @status = status
    @code = code
    @details = details
    @target_status = target_status
    super(message)
  end
end

def fetch_screenshot(uri, access_key:, open_timeout: 5, read_timeout: 60)
  request = Net::HTTP::Get.new(uri)
  request['X-Access-Key'] = access_key

  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = (uri.scheme == 'https')
  http.open_timeout = open_timeout
  http.read_timeout = read_timeout

  response = http.request(request)
  return response.body if response.is_a?(Net::HTTPSuccess)

  payload = JSON.parse(response.body) rescue {}
  error = payload['error'].is_a?(Hash) ? payload['error'] : payload
  target_status = error['target_status'] || error['upstream_status']

  raise ScreenshotApiError.new(
    status: response.code.to_i,
    code: error['code'] || error['error_code'] || 'unknown_error',
    message: error['message'] || error['error_message'] || 'Screenshot request failed',
    details: error,
    target_status: target_status
  )
end

endpoint = URI(ENV.fetch('SCREENSHOT_ENDPOINT'))
image = fetch_screenshot(
  endpoint,
  access_key: ENV.fetch('SCREENSHOT_ACCESS_KEY'),
  open_timeout: 5,
  read_timeout: 60
)
File.binwrite('shot.png', image)

Use an environment-backed secret such as SCREENSHOT_ACCESS_KEY; do not commit keys, place them in URLs that may be logged, or print full request headers. The wrapper intentionally returns only successful response bytes and raises for every non-2xx status.

Parsing JSON errors safely

Providers do not all use the same JSON envelope. Some return {"error":{"code":"...","message":"..."}}; others put code and message at the top level. The fallback extraction in the wrapper handles both shapes. Preserve the complete parsed object in restricted logs, but return a short actionable message to an end user.

Information	Why keep it
Provider HTTP status	Separates request failures (4xx) from potentially transient failures (5xx).
Provider error code	Usually identifies the exact corrective action.
Human-readable message	Useful for operators and support tickets.
Target HTTP status	Shows whether the destination site rejected or failed the navigation.
Request ID and timing	Allows provider support and your own logs to correlate incidents.

Retries with bounded exponential backoff

Retry only a documented transient provider failure, temporary storage failure, or a target 429 after honoring its rate-limit guidance. Never retry an invalid key, malformed option, invalid selector, missing permission, or a permanently unresolved hostname on every request.

def transient_error?(error)
  return true if error.status >= 500 && error.status <= 599

  retryable_codes = %w[
    internal_application_error
    temporary_storage_error
    network_error
  ]
  return true if retryable_codes.include?(error.code)

  error.code == 'host_returned_error' && [429, 502, 503, 504].include?(error.target_status.to_i)
end

def fetch_with_retries(uri, access_key:, attempts: 4, open_timeout: 5, read_timeout: 60)
  attempt = 0

  begin
    attempt += 1
    return fetch_screenshot(
      uri,
      access_key: access_key,
      open_timeout: open_timeout,
      read_timeout: read_timeout
    )
  rescue ScreenshotApiError => error
    raise unless transient_error?(error) && attempt < attempts

    # 1, 2, 4 seconds plus up to 250 ms of jitter.
    delay = [2 ** (attempt - 1), 30].min + rand * 0.25
    sleep(delay)
    retry
  rescue Net::OpenTimeout, Net::ReadTimeout => error
    raise if attempt >= attempts

    sleep([2 ** (attempt - 1), 30].min + rand * 0.25)
    retry
  end
end

Keep the total retry budget below the caller’s deadline. In a web request, four attempts with long render timeouts can exceed the browser or serverless function timeout. Pass an idempotency key when a provider supports one, especially if a failed response might hide a completed capture.

What each class of failure means

Code or condition	Likely cause	Action	Retry?
access_key_required, access_key_invalid, invalid signature	Missing, incorrect, expired, or wrongly signed credentials.	Fix secret injection, account configuration, or signing inputs.	No
request_not_valid	Malformed URL, unsupported option, invalid type, or missing required parameter.	Validate and correct the request.	No
Selector or element error	The selector is invalid or the element never appeared.	Check the selector, wait condition, and page state.	No, unless the page is expected to change and you issue a new request with a better wait strategy.
name_not_resolved	DNS failure, typo, private hostname, or DNS propagation delay.	Verify the hostname and public reachability. Retry after a real DNS change, not continuously.	Conditional
network_error	Connection reset, blocked automation, or unreachable target.	Check target availability, firewall rules, and whether automated access is allowed.	Conditional
host_returned_error	The target returned an HTTP error during navigation.	Inspect the target status. Handle target authentication, rate limits, or outages separately.	Depends on target status
timeout_error	Navigation or rendering exceeded a provider limit.	Reduce page weight or waits, tune rendering timeouts, or use an asynchronous job/webhook.	Only after changing the cause or when the timeout is known to be transient
internal_application_error or temporary storage failure	Provider-side transient problem.	Retry with bounded backoff and escalate if persistent.	Usually yes
HTTP 429 from the provider	Your API quota or provider rate limit was exceeded.	Honor Retry-After when present, reduce concurrency, and check quota.	After waiting

Distinguish provider failures from target-site failures

A provider response such as HTTP 502 does not necessarily mean the target returned 502. If the structured error includes a target status, log both values:

rescue ScreenshotApiError => error
  logger.error(
    event: 'screenshot_failed',
    provider_status: error.status,
    provider_code: error.code,
    target_status: error.target_status,
    details: error.details
  )

  if error.code == 'host_returned_error'
    # Apply the target site's policy, authentication, or rate-limit handling.
  elsif error.status >= 500
    # Apply bounded provider retry policy.
  else
    # Correct the request or credentials.
  end
end

A target 401 or 403 generally needs authorization or a policy decision. A target 429 needs rate-limit handling. Target 502, 503, and 504 may be transient and can be retried with backoff. A provider 401 caused by your key should never be treated like a target 401.

Timeouts and slow pages

• Set Ruby’s open_timeout for DNS and connection establishment.
• Set read_timeout longer than the provider’s expected render duration, while keeping it below your job or serverless deadline.
• Check the provider’s navigation or rendering timeout option separately from the client timeout.
• Reduce unnecessary delays, large assets, third-party scripts, and wait conditions.
• Use a selector wait only when the selector represents content required in the final image.
• For long or variable pages, use asynchronous jobs and signed webhooks when available.
• Use a proxy only when you are authorized to access the target and the evidence points to network or geo restrictions.

When a page can take 45 seconds to render, a Ruby read timeout of 10 seconds guarantees a client-side failure even if the provider is still working. Conversely, an unlimited read timeout can exhaust worker threads. Set a deadline for the whole operation and leave room for retries only when your request path can tolerate them.

Handling 429 responses and concurrency

Apply backoff at the queue or scheduler as well as inside an individual request. A process that launches 100 workers, each retrying four times, can amplify a rate-limit incident. Use a concurrency limit, honor Retry-After, add jitter, and expose queue depth and retry counts as metrics.

def retry_after_seconds(response)
  value = response['retry-after']
  return nil unless value

  Integer(value, exception: false)
end

If the provider reports quota exhaustion rather than a temporary rate limit, wait for the documented reset or change the plan. Do not retry every queued job immediately.

Request validation before the network call

• Require an absolute http or https URL.
• Reject credentials embedded in the target URL unless the provider explicitly supports them.
• Validate viewport, output format, scale, timeout, and page-range values against provider limits.
• Check that CSS selectors are non-empty and escaped correctly.
• Set a maximum URL length and normalize redirects if your application accepts user input.
• Keep provider access keys in environment variables or a secret manager.
• Use HTTPS for both your application and the screenshot API.

Operational logging and observability

Log a request identifier, provider status, provider code, target status, elapsed milliseconds, output format, and retry attempt. Redact access keys, cookies, authorization headers, and sensitive target query parameters. Separate counters for invalid requests, authentication failures, target errors, provider 5xx responses, timeouts, and successful captures make alert thresholds meaningful.

For privacy and security, avoid storing full screenshots or HTML in error logs by default. If you need a diagnostic artifact, apply a retention policy and restrict access.

Performance, reliability, and cost considerations

• Full-page captures and pages with lazy-loaded images require more rendering time and memory than a fixed viewport.
• Waiting for network idle can be slower than waiting for a specific content selector on pages with analytics or live connections.
• Caching identical URLs can reduce latency and spend when the page’s freshness requirements allow it.
• Retries increase both latency and provider usage when the provider bills attempts; confirm the provider’s billing rules.
• Asynchronous capture prevents a web request from holding a worker during a long render.
• Use a queue for bulk work and cap concurrency to protect both your application and target sites.
• Measure p50 and p95 end-to-end latency, timeout rate, target-status distribution, and retry success rate.

Or skip the browser setup

ScreenshotNeo provides a website screenshot API and MCP server. It accepts one GET request and returns PNG, JPEG, WebP, or PDF. Cookie and consent banners, newsletter popups, and chat widgets are removed before capture; bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and each response reports the result with X-Page-Verdict and X-Billed headers. Its MCP server includes take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients.

See the ScreenshotNeo API documentation for the request options.

curl -G 'https://api.screenshotneo.com/v1/shot' \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

import requests

r = requests.get(
    'https://api.screenshotneo.com/v1/shot',
    params={'access_key': 'YOUR_API_KEY', 'url': 'https://stripe.com'},
    timeout=90,
)
r.raise_for_status()
open('shot.webp', 'wb').write(r.content)

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot failed: ${res.status}`);
require('fs').writeFileSync('shot.webp', Buffer.from(await res.arrayBuffer()));

ScreenshotNeo supports full-page and element captures, dark mode, device presets, retina scale, custom CSS and JavaScript, click and wait actions, request blocking, headers, cookies, user agents, authorization, timezone, geolocation, transparent backgrounds, resizing, selectable cache TTLs, signed image links, asynchronous jobs with signed webhooks, bulk capture for up to 100 URLs per call, a usage API, and an OpenAPI specification. Parameter names used by other screenshot APIs also work, which can simplify migration.

Plans include 1,000 screenshots per month free with no card; paid plans start at $5 for 3,000 shots. Higher plans are $15 for 15,000, $39 for 60,000, $99 for 250,000, and $249 for 1,000,000; yearly billing gives two months free, and every feature is available on every plan.

Create a free ScreenshotNeo account and start with 1,000 screenshots a month without a card.

Troubleshooting checklist

1. JSON parse failure: confirm the response is an error before parsing; successful images and PDFs are binary.
2. Every request returns 401: verify the environment variable, key scope, signing inputs, and HTTPS endpoint.
3. Every request returns 400: validate URL encoding, option names, selector syntax, and required parameters.
4. Intermittent 5xx: capture the provider code and request ID, retry with a bounded budget, and check provider status information.
5. Target 403: determine whether authentication, robots policy, bot protection, or your authorization is responsible before changing proxy settings.
6. Target 429: honor the target’s rate limit, slow your queue, and avoid synchronized retries.
7. Ruby read timeout: compare client timeout, provider rendering timeout, page weight, and your outer job deadline.
8. Missing element: verify the selector in the final DOM and use a selector wait or a controlled delay.
9. DNS error: test resolution from the provider’s network and wait for DNS propagation after a real change.
10. Duplicate captures: add idempotency support if offered, or deduplicate jobs in your own queue.

FAQ

Should every 500 response be retried?

No. Retry only documented transient provider or storage failures, with a maximum attempt count. Persistent 500 responses need investigation.

Is a target 500 the same as a provider 500?

No. A target 500 means the destination site responded with an error during navigation. A provider 500 means the capture service failed. Their retry and escalation paths differ.

How should I handle a screenshot API timeout in a background job?

Set an outer job deadline, use bounded retries, reduce page work, and prefer asynchronous capture with a webhook when the provider supports it.

What should I store from an error?

Store the provider status, provider code, safe message, target status, request ID, attempt count, and elapsed time. Redact credentials, cookies, authorization headers, and sensitive URLs.

When is a proxy appropriate?

Only when you are authorized to access the target and diagnostics indicate a network, geo, or access-policy issue. A proxy does not fix invalid options or invalid credentials.