How to capture a website after it finishes loading in VisualScraper

Short answer: wait for the page state that proves the content you need is ready. In VisualScraper, look for a selector wait, JavaScript condition, page lifecycle event, network-idle option, or bounded delay if your version offers it. Exact VisualScraper controls and timeout limits could not be verified, so check the documentation for your version before relying on a particular option.

A page can finish its initial navigation before an application has fetched data or rendered it. Choose a readiness signal tied to the target content instead of assuming that the page’s first load event means everything is ready.

Choose a wait signal

Use the narrowest signal that tells you the content in the screenshot is ready. A generic page shell may appear before the data you care about.

Signal	What it tells you	Best fit	Limitation
`DOMContentLoaded`	The initial HTML document has been parsed.	You need an early document milestone.	Later scripts can still add or populate content. MDN: DOMContentLoaded
`load`	The document and dependent resources have completed loading.	Images and other page resources matter to the capture.	It does not prove that application data or hydration is complete. MDN: load event
Network idle	Network activity has been quiet for the tool’s defined interval.	Content appears after finite requests and the page becomes quiet.	Persistent requests may prevent quiet; quiet network activity does not prove the right content rendered.
CSS selector or JavaScript condition	A particular element or state is present.	You can identify a stable marker for the content you need.	The marker must indicate completed content, not just a placeholder.
Fixed delay	A chosen amount of time has passed.	No reliable readiness marker is available.	May be too short on a slow run and waste time on a fast one.

Browser automation tools document selector and condition waits as ways to target readiness more precisely: see shot-scraper’s usage guide and capture-website’s documentation. These examples explain general approaches; they do not confirm that VisualScraper exposes the same controls.

Set up a reliable capture

Identify the actual output. Confirm whether VisualScraper is capturing a screenshot, rendered HTML, or extracted data. Their capture points may differ.
Find a readiness marker. Choose an element or state that appears only once the content you need is present. Prefer a result row, heading, or completed-state element over a generic app container.
Use the matching wait control. If your VisualScraper version offers a selector or condition wait, use it. Otherwise choose a lifecycle event or network-idle option based on what the page does. Check that version’s documentation for exact labels and timeout behavior.
Handle interaction-triggered content. If the site reveals content after scrolling or clicking, perform that action first, then wait for the resulting content. A wait cannot substitute for the interaction that triggers loading.
Use a bounded delay only as a fallback. Start with a short delay and inspect captures from both fast and slow runs. Adjust based on observed results rather than treating one duration as universal.
Verify the output. Check that the target content appears in the screenshot. If rendered HTML is available, inspect it to confirm that the expected content or image URLs exist.

For content intended to load when visible, Google recommends that it load when visible without requiring user actions such as scrolling or clicking; that guidance concerns search crawling, not VisualScraper. For a capture workflow, scrolling or clicking can still be necessary when the specific page only reveals content after that interaction. See Google Search Central’s lazy-loading guidance.

Example wait patterns in browser automation

The following are illustrative patterns for common browser automation libraries, not VisualScraper configuration instructions. Use them only if you are implementing the capture in your own browser code. The selector and API names are specific to the libraries shown.

Playwright: wait for a target element

import { chromium } from 'playwright';

const browser = await chromium.launch({ headless: true });
const page = await browser.newPage({ viewport: { width: 1440, height: 1000 } });
try {
  await page.goto('https://example.com/dashboard', { waitUntil: 'domcontentloaded' });
  await page.locator('[data-testid="report-ready"]').waitFor({
    state: 'visible',
    timeout: 15000,
  });
  await page.screenshot({ path: 'capture.png', fullPage: true });
} finally {
  await browser.close();
}

Playwright: wait for an application state

await page.goto('https://example.com/dashboard', { waitUntil: 'domcontentloaded' });
await page.waitForFunction(() => {
  return document.querySelector('[data-testid="report-ready"]')?.textContent?.trim().length > 0;
}, { timeout: 15000 });
await page.screenshot({ path: 'capture.png', fullPage: true });

Choose a real marker from the target page. A selector that exists before its data is filled can still produce an early capture. Set a finite timeout so a missing marker becomes a diagnosable failure instead of an indefinitely stalled job.

Why a page may still look unfinished

JavaScript hydration: the initial HTML is present, but scripts have not yet populated or activated the visible application.
Asynchronous data: an API request fills in content after navigation. A lifecycle event can fire before that data arrives.
Lazy loading: an image or section loads only when it approaches the viewport. Scroll it into view if the capture flow needs to trigger it, then wait for the element or image to finish.
Interaction-gated content: a tab, menu, consent choice, or button must be activated before the target content exists.
Persistent network activity: analytics, polling, or streaming connections may keep a network-idle wait from completing. Use a content-specific signal when possible.
Weak readiness selector: the chosen element may represent a loading shell rather than the final result. Select a marker that changes only when the needed data is ready.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. One GET request returns an image or PDF. Its capture flow accepts cookie or consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before the shot; each cleanup step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. AI agents can use its MCP server tools: take_screenshot, get_page_info, and capture_pdf.

For a basic capture, save the response body as an image:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', res);

See the ScreenshotNeo API documentation for request options. It supports full-page shots, CSS selector capture, wait-for-selector, delay and network-idle waits, as well as custom JavaScript and CSS. You can also set viewport, device preset, dark mode, cookies, headers, user agent, timezone, geolocation, and caching. Images can be PNG, JPEG, or WebP; PDF capture supports paper size, margins, landscape, and page ranges. Async jobs, signed webhooks, bulk capture of up to 100 URLs per call, signed links for public image tags, and a usage API are available. The listed plans are 1,000 shots per month free with no card, Starter $5 for 3,000, Growth $15 for 15,000, Pro $39 for 60,000, Scale $99 for 250,000, and Business $249 for 1,000,000; yearly billing gives two months free, and every feature is on every plan.

Create a free ScreenshotNeo account for 1,000 screenshots a month with no card.

Troubleshooting

Symptom	Likely cause	What to try
Screenshot shows a spinner or skeleton	Capture happened at navigation completion, or the wait targets a shell.	Wait for a selector or condition that identifies the completed content. Confirm the marker does not exist during loading.
Some sections or images are missing	They load only when visible or after scrolling.	Scroll the relevant region into view, then wait for the content. Inspect rendered HTML when available.
Network-idle never completes	The page keeps connections or requests active.	Replace network-idle with a target selector/state wait, if available.
Wait succeeds but content is still stale or empty	The selector exists before data is populated, or the app updates it later.	Use a more specific condition, such as non-empty text or a completed status, and verify the capture output.
Content appears only after a click	The page requires an interaction to reveal or request it.	Trigger the required control before waiting for the resulting content.
Capture times out	The chosen condition never becomes true, the page failed, or the timeout is too short.	Check that the selector is correct and reachable, inspect whether the page loaded, and use a finite but suitable timeout for observed runs.
VisualScraper option name is unclear	Controls can vary by version, and its exact settings were not verified for this guide.	Check the documentation for the installed version; treat the lifecycle, selector, condition, and delay concepts here as general guidance.

Performance and reliability

Prefer a specific readiness condition. It can avoid waiting for unrelated page activity and reduce captures taken too early.
Keep waits bounded. A condition that never appears should fail within a known limit so the job can be diagnosed or retried.
Use network-idle selectively. It is useful when requests settle, but persistent background traffic can make it unreliable.
Make retries safe. For batch capture, record the URL, wait strategy, timeout, and failure reason so a retry can target the actual issue. Avoid retrying endlessly on a page that consistently fails.
Account for page behavior. Scrolling, clicking, consent handling, image loading, and full-page capture can all change when content becomes ready and how long a capture takes.
Do not choose a universal delay from an example. A timeout shown in another tool’s documentation is an example, not a recommended duration for VisualScraper or every site.

Frequently asked questions

Does DOMContentLoaded mean the website is fully rendered?

No. It signals that the initial document has been parsed. JavaScript can still fetch and render content afterward.

Is network-idle always the safest wait?

No. It may never occur on pages with persistent requests, and network quiet alone does not confirm that the target content is present.

Can I use these examples as VisualScraper settings?

No. The code shows general browser automation patterns. VisualScraper’s exact menu names and feature availability depend on its own version and documentation.

What if I cannot identify a readiness marker?

Use a bounded delay as a fallback, compare captures across different loading conditions, and refine the wait if you find a stable signal later.