ScreenshotNeo
Sign in Get a free key

BlogHow-to

How to Capture a Lazy-Loaded Page After Network Idle with Playwright

Learn why network idle does not guarantee complete content, then scroll, wait for page-specific signals, and capture a reliable full-page screenshot with Playwright.

By the ScreenshotNeo team4 October 202610 min read

Short answer: Playwright’s networkidle state does not mean that all lazy-loaded content is present. Scroll the page or the correct nested container to trigger loading, wait for a signal tied to the content you need, then take a full-page screenshot. Use networkidle only as an optional settling hint.

Playwright defines networkidle as at least 500 ms with no network connections and explicitly discourages it as a test readiness condition. Lazy loading may begin only after scrolling, while polling or analytics can keep requests active even when the content you need is ready. Playwright Page API: waitForLoadState

1. Install Playwright

This example uses TypeScript, Chromium, and Playwright Test assertions. Create a project and install the browser:

mkdir lazy-page-capture
cd lazy-page-capture
npm init -y
npm install -D @playwright/test
npx playwright install chromium

Save the next example as capture.ts. Run it with a TypeScript runner such as npx tsx capture.ts; install that runner with npm install -D tsx. Alternatively, put the same logic in a Playwright Test file and run it with npx playwright test.

2. Scroll, verify content, and capture

Replace the example URL, feed selector, and loading indicator with selectors from the site. The sample assumes a nested scrolling feed and that a loading indicator is shown while each batch loads. Its bounded loop stops when the container reaches a stable end, or fails clearly if the configured step limit is reached without reaching the end.

import { chromium, expect } from '@playwright/test';

const browser = await chromium.launch();
const page = await browser.newPage({ viewport: { width: 1440, height: 900 } });

try {
  await page.goto('https://example.com', {
    waitUntil: 'domcontentloaded',
    timeout: 30_000,
  });

  // Optional hint only. Pages with polling or streaming may never become idle.
  try {
    await page.waitForLoadState('networkidle', { timeout: 5_000 });
  } catch {
    console.log('Network did not become idle; continuing with content checks.');
  }

  // Replace these selectors and the readiness condition for the target site.
  const feed = page.getByTestId('feed');
  const cards = feed.locator('[data-testid="feed-card"]');
  const loading = page.getByTestId('loading-indicator');

  await expect(feed).toBeVisible();
  const maxSteps = 30;
  let previousCount = await cards.count();
  let stableEndChecks = 0;
  let reachedEnd = false;

  for (let step = 0; step < maxSteps; step++) {
    const before = await feed.evaluate(el => ({
      top: el.scrollTop,
      height: el.scrollHeight,
      viewport: el.clientHeight,
    }));

    await feed.evaluate(el => { el.scrollTop += el.clientHeight; });

    // A site-specific signal is stronger than a fixed sleep.
    await expect(loading).toBeHidden({ timeout: 5_000 });

    const currentCount = await cards.count();
    const after = await feed.evaluate(el => ({
      top: el.scrollTop,
      height: el.scrollHeight,
      viewport: el.clientHeight,
    }));
    const atEnd = after.top + after.viewport >= after.height - 1;
    const contentUnchanged = currentCount === previousCount;
    const heightUnchanged = after.height === before.height;

    if (atEnd && contentUnchanged && heightUnchanged) {
      stableEndChecks++;
      if (stableEndChecks >= 2) {
        reachedEnd = true;
        break;
      }
    } else {
      stableEndChecks = 0;
    }
    previousCount = currentCount;
  }

  if (!reachedEnd) {
    throw new Error(`Feed did not reach a stable end within ${maxSteps} scroll steps`);
  }

  // Ensure a meaningful target is present before saving the artifact.
  await expect(cards.first()).toBeVisible();
  await page.screenshot({ path: 'page.png', fullPage: true });
} finally {
  await browser.close();
}

The feed selector, card selector, and loading indicator are deliberately site-specific placeholders: there is no universal selector or completion signal for lazy content. If the page is document-scrolling rather than a nested feed, use the page-level variant below.

3. Choose the right loading and scrolling signals

Goal Recommended approach Why
Start navigation domcontentloaded or the default load Choose based on what the page needs; data and UI can still appear later.
Wait for a particular result Locator assertions such as toBeVisible(), a count change, or a known completion marker It checks the outcome needed for the capture and retries while waiting.
Get a brief network-settling hint Optionally wait for networkidle with a short timeout It may help on some pages, but neither idle traffic nor a timeout proves content completeness.
Load an infinite feed Scroll in bounded steps and check content after each step Scrolling often triggers the next batch. Recheck height and item count as the feed grows.
Capture one artifact page.screenshot({ fullPage: true }) Captures the document’s full scrollable page after the desired content has loaded.
Visual regression test expect(page).toHaveScreenshot() Playwright Test waits for two consecutive screenshots to match before comparing with the baseline.

Playwright’s navigation guide notes that modern pages can fetch data lazily and populate UI after the load event. Readiness depends on the site and its framework. Playwright navigation guide

Document-level scrolling

If the browser window owns the scroll position, scroll the document and check a signal such as a new card count, a target image becoming visible, or a loading marker disappearing:

const previousCount = await page.locator('.card').count();
await page.evaluate(() => window.scrollBy(0, window.innerHeight));
await expect(page.locator('.card')).toHaveCount(previousCount + 1, { timeout: 5_000 });

That count assertion assumes exactly one item is added. If the site adds a variable batch, wait for a count greater than the previous count instead, using a retrying assertion or polling helper. You can also bring a known bottom element into view with locator.scrollIntoViewIfNeeded(). For a nested panel, change that panel’s scrollTop rather than scrolling the window. Playwright input and scrolling guide

Nested scrolling containers

When the feed itself has a scrollbar, locate it and change its scroll position in the page context:

const feed = page.locator('.feed-panel');
await feed.evaluate(el => { el.scrollTop += el.clientHeight; });

Use a stable test ID or accessible locator if the site provides one. A visible page element inside the feed is not necessarily evidence that the window is the container that must scroll.

Wait for a real content condition

Prefer an assertion that describes the needed result. Examples include a specific image becoming visible, a loading marker disappearing, an item count increasing, or an application-provided “all results loaded” marker appearing. Locators auto-wait and web-first assertions retry until the condition passes or times out. Avoid a fixed delay as the main readiness rule: it can be too short on a slow run and waste time on a fast one. Playwright Locator API

4. Handle feeds without a natural end

An infinite feed has no objectively complete full-page screenshot unless you define what “complete” means. Pick a stopping condition before capture, such as reaching a known item, collecting a target number of cards, finding an end marker, or reaching a maximum duration. Also cap the number of scrolls. Recalculate scrollHeight as content arrives; the initial page height is not necessarily the final one.

The bounded-loop pattern in the sample is an implementation strategy, not a universal Playwright guarantee. If a site’s feed can remain unchanged temporarily while a request is pending, include its actual loading or completion signal in the check. If it can append duplicate cards, check stable item IDs or a target element instead of relying only on count.

5. Capture in a Playwright Test

For visual regression, use Playwright Test’s screenshot assertion after loading the content. Keep the same browser, viewport, and environment for baseline generation and comparison because rendering can vary by operating system, browser version, and other environment details.

import { test, expect } from '@playwright/test';

test('lazy feed screenshot', async ({ page }) => {
  await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });

  const feed = page.getByTestId('feed');
  const cards = feed.locator('[data-testid="feed-card"]');
  await expect(feed).toBeVisible();

  // Adapt this scroll and readiness assertion to the site's feed behavior.
  await feed.evaluate(el => { el.scrollTop = el.scrollHeight; });
  await expect(page.getByTestId('all-results-loaded')).toBeVisible({ timeout: 10_000 });
  await expect(cards.first()).toBeVisible();

  await expect(page).toHaveScreenshot('lazy-feed.png', { fullPage: true });
});

toHaveScreenshot() is a Playwright Test assertion and is not a general-purpose screenshot method. It waits for consecutive screenshots to stabilize before comparing with its expectation. For a one-off image file, use page.screenshot(). Playwright visual comparisons

6. Frames, images, and other edge cases

  • Content inside an iframe: Locate the relevant frame with page.frameLocator('iframe-selector') and use its locators to wait for content. Scrolling may still need to target the document or scrollable element inside that frame. A full-page screenshot of the main page does not make a cross-origin frame’s contents available to main-page locators.
  • Native lazy images: Scrolling the image into view can trigger loading. Before capture, check that required images are complete, for example by evaluating img.complete && img.naturalWidth > 0 for the images you care about. Broken images need a separate error or fallback check.
  • Images that load after a card appears: Card visibility alone may be too early. Wait for the target image’s loaded state or for a site-provided signal that its media is ready.
  • Virtualized lists: Some interfaces remove off-screen cards from the DOM. A full-page screenshot cannot include items that the application has not rendered at once. Capture sections in separate screenshots, change the application’s page size if supported, or use a site-specific export.
  • Sticky headers: Full-page capture behavior and fixed elements can make a header appear repeated or overlap content. Inspect the result; if needed, hide or restyle the sticky element for the screenshot with an intentional test-only style.
  • Animations and changing content: Disable or stabilize animations and dynamic values for visual comparisons. Playwright Test screenshot assertions disable animations by default, but a regular screenshot may require deliberate page styling or app configuration.
  • Very tall pages: Browser and image dimensions impose practical limits. If the capture is too large or memory-heavy, capture sections and stitch or review them separately rather than assuming every browser can produce an arbitrarily tall image.

7. Troubleshooting

Symptom Likely cause Fix
waitForLoadState('networkidle') times out Polling, analytics, streaming, or other persistent network activity Use a short optional timeout, then wait for the page-specific visible outcome. Do not treat the timeout as proof that the page is complete.
The screenshot is missing cards or images The scroll targeted the wrong element, or the script captured before the next batch finished Find the actual scrollable container, scroll it, and verify a new item or loaded image before capturing.
The loop never ends The feed is unbounded, its height keeps growing, or the stop condition is wrong Set an explicit target and maximum steps or time. Report failure or partial completion when the bound is reached.
toBeHidden() passes immediately The loader was never shown, or the selected locator does not identify the active loader Wait for a new item, an increased count, or another site-specific completion signal; validate the selector.
The content count does not change The page loads several items at once, reuses existing nodes, or has not triggered loading Use a greater-than condition, item IDs, a target item, or a site-provided marker. Confirm that the correct container scrolled.
Frame content is not found The locator is scoped to the main document while the content is inside a frame Use frameLocator() for the relevant iframe and check the frame’s own scroll behavior.
Screenshot assertion differs between machines Browser, operating system, fonts, viewport, or dynamic page state differ Generate and compare baselines in a consistent environment and stabilize content before the assertion.
Capture is clipped or fails on a long page Page dimensions exceed practical browser or image limits, or the page is virtualized Capture bounded sections, or use an application export. A full-page option cannot render content the app has removed from the DOM.

8. Performance, reliability, and cost

Every scroll step, readiness check, and full-page render adds work. Keep the initial navigation state useful, use a content condition instead of a long arbitrary sleep, and set timeouts and step limits so a run cannot wait indefinitely. A large page can consume substantial browser memory and produce a large image; limit the capture to the content needed or split it into sections.

Reliability comes from checking what the screenshot must contain. Network state is a weak proxy for application readiness, and a successful screenshot call only means the browser created an image, not that the intended lazy content loaded. Log the final item count or target marker and fail explicitly when the limit is reached before the goal.

With Playwright, the main cost is running and maintaining the browser automation environment: browser installation, compute time, storage, and retries. The exact cost depends on where and how often it runs; no universal price or benchmark applies. If browser setup and cleanup are the part you want to avoid, ScreenshotNeo offers per-month plans beginning with 1,000 free screenshots and paid plans from $5 for 3,000. See its website screenshot API and documentation.

Or skip the browser setup

ScreenshotNeo captures a URL with one request. Cookie and consent banners are accepted like a visitor, then more than 60 known consent platforms, newsletter popups, and chat widgets are removed before the shot; each step can be turned off. Bot checks, blank pages, and failed loads are never billed. An MCP server lets AI agents use screenshot tools. The free plan includes 1,000 screenshots a month with no card, and paid plans start at $5 for 3,000. See the ScreenshotNeo API documentation.

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp
import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`ScreenshotNeo returned ${res.status}`);
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer())));

Sign up free for 1,000 screenshots a month with no card.

FAQ

Does fullPage: true trigger lazy loading?

It requests a screenshot of the full scrollable page; it does not guarantee that scrolling-triggered content has loaded. Trigger and verify the content first.

Should I always wait for networkidle?

No. It is optional and can time out on normal pages with ongoing requests. A page-specific assertion is the better readiness check.

Can Playwright capture an infinite page completely?

Only after you define what complete means for that feed. Choose an end condition or a bounded target; an unbounded feed has no final state.

When should I use toHaveScreenshot() instead of page.screenshot()?

Use the assertion for a visual regression test with a baseline. Use page.screenshot() when you need an image artifact without baseline comparison.

Keep reading

7 min read

Full-Page Capture Chrome Plugin

Capture an entire webpage in Chrome with extensions, understand their limits, and automate clean full-page screenshots with ScreenshotNeo.

7 min read

Facebook Open Graph Debugger

Use Facebook’s Open Graph debugger to refresh stale link previews, verify metadata, and fix incorrect titles, descriptions, and images.