Capture Web Page Sections That Load After Scrolling Using Puppeteer

To capture sections that load as you scroll in Puppeteer, scroll the page in controlled steps, wait after each step for a page-specific signal that the new content is ready, and then take a full-page or element screenshot. page.evaluate() can run an asynchronous scroll routine in the page context; page.waitForFunction() can wait for a selector, content count, or loading state. Use a maximum scroll count or another clear stop condition for pages that keep extending.

There is no universal lazy-load selector or wait duration: the right readiness check depends on the site. Puppeteer provides the scrolling, waiting, and screenshot APIs, while the condition that means “this section is ready” must come from the page you are capturing. See the official Puppeteer screenshots guide, Page API, and page interactions guide.

1. Understand the capture sequence

A reliable capture has four steps:

1. Navigate to the page.
2. Scroll far enough to trigger its lazy-loading behavior.
3. Wait for an observable signal that the intended content has loaded.
4. Capture the whole page or the specific section.

A screenshot call does not itself prove that content triggered by scrolling has finished loading. Likewise, scrolling an element into view does not guarantee that its images, data, or other visual content are ready.

Goal	API	Use when
Capture the document	page.screenshot()	The result should include the whole page.
Capture one known section	ElementHandle.screenshot()	The result should isolate a particular element.
Wait for a page-specific condition	page.waitForFunction()	You can express readiness as a selector, count, or state.
Wait for network activity to quiet	page.waitForNetworkIdle()	Network quiet is a useful supporting signal for this site.

2. Set up Puppeteer

Install Puppeteer in a Node.js project:

npm install puppeteer

Save the following as capture.js. It is runnable as a general template; replace the URL, selector, and readiness condition for the target page. The example counts matching sections after each scroll and stops when the count no longer grows at the bottom, subject to a maximum number of scrolls.

const puppeteer = require('puppeteer');

async function capture() {
  const url = 'https://example.com';
  const sectionSelector = '.content-section'; // Replace for the target site.
  const maxScrolls = 30;

  const browser = await puppeteer.launch({ headless: true });
  try {
    const page = await browser.newPage();
    await page.setViewport({ width: 1440, height: 900 });
    await page.goto(url, { waitUntil: 'domcontentloaded' });

    let previousCount = await page.locator(sectionSelector).count();
    let stableAtBottom = 0;

    for (let attempt = 0; attempt < maxScrolls; attempt++) {
      await page.evaluate(() => {
        window.scrollBy(0, Math.floor(window.innerHeight * 0.8));
      });

      // Replace this predicate with a signal that means new content is ready.
      await page.waitForFunction(
        (selector, oldCount) =>
          document.querySelectorAll(selector).length > oldCount ||
          document.querySelectorAll('[aria-busy="true"]').length === 0,
        { timeout: 5000 },
        sectionSelector,
        previousCount
      ).catch(() => {});

      // A small pause can allow visual changes to paint after the predicate.
      await new Promise(resolve => setTimeout(resolve, 150));
      const currentCount = await page.locator(sectionSelector).count();
      const atBottom = await page.evaluate(() =>
        window.scrollY + window.innerHeight >= document.documentElement.scrollHeight - 2
      );

      if (currentCount > previousCount) {
        stableAtBottom = 0;
      } else if (atBottom) {
        stableAtBottom++;
      }
      previousCount = currentCount;

      if (stableAtBottom >= 2) break;
    }

    await page.screenshot({ path: 'page.png', fullPage: true });

    // For a specific section instead, wait for its content and screenshot it:
    // const section = await page.$(sectionSelector);
    // if (!section) throw new Error(`Section not found: ${sectionSelector}`);
    // await section.screenshot({ path: 'section.png' });
  } finally {
    await browser.close();
  }
}

capture().catch(error => {
  console.error(error);
  process.exitCode = 1;
});

Run it with node capture.js. The sample predicate is deliberately generic: an empty set of [aria-busy="true"] elements might say nothing about whether the specific content you need has appeared. Prefer a condition tied to the target section or its data.

3. Choose a readiness condition

Wait for a known section or element

If a section appears only after scrolling, wait for its selector to become visible or present. For example, after scrolling to a known section:

await page.evaluate(() => window.scrollBy(0, window.innerHeight * 0.8));
await page.waitForSelector('#pricing-section', { visible: true, timeout: 10000 });

For an existing element whose content changes, checking only that it exists may return too early. Wait for a meaningful property, such as non-empty text, a changed item count, or a loading marker disappearing.

Wait for a count or state change

waitForFunction() waits until its supplied function returns a truthy value. Capture the old state before scrolling, then wait for the state to change:

const before = await page.locator('.result-card').count();
await page.evaluate(() => window.scrollBy(0, window.innerHeight * 0.8));
await page.waitForFunction(
  oldCount => document.querySelectorAll('.result-card').length > oldCount,
  { timeout: 10000 },
  before
);

When the page replaces existing nodes instead of appending them, use a different observable signal, such as a changed label, a loading indicator becoming hidden, or a known image reaching complete with a nonzero natural width.

Use network idle as a supporting signal

Puppeteer exposes page.waitForNetworkIdle(). It can help when a page’s relevant requests settle after scrolling, but network quiet does not necessarily mean the intended content rendered, and some pages maintain background requests. Pair it with a page-specific condition when possible.

await page.evaluate(() => window.scrollBy(0, window.innerHeight * 0.8));
await page.waitForNetworkIdle({ timeout: 10000 }).catch(() => {});
await page.waitForFunction(
  () => document.querySelector('#next-section')?.textContent.trim().length > 0,
  { timeout: 10000 }
);

Scroll with a locator for a known target

Puppeteer recommends locators for selecting and interacting with elements. A locator can scroll a target into view as part of an interaction. This is useful when the section is known, but it remains separate from waiting for the section’s lazy content to finish rendering.

const target = page.locator('#target-section');
await target.scroll();
await page.waitForFunction(
  () => document.querySelector('#target-section')?.querySelectorAll('img').length > 0,
  { timeout: 10000 }
);
await target.screenshot({ path: 'target.png' });

4. Capture a full page or one section

Full-page screenshot

After your scroll routine has triggered the page’s loading behavior and its readiness checks have passed, capture the document:

await page.screenshot({ path: 'full-page.png', fullPage: true });

Full-page capture describes the screenshot area; it does not replace the scroll-and-wait sequence when the page only loads content as the viewport moves.

Element screenshot

For one section, obtain its handle and capture it after its content is ready:

const element = await page.$('#target-section');
if (!element) throw new Error('Target section was not found');
await element.screenshot({ path: 'section.png' });

Puppeteer’s element screenshot method tries to scroll a hidden element into view by default. Still wait for lazy-loaded content separately; the automatic scroll is not a readiness guarantee.

5. Adapt for infinite scroll, images, and dynamic pages

• Infinite scroll: use a maximum scroll count, a maximum elapsed time, or both. Stop when a known end marker appears or when the relevant content count remains stable at the bottom for multiple checks.
• Lazy images: scrolling can trigger image requests. If image completion matters, check the specific images’ complete and naturalWidth properties before capture.
• Virtualized lists: some pages remove off-screen rows from the DOM. A full-page screenshot may not contain items that were previously rendered and then removed. Capture each needed region during traversal or use the page’s own export/data interface where appropriate.
• Sticky headers and animations: sticky elements may appear repeatedly in stitched full-page output, and animations can make captures inconsistent. If the page allows it, use the appropriate CSS or application state before capture and keep the viewport consistent.
• Authentication and consent: the page may need a logged-in session or consent state to reveal content. Configure that state before scrolling and avoid exposing secrets in saved scripts or logs.

6. Troubleshooting

Symptom	Likely cause	Fix
Lower sections are missing	The script captured before scrolling triggered loading, or the wait checked the wrong thing.	Scroll in steps and wait for a target-specific count, selector, or state change before capture.
The wait times out	The predicate never becomes true, the selector is wrong, or the page does not append content.	Inspect the live DOM and choose a condition that matches how this page updates. Handle an expected timeout explicitly rather than treating it as success.
The loop never ends	The page keeps extending or the bottom condition is unstable.	Set a maximum iteration count and stop on a known end marker or stable no-growth condition.
Network-idle wait times out	Background requests keep the page active.	Use a page-specific readiness condition; use network idle only if it fits the site.
The element screenshot is blank or incomplete	The target exists, but its content has not loaded, or the selected element is not the content container.	Wait for the actual content and verify the selector before taking the element screenshot.
Images are missing	Image requests were triggered late, failed, or had not completed before capture.	Scroll the images into loading range and wait for the relevant image completion state.
Capture differs between runs	Dynamic content, animations, viewport differences, or timing change the rendered page.	Fix the viewport, wait for stable content, and control page state where possible.

7. Performance, reliability, and cost

Scrolling farther and waiting longer can improve completeness, but both increase capture time. Use a step size that triggers the page’s loading behavior, a condition that is specific enough to avoid premature capture, and a hard bound so an endless feed cannot consume unbounded time. Avoid repeated full-page captures inside the scroll loop; capture once after the needed content is ready, or capture individual regions when the page virtualizes its content.

For reliability, record which readiness condition passed and fail clearly when a required section never appears. A timeout should not silently turn into a successful-looking screenshot. The ideal timeout and scroll count are page-specific; the Puppeteer APIs do not define universal values for them.

Puppeteer’s cost depends on where and how you run the browser, including compute and execution time. This workflow has no per-shot price specified by the cited Puppeteer documentation. If you need a managed screenshot request instead of operating a browser, compare the service’s documented billing and failure behavior before adopting it.

8. Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. One GET request returns an image or PDF, and its full options and parameter reference are in the ScreenshotNeo docs.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

• Cookie banners and consent notices, newsletter popups, and chat widgets are removed before the shot; each cleanup step can be turned off.
• Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed. Response headers report the page verdict and billing status.
• An MCP server lets AI agents, including Claude and Cursor, take screenshots, inspect page information, and capture PDFs.
• The free plan includes 1,000 screenshots per month with no card. Paid plans start at $5 for 3,000 screenshots.

Sign up for ScreenshotNeo and get 1,000 screenshots a month free, with no card.

9. FAQ

Does fullPage: true trigger lazy loading?

Do not rely on it to trigger the site’s scroll-driven behavior. Scroll and verify the needed content first, then take the full-page screenshot.

Can I use a fixed delay instead of a condition?

A delay can allow rendering time, but it cannot tell whether the intended section arrived. Prefer an observable page-specific condition, with a short delay only as a supplementary measure.

Can I capture a section that is no longer in the DOM?

Not from its former element handle. For virtualized content, capture each region while it is rendered or use another source of the content.

Does Puppeteer prescribe one lazy-loading recipe?

No universal recipe is documented. Its APIs provide scrolling, evaluation, waits, and screenshots; the readiness predicate must match the target page.