Puppeteer Screenshot of an Infinite-Scroll Page in Node.js and TypeScript
Load an infinite-scroll feed with Puppeteer, stop safely when it is done, and capture the full page in Node.js or TypeScript.
To screenshot an infinite-scroll page with Puppeteer, first scroll the page to trigger more content, wait for the site to settle, and stop using a bounded, site-aware rule. Then call page.screenshot({ fullPage: true }). Screenshot capture and feed loading are separate operations: fullPage captures the page as it exists at capture time; it does not guarantee that every item in an endless feed has loaded.
The example below uses TypeScript and a document-height stability rule as a starting point. Prefer an end marker or known item count when the site exposes one. For a virtualized list, height alone can be misleading because the page may recycle visible elements.
1. Install Puppeteer and prepare the project
Puppeteer 25.12.0 documents Node.js 22.12 or later and TypeScript 5.0.1 or later when using TypeScript. Requirements can change between releases, so check the system requirements for the version you install. If you type-check dependencies, the documentation recommends targeting ES2022 or later.
npm install puppeteer
npm install --save-dev typescript tsx @types/node
Create tsconfig.json:
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"outDir": "dist"
},
"include": ["src/**/*.ts"]
}
Save the capture script below as src/capture.ts. Run it with npx tsx src/capture.ts, or compile with npx tsc and run the generated JavaScript with Node.
2. Scroll, wait, stop, and capture with TypeScript
import puppeteer from 'puppeteer';
const url = 'https://example.com/feed';
const output = 'feed.png';
const maxRounds = 30;
const stableRoundsNeeded = 3;
const networkIdleTimeoutMs = 5_000;
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
page.setDefaultNavigationTimeout(30_000);
await page.goto(url, { waitUntil: 'domcontentloaded' });
let previousHeight = await page.evaluate(
() => document.documentElement.scrollHeight,
);
let stableRounds = 0;
for (let round = 0; round < maxRounds && stableRounds < stableRoundsNeeded; round++) {
await page.evaluate(() => {
window.scrollTo(0, document.documentElement.scrollHeight);
});
// Helpful for pages that load after requests settle; not proof that the feed ended.
try {
await page.waitForNetworkIdle({
idleTime: 500,
timeout: networkIdleTimeoutMs,
});
} catch {
// Long-polling, analytics, or other continuing requests can prevent network idle.
}
const currentHeight = await page.evaluate(
() => document.documentElement.scrollHeight,
);
if (currentHeight <= previousHeight) {
stableRounds++;
} else {
stableRounds = 0;
}
previousHeight = currentHeight;
}
await page.screenshot({ path: output, fullPage: true });
console.log(`Saved ${output}; stopped after ${stableRounds} stable rounds.`);
} finally {
await browser.close();
}
Page.evaluate() runs its function in the page context; Puppeteer waits for a returned promise to resolve. The loop above deliberately has a maximum number of rounds. It treats repeated unchanged document height as a practical stopping signal, not a universal completion guarantee. Puppeteer documents Page.screenshot() and the fullPage option for full-page capture in its screenshot API.
3. Choose a stopping rule that matches the page
Infinite feeds do not have a general-purpose “finished” state. Choose a signal the target site actually provides, then retain a round or time limit as a safety cap.
| Stopping signal | When it fits | Limit |
|---|---|---|
| Explicit end marker | The page renders a stable element such as “You’re all caught up.” | The selector must reliably indicate the end rather than merely being present during loading. |
| Known item count | You know how many feed items should appear, or the page exposes a total. | Virtualized pages may not keep every item in the DOM at once. |
| Item-count stability | Ordinary feeds append items to the DOM. | Choose a selector for actual items, not wrappers that change for unrelated reasons. |
| Document-height stability | Simple pages that grow as more items append. | Can fail with virtualized lists, nested scrolling, or content that loads without changing total height. |
| Maximum rounds or elapsed time | Every automated capture, as a hard resource bound. | It stops the work safely but does not establish that the feed is complete. |
For an explicit end marker, Puppeteer provides waitForFunction() to wait until a page-context predicate becomes truthy. You can also check the predicate between scroll rounds. For example, adapt the selector to the target site:
const endSelector = '[data-testid="feed-end"]';
for (let round = 0; round < maxRounds; round++) {
const reachedEnd = await page.evaluate(
(selector) => document.querySelector(selector) !== null,
endSelector,
);
if (reachedEnd) break;
await page.evaluate(() => {
window.scrollTo(0, document.documentElement.scrollHeight);
});
await page.waitForNetworkIdle({ idleTime: 500, timeout: 5_000 }).catch(() => {});
}
await page.screenshot({ path: 'feed.png', fullPage: true });
For a known count or appended-item feed, use a site-specific predicate. This example waits for at least 50 matching elements after each scroll, with a timeout so the capture cannot wait forever:
const itemSelector = 'article.feed-item';
const expectedItems = 50;
for (let round = 0; round < maxRounds; round++) {
const count = await page.$$eval(itemSelector, (items) => items.length);
if (count >= expectedItems) break;
await page.evaluate(() => {
window.scrollTo(0, document.documentElement.scrollHeight);
});
try {
await page.waitForFunction(
({ selector, minimum }) =>
document.querySelectorAll(selector).length >= minimum,
{ timeout: 5_000 },
{ selector: itemSelector, minimum: Math.min(count + 1, expectedItems) },
);
} catch {
// No new item appeared within the budget; decide whether to retry or stop.
}
}
Validate selector behavior against the actual page. A count of DOM elements is not necessarily a count of all items ever loaded when the site virtualizes its feed.
4. Handle incremental scrolling, nested containers, and delayed loading
Some feeds load only after intermediate scroll events, not a jump to the bottom. Others scroll a nested element rather than the window. In those cases, change the scroll operation and keep the same bounded, site-aware stopping logic.
Scroll in steps
await page.evaluate(async () => {
const step = Math.max(300, Math.floor(window.innerHeight * 0.8));
const maxSteps = 20;
for (let i = 0; i < maxSteps; i++) {
const before = window.scrollY;
window.scrollBy(0, step);
await new Promise((resolve) => setTimeout(resolve, 250));
if (window.scrollY === before) break;
}
});
This example demonstrates stepped scrolling, but a single page-context loop is not a feed-completion test. For production captures, interleave steps with checks for an end marker, item count, or another signal from the site. Use a finite number of steps and an overall time budget.
Scroll a nested container
If the feed is inside a scrollable panel, identify that element and scroll it. Replace the selector with one verified on the target page.
const containerSelector = '.feed-scroll-container';
await page.evaluate((selector) => {
const container = document.querySelector(selector);
if (!(container instanceof HTMLElement)) {
throw new Error(`Scrollable container not found: ${selector}`);
}
container.scrollTop = container.scrollHeight;
}, containerSelector);
For incremental-only panels, repeat that operation in a bounded loop and inspect the panel’s item count or end marker. Full-page screenshots capture the page document; they do not necessarily turn a fixed-height, internally scrolling panel into a tall image of all its scroll contents. If the panel itself is the target, capture it as an element or use a page-specific approach that brings each panel segment into view.
Click a “Load more” control
If the page uses a button instead of automatic scrolling, click it and wait for a site-specific change. Check that the button still exists and is enabled before each click, and enforce a click limit.
const loadMoreSelector = 'button.load-more';
const maxClicks = 20;
for (let click = 0; click < maxClicks; click++) {
const button = await page.$(loadMoreSelector);
if (!button) break;
const enabled = await button.evaluate(
(element) => element instanceof HTMLButtonElement && !element.disabled,
);
if (!enabled) break;
const oldCount = await page.$$eval('article.feed-item', (items) => items.length);
await button.click();
try {
await page.waitForFunction(
(previousCount) => document.querySelectorAll('article.feed-item').length > previousCount,
{ timeout: 5_000 },
oldCount,
);
} catch {
break;
}
}
5. Equivalent JavaScript and command-line examples
The same approach works in JavaScript. Save as capture.mjs and run node capture.mjs after installing Puppeteer.
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
await page.goto('https://example.com/feed', { waitUntil: 'domcontentloaded' });
let previousHeight = 0;
let stableRounds = 0;
const maxRounds = 30;
for (let round = 0; round < maxRounds && stableRounds < 3; round++) {
await page.evaluate(() => window.scrollTo(0, document.documentElement.scrollHeight));
await page.waitForNetworkIdle({ idleTime: 500, timeout: 5_000 }).catch(() => {});
const height = await page.evaluate(() => document.documentElement.scrollHeight);
stableRounds = height <= previousHeight ? stableRounds + 1 : 0;
previousHeight = height;
}
await page.screenshot({ path: 'feed.png', fullPage: true });
} finally {
await browser.close();
}
cURL does not run Puppeteer or execute page JavaScript; it can only request an HTTP resource. Use it when you already have a rendered image endpoint. For a hosted screenshot call, see the ScreenshotNeo option below.
6. Capture options and useful configuration
The key option for this task is fullPage: true. Other screenshot options depend on the output you need and the Puppeteer version installed. Check the ScreenshotOptions API for the current supported fields.
| Need | Typical setting | Notes |
|---|---|---|
| Full document image | fullPage: true |
Captures the page’s full content height at capture time. |
| PNG output | Use a .png path or request a PNG buffer. |
Good default when sharp text or lossless output matters. |
| JPEG output | type: 'jpeg' and a quality value supported by the installed version. |
Usually smaller for photographic content; JPEG is lossy. |
| Page viewport | page.setViewport({ width, height, deviceScaleFactor }) |
Set before navigation when responsive layout depends on viewport size. |
| Clip or element capture | Screenshot an element handle or provide a clip, as supported by the API. | Useful when only a particular region is required. |
For example, configure a viewport before loading the page:
await page.setViewport({
width: 1440,
height: 900,
deviceScaleFactor: 1,
});
await page.goto(url, { waitUntil: 'domcontentloaded' });
For full-page output, note that image dimensions can become very large. Consider limiting feed depth, capturing sections separately, or using a compressed image format when file size matters.
7. Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Screenshot contains only the first items | The capture ran before scrolling triggered more loads, or the page uses a nested container or “Load more” button. | Scroll using the mechanism the site expects; wait for item count or an end marker before capture. |
| Loop stops while items are still loading | Height stayed constant briefly, or the feed appends content after a longer delay. | Use a site-specific item/end predicate and allow an appropriate per-round wait. Keep a total time limit. |
waitForNetworkIdle times out |
Long polling, analytics, streaming, or other requests keep the network active. | Treat network idle as a settling aid. Catch its timeout and use a site-specific predicate or bounded delay instead. |
| Height never stabilizes | The feed is unbounded, ads or layout changes alter height, or the site keeps loading content. | Use a hard round/time cap and define the desired amount of content explicitly. |
| Only visible list items exist in the DOM | The page virtualizes or recycles feed nodes. | Use an end marker, cursor/page state, or site-specific data signal. A full-page screenshot cannot restore items removed from the DOM. |
| Content is missing inside a panel | The panel, rather than the document, owns scrolling. | Scroll the panel element. Capture or stitch its contents using a strategy appropriate to that page. |
| Navigation timeout | The page did not reach the chosen navigation condition within the timeout. | Try domcontentloaded for script-heavy pages, set a suitable navigation timeout, and separately wait for the feed’s content signal. |
| Browser fails to launch in a container | Required browser dependencies or launch permissions are missing from the environment. | Install the dependencies required by Puppeteer’s browser setup for that environment, or use a hosted screenshot API. |
| Huge image or memory pressure | The accumulated document is very tall or has large images. | Limit the number of loaded items, reduce device scale, capture sections, or choose an output format suited to the content. |
8. Performance, reliability, and cost
Each scroll round adds browser work and may trigger page requests. A practical capture budget includes a maximum round count, a per-round wait, and an overall job deadline. Set the budget according to the amount of content you need; do not let an endless feed control runtime indefinitely.
For repeatable results, use a stable viewport, wait for a meaningful page condition, and keep the browser lifecycle in a try/finally block so it closes on errors. Record the stopping reason and loaded item count alongside the image. Network idle can improve settling, but Puppeteer documents it as network inactivity (default idle interval 500 ms), not evidence that an infinite feed has ended.
Local Puppeteer costs depend on the machine and runtime used to execute the browser; the dossier provides no benchmark or fixed cost figure. A hosted API replaces local browser setup with a service charge and its own capture behavior. Compare based on whether you need arbitrary feed-loading logic, how much content to capture, and how you want to operate browser infrastructure.
9. Or skip the browser setup
ScreenshotNeo provides a screenshot API and MCP server for developers. A single request can return an image or PDF, and its full-page capture loads lazy images. This one-call capture is useful when the page can be captured without a custom feed-specific loop; an unbounded feed still needs a site-aware completion rule if you must guarantee a particular number of items.
See the ScreenshotNeo API documentation for request options. Here is the cURL call from the API:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/feed -o shot.webp
Python:
import requests
r = requests.get(
"https://api.screenshotneo.com/v1/shot",
params={"access_key": "YOUR_API_KEY", "url": "https://example.com/feed"},
timeout=90,
)
open("shot.webp", "wb").write(r.content)
Node.js:
const q = new URLSearchParams({
access_key: 'YOUR_API_KEY',
url: 'https://example.com/feed',
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(({ writeFile }) =>
writeFile('shot.webp', Buffer.from(await res.arrayBuffer())),
);
ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before the shot. Bot checks, blank pages, failed loads, and cache hits are never billed, and the response identifies page verdict and billing status in headers. Its MCP server gives AI agents screenshot tools. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000.
Sign up for ScreenshotNeo’s free 1,000 screenshots a month, with no card required.
10. Frequently asked questions
Does fullPage: true scroll through an infinite feed?
No. It requests a full-page screenshot of content present at capture time. Scroll and wait for the feed before taking it.
How many rounds should I use?
There is no universal value. Set a maximum that fits your runtime budget, then stop earlier when a reliable site-specific end condition is met.
Can Puppeteer capture every item from a virtualized list?
Not from a single final DOM snapshot if the site removes earlier items as it scrolls. Use a site-specific data signal or capture strategy that preserves the portions you need.
Should I wait for networkidle0 or networkidle2?
For infinite feeds, neither network-idle condition proves completion. Use network settling only as an aid; base completion on page content or a known end state.


