How to Capture a Lazy-Loaded News Feed as One Full-Page Screenshot

To capture a lazy-loaded news feed as one full-page screenshot, first scroll through the live page in steps and wait for each batch of stories and images to appear. Then capture the page at its intended stopping point. A full-page screenshot captures the scrollable document; it does not necessarily trigger every scroll-based loader or include an undefined stream of future stories.

For a one-off capture, use Chrome DevTools’ Capture a full size screenshot command after scrolling through the feed. For repeatable captures, use Playwright: scroll and wait for the site’s content to load, then call page.screenshot({ fullPage: true }). Inspect the saved image from top to bottom for missing cards or blank images.

1. Set a clear stopping point

News feeds often keep adding stories as you scroll. Decide what the screenshot should represent before capturing it: for example, the stories available at a particular time, a date range, or the first set number of items. Record the cutoff if the image is being used for a report or reproducible workflow. A single screenshot cannot represent an infinite feed.

2. Capture the feed in Chrome DevTools

1. Open the news feed and wait for the first stories to finish loading.
2. Scroll down by part of the visible page. Pause and check whether new cards or images appear.
3. Repeat until you reach the chosen cutoff, the end of the feed, or a point where repeated scrolling adds no more items.
4. Open Chrome DevTools Device Mode’s screenshot menu and select Capture a full size screenshot.
5. Open the downloaded image and inspect the entire page. If cards or images are missing, return to the feed, scroll or wait for them to load, and capture again.

Chrome’s documented full-size screenshot command captures beyond the visible viewport. The feed-loading interaction is a separate step: scroll-triggered content may not exist yet simply because a full-size capture was requested. See the Chrome DevTools Device Mode documentation.

3. Automate it with Playwright

Playwright’s fullPage option captures the full scrollable page. For a feed that loads content on scroll, first scroll the live page and wait for the site’s own loading signal or for new items to appear. The selector and reliable wait condition depend on the particular news site.

Install Playwright and its Chromium browser in a Node.js project:

npm install playwright
npx playwright install chromium

Save this as capture-feed.js. Set FEED_URL to the feed and ITEM_SELECTOR to a selector matching its story cards. The script scrolls in viewport-sized steps, waits for the item count or page height to change, and stops when neither changes for several steps or it reaches the declared scroll limit.

const { chromium } = require('playwright');

const url = process.env.FEED_URL;
const itemSelector = process.env.ITEM_SELECTOR;
const maxScrolls = Number(process.env.MAX_SCROLLS || 100);
const quietLimit = Number(process.env.QUIET_STEPS || 3);

if (!url || !itemSelector) {
  throw new Error('Set FEED_URL and ITEM_SELECTOR before running this script.');
}

(async () => {
  const browser = await chromium.launch({ headless: true });
  const page = await browser.newPage({ viewport: { width: 1365, height: 900 } });

  try {
    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 60000 });
    await page.locator(itemSelector).first().waitFor({ state: 'visible', timeout: 30000 });

    let quietSteps = 0;
    for (let i = 0; i < maxScrolls && quietSteps < quietLimit; i++) {
      const before = await page.evaluate((selector) => ({
        count: document.querySelectorAll(selector).length,
        height: document.documentElement.scrollHeight,
      }), itemSelector);

      await page.evaluate(() => window.scrollBy(0, Math.floor(window.innerHeight * 0.8)));

      // This is a bounded settling interval, not proof that every site has finished loading.
      await page.waitForTimeout(700);

      const after = await page.evaluate((selector) => ({
        count: document.querySelectorAll(selector).length,
        height: document.documentElement.scrollHeight,
      }), itemSelector);

      quietSteps = after.count > before.count || after.height > before.height
        ? 0
        : quietSteps + 1;
    }

    // Wait for currently discovered images to finish or fail. Sites may add more images later.
    await page.evaluate(async () => {
      const images = Array.from(document.images);
      await Promise.all(images.map((img) => {
        if (img.complete) return Promise.resolve();
        return new Promise((resolve) => {
          img.addEventListener('load', resolve, { once: true });
          img.addEventListener('error', resolve, { once: true });
        });
      }));
    });

    await page.screenshot({ path: 'feed.png', fullPage: true, scale: 'css' });
    console.log('Saved feed.png. Inspect it for missing stories and blank images.');
  } finally {
    await browser.close();
  }
})().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});

Run it with the feed URL and a site-specific story-card selector, for example:

FEED_URL='https://example.com/news' ITEM_SELECTOR='article.story' node capture-feed.js

Replace the example URL and selector with values for the site you are capturing. The script’s 700 ms pause and quiet-step limit are tunable heuristics, not universal guarantees. If the feed loads slowly, uses a “Load more” button, or needs a consent action, adapt the script to the site’s actual behavior. Playwright documents the screenshot API and Page API.

4. Choose the screenshot scale and review the output

Playwright supports scale: 'css', which produces one output pixel per CSS pixel, and scale: 'device', which follows the device scale and can create larger output on high-density displays. Scale changes output dimensions; it does not load the feed. Use CSS scale when a smaller image is acceptable. Check that the screenshot includes the intended last story and that thumbnails are visible.

If the resulting image is extremely tall, consider whether one image is still useful for reading or review. For an endless feed, use the cutoff you defined; for a changing feed, capture after the intended content settles and note when the capture was made.

5. Troubleshoot missing content

Symptom	Likely cause	What to do
Stories are missing below the fold	The site loads them only after scrolling, but capture happened first.	Scroll the live page in steps, wait for cards to appear, then capture again.
Story cards appear but images are blank	Images have not loaded yet, or image requests failed.	Wait for image loading after cards appear and inspect the result. There is no universal wait duration.
The automation stops too early	The selector does not match story cards, or the quiet-step threshold is too low for a slow feed.	Check the selector in the page, increase QUIET_STEPS or MAX_SCROLLS, and use the site’s loading indicator when available.
The script never reaches the desired content	The site requires a button click, authentication, consent, or another interaction to continue.	Handle that interaction explicitly in the script and verify the page has the expected state before capture.
The screenshot is too large	The feed is long, or device-pixel scale produces more pixels than needed.	Use scale: 'css', set a reasonable cutoff, or capture separate sections if one tall image is impractical.
Headlines or layout differ between runs	The feed changed while loading or capturing.	Capture at a defined cutoff after content settles; for repeatable records, note the capture time and scope.

Scrolling before a full-page screenshot is a practical response to scroll-triggered loading; a Playwright issue discusses how off-screen capture and IntersectionObserver-based loading can interact. It is an implementation concern, not a guarantee about every site or a universal Playwright option: Playwright issue on lazy and scroll-triggered content.

6. Performance, reliability, and cost considerations

• Performance: Scrolling and waiting for each batch adds time, but captures content the full-page option alone may not have caused the site to load. Avoid an unnecessarily large scroll limit on endless feeds.
• Reliability: Prefer a site-specific signal such as an item count, loading indicator, or known end marker. A fixed delay is only a heuristic. Always inspect the output because images can fail even when story cards loaded.
• Output size: A long feed can produce a very tall image. CSS scale can reduce pixel dimensions compared with device scale on high-density displays, but it does not reduce the amount of page content.
• Cost: Chrome DevTools and a local Playwright run do not require a screenshot API service. If you capture through a hosted service, check its own billing rules and whether failed or cached captures are charged.

7. Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. It accepts one GET request with a URL and returns an image or PDF; its documentation lists the available options. For a normal page capture, the request is:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com/news -o feed.webp

This one-call example captures the URL as requested. For a news feed that loads stories only on scroll, confirm that your chosen capture settings and the page’s behavior produce the required content, then inspect the result. ScreenshotNeo’s listed options include full-page capture with lazy images loaded, wait conditions, custom JavaScript, and CSS, which can help tailor a capture to a page.

ScreenshotNeo removes cookie and consent banners, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and response headers identify the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000 screenshots.

Sign up for 1,000 free screenshots a month, with no card required.

8. FAQ

Does fullPage: true scroll the page for me?

It requests a screenshot of the full scrollable page. It does not guarantee that a site’s scroll-triggered feed items have been loaded first.

How do I know when to stop scrolling an endless feed?

Choose a declared scope, such as a date range or item count. If you have no fixed scope, stop after a defined number of steps with no new items and record that cutoff.

Should I use CSS or device scale?

Use CSS scale for one pixel per CSS pixel and generally smaller output dimensions; device scale follows device pixels and can produce a larger image. Neither choice changes feed loading.

Can I guarantee that every story image appears?

No universal wait or capture setting guarantees this across sites. Wait for the feed and its images, then inspect the saved screenshot for omissions or failed image loads.

Sources

• Playwright: Screenshots
• Playwright: Page API
• Chrome DevTools: Device Mode
• Playwright screenshot parameters
• Playwright issue on lazy and scroll-triggered screenshot content