How to capture a page that loads more results when you reach the footer

First load the results, then capture them. Scroll down a screenful at a time, pause for new items to appear, and repeat until the feed stops growing. Then take a full-page screenshot, print to PDF, or save the page for offline use. A browser’s capture timeout does not automatically scroll through an infinite feed, and no one capture method is guaranteed to include every site’s dynamic content.

1. Load the feed before capturing

Open the page and wait for its initial content to settle.
Scroll down about one screenful. Pause long enough for the next batch to load.
Repeat, checking that new results appear after each scroll.
Stop when the site shows its end state or scrolling no longer adds items. If older items disappear as new ones arrive, the site may be replacing content rather than retaining the whole feed.
Choose an output: screenshot for a visual record, PDF for a portable document, or a complete webpage save for offline browsing.

The scroll-and-pause sequence is practical guidance, not a browser-guaranteed algorithm. Feed behavior is controlled by each site. Some pages need longer waits, stop at a server-side limit, or load content only when a particular part of the page is visible.

2. Capture the loaded page

Full-page screenshot in Firefox

Firefox Developer Tools can capture an entire page or a selected element. Open Developer Tools, use its screenshot tool, and choose the full-page option. The Firefox console screenshot command also supports --fullpage. See Firefox’s screenshot documentation.

For a full-page console capture, open the Web Console and run:

:screenshot --fullpage

Confirm the page has finished appending the results before running the command. The screenshot captures the page state available to the browser; it does not itself exhaust the feed.

Chrome Headless screenshot or PDF

Chrome Headless can capture a screenshot, print a page to PDF, or dump the DOM after page scripts have run. Replace the example URL with the page you loaded. These commands open a browser session and wait up to the supplied timeout; they do not scroll the page to trigger every infinite-feed batch.

CHROME="$(command -v chromium || command -v chromium-browser || command -v google-chrome)"
URL="https://example.com/feed"

# Visual capture
"$CHROME" --headless --screenshot=feed.png --timeout=15000 "$URL"

# PDF capture without browser headers and footers
"$CHROME" --headless --print-to-pdf=feed.pdf --no-pdf-header-footer --timeout=15000 "$URL"

# Inspect the post-script DOM as serialized HTML
"$CHROME" --headless --dump-dom --timeout=15000 "$URL" > feed.html

These are separate runs. The timeout sets a maximum wait before capture, even if the page is still loading. It is not an instruction to scroll. The DOM dump is the serialized DOM after Chrome parses the page and runs scripts, not the original HTML source downloaded from the server. See Chrome Headless documentation.

Save a page for offline reading in Firefox

In Firefox, use Save Page and choose Web page, complete when you want the page along with pictures and other resources. Firefox also offers HTML only, text, and all files. A saved copy is useful for offline reading, but should not be assumed to preserve dynamic behavior or every later feed item. See Mozilla’s save-page instructions.

3. Choose the format for the job

Need	Use	Keep in mind
Visual record	Full-page screenshot	Check that all intended items were loaded and visible before capture.
Portable document	PDF	Page breaks and layout can differ from the live page; review the result.
Offline reading with resources	Firefox “Web page, complete”	It saves page resources, but a saved page is not a promise that scripts or future feed loading will work offline.
Diagnose missing content	DevTools Network panel or Chrome DOM dump	Inspection shows what loaded or exists in the current rendered state; site-specific loading logic may need investigation.

Chrome Headless supports --screenshot and --print-to-pdf; the Chrome DevTools Protocol also documents Page.captureScreenshot and Page.printToPDF. See the Page domain protocol reference.

4. If content is missing, inspect the rendered state

Compare the visible feed with the screenshot or PDF. Look for an item count, an end-of-feed marker, or gaps in the sequence.
Scroll again and wait. If more content appears, capture after the newly loaded items settle.
Open Chrome DevTools Network, enable Preserve log, then reproduce the scroll. The preserved request list can help identify whether requests occur as you reach the bottom. The Network panel can also capture screenshots during loading.
In Chrome Headless, use --dump-dom to inspect the post-script DOM. Compare its content with what the page displays. A DOM dump is evidence of the current rendered DOM, not a universal way to invoke a site’s loading behavior.
If a specific request or selector appears necessary, inspect that site’s implementation. Do not assume the same selector, endpoint, wait time, or scroll strategy will work on another site.

See Chrome DevTools Network documentation for request preservation and loading inspection.

5. Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server. Its one-request API returns a screenshot or PDF, and the options include full-page capture with lazy images loaded. For this kind of page, first make sure the feed’s results have actually loaded: a capture call should not be treated as a universal infinite-scroll traversal mechanism.

For a page that is ready to capture, here are runnable examples. Replace the target URL and put your API key in YOUR_API_KEY. See the ScreenshotNeo API documentation for parameters and response details.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://example.com/feed \
  -o feed.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com/feed"},
    timeout=90,
)
r.raise_for_status()
with open("feed.webp", "wb") as f:
    f.write(r.content)

Node.js

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://example.com/feed'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const bytes = new Uint8Array(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('feed.webp', bytes));

Cookie and consent banners, newsletter popups, and chat widgets are removed before capture; each step can be turned off.
Bot checks, blank pages, timeouts, failed loads, and cache hits are not billed. Response headers report the page verdict and billing status.
An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents and MCP clients.
The free plan includes 1,000 screenshots a month with no card. Paid plans start at $5 for 3,000 screenshots. Every feature is on every plan.

6. Troubleshooting

Symptom	Likely cause	What to try
The capture stops near the first screen	The feed was never scrolled, or the capture timeout expired while the page was still loading.	Load results manually in steps first. Increase the capture wait only when the page needs more time; a longer timeout still does not scroll the feed.
Some results are absent	The site has not appended them yet, has a server-side result limit, or replaces older items.	Scroll and wait again, check the site’s end state, and inspect the rendered DOM and Network panel.
Images are blank	Images may load lazily only when they approach the viewport, or may not have completed before capture.	Scroll through the feed so images enter view, pause for loading, then capture and review the output.
The PDF or screenshot looks different from the live page	Print layout, responsive sizing, or dynamic page state can change the result.	Review the output and use a screenshot when exact visual appearance matters. Capture after the page settles.
The offline copy lacks interactive content	Saving resources does not guarantee that scripts, requests, or future loading behavior will work offline.	Use the saved copy for reading the captured state; use a screenshot or PDF for a fixed record.
Headless Chrome command cannot find the browser	The executable name or path differs by installation.	Set `CHROME` to the installed Chrome or Chromium executable path.
ScreenshotNeo returns a non-image response or request error	The request may have an invalid key, URL, or other option, or the page may fail to load.	Check the response status and API response details, verify the URL and key, and consult the API docs. The API response includes verdict and billing headers.

7. Performance, reliability, and cost

Each manual scroll gives the page another opportunity to fetch and render results, so total time depends on the feed and its network behavior. A longer browser timeout helps only with waiting for a page that is still loading; it cannot make a site reveal results that require scrolling. For a reliable record, verify the feed’s end state and inspect the saved output.

Browser screenshots and PDFs have no per-capture API charge, but require the manual loading and review steps. ScreenshotNeo has a free allowance of 1,000 shots per month; paid plans are Starter $5 for 3,000, Growth $15 for 15,000, Pro $39 for 60,000, Scale $99 for 250,000, and Business $249 for 1,000,000. Yearly billing gives two months free. The service bills only clean shots; failed and cache-hit responses cost nothing. As with any capture route, confirm that the page state you need is present before relying on the output.

8. FAQ

Can I reach the end of every infinite feed automatically?

There is no universal browser capture command documented here that does so. Feed triggers and limits are site-specific; scrolling in steps and checking the result is the practical baseline.

Does a full-page screenshot include content that has not loaded?

No. It records the page state available to the browser at capture time. Load and verify the content first.

Should I use a screenshot, PDF, or saved webpage?

Use a screenshot for visual evidence, PDF for a portable document, and a complete webpage save for offline reading with resources.

Can I inspect what Chrome rendered instead of the original HTML?

Yes. Chrome Headless --dump-dom serializes the DOM after scripts execute. It does not tell you how to trigger every later feed request.