ScreenshotNeo

BlogHow-to

How to Screenshot a Page with Infinite Scroll Using Selenium in C#

Use Selenium in C# to load an infinite-scroll page before capturing it. Learn how to detect new content, stop safely, and stitch overlapping screenshots.

By the ScreenshotNeo team4 October 20269 min read

To screenshot an infinite-scroll page with Selenium in C#, repeatedly scroll the page or its actual scrollable feed container, wait until new content loads, and stop at an end marker or after content remains stable. Then save a screenshot. A single screenshot captures only the content currently available; for one tall image, capture overlapping viewport segments and stitch them with a separate image-processing step.

The scroll target, loading condition, and end signal depend on the site. Selenium provides JavaScript execution and screenshot APIs, but the documented APIs do not provide a universal infinite-scroll loader or a guaranteed one-call stitch operation. This guide uses Selenium 4 with .NET and ChromeDriver. Check the Selenium downloads page for the current release and keep browser and driver versions compatible.

1. Set up Selenium for C#

Create a .NET console project and add Selenium.WebDriver. ChromeDriver must be available to Selenium, and Chrome or Chromium must be installed in the environment.

dotnet new console -n InfiniteScrollCapture
cd InfiniteScrollCapture
dotnet add package Selenium.WebDriver

The official Selenium first-script guide shows the .NET WebDriver pattern. The examples below target a public demo-style page; replace its URL and selectors with those for the page you are authorized to capture.

2. Scroll, wait for new content, and save a screenshot

This runnable baseline scrolls the window in viewport-sized steps, waits for the number of feed items to increase, and stops after the page height and item count remain stable for several checks. It also has iteration and elapsed-time limits so a page without a reliable end signal cannot keep the process running indefinitely.

Change feedSelector to a selector that matches each loaded item, and adjust the stability conditions for the site. If the page exposes a reliable end marker or loading indicator, prefer that explicit signal over inferring completion from stable height.

using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System.Diagnostics;

const string pageUrl = "https://example.com/feed";
const string feedSelector = "article"; // Replace with the selector for one feed item.
const int maxSteps = 100;
const int stableChecksRequired = 3;

using var driver = new ChromeDriver();
driver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(60);
driver.Navigate().GoToUrl(pageUrl);

var js = (IJavaScriptExecutor)driver;
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(15));
var clock = Stopwatch.StartNew();
int stableChecks = 0;
long previousHeight = -1;
int previousCount = -1;

for (int step = 0; step < maxSteps && clock.Elapsed < TimeSpan.FromMinutes(5); step++)
{
    int beforeCount = driver.FindElements(By.CssSelector(feedSelector)).Count;
    long beforeHeight = Convert.ToInt64(js.ExecuteScript(
        "return Math.max(document.body.scrollHeight, document.documentElement.scrollHeight);"));

    // Scroll by most of one viewport so the site's lazy loader has a chance to trigger.
    js.ExecuteScript("window.scrollBy(0, Math.max(200, window.innerHeight * 0.8));");

    try
    {
        wait.Until(d => d.FindElements(By.CssSelector(feedSelector)).Count > beforeCount);
    }
    catch (WebDriverTimeoutException)
    {
        // A timeout may mean the end was reached, the selector is wrong,
        // or the site uses a different loading signal. Stability checks decide below.
    }

    // Let layout settle, then compare both document height and item count.
    Thread.Sleep(300);
    int currentCount = driver.FindElements(By.CssSelector(feedSelector)).Count;
    long currentHeight = Convert.ToInt64(js.ExecuteScript(
        "return Math.max(document.body.scrollHeight, document.documentElement.scrollHeight);"));

    if (currentCount == previousCount && currentHeight == previousHeight && currentCount == beforeCount)
        stableChecks++;
    else
        stableChecks = 0;

    previousCount = currentCount;
    previousHeight = currentHeight;

    if (stableChecks >= stableChecksRequired)
        break;
}

// Save the currently loaded page as a PNG. This is not guaranteed to include
// all content beyond the current browser viewport as one tall image.
driver.GetScreenshot().SaveAsFile("infinite-scroll.png");

Thread.Sleep above is only a short layout-settling pause after an explicit wait. For better reliability, replace it with a condition tied to the page—for example, a loading spinner disappearing, an item count changing, or an end-of-feed marker becoming visible.

Make the stopping condition site-specific

  • End marker: Stop when a known “end of results” element appears. This is the clearest completion condition when the site exposes one.
  • Loading indicator: Wait for the loader to appear and then disappear, or for the next batch of items to be added.
  • Item count: Record the count before scrolling and wait for it to increase. If the feed replaces items instead of appending them, watch a stable item identifier or another page-specific signal.
  • Scroll height: Compare the document height before and after scrolling. Height alone is not enough: a feed can load more items without increasing height, or grow after delayed images finish loading.
  • Bounded fallback: Keep a maximum number of scrolls and a time limit even when using an end marker. This protects against broken selectors and pages that keep producing content.

3. Find the element that actually scrolls

Many pages scroll the browser window, but some keep the feed inside a nested panel with its own scrollTop and scrollHeight. If scrolling the window does nothing, inspect the page in the browser’s developer tools and identify the container whose scroll position changes as the feed moves.

For a container such as div.feed, replace the window scroll script with a container scroll, and measure that container rather than the document:

const string containerSelector = "div.feed"; // Replace with the real scroll container.

js.ExecuteScript(@"
  const el = document.querySelector(arguments[0]);
  if (!el) throw new Error('Scroll container not found');
  el.scrollTop += Math.max(200, el.clientHeight * 0.8);
", containerSelector);

long containerHeight = Convert.ToInt64(js.ExecuteScript(@"
  const el = document.querySelector(arguments[0]);
  return el ? el.scrollHeight : 0;
", containerSelector));

Use the same container in your completion checks. A window height that remains unchanged says nothing about a nested feed’s progress.

4. Capture the whole feed as one tall image

After loading the content, a normal WebDriver screenshot represents the current browsing context; it is not a universal promise that an arbitrarily tall document will be captured in one image. For a tall image, move through the loaded content in overlapping viewport-sized segments, save each segment, then stitch them with an image-processing library or tool outside WebDriver.

  1. Load the feed to the desired endpoint first. Record the final item count and confirm the end condition.
  2. Return to the top and capture viewport images at fixed vertical offsets.
  3. Move by less than one viewport height so adjacent captures overlap. The overlap gives you material to align and inspect at seams.
  4. Stitch the segments in a separate image-processing step, accounting for the overlap. Review the result for missing or duplicated rows.

A basic capture loop can save viewport segments. This example assumes a window-scrolling page and a fully loaded document; adapt it for nested containers and use an image library to assemble the files.

long totalHeight = Convert.ToInt64(js.ExecuteScript(
    "return Math.max(document.body.scrollHeight, document.documentElement.scrollHeight);"));
long viewportHeight = Convert.ToInt64(js.ExecuteScript("return window.innerHeight;"));
long stepHeight = Math.Max(1, (long)(viewportHeight * 0.8));

js.ExecuteScript("window.scrollTo(0, 0);");
int segment = 0;

for (long y = 0; y < totalHeight; y += stepHeight)
{
    js.ExecuteScript("window.scrollTo(0, arguments[0]);", y);
    Thread.Sleep(300); // Prefer a site-specific condition for fonts and layout to settle.
    driver.GetScreenshot().SaveAsFile($"segment-{segment++:D3}.png");
}

For a stable, bounded content region, Selenium also supports screenshots of an element. This can be simpler than capturing the whole document when the target content is contained in one element. It does not solve loading the feed, and a very tall element may still be constrained by browser behavior or memory. See Selenium’s screenshot documentation for current-context and element capture examples.

5. Handle lazy content and changing layouts

  • Lazy-loaded images: Scroll far enough to trigger them, then wait for image completion before capturing. An image’s complete property and nonzero naturalWidth can help distinguish loaded images from placeholders; decide which images matter before waiting for every asset.
  • Sticky headers: A fixed header can appear in every segment and cover content. Account for it when choosing overlap and when stitching, or temporarily hide it with page-specific CSS if that is appropriate for your capture.
  • Animations and ads: They may change between segments and create inconsistent output. Disable or wait for animation where possible, and consider whether dynamic ad content should be captured.
  • Changing feeds: New items may arrive while you scroll, shifting earlier content. Capture a stable snapshot where possible and compare item identifiers before and after segmentation.
  • Very long pages: Many segments consume time and memory. Set a maximum page depth or item count when a complete archive is not required.

6. Troubleshooting

Symptom Likely cause Fix
Screenshot contains only the first screen The screenshot was taken before scrolling triggered more loads, or the feed uses a nested scroller. Run the scroll-and-wait loop first. Inspect which element scrolls and target it directly.
The loop stops immediately The item selector matches nothing, the content was already present, or the wait checks the wrong signal. Verify the selector in developer tools and log item count, scroll height, and marker state at each step.
The loop never finishes The page keeps adding items, the end marker is absent, or the stable condition can never be met. Use an application-specific stop signal plus maximum steps and elapsed-time bounds.
Window scrolling has no effect A nested element owns the scroll position. Find the feed container and update its scrollTop; measure its scrollHeight.
Some items or images are missing The next batch or lazy asset had not finished loading when capture began. Wait for a new item, loader disappearance, and required images to load before each capture.
Stitched image has repeated or missing rows Segments had insufficient overlap, the page shifted, or sticky content obscured a seam. Increase overlap, align using stable visual landmarks, and capture only after the layout stops moving.
ChromeDriver cannot start or navigate Browser and driver are incompatible, browser is missing, or the runtime environment blocks browser startup. Install Chrome or Chromium, use a compatible Selenium/driver setup, and inspect the driver’s startup error. In containers, configure browser sandboxing and display/headless requirements for that environment.
Screenshot file is overwritten SaveAsFile writes to the requested path and overwrites an existing file there. Use unique filenames or check the output directory before saving.

7. Performance, reliability, and cost

Capture time grows with the number of scroll steps, the site’s load time, and the number of output segments. Scrolling by a large fraction of the viewport reduces round trips, while overlap adds capture work in exchange for easier seam checks. For a repeatable job, log each step’s item count, scroll position, elapsed time, and stop reason.

Reliability depends more on the page-specific wait and stop conditions than on the screenshot call. Network speed, rate limits, authentication, lazy assets, and content that changes during capture all affect the result. Use a bounded wait and retry only transient navigation or load failures; repeated retries against a page that rejects automation will not make the capture complete.

With self-hosted Selenium, there is no per-screenshot API charge from Selenium itself, but you pay for the machine, browser runtime, engineering time, and image processing. Long captures use more CPU, memory, and wall time. Reduce work by limiting the capture depth, blocking irrelevant resources when permitted, and avoiding stitching when separate viewport images meet the requirement.

Or skip the browser setup

ScreenshotNeo can capture a page with one API request. It returns an image or PDF, and its options include full-page capture with lazy images loaded. See the ScreenshotNeo API documentation for parameters.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo accepts cookie banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers report the page verdict and billing status. Its MCP server gives AI agents tools for screenshots, page information, and PDF capture. The free plan includes 1,000 shots a month with no card; paid plans start at $5 for 3,000 shots. Learn about ScreenshotNeo or sign up free for 1,000 screenshots a month, with no card.

FAQ

Can Selenium load every infinite-scroll page?

Only if the page makes additional content available through browser interactions that your script can trigger, and you can identify a valid completion condition. Feeds requiring user gestures, authentication, or site-specific API state may need additional handling.

Does a WebDriver screenshot automatically capture the complete document?

Do not assume so for an arbitrarily tall infinite feed. Load the content first, then use segment captures and stitching when one tall image is required.

Should I use an item count or scroll height to detect completion?

Prefer an explicit end marker or loading signal. Item count and scroll height are useful fallback signals, but either can remain unchanged temporarily while the page is still loading.

Can I capture just the feed instead of the whole page?

Yes, if the feed is a stable DOM element. Selenium supports element screenshots; the infinite-scroll content still needs to be loaded, and tall-element limits can vary by browser.