How to Capture a Lazy-Loaded Page with Selenium in Java

To capture a lazy-loaded page with Selenium in Java, scroll through it in steps, wait after each step for a condition tied to the content you expect, and save the screenshot only when that condition is satisfied. A page reaching document.readyState == "complete" does not prove that JavaScript-driven content has finished appearing. The selector and completion rule must match the site you are capturing.

1. Set up Selenium Java

This example uses Maven, Java 17 syntax, Chrome, and Selenium’s Java API. It saves a screenshot of the current browser context to lazy-page.png. Replace the URL and .result-item with a page and selector you are authorized to access.

<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.XX.X</version>
</dependency>

Replace 4.XX.X with the Selenium 4 version selected for your project. Keep the Selenium version, browser, and driver compatible according to your environment. This code relies on Selenium 4 APIs. Ensure Chrome is installed and the driver is available through your environment’s driver setup.

2. Scroll, wait for new content, and capture

The loop advances by roughly 80% of the viewport so successive regions enter view. After each advance, it waits for the count of matching items to increase. Two rounds without an increase stop the loop, subject to a maximum-round safety bound. This quiet-round rule is only a practical heuristic: it does not prove that a page is complete unless the site’s behavior supports it.

import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.time.Duration;
import java.util.List;

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.TimeoutException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.support.ui.WebDriverWait;

public class LazyPageScreenshot {
    public static void main(String[] args) throws Exception {
        String url = "https://example.com/listing"; // Replace with the target URL.
        By itemSelector = By.cssSelector(".result-item"); // Replace with the real item selector.
        int maximumRounds = 20;
        int requiredQuietRounds = 2;

        WebDriver driver = new ChromeDriver();
        try {
            driver.manage().timeouts().implicitlyWait(Duration.ZERO);
            driver.get(url);

            JavascriptExecutor js = (JavascriptExecutor) driver;
            WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
            int previousCount = driver.findElements(itemSelector).size();
            int stableRounds = 0;

            for (int round = 0;
                 round < maximumRounds && stableRounds < requiredQuietRounds;
                 round++) {

                js.executeScript("window.scrollBy(0, Math.max(300, window.innerHeight * 0.8));");

                try {
                    final int countBeforeScroll = previousCount;
                    wait.until(d -> d.findElements(itemSelector).size() > countBeforeScroll);
                    previousCount = driver.findElements(itemSelector).size();
                    stableRounds = 0;
                } catch (TimeoutException noNewItemsWithinTimeout) {
                    // Could mean there are no more items, or that the selector or wait is wrong.
                    stableRounds++;
                }
            }

            File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
            Files.copy(screenshot.toPath(), Path.of("lazy-page.png"),
                    StandardCopyOption.REPLACE_EXISTING);
            System.out.println("Saved lazy-page.png; matched items: "
                    + driver.findElements(itemSelector).size());
        } finally {
            driver.quit();
        }
    }
}

The Maven snippet and Java listing are templates: the URL, selector, Selenium version, browser setup, and completion condition need to match your project and target. The listing waits for newly added matching elements. If the page replaces existing elements instead of appending them, a count may not increase; wait for a known target to become visible, a loading indicator to disappear, a known total to be reached, or an explicit end marker instead.

3. Choose a completion condition that reflects the page

Use a wait condition that corresponds to the result you actually need. Selenium explicit waits poll for a condition; they are generally a better fit than sleeping for a fixed duration. Selenium cautions against mixing implicit and explicit waits because the resulting timing can be unpredictable. The example sets the implicit wait to zero.

Page behavior	Useful completion signal	Limitation
New cards append to a list	Matching element count increases after a scroll	Does not establish that the final page item has loaded
The page has a known number of results	Count reaches that expected total	Requires a reliable total from the page or another source
A target section is the goal	Target element is visible or present	Presence alone may not mean images or text inside it are ready
A loading indicator marks requests	Indicator disappears, followed by a content check	Some sites remove it before all visual assets finish rendering
The page marks the end of results	Explicit end marker appears	Only works if the site provides a meaningful marker

If you need a target to be visible, for example, use wait.until(ExpectedConditions.visibilityOfElementLocated(targetSelector)) after scrolling toward it. Import org.openqa.selenium.support.ui.ExpectedConditions. A fixed sleep can be useful for a narrowly understood animation delay, but it is a poor primary test for asynchronous completion: too short can capture early, and too long wastes time.

For an asynchronous script, Selenium Java also provides executeAsyncScript; the script must call Selenium’s injected completion callback, and the script timeout must be suitable. For ordinary scroll-triggered loading, a synchronous scroll followed by an explicit DOM condition is usually easier to reason about.

4. Scroll the correct container

The sample scrolls the document with window.scrollBy. Some interfaces put results inside a nested element with its own scrollbar. In that case, advancing the window will not trigger the container’s lazy loading. Locate the scrollable element and scroll it instead, then wait on the same content condition.

WebElement panel = driver.findElement(By.cssSelector(".results-panel"));
((JavascriptExecutor) driver).executeScript(
    "arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].clientHeight * 0.8;",
    panel
);

For pages that respond to native scrolling or user-like interaction, use Selenium interactions such as scrolling toward a known element. The best choice depends on whether the page listens to document scrolling, a nested scroll container, or interaction events. Selenium’s JavaScript executor runs in the currently selected frame or window. If content lives in an iframe, switch to it before locating elements or running scripts.

5. Decide what “capture” means

Screenshot the current context

TakesScreenshot with OutputType.FILE obtains a screenshot file. Driver and browsing-context behavior can differ, so inspect the result in the actual browser and driver you use. Do not assume this produces a full-page image; it may represent the current viewport.

Capture one element

If you need just a component, find it and use the element screenshot API:

WebElement card = driver.findElement(By.cssSelector(".result-item"));
File cardScreenshot = card.getScreenshotAs(OutputType.FILE);
Files.copy(cardScreenshot.toPath(), Path.of("result-item.png"),
        StandardCopyOption.REPLACE_EXISTING);

Wait until the element is in the required state before taking this screenshot. Confirm whether the element image includes the full element in your target driver. If the element extends beyond the visible area or contains lazy images, scroll it into view and check the rendered output.

Extract rendered content instead of an image

If “capture” means text or DOM data, locate and extract the specific elements after the wait condition. Do not treat getPageSource() as proof that JavaScript-modified content is represented: Selenium’s Java API does not guarantee that the returned source reflects modifications made after load.

6. Options to adapt

Setting or choice	What it changes
Item selector	Defines what the wait counts or checks; use the actual repeated item or a more reliable target.
Wait duration	Sets how long each condition can take before timing out; tune it to the site’s response behavior and your capture deadline.
Scroll step	Controls how much of the page is advanced each round. Smaller steps may trigger more intermediate regions; larger steps reduce iterations but can skip site-specific trigger points.
Maximum rounds	Bounds work when the page keeps adding content or a completion condition never occurs.
Quiet rounds	Provides a stopping heuristic when no explicit end signal exists; it is not a completeness guarantee.
Screenshot scope	Choose current context, a single element, or a browser-specific full-page capability according to the output required.

Prefer a known total or explicit end marker over quiet rounds whenever the site exposes one. If the selector matches content already on the page but not the newly loaded items, the count condition can time out despite successful loading.

7. Troubleshooting

Symptom	Likely cause	Fix
The first screenshot is incomplete	Navigation completed before the page’s JavaScript added the expected content.	Wait for a page-specific content condition before capturing; ready state alone is insufficient.
Scrolling stops loading results	The site loads as successive regions approach the viewport, or uses a nested scroller.	Scroll in steps, identify the actual scroll container, and verify which action triggers loading on that page.
The wait times out while content appears on screen	The selector or condition is wrong, or the content is in a frame that is not selected.	Inspect the live DOM, verify the selector, and switch to the correct frame before waiting.
The item count never increases	The page replaces items, changes a different element, or the selector misses new content.	Wait for the relevant target, known total, loading state transition, or end marker instead of a count increase.
The image is only the visible viewport	The driver’s screenshot behavior does not provide the full-page scope you expected.	Inspect the output and use a full-page mechanism supported by the browser and driver in your environment.
Page source seems stale	`getPageSource()` is not guaranteed to reflect JavaScript modifications after load.	Verify the actual located elements, their text, or the rendered screenshot.
JavaScript execution fails	The script may be running in the wrong frame/window or touching cross-domain content.	Select the intended browsing context and avoid cross-domain frame access that the browser disallows.
The loop takes too long	Each scroll may consume the full wait timeout, or the page continually adds results.	Use a more specific completion signal, tune the wait and round limits, and stop at a known target or end condition.

8. Performance, reliability, and cost

Capture time grows with the number of scroll-and-wait rounds and the time each condition takes. A specific signal, such as a known result count, usually avoids waiting through repeated timeouts at the end of a list. Set maximum rounds and timeouts so an unbounded feed cannot run indefinitely. Consider browser startup, network access, and page rendering as part of the work; this recipe makes no performance guarantee for a particular site.

Reliability depends on matching the selector and wait to the site’s behavior. A timeout can indicate a broken selector, a changed page, an unselected frame, or simply that no more content exists. Log the final item count and whether the stop condition was a target, known total, end marker, or timeout. Inspect representative screenshots from the browser and driver you deploy, especially when full-page output matters.

With Selenium, the main cost is the browser and compute capacity you operate, plus the engineering effort to maintain browser, driver, selectors, and site-specific waits. No Selenium cost or runtime benchmark is asserted here.

9. Or skip the browser setup

If you need a screenshot without managing a Selenium browser, ScreenshotNeo is a website screenshot API and MCP server. Its capture options include full-page screenshots with lazy images loaded and waiting for a selector, delay, or network idle. It also supports PNG, JPEG, WebP, and PDF. See the ScreenshotNeo API documentation for request options.

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const bytes = new Uint8Array(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', bytes));

ScreenshotNeo removes cookie banners, popups, and chat widgets before the shot. Bot checks, blank pages, and failed loads are never billed. Its MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month, with no card.

10. FAQ

Can I know that every lazy-loaded item has appeared?

Only if the page provides a trustworthy completion signal, such as a known total or end marker. A few quiet scroll rounds are a heuristic, not proof.

Does this method capture lazy-loaded images?

It triggers loading by bringing page regions into view, but the example waits for matching content elements, not for every image to finish decoding. If image readiness matters, add a page-specific condition that checks the relevant image state before capture.

Does `document.readyState` equal `complete` mean the screenshot is ready?

No. JavaScript can add content after navigation reaches that state. Wait for the result your capture requires.

Can I use the same loop on every site?

The scrolling pattern is reusable, but selectors, scroll containers, frame selection, and completion conditions are site-specific.