ScreenshotNeo

BlogHow-to

How to run Selenium screenshot tests in a Jenkins pipeline

Capture Selenium screenshots in Jenkins, preserve them when tests fail, and add visual comparison without confusing evidence with an assertion.

By the ScreenshotNeo team4 October 202610 min read

Run your existing Selenium test command in a Jenkins Pipeline stage on an agent that can run the selected browser. In the test, save a screenshot from the active WebDriver session to a predictable directory in the workspace. Use a Pipeline post section to archive screenshots and reports even when tests fail. Screenshots provide evidence; they do not constitute a visual regression test until you compare them with reviewed baselines using a chosen comparison method.

This guide uses Java, Maven, Selenium 4, and a Jenkins Declarative Pipeline as a concrete example. Adapt the test command and screenshot hook to your language and test runner. Selenium describes WebDriver as its browser automation interface and says its bindings use Selenium Manager by default for browser and driver management. Your Jenkins agent still needs a usable browser environment and access to the test site. See Selenium’s documentation.

1. Prepare the test and Jenkins agent

  1. Choose the agent label or container image that has the browser and system dependencies your tests require. Verify it can reach the application under test.
  2. Keep the test command consistent with local development where practical. The example runs mvn test.
  3. Choose a stable workspace path for screenshots. This example uses target/screenshots, which is also a convention documented by Jenkins’ UI Test Capture plugin; you can use another path if your runner expects it.
  4. Decide whether a local browser on one agent is enough. If you need multiple machines or browser and operating system combinations, Selenium Grid is designed to distribute tests across them. Grid adds environment setup and operational work. Read the Selenium overview.

Do not assume Selenium Manager installs every browser your agent needs. Check browser availability, permissions, network access, and any organization-specific proxy or container requirements on the actual Jenkins agent.

2. Capture a screenshot from the WebDriver session

Capture at a meaningful checkpoint, such as immediately after navigating to a page or in a test failure hook. Use the same WebDriver instance that performed the test so the screenshot reflects that browser session.

Java example with JUnit 5

This test creates the output directory, saves a PNG after navigation, and quits the driver reliably. It assumes the project has Selenium and JUnit 5 dependencies and that the agent can start Chrome. Replace the example URL with your test environment.

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

class CheckoutScreenshotTest {
    private WebDriver driver;

    @Test
    void captureCheckoutPage() throws IOException {
        driver = new ChromeDriver();
        driver.manage().window().setSize(
            new org.openqa.selenium.Dimension(1365, 900)
        );
        driver.get("https://example.com/checkout");

        Path screenshotDir = Path.of("target", "screenshots");
        Files.createDirectories(screenshotDir);
        Path destination = screenshotDir.resolve("checkout.png");
        Files.copy(
            ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE).toPath(),
            destination,
            StandardCopyOption.REPLACE_EXISTING
        );

        // Put assertions for the page state here. A saved screenshot alone
        // does not check that the page is correct.
    }

    @AfterEach
    void closeBrowser() {
        if (driver != null) {
            driver.quit();
        }
    }
}

For failure evidence across many tests, put screenshot capture in your test framework’s failure hook or listener, where it can access the failing test’s driver. Avoid one shared filename for parallel tests: include a sanitized test name or unique identifier so workers do not overwrite each other’s files. Keep the artifact directory inside the Jenkins workspace so the Pipeline can archive it.

3. Run tests and retain screenshots in Jenkins

This Declarative Pipeline runs the Maven test command and archives screenshots and Surefire reports in post. The always condition runs after either a pass or failure. Jenkins documents post conditions including always, failure, and unsuccessful; choose the condition that matches your evidence-retention policy. See Pipeline syntax.

pipeline {
    agent { label 'selenium-chrome' }

    stages {
        stage('Selenium screenshot tests') {
            steps {
                sh 'mvn -B test'
            }
        }
    }

    post {
        always {
            archiveArtifacts(
                artifacts: 'target/screenshots/**/*.png,target/surefire-reports/**/*.xml',
                allowEmptyArchive: true
            )
        }
    }
}

Use allowEmptyArchive: true when no screenshot is expected on some runs, for example when a setup failure happens before the browser starts. If screenshots should exist on every run, omit it so a missing directory is visible as a pipeline problem. Confirm artifact retention settings in your Jenkins installation; archiving makes files available as build artifacts, but retention and storage limits are installation choices.

Publish only after a failed build

If successful-run screenshots are not useful, use failure for that archive step. If you want evidence for any non-successful result, Jenkins also supports unsuccessful. You can archive reports unconditionally and screenshots only on failures by using separate post conditions. The exact artifact syntax and available reporting plugins depend on your Jenkins installation.

Optional screenshot reporting plugins

The Jenkins UI Test Capture plugin documents a target/screenshots convention and associating images with test result data, including viewing failed-test screenshots. Its behavior and maintenance status are plugin-specific, so check its current page and compatibility before adopting it.

The Selenium HTML Report plugin documents scanning a Selenium result directory for HTML files and copying them under seleniumReports in the build root. Check current compatibility and maintenance before selecting it. A Selenium Jenkins plugin page describing Selenium 3 Grid integration displays an unresolved security warning and adoption notice; it is not required for the pipeline shown here, and its current status should be reviewed before use: Jenkins Selenium plugin page.

4. Make screenshots useful and repeatable

  • Set the viewport. Use an explicit browser window size for screenshots that need consistent dimensions. A fixed viewport helps, but it does not make rendering identical across different browsers, fonts, operating systems, or device scale factors.
  • Wait for the state you intend to capture. Prefer an explicit wait for a meaningful element or state over an arbitrary sleep. Capture after the page has reached the checkpoint your test is meant to document.
  • Use deterministic test data. Timestamps, rotating content, randomized data, animations, and asynchronous updates can create image differences unrelated to a code regression.
  • Name files per test and run context. Include a sanitized test identifier and, where useful, a build or worker identifier. Avoid embedding secrets or personal data in filenames or images.
  • Capture the right scope. A normal WebDriver screenshot generally captures the current browser view. If you need a full-page image, verify how your chosen browser and capture method handle page height; do not assume a viewport screenshot contains content below the fold.
  • Keep artifact volume bounded. Capturing every step in every test can create large builds. Capture at selected checkpoints or on failures, and align Jenkins retention with the team’s debugging needs.

5. Add visual regression comparison separately

A screenshot is an output artifact. To make it a visual test, compare the new image with a reviewed baseline using a selected image comparison tool, then define what differences should fail the build. Selenium and Jenkins do not prescribe a universal comparison library or acceptable pixel-difference threshold in the sources cited here.

  1. Generate baselines in a controlled environment and review them before accepting them.
  2. Keep browser, viewport, operating system, fonts, locale, timezone, and test data consistent between baseline and current runs where possible.
  3. Choose a comparison approach and tolerance appropriate to the page. Document how dynamic regions are handled.
  4. Publish the current screenshot and comparison output on both pass and failure where that helps reviewers understand a change.
  5. Update baselines deliberately after review; do not automatically bless every changed screenshot.

For broader browser and operating system coverage, Selenium Grid can distribute execution across machines and configurations. Grid is useful when that coverage is required, but brings additional setup and capacity management compared with one controlled agent. Selenium’s overview describes Grid’s purpose.

6. Use ScreenshotNeo when a screenshot does not need your test browser

The Selenium approach is appropriate when the test must interact with the application through WebDriver, verify authenticated or stateful flows, or inspect the exact browser session. For a direct page capture that does not need those interactions, ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. Its one-request API can return an image or PDF. See the ScreenshotNeo API documentation.

Or skip the browser setup

Use this cURL request as a standalone capture or a separate Jenkins step. Replace the example URL and provide your API key through your Jenkins credentials setup rather than committing it to source control.

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Equivalent Python and Node.js requests are available when those fit your pipeline scripts:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)
const q = new URLSearchParams({
  access_key: 'YOUR_API_KEY',
  url: 'https://stripe.com'
});
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await Bun.write('shot.webp', new Uint8Array(await res.arrayBuffer()));

ScreenshotNeo removes cookie banners, newsletter popups, and chat widgets before capture. Bot checks, blank pages, failed loads, and cache hits are not billed; response headers report the page verdict and billing status. Its MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 screenshots.

Sign up free for 1,000 screenshots a month, with no card required.

7. Troubleshooting

Symptom Likely cause What to check or change
Browser fails to start in Jenkins The agent lacks the browser, required system libraries, permissions, or a compatible runtime configuration. Run the test command on the same agent image; verify browser availability and agent permissions. Selenium Manager handles browser and driver management by default in Selenium bindings, but does not guarantee the chosen agent has every required browser environment.
Tests pass locally but cannot reach the site in CI The agent has different network access, DNS, proxy, firewall, or test-environment credentials. Check connectivity from the Jenkins agent and configure required access through the team’s normal secrets and network setup.
Screenshot directory is missing from the build The test failed before capture, the path differs from the archive glob, or the browser hook did not run. Check the test log and workspace path; create the directory before writing. Use allowEmptyArchive: true only if an absent artifact is an expected outcome.
Screenshot is blank or shows the wrong state Capture happened before navigation or rendering completed, or the test reached an unexpected page. Wait for the expected state, inspect the test URL and assertions, and capture at the intended checkpoint. Preserve failure logs alongside the image.
Parallel tests overwrite each other’s images Workers write to the same filename. Use unique, sanitized names per test and worker, or separate output directories.
Images differ across builds without a meaningful UI change Viewport, browser, OS, fonts, locale, timezone, dynamic data, or animation differs. Standardize the rendering environment and test state; handle known dynamic regions in the chosen comparison workflow.
Jenkins reports no artifacts after a test failure Archiving was placed only in normal steps or the post condition does not cover the result. Move artifact handling into post and select always, failure, or unsuccessful according to the retention policy.
Build storage grows quickly Every test or checkpoint creates an artifact and retention is broad. Capture only useful checkpoints, archive failures when appropriate, and review build artifact retention settings.

8. Performance, reliability, and cost considerations

  • Performance: Browser startup and page rendering add work to a test run. Capture only the checkpoints that aid debugging or comparison, and use parallel execution only when the agent capacity and test design support it. The sources cited here do not establish a universal runtime or speedup.
  • Reliability: A screenshot is only as dependable as the browser environment, page state, and capture hook. Preserve logs and test reports as well as images, and make post-run artifact handling resilient to test failures.
  • Jenkins storage: Images and reports consume artifact storage. Set retention to fit the team’s debugging and audit needs; exact retention behavior depends on Jenkins configuration.
  • Grid operations: Grid supports distributed execution across machines and browser/OS combinations. It also requires appropriate nodes, capacity, and troubleshooting. The research sources do not provide a general cost or runtime comparison.
  • ScreenshotNeo API cost: If a direct URL capture suits the task, ScreenshotNeo’s free tier includes 1,000 shots per month without a card. Paid plans are Starter $5 for 3,000, Growth $15 for 15,000, Pro $39 for 60,000, Scale $99 for 250,000, and Business $249 for 1,000,000. Yearly billing gives two months free, and every feature is on every plan. Only clean shots are billed.

FAQ

Does saving a screenshot make a Selenium test a visual regression test?

No. It preserves evidence. A visual regression test also needs a baseline, an image comparison method, and a policy for acceptable changes.

Should screenshots be taken on every run?

That depends on whether successful-run images help review or debugging. Many teams capture selected checkpoints or failure evidence to control artifact volume.

When should I use Selenium Grid?

Use Grid when the job needs execution distributed across machines or coverage across browser and operating system combinations. A single agent is simpler when one controlled environment is sufficient.

Can ScreenshotNeo replace Selenium for an end-to-end test?

No. A direct screenshot API captures a page URL; Selenium remains the fit when the test must perform browser interactions or verify a particular WebDriver session.