ScreenshotNeo
Sign in Get a free key

BlogHow-to

How to Run Website Screenshot Change Checks with Python on Ubuntu

Build repeatable visual checks on Ubuntu with Playwright and pytest: create reviewed baselines, compare changes, and investigate noisy diffs.

By the ScreenshotNeo team4 October 202611 min read

Use Playwright for Python to open the page, capture a screenshot, and compare it with a reviewed baseline. Run both baseline creation and later checks with the same Ubuntu release, browser version, viewport, and page state so rendering differences are less likely to look like application changes. The examples below use pytest and Pillow for an explicit pixel-difference check.

Playwright lists Ubuntu 22.04, 24.04, and 26.04 among its supported Linux environments. Check the current Playwright for Python installation guide for your selected Playwright version, Ubuntu release, architecture, and browser dependencies.

1. Install Playwright and Chromium on Ubuntu

Create an isolated environment, install the browser automation library, pytest plugin, and Pillow, then install Chromium and its system dependencies:

sudo apt-get update
sudo apt-get install -y python3 python3-venv python3-pip
mkdir screenshot-checks
cd screenshot-checks
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install playwright pytest-playwright Pillow
python -m playwright install --with-deps chromium

For repeatable CI runs, pin package versions in a requirements file after selecting versions that work in your environment. Keep the Playwright package and its browser binaries aligned: after changing the package version, run the corresponding browser installation command again. The official install guide documents installation and supported environments.

2. Configure pytest and the screenshot baseline

Use this small project layout. The baseline is committed to version control after a person reviews it; test output goes into a separate artifacts directory.

screenshot-checks/
├── conftest.py
├── pytest.ini
├── requirements.txt
├── tests/
│   └── test_visual.py
└── visual/
    └── baseline/
        └── homepage.png

Record the dependencies you chose so another machine can recreate the environment:

playwright
pytest-playwright
Pillow

After confirming compatible versions, replace these unpinned names with the selected version numbers. Add pytest configuration in pytest.ini:

[pytest]
testpaths = tests
addopts = -ra

Create conftest.py to set a consistent viewport and preserve a full-page screenshot if a test fails:

import pytest


@pytest.fixture(scope="session")
def browser_context_args(browser_context_args):
    return {
        **browser_context_args,
        "viewport": {"width": 1440, "height": 900},
        "device_scale_factor": 1,
    }


@pytest.fixture

def browser_context(context, request):
    yield context
    if request.node.rep_call.failed:
        context.pages[0].screenshot(
            path="artifacts/failure.png",
            full_page=True,
        ) if context.pages else None


@pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    setattr(item, "rep_" + report.when, report)

The fixture above writes into artifacts, so create the directory before running, or use the following safer version of the failure capture fixture, which creates it automatically:

import os
import pytest


@pytest.fixture(autouse=True)
def capture_failure(page, request):
    yield
    if getattr(request.node, "rep_call", None) and request.node.rep_call.failed:
        os.makedirs("artifacts", exist_ok=True)
        page.screenshot(path="artifacts/failure.png", full_page=True)


@pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    setattr(item, "rep_" + report.when, report)

Use the second version as the actual conftest.py. It relies on pytest-playwright’s page fixture and the report hook to save a failure capture. The plugin supports browser selection and screenshot options, including full-page screenshots on failure; see the pytest plugin reference.

3. Add a runnable visual comparison test

This test captures a full-page image of a URL from TARGET_URL. Set UPDATE_BASELINE=1 only when intentionally creating or replacing a reviewed reference. Normal runs compare each pixel and fail if the number of changed pixels exceeds the configured allowance.

import os
from pathlib import Path

from PIL import Image, ImageChops
from playwright.sync_api import expect

BASELINE = Path("visual/baseline/homepage.png")
ACTUAL = Path("artifacts/homepage-actual.png")
DIFF = Path("artifacts/homepage-diff.png")


def test_homepage_visual(page):
    url = os.environ.get("TARGET_URL", "https://example.com")
    page.set_viewport_size({"width": 1440, "height": 900})
    page.goto(url, wait_until="networkidle", timeout=60_000)
    page.locator("body").wait_for(state="visible")

    # Replace this with selectors that make your site's capture state stable.
    page.add_style_tag(content="""
        *, *::before, *::after {
            animation: none !important;
            caret-color: transparent !important;
            transition: none !important;
        }
    """)

    image_bytes = page.screenshot(full_page=True, animations="disabled")
    BASELINE.parent.mkdir(parents=True, exist_ok=True)
    ACTUAL.parent.mkdir(parents=True, exist_ok=True)

    if os.environ.get("UPDATE_BASELINE") == "1":
        BASELINE.write_bytes(image_bytes)
        return

    assert BASELINE.exists(), (
        f"Missing {BASELINE}. Review a screenshot, then run with UPDATE_BASELINE=1."
    )
    ACTUAL.write_bytes(image_bytes)

    with Image.open(BASELINE) as expected_file, Image.open(ACTUAL) as actual_file:
        expected = expected_file.convert("RGBA")
        actual = actual_file.convert("RGBA")

    assert expected.size == actual.size, (
        f"Image dimensions differ: baseline={expected.size}, actual={actual.size}. "
        "Check page content, viewport, and full-page capture behavior."
    )

    diff = ImageChops.difference(expected, actual)
    # A pixel counts as changed if any RGBA channel differs by more than 12.
    changed = diff.convert("RGB").point(
        lambda channel: 255 if channel > 12 else 0
    ).convert("L").point(lambda value: 255 if value else 0)
    changed_pixels = sum(1 for value in changed.getdata() if value)
    allowed_changed_pixels = 100

    DIFF.parent.mkdir(parents=True, exist_ok=True)
    diff.save(DIFF)
    assert changed_pixels <= allowed_changed_pixels, (
        f"Visual change: {changed_pixels} pixels exceed the allowance of "
        f"{allowed_changed_pixels}. Inspect {BASELINE}, {ACTUAL}, and {DIFF}. "
        "If the change is intentional, review it before updating the baseline."
    )

In the Python file, write the comparison operators as normal Python characters: the line channel > 12 should be channel > 12 in HTML source rendered as code, and the assertion should use changed_pixels <= allowed_changed_pixels. The entities shown in this article represent > and <=.

This deliberately simple comparator is useful for a first working check, but it does not understand layout or text. A small antialiasing shift can alter many pixels; conversely, a threshold can conceal a subtle change. Tune the channel threshold and changed-pixel allowance for the page, and review the actual and diff images when a check fails. Playwright’s Python screenshot API can save captures to a file or return bytes, and supports full-page and element screenshots; see Screenshots | Playwright Python.

4. Create, review, and update the baseline

  1. Set the target URL and create the initial capture.
  2. Open visual/baseline/homepage.png and check that it represents the intended page, state, and content.
  3. Commit that reviewed baseline with the test.
  4. Run the test normally on the same environment. Inspect artifacts/homepage-actual.png and artifacts/homepage-diff.png after a failure.
  5. For an intentional UI change, inspect the new screenshot, then regenerate and review the baseline in the same environment.
mkdir -p artifacts
TARGET_URL="https://example.com" UPDATE_BASELINE=1 pytest -q
# Review visual/baseline/homepage.png before committing it.
TARGET_URL="https://example.com" pytest -q

Baseline generation is setup, not proof that the page is correct. Playwright’s visual comparison guidance describes the reference-image workflow and notes that rendering can vary with the host OS, browser version, settings, hardware, power source, and headless mode. Keep reference creation and comparison runs consistent, and inspect changes before accepting them: Visual comparisons.

5. Choose capture scope and browser coverage

Need Capture or test choice Trade-off
Catch broad page layout changes page.screenshot(full_page=True) Captures the whole document, including content far below the fold; dynamic or personalized sections can add noise.
Check a component in isolation page.locator(".pricing-card").screenshot() Reduces unrelated page changes; the selector must match a visible, stable element.
Check viewport behavior Set a fixed viewport and use the default viewport screenshot Tests the visible browser viewport, not the entire page.
Check browser-specific rendering Run selected Chromium, Firefox, or WebKit projects Maintain separate baselines per browser because rendering differs.

The pytest plugin uses Chromium by default and can select Firefox or WebKit. Add browser coverage where it matches your users and compatibility risks; avoid multiplying runs without a concrete need. Its configuration and screenshot options are in the official plugin reference.

To capture a component instead, replace the full-page line with:

image_bytes = page.locator(".pricing-card").screenshot()

For a screenshot saved directly by Playwright rather than returned as bytes, use page.screenshot(path="artifacts/page.png", full_page=True). The screenshot API documents capture scope and output options at Playwright Screenshots.

6. Stabilize the page before capture

  • Wait for meaningful readiness. networkidle can be unsuitable for pages with polling, analytics, or long-lived requests. Prefer a page-specific condition, such as page.get_by_role("heading", name="Pricing").wait_for(), after navigation if that better represents a ready page.
  • Control data. Use a test account or deterministic fixture data. Avoid dates, randomized recommendations, rotating banners, and personalized content where possible.
  • Handle fonts and images. Wait for key images and fonts to load if they affect the target. Lazy-loaded images may require scrolling through the page before a full-page capture.
  • Disable motion. Playwright’s screenshot call accepts animations="disabled"; a CSS override can also hide blinking cursors and transitions.
  • Isolate third parties. Ads, chat tools, cookie consent, and remote content can vary. In a test environment, disable or stub them when they are outside the visual contract being tested.
  • Fix the rendering environment. Use the same Ubuntu image, browser version, viewport, device scale factor, fonts, and headless setting for baseline and comparison runs.

There is no single readiness condition that works for every site. Choose a condition that means the specific content under test is in its intended state.

7. Use a perceptual or snapshot comparator when appropriate

The Pillow example compares pixels with a small threshold and a changed-pixel allowance. It is transparent and easy to customize, but can be sensitive to font rasterization and antialiasing. For a larger suite, define the comparison policy explicitly: per-channel tolerance, allowed changed area, ignored dynamic regions, and whether dimensions must match.

Playwright’s documented snapshot comparison flow is part of its Playwright Test runner documentation. In Python, you can capture screenshot bytes through the Python API and feed them to a separate image-diff tool when you need custom processing. Keep the baseline process reviewable whichever comparator you choose.

8. Troubleshooting

Symptom Likely cause Fix
Browser executable missing The installed Playwright version has no matching browser binary. Run python -m playwright install chromium after installing or changing Playwright. On a clean Ubuntu runner, use --with-deps to install browser system dependencies.
Page navigation times out The site never reaches the chosen load condition, or the URL is unreachable from the runner. Check network access and the URL. Use a page-specific readiness wait instead of requiring network idle when the site keeps connections open.
Every run has a visual diff Different OS, browser build, viewport, fonts, device scale factor, data, or animation state. Align the environment and page state, disable motion, and separate browser baselines.
Baseline is missing The normal test run happened before reference creation. Generate a baseline explicitly, inspect it, then commit it. Do not treat automatic baseline creation during a failing CI run as approval.
Screenshot dimensions differ Content height changed, a lazy section loaded differently, or viewport/capture scope changed. Check full-page versus viewport capture, wait for content, and verify the same viewport and page data.
Diff image is too noisy Antialiasing or dynamic content affects many neighboring pixels. Stabilize content and environment first. Then tune the per-channel threshold or allowed pixel count and validate the trade-off against a known intentional change.
Failure screenshot is absent The artifacts directory was unavailable, or the report hook/fixture was not loaded. Use the provided conftest.py, create the output directory, and confirm pytest discovers the file.
Element screenshot fails The selector matched no visible element or matched multiple unstable elements. Use a specific locator, wait for it to become visible, and confirm the element exists in the test data.

9. Performance, reliability, and cost

Each browser launch and page load adds time, so reuse pytest-playwright’s managed fixtures instead of launching a new browser for every assertion. Keep the suite focused: test shared templates and high-value pages, use element captures for component-level checks, and add browser engines based on actual compatibility needs. Full-page screenshots consume more time and memory for long pages than a viewport or element capture.

Visual checks are reliable when the input state is controlled and the reference is reviewed. A passing pixel threshold does not prove accessibility, correct behavior, or semantic correctness; pair visual checks with functional assertions where needed. Preserve actual and diff images as CI artifacts so a failure can be diagnosed without rerunning it.

Playwright is open source; this workflow’s direct infrastructure cost is the Ubuntu runner time, artifact storage, and maintenance of pinned environments and baselines. The research sources provide no benchmark or universal runtime estimate, so measure on your own pages and CI runners.

Or skip the browser setup

ScreenshotNeo is a website screenshot API and MCP server from Yorker Media. One GET request returns an image or PDF. Cookie banners are accepted and removed along with known consent platforms, newsletter popups, and chat widgets before capture; individual steps can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and the response identifies the page verdict and billing status in headers.

For a direct capture you can use this cURL request (replace the URL and API key):

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', Buffer.from(await res.arrayBuffer())));

See the ScreenshotNeo API documentation for request options. An MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up free and make your first capture.

Frequently asked questions

How do I compare website screenshots with Python?

Capture the page with Playwright, keep a reviewed reference image, and compare each later capture with that reference. The example uses Pillow; you can substitute a diff tool if you need a different tolerance policy.

How do I run Playwright visual tests on Ubuntu?

Install the Python package and browser binaries in a virtual environment, then run the tests with pytest. Confirm your Ubuntu release and architecture against Playwright’s current supported-platform documentation.

How do I update screenshot baselines?

Review the current actual and diff images first. If the change is intended, regenerate the reference in the same environment and commit the reviewed image alongside the UI change.

Should I capture the full page or an element?

Use full-page capture for page-wide layout and element capture when a component’s appearance is the contract you need to protect.

Keep reading

7 min read

Full-Page Capture Chrome Plugin

Capture an entire webpage in Chrome with extensions, understand their limits, and automate clean full-page screenshots with ScreenshotNeo.

7 min read

Facebook Open Graph Debugger

Use Facebook’s Open Graph debugger to refresh stale link previews, verify metadata, and fix incorrect titles, descriptions, and images.