ScreenshotNeo

BlogEngineering

Why Cloudflare Treats Headless Selenium Chrome Differently and How to Test It

Cloudflare does not use a single Selenium rule. Learn which browser signals matter and how to run a controlled, repeatable test on your own zone.

By the ScreenshotNeo team30 September 20269 min read

Why Cloudflare Treats Headless Selenium Chrome Differently and How to Test It

Short answer: Cloudflare can treat headless Selenium Chrome differently because its bot defenses combine request, header, session and browser evidence. Cloudflare documents several detection engines, including heuristics, JavaScript Detections (JSD) and plan-dependent machine learning. JSD explicitly identifies headless browsers and malicious fingerprints, but Cloudflare does not publish a deterministic “Selenium equals bot” rule. A challenge therefore does not prove that headless mode alone caused it.

The reliable way to investigate is to compare a normal interactive Chrome session with Selenium-controlled Chrome against a domain you administer. Keep the URL, Chrome version, network, account, cookies and timing as similar as practical. Record the HTTP result, challenge behavior, available security-event fields, JSD outcome and bot score. Repeat the comparison because session state and bot scoring can change between requests.

What Cloudflare is actually evaluating

Cloudflare describes a layered system rather than one browser switch. Heuristics run broadly. JavaScript Detections inject a lightweight, invisible script into HTML responses to identify headless browsers and other malicious fingerprints. Business and Enterprise Bot Management can add machine learning based on request, session and browser features. Which engines and fields you can inspect depends on the zone’s plan and configuration. See Cloudflare’s bot detection engines documentation.

Cloudflare also describes a __cf_bm cookie that smooths bot scores using a user’s request pattern. That means a fresh browser profile, a warmed-up session and a session that changed IP can produce different observations. Browser rendering characteristics, headers and JavaScript behavior can all contribute evidence. None of those facts establishes that every Selenium session is detected.

JSD is not the same as the final decision

JSD runs after the browser receives HTML and executes the injected script. Cloudflare says the first request generally has no JSD result because the browser must first receive the page and run the script. A JSD pass does not prevent other heuristics from assigning a low bot score. A JSD failure also does not block traffic by itself: enforcement requires a WAF custom rule that uses the field. Cloudflare recommends managed challenges and warns that first-page requests, native mobile or API clients and WebSocket endpoints may legitimately lack a JSD result. Read the JSD documentation before interpreting a missing value as a failure.

Bot scores need careful interpretation

When Bot Management evaluates a request, Cloudflare documents scores from 1 through 99. A score of 0 means the request was not evaluated; it does not mean “safe.” Cloudflare also documents that a missing or empty User-Agent can produce score 1. Check your proxy, Selenium capabilities and any extension that edits headers before blaming headless Chrome. See Cloudflare’s bot score reference.

Design a controlled test on a zone you own

  1. Define the boundary. Use a staging hostname or test page you administer. Do not probe a third-party site to infer its private detection rules. Write down the Cloudflare plan, enabled bot products, JSD status and WAF rules.
  2. Create a stable page. Prefer a small HTML page that you control. Avoid changing content, login state or redirects while comparing modes. Record the exact URL and query string.
  3. Capture a baseline. Open the page in ordinary interactive Chrome. Save the final URL, status code, response headers, cookies, any interstitial challenge and the fields visible in Cloudflare Security Events or logs. Do not treat the first navigation as a JSD verdict.
  4. Run Selenium. Use the same Chrome major version, operating system, outbound IP, account and approximate timing. Start with a clean profile, then repeat with a persistent profile so you can distinguish session effects from browser-mode effects.
  5. Compare like-for-like requests. Examine the first HTML navigation separately from later requests made after JavaScript executes. Compare challenge behavior, JSD result, bot score and WAF action. Keep a timestamp for every run.
  6. Repeat. Run multiple sessions on different minutes. A single challenge is an observation, not proof of a Selenium-only cause.
A controlled comparison keeps browser, request, session and network variables visible.
A controlled comparison keeps browser, request, session and network variables visible.

Runnable Selenium test in Python

Install Selenium and a compatible Chrome or Chromium installation:

python -m pip install selenium

The script below visits the same URL in visible and headless modes, records navigation timing and saves the final HTML. It does not bypass a challenge; it measures what your own zone returns.

from pathlib import Path
from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

TARGET = "https://staging.example.com/test-page"


def run(headless: bool, label: str) -> None:
    options = Options()
    if headless:
        options.add_argument("--headless=new")
    options.add_argument("--window-size=1440,1000")
    options.add_argument("--disable-gpu")
    # Keep a normal, explicit User-Agent while testing.
    options.add_argument(
        "--user-agent=Mozilla/5.0 (X11; Linux x86_64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/126.0.0.0 Safari/537.36"
    )

    driver = webdriver.Chrome(options=options)
    try:
        driver.get(TARGET)
        # Allow post-navigation JavaScript and JSD time to run.
        sleep(5)
        print({
            "mode": label,
            "title": driver.title,
            "url": driver.current_url,
            "user_agent": driver.execute_script("return navigator.userAgent"),
            "ready_state": driver.execute_script("return document.readyState"),
        })
        Path(f"{label}.html").write_text(driver.page_source, encoding="utf-8")
        driver.save_screenshot(f"{label}.png")
    finally:
        driver.quit()


run(False, "interactive")
run(True, "headless")

For a useful experiment, add a third run using a persistent Chrome profile, then a run from the same profile after a normal interactive visit. Keep those runs labeled clearly. A profile change can alter cookies and session signals, so it is a variable to record, not a hidden fix.

What to record from Cloudflare

Observation Why it matters
HTTP status and final URL Separates redirects, origin failures and challenge responses.
Challenge or interstitial type Shows that Cloudflare required a client interaction, but not which signal caused it.
JSD result Useful only after a browser has received HTML and executed the script.
Bot score Score 0 means not evaluated; low scores can result from several signals.
User-Agent and client IP Detects proxy mistakes, empty headers and changed network paths.
WAF action and rule ID Shows whether a custom rule enforced a JSD or score condition.
Timestamp and request ID Lets you match browser output with Security Events or logs.

Cloudflare’s challenge documentation explains that challenge outcomes depend on the request and the configured action. A challenged request is therefore evidence about that request and configuration, not a universal Selenium classification.

Common confounders and edge cases

First navigation versus a later request

The initial HTML response may arrive before JSD has produced an outcome. Compare a later browser-facing request, such as an image, API call or reload, separately. Do not label a missing first-request JSD field as “failed.”

Changed IP or proxy path

If the challenge page is fetched through one proxy and the follow-up request through another, Cloudflare sees a different session context. Keep the outbound address stable. Record VPN, corporate proxy and IPv4/IPv6 changes.

Missing or modified headers

An empty User-Agent can receive score 1. Header rewriting extensions, reverse proxies and custom Selenium capabilities can also change Accept-Language, client hints or cookies. Capture the headers at the edge where possible.

Browser version and rendering differences

Cloudflare’s engineering material discusses client-side browser signals, including rendering output affected by hardware and software. A different Chrome build, operating system, graphics stack or extension can change those signals. Keep the environment constant when comparing modes.

Legitimate clients without JSD

Native mobile apps, API clients, WebSocket endpoints and some first requests may not execute JSD. A WAF rule that blocks every missing result can catch legitimate traffic. Follow Cloudflare’s guidance and test the rule against known-good clients.

Cloudflare Browser Run

Cloudflare’s Browser Run service sends identifying headers and method-specific bot-detection IDs. Cloudflare recommends a WAF skip rule when scanning a zone you own, so the zone’s protection does not interfere. That advice is specific to Browser Run and is not a Selenium bypass method. See the automatic request headers documentation.

Troubleshooting checklist

Symptom Likely cause Fix
Headless gets a challenge, interactive Chrome does not Different browser signals, session state, timing or network. Repeat with the same Chrome build, IP, profile state and wait period. Inspect score and WAF fields.
Both modes are challenged Zone policy, IP reputation, rate limit or an origin rule. Check Security Events, WAF rules and rate limits before attributing it to Selenium.
JSD is empty First request, non-HTML response, unsupported client or script did not execute. Load an HTML page, wait for execution, then inspect a subsequent request.
Bot score is 0 Bot Management did not evaluate the request. Do not call it safe or blocked; verify plan coverage and the request type.
Score is 1 unexpectedly Missing or empty User-Agent, or another strong signal. Inspect edge and browser headers, proxy rewrites and extensions.
Results change between runs Cookies, __cf_bm, IP, timing or adaptive scoring. Log all variables and use repeated, comparable sessions.
Selenium cannot start Chrome Driver/browser version mismatch or unavailable binary. Install a matching Chrome/Chromium and Selenium Manager-supported driver; print versions before the test.

Performance, reliability and cost of this test

Headless mode usually reduces local display overhead, but it does not guarantee faster Cloudflare decisions. Network latency, JavaScript execution, challenge interaction and origin response time dominate many runs. Measure navigation time, time to document.readyState, and time until your target selector appears. Use the same explicit timeout and avoid comparing one warm profile with one cold profile.

For reliability, store raw HTML, screenshots, timestamps and request identifiers. Keep test pages deterministic and run a small matrix rather than one long batch. If your test changes WAF rules, apply the change to a staging zone first and document the exact rule expression. Cloudflare features and fields vary by plan, so verify current availability in the official documentation.

Or skip the browser setup

If your goal is a clean screenshot rather than measuring Cloudflare behavior, ScreenshotNeo provides a website screenshot API and MCP server. It accepts one GET request and returns PNG, JPEG, WebP or PDF. Before capture it accepts consent banners and removes more than 60 known consent platforms, newsletter popups and chat widgets. Bot checks, blank pages, timeouts, failed loads and cache hits are not billed, and the response identifies the result with X-Page-Verdict and X-Billed headers.

See the ScreenshotNeo API documentation for all options. The same endpoint supports full-page capture with lazy images, CSS-selector element capture, dark mode, device presets, custom viewports, retina scale, PDF paper and margin settings, custom CSS and JavaScript, clicks, waits, blocked requests, headers, cookies, user agents, authorization, timezone, geolocation, transparent backgrounds, resizing, cache TTL, signed image links, asynchronous webhooks, bulk capture and usage reporting.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" \
  -d access_key=YOUR_API_KEY \
  --data-urlencode url=https://stripe.com \
  -o shot.webp

Python

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
r.raise_for_status()
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot failed: ${res.status}`);
const buffer = Buffer.from(await res.arrayBuffer());
require('fs').writeFileSync('shot.webp', buffer);

ScreenshotNeo also provides an MCP server with take_screenshot, get_page_info and capture_pdf tools for Claude, Cursor and other MCP clients. Plans include 1,000 free shots per month with no card, then Starter at $5 for 3,000; yearly billing gives two months free. Paid plans start at $5 and every feature is available on every plan. Create a free ScreenshotNeo account.

FAQ

Does Cloudflare always detect headless Chrome?

No. Cloudflare uses multiple signals and does not publish a deterministic Selenium-only rule. Outcomes depend on the request, session, browser and zone configuration.

ScreenshotNeo clears common consent and overlay elements before capture.
ScreenshotNeo clears common consent and overlay elements before capture.

Does passing JSD prove the browser is trusted?

No. Cloudflare says JSD can pass while other heuristics still assign a low bot score.

Can I test this against any public website?

Use a zone you control. Testing a third-party site can violate its terms and cannot reveal its private WAF configuration reliably.

Why did a score of zero look like a successful request?

Score 0 means Bot Management did not evaluate that request. It is not a safety rating.

What is the most useful comparison?

Compare interactive and Selenium-controlled Chrome on the same page, with the same network, Chrome version, account and timing, then repeat enough times to expose session and timing effects.