How Selenium WebDriver Works: The Client-Server Transport Layer
Follow a Selenium command from your test through HTTP to the browser, and learn how sessions, Grid, and WebDriver BiDi fit together.
Selenium WebDriver lets a test control a browser through a language-level API. Underneath that API, the classic WebDriver protocol sends commands over HTTP from a local end to a remote end. A session ID ties later commands to the browser session created at the start. With Selenium Grid, Grid routes those requests to a remote WebDriver end node. WebDriver BiDi adds a WebSocket channel for bidirectional communication, including browser events.
This is the client-server transport layer: how commands travel, how the server knows which session they belong to, and how responses get back to your test. It does not require the test author to construct HTTP messages for normal Selenium use. Selenium’s binding does that work.
How does Selenium WebDriver communicate with the browser?
The short version is: your test calls a Selenium binding, the binding sends a protocol command over HTTP, the remote end runs it against the browser, and the binding turns the response into a return value or exception.
- Start a session. The binding starts or connects to a WebDriver remote end and requests a session with browser options or capabilities. This corresponds to the WebDriver New Session command.
- Receive a session ID. The remote end creates the session and returns an identifier. The binding keeps it so subsequent commands target that session.
- Send commands. Calls such as navigation, finding an element, or clicking become HTTP requests. The HTTP method and URL identify the protocol endpoint and command.
- Run the command remotely. The remote end executes the requested steps against the browser and returns a response. A successful response contains command results; an unsuccessful one describes a WebDriver error that the binding exposes to the program.
- Close the session. Calling
quit()sends Delete Session. The remote end removes the session and may close its browser process.
The W3C Recommendation defines a session as the connection between a local end and a specific remote end. In direct local use, the driver service and browser are on the same machine as the test. In remote use, the client sends requests to a remote endpoint, often Selenium Grid, which routes them to a node running WebDriver and the browser.
What happens when I call driver.get()?
At the API level, driver.get(url) looks like a method call. At the transport level, the binding encodes it as a WebDriver navigation command associated with the active session. The remote end receives the HTTP request, runs the navigation steps in the browser, and replies when the command completes or fails. The binding then returns control to your code.
The exact endpoint path is determined by the WebDriver protocol and the remote end’s routing. The 2026 WebDriver 2 document is a Working Draft; it describes remote ends that can use a URL prefix, such as routing New Session through POST /wd/session instead of POST /session. Treat that detail as draft text, not as a change to the 2018 Recommendation.
Run a local Selenium session in Python
Install Selenium in your Python environment, ensure a compatible browser and driver setup is available, then run this script. It opens a page, prints its title, and always asks Selenium to end the session even if navigation or title retrieval fails.
from selenium import webdriver
options = webdriver.ChromeOptions()
# Add browser-specific options here, for example:
# options.add_argument("--headless")
driver = webdriver.Chrome(options=options)
try:
driver.get("https://example.com")
print(driver.title)
finally:
driver.quit()The binding hides the protocol encoding and HTTP exchange. You configure the requested browser session through options; the driver object owns the session ID and uses it for commands. Call quit() when the test is finished. Avoid relying on process exit to clean up a session, especially when the remote browser is on another machine.
Run against a remote WebDriver endpoint
For a Selenium Grid or another remote WebDriver endpoint, provide the endpoint URL and browser options. The client-side API remains familiar; the network destination and browser execution location change.
from selenium import webdriver
options = webdriver.ChromeOptions()
# Configure capabilities/options supported by the remote browser.
driver = webdriver.Remote(
command_executor="http://localhost:4444",
options=options,
)
try:
driver.get("https://example.com")
print(driver.title)
finally:
driver.quit()Replace the example endpoint with the address of your Grid or remote WebDriver service. The endpoint may be configured with a path prefix by the deployment. Keep credentials and any authenticated endpoint details out of source control.
See the HTTP transport with cURL
Normally, use a Selenium binding. Raw HTTP is useful for understanding the protocol or diagnosing a server. The following illustrates the New Session request and then a navigation request. It assumes a WebDriver server listening at localhost:4444, the conventional session endpoint, and a JSON response containing a session ID. Remote deployments may use a different host or path prefix.
curl -i -X POST http://localhost:4444/session \
-H 'Content-Type: application/json' \
-d '{"capabilities":{"alwaysMatch":{"browserName":"chrome"}}}'Read the returned session identifier from the response, then substitute it below. The session ID is part of the command path so the remote end can associate the navigation with the established session.
curl -i -X POST http://localhost:4444/session/SESSION_ID/url \
-H 'Content-Type: application/json' \
-d '{"url":"https://example.com"}'End the session with Delete Session:
curl -i -X DELETE http://localhost:4444/session/SESSION_IDThese examples show the shape of the exchange, not a replacement for the binding’s error handling or capability negotiation. A server can return a protocol error, and an intermediary can reject or fail a request before it reaches the browser.
How Selenium Grid changes the route
With local WebDriver, the test and driver service run locally and the browser is controlled on that machine. With Remote WebDriver, the client connects to a remote endpoint. If that endpoint is Grid, Grid routes the request to a suitable node where WebDriver controls the browser. The response follows the route back to the client.
Grid adds a network hop and moves browser execution; it does not change the basic WebDriver API concept. This matters when diagnosing latency and connectivity: distinguish failure to reach the Grid endpoint from a command failure at the browser node. Selenium’s remote documentation describes the client-to-Grid-to-end-node path.
What is a WebDriver session ID?
The session ID identifies the active browser automation session at the remote end. Session creation returns it; later commands carry it so the server can find the right browser context. It is not a browser window identifier, a URL, or a reusable login token. When the session is deleted, commands using that session ID should no longer succeed.
Keep the WebDriver object that owns the session for the lifetime of the test. Do not manually mix session IDs between parallel tests. Each parallel browser session needs its own session context, and every session should be closed when its work is done.
Classic WebDriver and WebDriver BiDi
| Aspect | Classic WebDriver | WebDriver BiDi |
|---|---|---|
| Transport model | HTTP request and response commands | WebSocket channel supporting bidirectional communication |
| Typical interaction | Client sends a command and receives its result | Client and browser can exchange messages, including streamed events |
| Best mental model | Sequential command protocol | Command interaction plus event streaming |
| Availability | Core WebDriver protocol | Feature support varies across browser and Selenium implementations |
BiDi complements the classic command model; it does not mean every Selenium command has moved to WebSocket. Check the Selenium and browser documentation for the specific BiDi features supported by the versions you deploy. Selenium describes BiDi as adding WebSocket communication and event streaming.
Reliability, performance, and cost considerations
Reliability
- Always close sessions. Put
quit()in afinallyblock or equivalent cleanup hook so failed assertions do not leave remote browsers running. - Keep session ownership clear. A session ID is the context for commands. Sharing one driver across unrelated parallel work can interleave operations and make failures hard to interpret.
- Separate transport failures from browser failures. A connection refusal or timeout reaching the endpoint differs from a valid WebDriver error returned by the remote end.
- Account for intermediary behavior. Grid, proxies, and network policies can affect routing, allowed paths, timeouts, and connection availability.
Performance
Each classic command requires a request and response, so excessive fine-grained calls can add transport overhead, especially when the client and browser are separated by a network or Grid. Prefer meaningful operations and avoid repeatedly fetching the same state in a tight loop. Browser navigation and page behavior can dominate elapsed time; protocol transport is only one part of total test time. No universal latency figure applies across local, Grid, and hosted deployments.
Cost
The WebDriver protocol itself does not set a service price. Your cost depends on where browsers run, how long sessions occupy machines, and whether you use a hosted browser or Grid provider. Close sessions promptly and avoid retry loops that create duplicate sessions. If you only need a rendered website image or PDF rather than browser interaction, a screenshot API may fit the task with less browser infrastructure to operate.
Troubleshooting common transport problems
| Symptom | Likely cause | What to check or fix |
|---|---|---|
| Connection refused when creating a driver | No driver service or remote endpoint is listening at the configured address. | Confirm the service is running, the hostname and port are correct, and the client can reach that network address. |
| Remote session creation fails | The endpoint, requested browser, or capabilities are unsupported or incorrectly configured. | Check the endpoint URL and browser options against the remote service’s supported configuration. Inspect the server’s returned WebDriver error. |
| Unknown command or 404 response | The request reached the wrong route, possibly because the endpoint path or prefix is wrong. | Use the service’s configured WebDriver base URL. Do not assume every deployment exposes the same path prefix. |
| Invalid session ID | The session was deleted, never created successfully, or the command uses the wrong session identifier. | Create a fresh session and ensure commands use the ID returned for that same session. |
| Command hangs or times out | The remote end, Grid route, browser, or page is slow or unreachable; a network intermediary may also be involved. | Identify whether the timeout is in client-to-server connectivity, WebDriver command handling, or page loading. Review the relevant configured timeout and service logs. |
| Browser closes unexpectedly | The session ended, the browser process failed, or the remote node became unavailable. | Check node and browser logs, avoid sending more commands after quit, and create a new session after a node failure. |
| Raw cURL works but Selenium fails (or the reverse) | The binding may negotiate different capabilities or use a different endpoint URL than the manual request. | Compare the actual remote URL and requested capabilities. Prefer the binding for normal use; use server-side logs to inspect protocol errors. |
Or skip the browser setup
If the task is to capture a page rather than interact with it, ScreenshotNeo is a website screenshot API and MCP server. One GET request returns a PNG, JPEG, WebP, or PDF. Its API call does not use Selenium or create a WebDriver session.
cURL:
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpPython:
import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)Node.js:
const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);ScreenshotNeo accepts cookie or consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each cleanup step can be turned off. Bot checks, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000. See the ScreenshotNeo API documentation and sign up for 1,000 free screenshots a month with no card.
FAQ
Does Selenium send commands directly to the browser?
The binding sends WebDriver protocol commands to a remote end. A browser driver or remote WebDriver implementation performs the browser control. In Grid deployments, Grid routes the request to the end node.
Is the session ID sent with every command?
Commands that operate within a session identify that session in their protocol route. Session creation and session deletion are special lifecycle commands.
Does BiDi replace HTTP WebDriver?
No. Selenium documents BiDi as adding a WebSocket channel for bidirectional communication and event streaming alongside the classic command interaction.
Can I use raw HTTP instead of Selenium bindings?
Yes, a client can send protocol requests directly, but then it must handle session creation, IDs, endpoint paths, response parsing, errors, and cleanup. Bindings are generally simpler for test code.
Sources
- W3C WebDriver Recommendation — sessions, session IDs, command behavior, and teardown.
- W3C WebDriver 2 Working Draft, 28 May 2026 — request routing and URL prefix example; this is draft text.
- W3C WebDriver standards index — publication status of the Recommendation and Working Draft.
- Selenium documentation: Driver Sessions, Remote WebDriver, and WebDriver BiDi.
