How to Run Parallel Selenium Tests with Python unittest

Direct answer: Keep your tests as unittest.TestCase classes, then run them with pytest and pytest-xdist. pytest supports unittest tests, and xdist runs tests in multiple worker processes. Each test should create and close its own Selenium WebDriver session.

python -m pip install pytest pytest-xdist selenium
pytest -n 4

Start with a small worker count and adjust it based on the CPU, memory, browser capacity, and run time available in your target environment. Use Selenium Grid when you need remote machines, browser versions, or operating systems.

1. Install the parallel test runner

Install pytest, pytest-xdist, and Selenium in the same Python environment that runs your suite:

python -m pip install pytest pytest-xdist selenium

Check the installed commands and versions:

python -m pytest --version
python -m pip show pytest-xdist selenium

pytest discovers unittest-style test classes and methods, so you do not need to rewrite them as pytest functions. Keep normal unittest naming conventions: test files should generally match test_*.py or *_test.py, classes should start with Test unless they inherit from unittest.TestCase, and methods should start with test_.

2. Make each test own its WebDriver

Create a fresh browser session in setUp, and register quit immediately with addCleanup. unittest runs cleanups even when the test fails, which helps prevent leaked browser processes.

import unittest
from selenium import webdriver


class SearchTests(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.addCleanup(self.driver.quit)

    def test_search_page_title(self):
        self.driver.get("https://example.com")
        self.assertIn("Example", self.driver.title)

    def test_page_has_heading(self):
        self.driver.get("https://example.com")
        heading = self.driver.find_element("tag name", "h1")
        self.assertTrue(heading.is_displayed())


if __name__ == "__main__":
    unittest.main()

Save this as test_search.py, then run it serially with python -m unittest or python -m pytest. When using xdist, invoke pytest; the standard unittest command does not provide xdist’s worker scheduling.

Keep driver creation in per-test setup. Do not store one shared driver in a module global or class variable: workers and tests should not issue concurrent commands to the same session.

3. Run the suite with multiple workers

From the project root, run a fixed number of worker processes:

python -m pytest -n 4

The -n option is an alias for --numprocesses. You can also select automatic worker detection:

python -m pytest -n auto

Automatic detection uses the available physical CPU core count. For browser suites, this is only a starting point: each worker can launch a browser process, so memory, CPU contention, and remote session limits can become the true constraint. A fixed count is usually easier to keep stable in CI.

Useful invocations:

# Run the suite in four worker processes
python -m pytest -n 4

# Run a particular file in four workers
python -m pytest -n 4 tests/test_search.py

# Select tests by keyword and distribute those tests
python -m pytest -n 4 -k search

# Show more detail for failures
python -m pytest -n 4 -v

pytest-xdist distributes tests among worker processes. The exact completion time depends on test duration, startup overhead, and the resources available; parallel execution does not guarantee a proportional speedup.

4. Keep parallel tests independent

Parallelism exposes hidden dependencies that a serial run can conceal. Before increasing workers, check that tests can run in any order and at the same time.

Use separate browser sessions. Create the driver per test and always quit it.
Isolate mutable test data. Give tests unique records, usernames, temporary directories, or other resources when they create or modify them.
Avoid order dependencies. A test should establish the state it needs rather than rely on another test having run first.
Do not share mutable globals. Worker processes are separate, and relying on in-memory state shared by serial tests will fail under distribution.
Make collection deterministic. Workers must collect the same tests in the same order. Avoid test discovery that changes based on timing, external state, or worker-specific data.
Account for external limits. A test environment, account pool, or remote browser service can have a lower concurrency limit than the test runner.

These are practical consequences of running tests concurrently in separate processes. If a test is inherently stateful, mark or organize it so it runs in a controlled way, and keep the rest of the suite parallel.

5. Choose between local workers and Selenium Grid

Need	Approach	What it provides
Parallelize a unittest suite on one machine	pytest + pytest-xdist	Test discovery and scheduling across local worker processes.
Remote machines or multiple browser and platform configurations	Selenium Grid	Remote WebDriver sessions routed to browser instances.
Local scheduling with remote browser capacity	pytest-xdist + Grid	xdist distributes tests while each test requests a remote session from Grid.

Grid is useful for parallel testing across browser types, versions, and operating systems. Its execution time depends on the number and capacity of available nodes. When combining xdist and Grid, make sure worker concurrency does not exceed the Grid session capacity.

Here is a minimal remote-driver variation. Start or connect to your Grid and use its actual WebDriver endpoint in place of the example address:

import unittest
from selenium import webdriver
from selenium.webdriver.chrome.options import Options


class RemoteSearchTests(unittest.TestCase):
    def setUp(self):
        options = Options()
        self.driver = webdriver.Remote(
            command_executor="http://localhost:4444",
            options=options,
        )
        self.addCleanup(self.driver.quit)

    def test_search_page(self):
        self.driver.get("https://example.com")
        self.assertIn("Example", self.driver.title)

Then schedule those cases with a worker count that fits the available Grid sessions:

python -m pytest -n 4

See the official Selenium Grid overview and Grid applicability guidance for remote execution and when Grid fits.

6. Tune worker count for time and capacity

Establish a serial baseline. Run the same selection without -n and record elapsed time and failures.
Try a modest count. Start with two workers, then increase to four if the machine or Grid has capacity.
Watch resource use. Browser processes consume CPU and memory; monitor the CI runner and browser nodes during a representative run.
Compare repeat runs. A single fast run can be misleading if tests or infrastructure are variable. Compare stable, equivalent runs.
Set a capacity ceiling. Keep the configured worker count within local resources and remote session limits.

More workers can increase contention, browser startup overhead, and failures caused by overloaded dependencies. Treat concurrency as a capacity setting, not a target to maximize. Selenium’s Grid documentation includes an illustrative relationship between test count, average test time, and node count; it is not a benchmark or a promise of linear speedup.

7. Troubleshooting

Symptom	Likely cause	Fix
`pytest: error: unrecognized arguments: -n`	pytest-xdist is not installed in the active environment.	Run `python -m pip install pytest-xdist` with the same Python interpreter used to invoke pytest.
No tests are collected	File, class, or method names do not match discovery conventions, or the command runs from the wrong directory.	Use a `test_.py` file and `test_` methods; run pytest from the repository root or pass the test path explicitly.
Browser or driver startup fails	The browser is unavailable, incompatible with the driver setup, or the environment cannot launch it.	Check browser availability and Selenium’s driver configuration in that environment. For remote execution, verify the Grid endpoint and node readiness.
Tests pass serially but fail with workers	Shared mutable test data, order assumptions, or resource contention.	Make test state independent, allocate unique data per test, and lower the worker count to distinguish collisions from capacity problems.
Grid reports no available slot or session	Worker concurrency exceeds Grid capacity, or nodes are unavailable.	Reduce `-n` to the number of sessions the Grid can serve, and check node availability.
Browser processes remain after failures	The driver is not reliably closed.	Register `self.driver.quit` with `self.addCleanup` immediately after creating the driver.
Parallel run is slower than serial	Startup overhead or CPU, memory, network, or service contention outweighs scheduling gains.	Try fewer workers, compare representative runs, and check the constrained resource before raising concurrency again.
Workers disagree about collected tests	Collection depends on unstable external or runtime state.	Make test discovery deterministic and independent of worker identity and timing.

8. Screenshot a page without maintaining browser setup

For test assertions, keep Selenium and unittest. For a screenshot artifact or visual reference of a URL, ScreenshotNeo provides a website screenshot API and MCP server for developers. Its API accepts one GET request and returns an image or PDF. The examples below use the documented API; see the ScreenshotNeo API documentation for request options.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://example.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://example.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);
if (!res.ok) throw new Error(`Screenshot request failed: ${res.status}`);
const bytes = new Uint8Array(await res.arrayBuffer());
await import('node:fs/promises').then(fs => fs.writeFile('shot.webp', bytes));

ScreenshotNeo accepts cookie and consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each of those steps can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers report the page verdict and billing status. Its MCP server includes take_screenshot, get_page_info, and capture_pdf for AI agents. The free plan includes 1,000 shots a month with no card; paid plans start at $5 for 3,000 shots.

FAQ

Do I have to rewrite unittest tests to use pytest?

No. pytest supports unittest test cases. You can keep the existing test classes and use pytest-xdist for process-based execution.

Can I use `unittest.main()` and xdist together?

Use pytest to launch the distributed run, for example python -m pytest -n 4. The unittest.main() block remains useful for direct unittest runs, but unittest itself does not provide xdist’s worker scheduling.

Should I always use `-n auto`?

No. It detects physical CPU cores, but browser memory use or Grid capacity may support fewer concurrent sessions. Choose a count that fits the environment.

Does parallel execution make the suite proportionally faster?

Not necessarily. Scheduling overhead, browser startup, shared dependencies, and resource limits affect the result. Measure in the environment where the suite runs.

Can xdist run tests on multiple operating systems?

xdist provides worker processes; Selenium Grid is the layer for remote machines and browser or platform configurations.

Sources

Or skip the browser setup

Make one GET request to get a screenshot without installing or managing a browser for that capture:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.com -o shot.webp

ScreenshotNeo removes cookie banners, popups, and chat widgets before the shot. Bot checks, blank pages, and failed loads are never billed. An MCP server lets AI agents take screenshots. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. See the API docs, then sign up for 1,000 free screenshots a month.