How to Build a Web Bot with Selenium and Python

Selenium lets a Python program send commands to a real browser through WebDriver. A small bot can open a page, locate a stable element, interact with it, wait for the resulting page state, check the outcome, and close the browser. Selenium does not grant access to a website or decide whether a particular automation task is permitted; choose a target and actions you are authorized to automate.

This guide builds a local, single-browser script. It uses Selenium Manager for ordinary browser-driver setup, condition-based waits for dynamic pages, and try/finally so the browser session is closed even when a step fails. The examples use a placeholder URL and locator: replace them with a page and element you have verified.

1. Install Python and Selenium

The current Selenium Python API documentation lists Python 3.10 or newer and documents installation with pip. Check the current Python API documentation for supported Python versions and browsers when setting up a new environment. A virtual environment keeps this project’s packages separate.

python -m venv .venv

# macOS or Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

python -m pip install -U selenium

On most supported local configurations, Selenium Manager handles obtaining the browser driver when the script creates a WebDriver. A separate Selenium server is not required for a local browser. Install a supported browser, then let Selenium Manager handle the usual driver setup. Manual browser or driver configuration remains an option when your environment requires it.

2. Build a small browser bot

This example opens a page, waits for its heading, finds a form field and button, submits text, waits for a visible result, verifies that result, and always ends the session. It follows the basic workflow in Selenium’s first-script guide.

Replace PAGE_URL, FIELD_LOCATOR, SUBMIT_LOCATOR, and RESULT_LOCATOR with values inspected on your permitted target page. The example assumes the page has a text input with the name q, a submit button, and an element with the ID result; those are illustrative selectors, not a claim about a real site.

from selenium import webdriver
from selenium.common.exceptions import TimeoutException, WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

PAGE_URL = "https://example.com/search"
FIELD_LOCATOR = (By.NAME, "q")
SUBMIT_LOCATOR = (By.CSS_SELECTOR, "button[type='submit']")
RESULT_LOCATOR = (By.ID, "result")


def main():
    driver = webdriver.Chrome()
    try:
        driver.get(PAGE_URL)
        print(f"Page title: {driver.title}")

        wait = WebDriverWait(driver, 15)
        field = wait.until(EC.visibility_of_element_located(FIELD_LOCATOR))
        field.send_keys("selenium")

        submit = wait.until(EC.element_to_be_clickable(SUBMIT_LOCATOR))
        submit.click()

        result = wait.until(EC.visibility_of_element_located(RESULT_LOCATOR))
        result_text = result.text.strip()
        if not result_text:
            raise RuntimeError("The result element appeared but contained no visible text")

        print(f"Result: {result_text}")
    except TimeoutException as exc:
        raise RuntimeError(
            "Timed out waiting for a page element. Check the URL, locator, "
            "page state, and whether the element is inside a frame."
        ) from exc
    except WebDriverException as exc:
        raise RuntimeError(
            "WebDriver could not complete the browser operation. Check the browser "
            "installation, startup output, and network or page error."
        ) from exc
    finally:
        driver.quit()


if __name__ == "__main__":
    main()

Save this as bot.py and run python bot.py. The sample is structurally runnable after you provide selectors and a URL that match a page you can access. The official Selenium first-script guide shows the same core lifecycle: start a session, navigate, locate and interact, inspect a result, then end the session.

What each step does

webdriver.Chrome() starts a Chrome WebDriver session. Selenium Manager normally handles driver setup for supported configurations.
driver.get(PAGE_URL) navigates the browser to the page.
By.NAME, By.CSS_SELECTOR, and By.ID describe how Selenium should locate elements.
WebDriverWait(...).until(...) waits for the specific state the next operation needs.
send_keys types into the field; click activates the button.
The result is read and checked so the script reports a concrete outcome rather than treating a click as proof of success.
The finally block calls quit() on success or failure, ending the whole WebDriver session.

3. Choose stable locators

A locator is the rule Selenium uses to find an element. Prefer an ID, name, or CSS selector tied to a stable attribute when the page offers one. A CSS selector such as button[data-testid='save'] can be clearer and less fragile than a selector based on generated classes. Avoid absolute XPath paths and classes that appear to be generated for a particular build unless you have verified they are stable.

Locator	Example	Good fit
ID	`(By.ID, "result")`	An element has a unique, stable ID.
Name	`(By.NAME, "q")`	A form field has a stable name attribute.
CSS selector	`(By.CSS_SELECTOR, "button[type='submit']")`	You can anchor on a stable attribute or relationship.
Tag name	`(By.TAG_NAME, "h1")`	You need a simple element such as a page heading.
XPath	`(By.XPATH, "//button[normalize-space()='Continue']")`	A stable text or relationship is needed and CSS is insufficient.

Inspect the page in browser developer tools to confirm the element and its attributes. If a selector finds multiple elements, narrow it using a stable parent or attribute. If the target is inside an iframe, switch into that frame before locating it, then switch back with driver.switch_to.default_content() when finished.

4. Wait for the page state you need

A navigation reaching its configured document readiness state does not guarantee that JavaScript-driven content has appeared or that a later update has finished. Selenium’s waiting strategies documentation describes timing races as a common source of flaky browser automation. Wait for the condition required by the next action, such as visibility before reading text or clickability before clicking.

wait = WebDriverWait(driver, 15)

# The element exists in the DOM and is visible
panel = wait.until(
    EC.visibility_of_element_located((By.ID, "status-panel"))
)

# The control is available to click
save_button = wait.until(
    EC.element_to_be_clickable((By.CSS_SELECTOR, "button[data-action='save']"))
)
save_button.click()

# A known state appears after the action
wait.until(
    EC.text_to_be_present_in_element((By.ID, "status"), "Saved")
)

Explicit waits poll for the specified condition and proceed when it is met or time out. A fixed time.sleep(5) always waits five seconds: it may waste time when the page is ready sooner and still fail when the page takes longer. Selenium project guidance recommends condition-based waits and advises against mixing implicit and explicit waits in one session because their combined timing can be confusing.

Useful expected conditions include presence_of_element_located when the element need only exist in the DOM, visibility_of_element_located when it must be visible, element_to_be_clickable before clicking, url_contains after navigation, and text_to_be_present_in_element after a content update.

5. Handle common page details

Stale elements

A saved WebElement reference can become stale after a page update replaces that DOM node. Wait for the new state and locate the element again instead of reusing the old reference. Keep the locator tuple so you can repeat the lookup.

Frames

Elements inside an iframe are not located from the top-level document. Wait for and switch into the frame, interact with its contents, then return to the default content:

frame = wait.until(
    EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe.payment"))
)
try:
    wait.until(EC.visibility_of_element_located((By.NAME, "account"))).send_keys("value")
finally:
    driver.switch_to.default_content()

New tabs or windows

After an action opens another tab, wait until the number of window handles increases, then switch to the new handle before looking for its elements. Do not assume the original tab is still the active context.

original = driver.current_window_handle
wait.until(lambda d: len(d.window_handles) > 1)
new_handle = next(handle for handle in driver.window_handles if handle != original)
driver.switch_to.window(new_handle)

Dialogs and alerts

Browser JavaScript alerts are not ordinary page elements. Wait for an alert, then read or accept it with Selenium’s alert interface. For custom HTML dialogs, locate the dialog as a normal DOM element.

Cleanup

driver.quit() ends the full browser session. driver.close() closes only the current window or tab, so it is not the general cleanup operation when the session should end. Selenium guidance recommends deterministic cleanup and a fresh session for each test. For a standalone script, the try/finally pattern above provides a reliable place to quit.

6. Troubleshooting

Symptom	Likely cause	Fix
`NoSuchElementException`	The selector does not match, the content has not loaded, or the element is in a frame.	Inspect the live DOM, correct the locator, add a condition-based wait, or switch into the frame first.
`TimeoutException`	The expected condition never became true within the timeout.	Check the URL and locator, confirm the expected state actually occurs, and check overlays, frames, or navigation errors. Increase the timeout only if the page legitimately needs longer.
`ElementClickInterceptedException`	An overlay, banner, or another element covers the target, or the page has not reached a stable state.	Wait for the covering element to disappear or for the intended control to become clickable. Verify that the click matches the page’s normal interaction.
`StaleElementReferenceException`	The page rerendered and replaced the element after it was found.	Wait for the update and locate the element again using its locator.
Browser or driver startup error	The browser is missing, unsupported, or the environment cannot obtain or launch a compatible driver.	Confirm a supported browser is installed, review Selenium Manager and startup output, and check current Selenium support guidance. Use manual driver configuration only when the environment requires it.
The click succeeds but nothing changes	The wrong control was selected, validation failed, or the script checked too early.	Wait for a meaningful result condition and inspect visible validation messages or the page state before assuming success.
Works locally but fails in deployment	The deployed environment may have a different browser, display setup, network access, or permissions.	Check the browser installation and runtime environment there; consider remote WebDriver if the browser must run on another host.

7. Performance, reliability, and cost

Browser sessions are heavier than simple script operations because Selenium controls a real browser. For a small task, keep one session for the sequence of related steps and avoid repeatedly launching browsers inside a loop. Wait for the required condition rather than adding long fixed sleeps. If the task is a test suite, isolate cases with fresh sessions where practical so one case’s browser state does not leak into another.

Reliability comes primarily from stable locators, waits tied to actual page conditions, clear result checks, and cleanup. A timeout is a useful failure signal: report which state did not appear rather than silently continuing with an incomplete page. Browser automation has no universal runtime or cost figure; those depend on the machine, page, network, and execution environment.

A local Selenium script does not require Grid. Consider Selenium Grid or Remote WebDriver when browsers need to run on another machine or when sessions need to be distributed. The Selenium Python API documentation links to Remote WebDriver guidance for that later deployment step.

8. Or skip the browser setup

If your task is to capture a page as an image or PDF rather than interact with browser controls, ScreenshotNeo is a website screenshot API and MCP server. It takes a URL in one GET request, so you do not need to install Selenium or manage a local browser for a screenshot.

cURL:

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python:

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

Node.js:

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' });
const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

See the ScreenshotNeo API documentation for request options. Cookie banners, newsletter popups, and chat widgets are removed before the shot; each step can be turned off. Bot checks, blank pages, timeouts, and failed loads are never billed, and cache hits cost nothing. An MCP server lets AI agents use take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots a month with no card; paid plans start at $5 for 3,000. Sign up for 1,000 free screenshots a month, with no card required.

Frequently asked questions

Does Selenium run a real browser?

Yes. Selenium’s Python WebDriver bindings send commands to a browser session.

Do I need Selenium Grid to run this example?

No. The tutorial uses a browser on the same machine. Grid and Remote WebDriver are optional when a browser needs to run elsewhere or sessions need to be distributed.

Can Selenium automate any website?

Selenium provides browser controls, not permission to access a site. Use it only for pages and actions you are authorized to automate.

Should I use `close()` or `quit()`?

Use quit() to end the whole WebDriver session. close() closes only the current window or tab.