How to Automate Web Forms with Python and Selenium

Use Selenium WebDriver to open the form, locate each control, enter values, click the form’s submit button, and wait for an observable success state. The click alone does not prove the submission worked. The example below uses explicit waits so it can handle controls that appear after JavaScript runs.

1. Install Selenium and prepare the browser

Use Python 3 and Selenium 4. Install the package in your project environment:

python -m pip install selenium

Selenium’s Python examples use Chrome with webdriver.Chrome(). Use a browser supported by your installed Selenium version. Selenium Manager can assist with driver setup in current Selenium releases; if your environment manages browser drivers another way, follow that setup and keep the browser and driver compatible. See the Selenium getting started guide.

Before automating a form, check that you are authorized to submit it and understand whether the action creates an account, sends a message, or changes stored data. Use a test environment when available. Do not build automation to evade CAPTCHA or access controls.

2. Inspect the form and choose stable locators

Open the page in a normal browser and inspect the form markup. Prefer stable attributes such as an input’s name, a meaningful id, or a deliberate CSS selector. Avoid brittle selectors tied to generated class names or a long chain of parent elements.

Control or goal	Typical Selenium locator	Example
Field with a name	`By.NAME`	`(By.NAME, "email")`
Field with an id	`By.ID`	`(By.ID, "message")`
Submit button	`By.CSS_SELECTOR`	`(By.CSS_SELECTOR, "button[type='submit']")`
Confirmation element	`By.ID` or another inspected locator	`(By.ID, "confirmation")`

These selectors are examples, not selectors guaranteed to exist on a particular site. Replace them with selectors from the page you are automating.

3. Fill, submit, and verify with Python

This runnable structure follows Selenium’s official first-script pattern. Replace the example URL and all placeholder locators with those from your form. The confirmation text is checked after submission so a failed or incomplete action does not silently look successful.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
from selenium.common.exceptions import TimeoutException

FORM_URL = "https://example.com/form"

# Replace these values after inspecting the target form.
NAME_LOCATOR = (By.NAME, "name")
EMAIL_LOCATOR = (By.NAME, "email")
SUBMIT_LOCATOR = (By.CSS_SELECTOR, "button[type='submit']")
CONFIRMATION_LOCATOR = (By.ID, "confirmation")


def main():
    driver = webdriver.Chrome()
    wait = WebDriverWait(driver, 10)

    try:
        driver.get(FORM_URL)

        name_field = wait.until(EC.visibility_of_element_located(NAME_LOCATOR))
        name_field.clear()
        name_field.send_keys("Example User")

        email_field = wait.until(EC.visibility_of_element_located(EMAIL_LOCATOR))
        email_field.clear()
        email_field.send_keys("user@example.com")

        submit_button = wait.until(EC.element_to_be_clickable(SUBMIT_LOCATOR))
        submit_button.click()

        confirmation = wait.until(
            EC.visibility_of_element_located(CONFIRMATION_LOCATOR)
        )
        print("Form result:", confirmation.text)
    except TimeoutException as exc:
        raise RuntimeError(
            "The expected form control or confirmation did not appear. "
            "Check the URL, locators, page state, and timeout."
        ) from exc
    finally:
        driver.quit()


if __name__ == "__main__":
    main()

The example’s target URL and selectors are placeholders. For a real form, define success in terms of the page’s actual outcome: a confirmation message, a changed URL, a completed state, or another visible result. A button becoming disabled may be useful evidence, but by itself it may not mean the server accepted the submission.

4. Handle common form controls

Text, email, password, and multiline fields

Use send_keys on an editable, keyboard-interactable element. Use clear() first when the field may contain a default value. Do not print or log password values. If a field is read-only or disabled, Selenium cannot type into it as an ordinary editable control.

field = wait.until(EC.visibility_of_element_located((By.NAME, "message")))
field.clear()
field.send_keys("A short message")

Checkboxes and radio buttons

Click only when the current state differs from the desired state. Checking the state first prevents a second run from accidentally unchecking a box that was already selected.

checkbox = wait.until(EC.element_to_be_clickable((By.ID, "terms")))
if not checkbox.is_selected():
    checkbox.click()

Native select menus

For a standard HTML <select>, use Selenium’s Select helper. This helper is for native select elements; custom JavaScript dropdowns usually need their visible control opened and an option clicked like other page elements.

from selenium.webdriver.support.ui import Select

country = wait.until(EC.visibility_of_element_located((By.NAME, "country")))
Select(country).select_by_value("US")
# Alternatives: select_by_visible_text("United States") or select_by_index(0)

File inputs

For a standard file input, send an absolute file path to the input. The file must be available to the machine running the browser; remote browser setups may require their own file transfer mechanism.

from pathlib import Path

upload = wait.until(EC.presence_of_element_located((By.NAME, "attachment")))
upload.send_keys(str(Path("report.pdf").resolve()))

Forms with several similar fields

If a page repeats controls, locate the relevant form container first, then search within it. This reduces ambiguity when multiple fields share a label or selector.

form = driver.find_element(By.CSS_SELECTOR, "form#contact")
email = form.find_element(By.NAME, "email")

5. Wait for the right page state

A completed navigation does not guarantee that JavaScript has rendered the form or that a post-submit response is ready. Selenium’s waiting guide describes synchronization as a common source of flaky browser automation. Use an explicit wait for the condition needed next, such as visibility of a field, clickability of a button, or presence of a confirmation.

wait = WebDriverWait(driver, 10)
field = wait.until(EC.visibility_of_element_located((By.NAME, "email")))
# After submitting, wait for the actual success condition:
wait.until(EC.visibility_of_element_located((By.ID, "confirmation")))

WebDriverWait takes a timeout in seconds; Selenium’s Python API documents a default polling interval of 0.5 seconds. Choose a timeout suited to the page and environment. A longer timeout can tolerate slower responses, but it cannot fix an incorrect selector or a result that never occurs.

Why not sleep after every action?

A fixed sleep guesses how long the page needs. If it is too short, the next step races the page; if it is too long, every run waits unnecessarily. Explicit waits stop when the requested condition becomes true or the timeout expires. Avoid mixing implicit and explicit waits because their combined timing can be difficult to reason about. Selenium’s waiting strategies explain the synchronization issue and available approaches.

Useful expected conditions

presence_of_element_located: the element exists in the DOM, whether or not visible.
visibility_of_element_located: the element exists and is displayed.
element_to_be_clickable: the element is visible and enabled for clicking.
text_to_be_present_in_element: expected text appears in an element.
staleness_of: an old element reference is no longer attached after a page update.

Pick the condition that expresses what the next action needs. For example, presence alone is insufficient if the field is hidden behind a collapsed panel.

6. Verify the submission, not just the click

After clicking, wait for evidence tied to the result you need. Selenium’s official example reads a response message after submitting its demonstration form. Depending on the application, verification may be:

A visible success message with expected text.
A success state or receipt identifier.
A URL change to a confirmation page.
A form transition to a completed state.

For a URL transition, capture the old URL and wait for it to change:

from selenium.webdriver.support import expected_conditions as EC

old_url = driver.current_url
submit_button.click()
wait.until(EC.url_changes(old_url))

A URL change establishes navigation, not necessarily business success. If possible, also inspect the confirmation content or another application-specific outcome.

7. cURL, Python requests, and Node.js alternatives

Selenium is appropriate when the workflow depends on browser behavior such as rendered controls, client-side validation, or interactive steps. If the site exposes an authorized HTTP form endpoint and you understand its required fields, a direct HTTP request can be simpler. Do not assume a browser form can be reproduced by guessing a POST URL: sites may use CSRF tokens, cookies, hidden fields, or JavaScript-generated values. Use the site’s documented API when one exists.

cURL: inspect or call a known endpoint

This is a template for an endpoint you are authorized to use; it is not a generic submission URL. Supply the method, endpoint, headers, cookies, and form fields required by that service.

curl --fail-with-body -X POST "https://example.com/known-form-endpoint" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  --data-urlencode "name=Example User" \
  --data-urlencode "email=user@example.com"

Python requests: known form endpoint

import requests

url = "https://example.com/known-form-endpoint"
payload = {"name": "Example User", "email": "user@example.com"}

with requests.Session() as session:
    response = session.post(url, data=payload, timeout=30)
    response.raise_for_status()
    print("HTTP status:", response.status_code)
    print(response.text[:500])

For a real flow, preserve required session cookies and tokens and verify the response according to the endpoint’s documented behavior. A successful HTTP status does not always mean the form’s intended business action succeeded.

Node.js: known form endpoint

const body = new URLSearchParams({
  name: 'Example User',
  email: 'user@example.com'
});

const response = await fetch('https://example.com/known-form-endpoint', {
  method: 'POST',
  headers: { 'content-type': 'application/x-www-form-urlencoded' },
  body
});

if (!response.ok) {
  throw new Error(`Request failed: ${response.status} ${response.statusText}`);
}
console.log((await response.text()).slice(0, 500));

These HTTP examples do not replace Selenium for browser-only interactions. An endpoint that works in a browser may depend on state established earlier in the session.

8. Troubleshooting

Symptom	Likely cause	What to do
`NoSuchElementException`	Wrong selector, wrong page, or the element has not appeared yet.	Inspect the live DOM and URL; use an explicit wait for presence or visibility; confirm the element is inside the expected frame or form.
`TimeoutException`	The awaited condition did not become true before the timeout.	Check selector and expected state first. Increase the timeout only when the page is legitimately slow.
`ElementNotInteractableException` or invalid element state	The field is hidden, disabled, read-only, or not keyboard-interactable.	Wait for visibility and enabled state; open the relevant panel; verify the correct editable control was located.
`ElementClickInterceptedException`	An overlay, banner, or another element covers the target.	Wait for the overlay to clear or handle the page’s intended dialog, then wait for the button to be clickable. Do not use JavaScript clicks as a default workaround.
Click returns but nothing seems to happen	Wrong button, client-side validation error, or asynchronous response.	Check field validation messages, confirm the chosen button belongs to the form, then wait for a meaningful post-submit state.
Stale element reference	The page re-rendered and replaced the element after it was located.	Wait for the update, then locate the element again instead of reusing the old reference.
Browser or driver fails to start	Browser installation, driver compatibility, or environment setup problem.	Confirm the browser is installed and supported, update Selenium, and follow the browser-specific setup documentation for the environment.
Form works manually but automation is blocked	The site may restrict automation or require a verification step.	Use an approved API or test environment, or ask the site owner for an automation-friendly flow. Do not try to bypass CAPTCHA or access controls.
Duplicate submissions on rerun	The script retries after an uncertain result or blindly repeats a non-idempotent action.	Check for an existing receipt or completion state before retrying. Use a test account and an application-provided idempotency mechanism when available.

9. Performance, reliability, and cost

Browser automation starts a browser process and waits on page rendering, so it is generally heavier than a direct request to a documented endpoint. Keep one browser session for a sequence of related steps and always close it with driver.quit() in a finally block. Set explicit timeouts, avoid unnecessary page reloads, and wait only for states the workflow needs.

For reliability, use stable locators, condition-based waits, and explicit success checks. Record enough diagnostic context to investigate failures, such as the current URL and a sanitized error message, but do not log passwords, session cookies, authorization headers, or personal form data. Retry only failures that are safe to retry: submitting a form twice can create duplicate actions.

There is no universal cost or speed figure for Selenium form automation. Resource use depends on the browser, page, infrastructure, and volume. Account for browser compute, maintenance when the page changes, test data, and the effect of repeated submissions. An HTTP API may be cheaper to operate when the service documents one and it supports the required action.

10. Inspect the page before automating it

When a form fails, a screenshot can help show whether a consent banner, popup, chat widget, or unexpected page state covers the controls. Capture only pages you are permitted to access, and avoid exposing private form data in artifacts. ScreenshotNeo is a website screenshot API and MCP server from ScreenshotNeo; it can return PNG, JPEG, WebP, or PDF captures through one GET request. Its API documentation describes the available parameters.

11. Or skip the browser setup

If your goal is to inspect a page rather than submit its form, ScreenshotNeo takes a screenshot with one API request. This does not automate form submission; Selenium remains the right tool for filling and submitting browser forms.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

See the ScreenshotNeo API docs for the Python and Node.js forms of the same request and its options. Before the capture, it accepts cookie and consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets; each step can be turned off. Bot checks and CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and response headers identify the page verdict and whether the capture was billed. AI agents can use its MCP server, including take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots per month with no card; paid plans start at $5 for 3,000 shots.

Create a free ScreenshotNeo account and get 1,000 screenshots a month with no card.

12. FAQ

Can Selenium submit a form without clicking its button?

Selenium 4’s interaction guide recommends clicking the applicable submission button instead of using the submit() method. This follows the page’s visible form interaction.

Should I use implicit or explicit waits?

Use explicit waits when a step depends on a particular page condition. They make the required state clear and are easier to tune for dynamic forms.

How do I know whether a submission really succeeded?

Wait for and inspect an application-specific result, such as confirmation text or a receipt state. A click or HTTP status alone may not establish the intended outcome.

Can I automate any online form?

No. A site may prohibit automation, require human verification, or expose no supported way to submit it. Follow the site’s rules and use documented APIs or authorized test flows when available.