How to Find HTML Elements by Class with BeautifulSoup

Use soup.find_all(class_="target") to find every HTML element with a class, or soup.find(class_="target") to return only the first match. If you prefer CSS selector syntax, use soup.select(".target") or soup.select_one(".target").

BeautifulSoup treats a class as one value in an element’s list of classes. That means a search for class_="card" also matches <div class="card featured">. Use a compound CSS selector such as .card.featured when both classes are required.

Install BeautifulSoup and parse HTML

Install the parser and an HTTP client if you are loading a page from the web:

python -m pip install beautifulsoup4 requests

Then create a soup object. The parser choice affects how malformed markup is repaired, so use the parser that matches your deployment environment.

from bs4 import BeautifulSoup

html = """
First
Second
"""
soup = BeautifulSoup(html, "html.parser")

The official Beautiful Soup documentation documents the class_ shortcut and CSS selector support. It identifies class_ as available since Beautiful Soup 4.1.2 and SoupSieve-backed selectors as available since 4.7.0; check the version installed in your own environment.

Find every element with one class

Call find_all() with class_ when you need all matching tags:

BeautifulSoup offers both its search API and CSS selectors for class matching.

cards = soup.find_all(class_="card")

for card in cards:
    print(card.get_text(strip=True))

# First result: First
# Second result: Second

This search is not limited to a particular tag. It can return a div, article, a, or any other element carrying that class. To narrow it to one tag name, pass the name first:

divs = soup.find_all("div", class_="card")
links = soup.find_all("a", class_="card")

Use find() when you only want the first matching element. It returns a Tag or None if nothing matches:

first_card = soup.find(class_="card")
if first_card is not None:
    print(first_card.get_text(" ", strip=True))

Use CSS class selectors with select()

select() uses CSS selector syntax through SoupSieve. A class selector starts with a dot:

cards = soup.select(".card")
first_card = soup.select_one(".card")

CSS syntax becomes useful when the class search also expresses structure:

# A featured card that is also a card
featured = soup.select(".card.featured")

# Cards only inside a section
section_cards = soup.select("section.results .card")

# A card whose direct child is a heading
with_heading = soup.select(".card > h2")

# Cards that contain a link
linked_cards = soup.select(".card:has(a)")

For a plain class filter, find_all(class_=...) and select(...) are both direct. CSS selectors are concise for combinations of tags, classes, descendants, siblings, and attributes. The documentation describes select() as a convenience over equivalent BeautifulSoup searches; it does not establish that CSS selection is faster.

Match multiple classes correctly

HTML permits multiple class values:

html = '<p class="body strikeout">Text</p>'
soup = BeautifulSoup(html, "html.parser")

A single-class search matches when that class appears anywhere in the element’s class list:

soup.find_all(class_="body")       # matches
soup.find_all(class_="strikeout")  # also matches

To require both classes, use a compound selector:

matches = soup.select("p.body.strikeout")

Do not pass class="body" as a Python keyword argument: class is reserved syntax. Use class_ instead. An alternative is an attribute mapping:

matches = soup.find_all(attrs={"class": "body"})

Passing the whole string "body strikeout" to class_ tests the class attribute string as shown in the documentation. It is order-sensitive in that example, so it is not a reliable way to express “has both classes.” Prefer p.body.strikeout.

Extract text, attributes, and HTML safely

A Tag contains markup as well as text. Choose the extraction method that matches your data:

card = soup.select_one(".card")

if card:
    text = card.get_text(" ", strip=True)
    href = card.get("href")              # None when absent
    classes = card.get("class", [])      # always use a default
    inner_html = card.decode_contents()
    outer_html = str(card)

    print(text, href, classes)

get_text(" ", strip=True) inserts spaces between descendant text nodes and trims surrounding whitespace. Use separator="\n" when line boundaries matter. Treat extracted HTML as untrusted input if you later render it.

Complete example: scrape cards by class

This script fetches a page, checks the response, finds every product card, and handles missing fields without raising an exception:

from urllib.parse import urljoin

import requests
from bs4 import BeautifulSoup

url = "https://example.com/products"
response = requests.get(
    url,
    headers={"User-Agent": "class-extractor/1.0"},
    timeout=20,
)
response.raise_for_status()

soup = BeautifulSoup(response.text, "html.parser")
results = []

for card in soup.select(".product-card"):
    name_tag = card.select_one(".product-name")
    link_tag = card.select_one("a[href]")
    price_tag = card.select_one(".price")

    results.append({
        "name": name_tag.get_text(" ", strip=True) if name_tag else None,
        "price": price_tag.get_text(" ", strip=True) if price_tag else None,
        "url": urljoin(url, link_tag["href"]) if link_tag else None,
        "classes": card.get("class", []),
    })

for item in results:
    print(item)

Replace the selectors with the classes present in your document. A successful HTTP response does not guarantee that the desired elements exist: the page may have changed, require JavaScript, or return a consent page instead of the content you expected.

Classes, attributes, and dynamic pages

Classes are not unique IDs

A class normally describes a reusable style or component. Expect zero, one, or many matches. Use find_all() or select() for collections, and reserve find() or select_one() for a deliberate first-match rule.

Exact class-list checks

If you need to inspect the parsed class list yourself, compare sets:

for tag in soup.find_all(class_=True):
    classes = set(tag.get("class", []))
    if {"card", "featured"}.issubset(classes):
        print(tag)

This is useful when you need custom logic, such as allowing any order while also recording extra classes.

JavaScript-rendered content

BeautifulSoup parses the HTML you give it; it does not execute JavaScript. If the server response contains an empty app shell and the browser adds cards later, no class query can find those cards in the original response. Use the site’s underlying data endpoint when available, or capture rendered HTML with a browser automation tool before parsing.

Changing class names

Generated CSS-module names and hashed classes are fragile selectors. Prefer stable attributes such as data-testid, semantic tags, or a parent structure when the site provides them. Keep selectors in configuration so a markup change does not require editing parsing logic throughout your code.

Or skip the browser setup

If your goal is to obtain a clean image or PDF of a rendered page before inspecting it, ScreenshotNeo provides a single GET request. It can wait for rendered content, capture a full page or one CSS-selected element, and return PNG, JPEG, WebP, or PDF. The API accepts the same kinds of URL and selector parameters used by other screenshot services, which can simplify migration. See the ScreenshotNeo documentation for the complete option list.

A rendering service can clean overlays before you inspect or capture a page.

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' }); const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

Cookie banners, newsletter popups, and chat widgets are removed before the shot. Bot checks, blank pages, failed loads, timeouts, and cache hits are not billed, and response headers identify the page verdict and billing status. An MCP server lets Claude, Cursor, and other MCP clients call take_screenshot, get_page_info, and capture_pdf. The free plan includes 1,000 screenshots each month with no card; paid plans start at $5 for 3,000. Create a free ScreenshotNeo account.

Options for real scraping jobs

Need	Recommended approach
All elements with one class	`find_all(class_="name")` or `select(".name")`
First matching element	`find(class_="name")` or `select_one(".name")`
Specific tag and class	`find_all("a", class_="name")`
Several required classes	`select(".one.two")`
Nested or structural rule	CSS `select()` syntax
Custom order-insensitive logic	Inspect `tag.get("class", [])` and compare sets

Troubleshooting

“Why is my result empty?”

Wrong class spelling or case: class values are case-sensitive. Print a sample with [(t.name, t.get("class")) for t in soup.find_all(class_=True)][:20].
Wrong document: print response.url, status code, and the first few hundred characters of response.text. Redirects and consent pages are common surprises.
JavaScript content: inspect the raw response. If the class is absent, obtain rendered HTML or an API response.
Parser differences: malformed markup can be repaired differently. Try the parser supported by your deployment and keep it consistent.

“Why did I get too many elements?”

Add a tag name, parent selector, or compound selector.
Remember that a class is reusable and a single class search also matches elements carrying additional classes.
Use select_one() only when first-match behavior is actually correct; otherwise filter the complete result set explicitly.

“Why does `class="..."` fail?”

Python reserves class. Write class_=... or attrs={"class": ...}.

“Why does a two-class string fail in one order?”

The documentation’s whole-string example compares the class attribute string, so order matters. Use .first.second or set-based checking when both classes are required regardless of order.

“Why does my selector raise a CSS error?”

Escape punctuation in generated class names and verify brackets, quotes, and combinators. Start with a simple selector such as .card, then add one condition at a time.

Performance, reliability, and cost

Most runtime is spent downloading and parsing the document, not choosing between find_all() and select(). Fetch once, parse once, and reuse the soup object for related queries. Narrow searches to a parent tag when the document is large:

results = soup.select_one("main").select(".card")

Set network timeouts, call raise_for_status(), and record the URL, parser, selector, match count, and response status in logs. Retry transient network failures with bounded backoff, but do not blindly retry deterministic 4xx responses. Cache source HTML when legally and operationally appropriate so repeated extraction does not repeatedly fetch the same page.

BeautifulSoup itself has no per-element or API usage charge. Your costs come from network traffic, hosting, browser rendering if needed, and any third-party service. If you use ScreenshotNeo, only clean shots are billed; failed loads, bot checks, blank pages, timeouts, and cache hits are free, and the response includes X-Page-Verdict and X-Billed headers. Choose a cache TTL, wait strategy, viewport, device preset, resource blocking, and concurrency level that fit the page and your workload.

Testing checklist

Test zero matches, one match, and many matches.
Include an element with extra classes and verify single-class behavior.
Include reversed class order and verify compound-selector behavior.
Test missing child elements and absent attributes.
Test malformed HTML with the parser used in production.
Test a JavaScript-rendered page separately from a server-rendered page.
Assert that your selector still identifies the intended semantic element after a fixture update.

FAQ

What is the shortest BeautifulSoup class query?

soup.find_all(class_="name") returns all matches. Use soup.find(class_="name") for the first.

Is `select()` better than `find_all()`?

Neither is universally better. Use the API that makes your selector easiest to read; choose CSS when the query expresses structure or multiple classes.

Can BeautifulSoup find hidden elements?

It can find any elements present in the HTML string, including hidden markup. It cannot discover content added later by JavaScript.

How do I select an element with a class containing punctuation?

Use a CSS escape for punctuation or inspect the class list in Python and filter it directly. Generated names are often better handled through stable attributes.

Which versions support these methods?

The cited documentation states the class_ shortcut threshold as Beautiful Soup 4.1.2 and SoupSieve CSS selector support as 4.7.0. Verify your installed package versions before relying on a feature.