CSS selectors are the query language every scraping stack understands. Browsers, querySelectorAll, BeautifulSoup, Cheerio, Scrapy, Playwright, Selenium, and jsoup all accept the same core syntax, which makes a selector the most portable thing you can write down about a page's structure. This cheat sheet collects the syntax you'll actually use — each with a copy-paste example — plus the parts that quietly differ between engines and the two questions people ask most that CSS genuinely cannot answer.
Key Takeaways
tag.class[attr]chained without spaces means "all of these on one element"; add a space and you've asked for a descendant instead- Commas make a selector list (OR), and they're how you pull several different element types in one query
- Attribute selectors have seven operators —
=,~=,|=,^=,$=,*=, plus theicase-insensitivity flag :nth-child(n)counts all siblings;:nth-of-type(n)counts only siblings of the same tag — mixing them up is the top nth bug- CSS has no
:contains()selector. Text matching belongs to XPath, or to a tool-specific extension like Cheerio's:contains()or Playwright's:has-text() - Selector support is not uniform across parsers:
:has()works in browsers, Soup Sieve, Cheerio, andcssselect, while:not()with a comma-separated list works everywhere exceptcssselect(so Scrapy and lxml) - Test against real markup before shipping — the free XPath & CSS selector tester runs a selector against pasted HTML or a live-rendered URL
The cheat sheet
| Selector | Matches |
p | Every <p> element |
.price | Every element with class="price" |
#main | The element with id="main" |
* | Every element |
.card.featured | Elements with both classes |
a.btn | <a> elements that also have class="btn" |
h1, h2, h3 | Selector list: all three tags (OR) |
div p | <p> anywhere inside a <div> (descendant) |
div > p | <p> that is a direct child of a <div> |
h2 + p | The <p> immediately after an <h2> |
h2 ~ p | Every <p> after an <h2>, same parent |
a[href] | Links that have an href attribute |
a[href="/pricing"] | Exact attribute value |
a[href^="/product/"] | Value starts with |
a[href$=".pdf"] | Value ends with |
a[href*="utm_"] | Value contains substring |
div[class~="card"] | Value is one whitespace-separated word |
| `html[lang\ | ="en"]` |
a[href$=".PDF" i] | Case-insensitive value match |
li:first-child | First child of its parent |
li:last-child | Last child of its parent |
li:nth-child(2) | Second child |
li:nth-child(odd) | 1st, 3rd, 5th … |
li:nth-child(3n+1) | Every third, starting at 1 |
p:nth-of-type(2) | Second <p> among its siblings |
div:not(.ad) | <div> without that class |
div:has(> img) | <div> that directly contains an image |
td:empty | Cells with no children or text |
input:checked | Checked checkboxes and radios |
The rest of the guide unpacks each group, and flags where a given engine disagrees.
Basic selectors and how they chain
The four primitives are the tag name, .class, #id, and *. What matters for scraping is what happens when you put them next to each other:
div.product /* a div that ALSO has class "product" */
.product.sale /* both classes on one element */
a#logo /* an <a> with that id */
input[type="email"] /* tag plus attribute condition */
No space means "all of these conditions on the same element". Add a space and the meaning changes completely — div .product is "an element with class product somewhere inside a div". That single character is the most common selector typo there is.
A class attribute holding several values matches on any one of them: <div class="card featured sale"> is selected by .card, by .featured, and by .card.sale. Unlike XPath's @class, CSS never compares the whole attribute string, which is why the awkward contains(concat(' ', @class, ' '), ' card ') idiom has no CSS equivalent — you just write .card.
Selector lists: matching multiple element types at once
Commas build a selector list, and each part is evaluated independently:
h1, h2, h3 /* all three heading levels */
.price, .sale-price, [data-price] /* any of three ways a price is marked up */
article > p, article > ul /* both direct-child shapes */
Results come back in document order, not in the order you listed the selectors — if you need to know which branch matched, run separate queries instead. In a scraper this is the pattern that survives a redesign: list every markup variant a site has used for a field, and the selector keeps working through an A/B test.
One sharp edge worth knowing: in CSS Selectors Level 3, an unrecognized selector anywhere in a list invalidated the whole list. Level 4 added :is() and :where(), which forgive unknown selectors inside them — :is(.price, .cost, :some-future-thing) still matches the first two — and they also let you factor a shared prefix out: article :is(h2, h3, h4). Both work in current browsers, Soup Sieve, and cssselect, but a plain comma list is still the most portable thing you can write.
Combinators: descendants, children, and siblings
Four combinators cover every structural relationship CSS can express:
| Combinator | Syntax | Meaning |
| Descendant | a b | b at any depth inside a |
| Child | a > b | b is a direct child of a |
| Adjacent sibling | a + b | b immediately follows a |
| General sibling | a ~ b | b follows a, same parent |
.results .product-title /* any depth — resilient to wrapper divs */
.results > .product /* direct children only — resilient to nesting */
dt + dd /* the value right after a label */
h2 ~ p /* every paragraph in a section after its heading */
Descendant vs. child is a real trade-off rather than a style preference. The descendant combinator survives a designer adding a wrapper <div>; the child combinator protects you from accidentally matching a nested copy of the same structure — a product card inside a "related products" carousel inside a product card. Use > when the page nests the same component in itself, and a space otherwise.
+ is the workhorse for label/value layouts (dt + dd, .label + .value, th + td). Note that all sibling combinators look forward only. There is no "previous sibling" combinator and no parent combinator; for those you need XPath's axes or :has(), covered below.
Attribute selectors
Attribute selectors are the most underused tool in scraping, and the most durable — attributes like href, data-id, and itemprop change far less often than class names.
| Syntax | Matches when the attribute |
[attr] | exists at all |
[attr="val"] | equals val exactly |
[attr~="val"] | contains val as a whitespace-separated word |
| `[attr\ | ="val"]` |
[attr^="val"] | starts with val |
[attr$="val"] | ends with val |
[attr*="val"] | contains val anywhere |
a[href^="https://"] /* absolute links only */
a[href$=".pdf"] /* document downloads */
a[href*="/product/"] /* product URLs anywhere in the path */
[data-testid="add-to-cart"] /* test hooks make excellent scraping hooks */
img[src][alt] /* has both attributes */
input[name="email"][required] /* two conditions, same element */
[itemprop="price"] /* microdata, when the site publishes it */
Attribute values are case-sensitive in HTML (attribute names are not). Append the i flag inside the brackets for a case-insensitive comparison:
a[href$=".PDF" i] /* matches .pdf, .PDF, .Pdf */
[data-state="OPEN" i]
The i flag works in browsers and in Soup Sieve (BeautifulSoup), but cssselect — which is what Scrapy and lxml use — rejects it with a SelectorSyntaxError. There, match case-insensitively with XPath's translate() or filter in Python. Quotes around the value are optional only when it's a valid identifier — [href^=/product/] is invalid because of the slashes. Always quote, and you never have to think about it.
Pseudo-classes: the nth family
:nth-child() and friends select by position among siblings.
tr:first-child /* header row, usually */
tr:last-child /* totals row, usually */
li:nth-child(2) /* second item */
li:nth-child(odd) /* 1, 3, 5 … — zebra rows */
li:nth-child(even) /* 2, 4, 6 … */
li:nth-child(3n) /* every third */
li:nth-child(3n+1) /* 1, 4, 7 … — first item of each row of three */
li:nth-child(-n+5) /* the first five */
tr:nth-last-child(2) /* second from the end */
p:only-child /* the sole child of its parent */
The An+B formula runs n = 0, 1, 2, … and keeps positive results, so -n+5 yields 5, 4, 3, 2, 1 — a clean way to cap a list. Indexes are 1-based, unlike almost every array you'll assign the results to.
The distinction that causes the most wasted debugging:
<div>
<h2>Specs</h2>
<p>First paragraph</p>
<p>Second paragraph</p>
</div>
Here p:nth-child(1) matches nothing — the first child is the <h2>, and :nth-child counts every sibling regardless of tag, then filters by p. p:nth-of-type(1) matches "First paragraph", because :nth-of-type counts only the <p> siblings. When a container mixes element types, :nth-of-type is nearly always what you meant. Same relationship holds for :first-child vs :first-of-type and :last-child vs :last-of-type.
A Level 4 addition, :nth-child(An+B of S), filters before counting — li:nth-child(2 of .in-stock) is the second in-stock item, not "the second item, if it happens to be in stock". Browser support is current-generation only and parser support is thin, so treat it as a browser-automation tool rather than something to put in a Scrapy spider.
:not() and :has()
:not() excludes, and it's how you skip the rows that aren't data:
tr:not(.header)
li:not(:first-child)
a:not([href^="#"]) /* real links, not in-page anchors */
div.item:not(.sold-out)
Selectors Level 4 allows a comma-separated list inside :not(), which reads much better than chaining:
p:not(.ad, .promo, .disclaimer) /* Level 4 */
p:not(.ad):not(.promo):not(.disclaimer) /* Level 3, works everywhere */
This is a place where engines genuinely disagree. Browsers, Soup Sieve (BeautifulSoup), and Cheerio accept the list form; cssselect — which is what Scrapy and lxml use — raises a SelectorSyntaxError on it. If your code has to run in both, write the chained Level 3 form.
:has() is the long-awaited "parent selector": it matches an element based on what it contains.
div.card:has(img) /* cards that have an image */
div.card:has(> h2) /* … a direct child h2 */
tr:has(td.error) /* rows containing an error cell */
article:has(.price):not(:has(.sold-out))
li:has(+ li.active) /* the item before the active one */
That last pattern is the trick for looking backwards: CSS has no previous-sibling combinator, but :has(+ x) expresses "the element whose next sibling is x", which is the same thing.
Support is better than its reputation: :has() works in all current browsers, in Soup Sieve, in Cheerio, and — verified against the current release — in cssselect, which translates div:has(> p) into the XPath descendant-or-self::div[./p]. What it does not do is make CSS text-aware, which is the next section.
Selecting elements that contain specific text
This is the most-asked CSS selector question, and the honest answer is that CSS cannot do it. There is no :contains() in any CSS specification. A :contains() pseudo-class appeared in a CSS Selectors Level 3 working draft and was removed before the spec was finalized; it never shipped in a browser. document.querySelectorAll('a:contains("Next")') throws a SyntaxError.
What exists instead, ranked by how much of your stack it works in:
XPath — the standards-based answer, available in browsers, Selenium, Playwright, lxml, and Scrapy:
//button[contains(., 'Add to cart')]
//h2[normalize-space()='Specifications']
//a[starts-with(., 'Download')]
normalize-space() handles the whitespace that makes exact text matches fail. The XPath cheat sheet covers the text functions in depth.
Tool-specific extensions — non-standard pseudo-classes that only work inside one library:
| Tool | Syntax | Notes |
| jQuery / Cheerio | a:contains("Next") | Substring, case-sensitive |
| Playwright | a:has-text("Next") | Substring, case-insensitive; also :text-is() for exact |
| Puppeteer | a::-p-text("Next") | Puppeteer's own p-selector syntax |
| Selenium | — | No CSS equivalent; use XPath or filter in code |
| BeautifulSoup | — | soup.find_all('a', string='Next') instead of select() |
| jsoup | a:contains(Next) | jsoup extension; also :matchesOwn() for regex |
None of these are portable. A :contains() selector copied from a jQuery answer into querySelectorAll will throw, and the same selector in Playwright silently means something different (:has-text is case-insensitive, :contains is not).
Filter in your own code — the approach that always works and is easiest to read:
const next = [...document.querySelectorAll('a')]
.find(a => a.textContent.trim() === 'Next');
next_link = next(
(a for a in soup.select('nav a') if a.get_text(strip=True) == 'Next'),
None,
)
Select structurally with CSS, then filter on text in the host language. It costs two lines and it behaves identically everywhere, which is worth more than a clever one-liner in a scraper you'll maintain for a year.
Selecting by width or height
The other question CSS can't answer the way people expect. img[width="300"] matches the HTML attribute <img width="300"> — a string comparison against markup. It does not match an image that happens to render 300 pixels wide, and it won't match <img style="width:300px"> or an image sized by a stylesheet at all.
CSS has no selector for computed layout size, and this is by design: styles determine size, so selecting on size would be circular. The features that come close solve different problems — media queries respond to the viewport, and container queries to an ancestor's size; neither selects an element by its own rendered dimensions.
If you need rendered size, measure it after layout:
const wide = [...document.querySelectorAll('img')]
.filter(img => img.getBoundingClientRect().width > 300);
This only works in a real browser context — Playwright, Puppeteer, Selenium, or a rendering API. An HTML parser like BeautifulSoup or Cheerio never runs layout, so it has no size to report; there, the attributes and inline styles in the markup are genuinely all the information that exists.
Writing selectors that survive a redesign
Scraping selectors break for one reason: they encode something the site didn't promise to keep. A rough stability ranking, most durable first:
data-*attributes and test hooks —[data-testid="price"],[data-product-id]. Written for machines, changed rarely.- Semantic attributes and microdata —
[itemprop="price"],[rel="next"],time[datetime]. - IDs — stable when authored, meaningless when generated (
#ember1043). - Semantic tags plus structure —
article > h2,nav a. - Human-written class names —
.product-titleis reasonable;.text-sm.font-bold.mb-2is a utility-framework accident. - Positional selectors —
:nth-child(3)breaks when one row is inserted. - Long descendant chains —
body > div > div > div:nth-child(2) > span, which is what a browser's "Copy selector" gives you. Never ship it.
Three concrete heuristics:
- Prefer one strong anchor to a long path.
[data-testid="price"]beats.container .row .col-md-6 .priceeven though both work today. - Anchor on the container, then query within it. Find the card, then pull fields relative to it. One broken selector fails one field instead of the whole scrape.
- Treat hashed classes as expired on arrival.
.css-1x2y3z4,.jsx-2847, and Tailwind utility stacks are build output. If they're all a site gives you, reach for an attribute or the element's position relative to a stable landmark.
Then make failure loud. A selector that matches nothing should raise, not write an empty string into your dataset — silent selector rot is how a pipeline produces a month of blank columns before anyone notices.
Per-tool syntax
The same selector string, in the eight places you're most likely to type it:
| Tool | Call |
| Browser JS | document.querySelectorAll('.price') |
| BeautifulSoup | soup.select('.price') / soup.select_one('.price') |
| Cheerio | $('.price') |
| Playwright | page.locator('.price') |
| Puppeteer | page.$$('.price') |
| Selenium | driver.find_elements(By.CSS_SELECTOR, '.price') |
| Scrapy | response.css('.price::text').getall() |
| jsoup | doc.select(".price") |
| Nokogiri | doc.css('.price') |
| WebScraping.AI | GET /selected?selector=.price |
Scoping a query to an element you already have needs :scope in the browser, and it's a real trap:
card.querySelectorAll('> p'); // SyntaxError — invalid selector
card.querySelectorAll(':scope > p'); // correct
card.querySelectorAll('p'); // any depth inside the card
Soup Sieve accepts :scope too, so card.select(':scope > p') is the BeautifulSoup equivalent. In Cheerio you'd use $(card).children('p'), and in Scrapy a relative response.css() call is already scoped to the element you called it on.
A worked example in Python:
from bs4 import BeautifulSoup
soup = BeautifulSoup(html, "html.parser")
for card in soup.select("div.product-grid > div.product-card"):
name = card.select_one("[data-testid='title'], h3")
price = card.select_one(".price:not(.price--old)")
link = card.select_one("a[href^='/product/']")
if not (name and price):
raise ValueError(f"selector drift in card: {card.get('data-id')}")
print(name.get_text(strip=True), price.get_text(strip=True), link["href"])
And in Node with Cheerio:
const $ = cheerio.load(html);
const products = $('div.product-grid > div.product-card').map((_, el) => ({
name: $(el).find('[data-testid="title"], h3').first().text().trim(),
price: $(el).find('.price:not(.price--old)').first().text().trim(),
url: $(el).find('a[href^="/product/"]').attr('href'),
})).get();
Escaping class names and special characters
Utility-first CSS frameworks generate class names full of characters that mean something to a selector parser. class="md:flex" cannot be selected with .md:flex — the parser reads :flex as a pseudo-class. Escape with a backslash:
.md\:flex /* class="md:flex" */
.w-1\/2 /* class="w-1/2" */
.\32 col /* class="2col" — a leading digit needs a hex escape */
.p-\[13px\] /* class="p-[13px]" */
In JavaScript, don't hand-roll it — CSS.escape() exists for exactly this:
document.querySelectorAll('.' + CSS.escape('md:flex'));
And remember the backslash has to survive your host language's own string rules: in Python, use a raw string (r".md\:flex"), and in a JavaScript string literal you'll need '.md\\:flex'. An attribute selector sidesteps the whole problem — [class~="md:flex"] needs no escaping at all, because the value is already inside quotes.
CSS selectors vs XPath
Both query a parsed document tree; the difference is expressiveness against readability.
| CSS | XPath | |
| Class/id targeting | .price | //*[contains(concat(' ',@class,' '),' price ')] |
| Match by text | Not possible | //a[contains(., 'Next')] |
| Select a parent | :has() (approximate) | parent::, ancestor:: |
| Previous sibling | :has(+ x) (approximate) | preceding-sibling:: |
| Select an attribute value | Not possible | //a/@href |
| Functions, counts, ranges | No | count(), position(), not() |
| Support in parsers | Universal | Universal |
CSS wins on brevity for the 80% case — class, id, attribute, and structural selection — and it's the syntax every tool accepts without a prefix. XPath wins the moment you need text matching, upward traversal, or a condition CSS has no grammar for. Performance is not a tiebreaker: both evaluate in microseconds against a parsed tree, and network time dominates a scrape by several orders of magnitude.
Most production scrapers use both, picking per field rather than per project. See the XPath cheat sheet for the other half of the comparison.
Using CSS selectors with WebScraping.AI
A correct selector still returns nothing when the HTML it targets never arrives — JavaScript-rendered content, anti-bot walls, and IP blocks all happen before your parser runs. WebScraping.AI handles that fetch layer and can apply the selector server-side, so you get back just the region you asked for:
# HTML of the first matching element
curl -G https://api.webscraping.ai/selected \
-d api_key=YOUR_API_KEY \
-d url=https://example.com/products \
-d js=true \
-d 'selector=.product-grid'
# Several regions in one request — returns a JSON array, one entry per selector
curl -G https://api.webscraping.ai/selected-multiple \
-d api_key=YOUR_API_KEY \
-d url=https://example.com/products \
-d 'selectors[]=h1' \
-d 'selectors[]=.price' \
-d 'selectors[]=a[href^="/product/"]'
Two behaviors worth knowing before you build on it. /selected returns the inner HTML of the first match — if your selector matches twenty product cards, use /selected-multiple, or fetch /html and iterate locally. And because the page is rendered in a real browser, modern selectors work: div:has(> p a[href*="iana"]) resolves correctly against a live page, which is not true of every parser you might run the same string through.
For a page whose markup changes weekly and where no selector stays valid, the /ai/fields endpoint skips selectors entirely — describe the fields you want and the model extracts them from the rendered page.
Frequently asked questions
Is there a CSS selector for elements containing certain text?
No. :contains() was dropped from CSS Selectors Level 3 before the spec was finalized and never shipped in a browser. Use XPath (//a[contains(., 'Next')]), a tool-specific extension (Cheerio :contains(), Playwright :has-text(), jsoup :contains()), or select structurally and filter on text in your own code.
What's the difference between :nth-child and :nth-of-type?
:nth-child(n) counts every sibling and then checks the tag; :nth-of-type(n) counts only siblings with the same tag. In a <div> containing an <h2> followed by two <p>s, p:nth-child(1) matches nothing while p:nth-of-type(1) matches the first paragraph.
How do I select an element with two classes?
Chain them without a space: .card.featured. A space (.card .featured) means "a .featured element inside a .card", and a comma (.card, .featured) means "either one".
Can CSS select a parent element?
:has() does it — div:has(> img) selects a div by its contents — and it's supported in current browsers, Soup Sieve, Cheerio, and cssselect. For older parsers, or for anything needing ancestor::-style traversal several levels up, XPath remains the reliable option.
Are CSS selectors case-sensitive?
Tag names are not, in HTML — DIV and div both match <div>. Class names, IDs, and attribute values are. Attribute names are not. Add the i flag inside brackets for a case-insensitive value match: [href$=".PDF" i].
Why does my selector work in DevTools but not in my scraper?
Usually one of three things. DevTools queries the live DOM after JavaScript ran, while your scraper parses the raw HTML response — view source and check the element is actually there. Or the content sits inside an <iframe>, which selectors cannot cross. Or you're using a pseudo-class your parser doesn't implement, such as :not() with a comma-separated list under Scrapy.
How do I select the first element with a CSS selector?
Use your library's single-element call — querySelector, select_one, .first() — rather than a positional pseudo-class. li:first-child means "first child of each parent", so on a page with five lists it returns five elements, not one.