Is Web Scraping Legal? What the Case Law Actually Says

"Is web scraping legal?" has no single answer, because it isn't one legal question. It's five, and they have different answers, different jurisdictions, and different worst-case outcomes. A scraper can be entirely clear of the US computer-crime statute and still lose a contract claim — that is precisely what happened to hiQ Labs, the company usually cited as proof that scraping is legal.

This guide walks through each of the five axes with the cases that actually decided them, current as of July 2026.

This is not legal advice. It is a developer-oriented summary of publicly reported case law and regulatory guidance, written by engineers, not lawyers. Case law changes, applies differently to different facts, and varies by jurisdiction. Before you build a business on scraped data, get advice from a qualified lawyer in your jurisdiction.

Key Takeaways

• Scraping publicly accessible pages is not a US computer-crime violation. Van Buren (2021) and hiQ (2022) settled that: a public page has "no gates to lift or lower."
• hiQ still lost. In December 2022 it accepted a $500,000 consent judgment and a permanent injunction, and deleted its scraped data — on breach of contract, not on the public-data CFAA theory it won. People routinely misreport this case as a clean win for scrapers.
• Logging in changes everything. Every scraping case where the defendant lost badly involved credentials, fake accounts, or gated content. Meta v. Bright Data (2024) turned on exactly this: logged-out scraping of public pages did not breach Meta's terms.
• The EU is stricter on contract and data, looser on nothing. Under Ryanair v. PR Aviation (CJEU, 2015), a site owner can restrict scraping by contract even when no database right exists — a lever US law gives more grudgingly.
• "Publicly available" is not a GDPR defence. Clearview AI scraped only public images and has drawn fines of €20M (France), €20M (Italy), and €30.5M (Netherlands).
• robots.txt is not a law in the US, but it is nearly one in the EU for AI training: Article 4 of the DSM Directive makes a machine-readable opt-out legally operative.

The five questions, at a glance

Work out which of these your project actually touches. Most scraping projects touch two or three, not all five.

The uncomfortable truth is that axes 1 and 5 — the ones people worry about most — are now the hardest for a site owner to win on. Axes 2, 3, and 4 are where scrapers actually lose.

1. Does scraping violate the CFAA?

The Computer Fraud and Abuse Act (18 U.S.C. § 1030) criminalises accessing a computer "without authorization" or in a way that "exceeds authorized access." For two decades site owners argued that a terms-of-service ban converted ordinary scraping into a federal crime. That argument is now dead for public pages.

Van Buren v. United States (Supreme Court, June 2021) read "exceeds authorized access" narrowly. Justice Barrett's 6–3 majority adopted a gates-up-or-down test: liability attaches when you access an area of a system that is off-limits to you, not when you access something you're allowed to see for a reason the owner dislikes. Misusing data you were entitled to retrieve is not a CFAA violation.

hiQ Labs v. LinkedIn (Ninth Circuit, April 2022) applied that on remand. Public profiles, the court reasoned, present "no gates to lift or lower in the first place" — a defining feature of a public website is that it's open to anyone with a browser. Scraping them falls outside the CFAA even after a cease-and-desist letter and even against the site's express wishes.

The line the case law draws is technical, not contractual:

• Gate up — the page renders for an anonymous browser with no credentials. Outside the CFAA.
• Gate down — login, paywall, API key, or an access control you have to defeat. Squarely inside it.

Ryanair v. Booking.com shows how much that distinction matters. Booking.com reached the password-protected "myRyanair" area through a third party. A Delaware jury found a CFAA violation in July 2024 — and awarded exactly $5,000, the statutory minimum. Judge Bryson then granted judgment as a matter of law for Booking.com, finding Ryanair hadn't proven the $5,000 in "loss" the statute requires. Ryanair appealed to the Third Circuit, and as of July 2026 the appeal is pending. Note the fact pattern: this was never a public-data case.

The practical rule: if your scraper needs a password, a session cookie you didn't earn honestly, or a bypass of a technical access control, you have left the safe zone. If it fetches what any anonymous visitor's browser fetches, the CFAA is very likely not your problem.

2. Are website terms of service legally binding on a scraper?

This is where hiQ actually lost, and it is the axis most developers underestimate.

Winning the CFAA appeal in April 2022 did not end hiQ's case. In November 2022 the district court found hiQ had breached LinkedIn's User Agreement. On 8 December 2022 the parties entered a consent judgment: $500,000 against hiQ, a permanent injunction requiring it to stop scraping LinkedIn, and an obligation to delete the source code, data, and algorithms built from it. hiQ, the company whose name is invoked to prove scraping is legal, was put out of the LinkedIn-scraping business by contract law.

The stipulated judgment also covered CFAA liability — but on a specific basis worth understanding, because it doesn't undercut the Ninth Circuit's public-data holding: it rested on hiQ using fake accounts to reach password-protected pages. Gate down.

When are you actually bound?

A contract needs assent. Courts distinguish:

• Clickwrap — you affirmatively clicked "I agree." Generally enforceable.
• Browsewrap — a "Terms" link in the footer, no assent required. Frequently unenforceable without actual or inquiry notice (Nguyen v. Barnes & Noble, 9th Cir. 2014).
• Account holders — if you registered, you clicked something. You're bound, logged in or not.

Meta v. Bright Data (N.D. Cal., 23 January 2024) is the clearest modern statement. Judge Edward Chen granted summary judgment for Bright Data: it had scraped only logged-out, public pages, and Meta produced no evidence it had scraped while logged into an account. Meta's terms therefore didn't reach the conduct. A detail did real work here — Meta had removed language in 2009 that purported to bind everyone who visited Facebook, user or not.

The practical rule: an account is a liability. If you never create one, never accept terms, and only fetch logged-out pages, the strongest weapon site owners have is substantially blunted. If you do hold an account, assume its terms bind your scraping everywhere on that domain.

Does robots.txt have legal force?

In the US, no court has held that ignoring robots.txt is by itself unlawful. It is not a contract and not an access control. But it is evidence — of the site's expressed wishes, and of your state of mind if a dispute reaches discovery.

In the EU it is much closer to binding. Article 4 of the DSM Directive (EU) 2019/790 permits text and data mining of lawfully accessible content unless the rightsholder has reserved it in a machine-readable form — and robots.txt is the canonical way to do that. Ignore a TDM opt-out and you lose the exception you were relying on.

A newer front: Reddit v. Perplexity AI (S.D.N.Y., filed 22 October 2025) pleads DMCA § 1201 anti-circumvention rather than CFAA, alleging defendants rotated IPs and masked identities to defeat rate limits and anti-bot systems. A motion to dismiss was argued on 23 July 2026 and remains pending. If a circumvention theory succeeds, anti-bot evasion becomes a distinct legal risk from access itself — worth watching closely.

3. Copyright, database rights, and AI training

Facts aren't copyrightable. Prices, availability, ratings, addresses, and specifications are facts. Copying them is a weak infringement case, which is why price monitoring and property listing aggregation are established industry practice.

Creative expression is different. Article text, photographs, reviews, and product descriptions are protected. Storing them internally to derive facts is one thing; republishing them is another, and it is the fastest way to a takedown.

In the EU and UK there's a second layer. The Database Directive (96/9/EC) grants a sui generis right — independent of copyright — to anyone who made a substantial investment in obtaining, verifying, or presenting a database's contents. Extracting a substantial part infringes even if every individual item is an unprotectable fact. The UK retained an equivalent right after Brexit. There is no US analogue.

And EU contract law reaches further than US contract law here. In Ryanair v. PR Aviation (CJEU, C-30/14, 15 January 2015), the court held that because Ryanair's flight database was protected by neither copyright nor the sui generis right, the Directive's protections for lawful users didn't apply — leaving Ryanair free to restrict screen-scraping by contract. The counterintuitive result: in the EU, having no IP right in your data can leave you with more freedom to impose contractual limits.

Scraping for AI training

Two 2025 US decisions frame this, and both are narrower than the headlines suggested:

• Bartz v. Anthropic (N.D. Cal., June 2025). Judge Alsup held that training an LLM on lawfully acquired books was transformative fair use — but that downloading and retaining over seven million pirated books was not. The piracy claims settled for $1.5 billion, roughly $3,000 per work, with final approval granted.
• Kadrey v. Meta (N.D. Cal., June 2025). Judge Chhabria ruled for Meta on the record the plaintiffs actually built, while explicitly warning the opinion was not broad permission to train on copyrighted work.

The pattern is consistent and it is the single most useful thing to take from this section: courts have been relatively receptive to training as a use, and hostile to how the data was acquired. Acquisition is the exposed flank. If you're assembling a RAG knowledge base or fine-tuning dataset, your provenance records matter more than your fair-use argument.

The EU AI Act now adds disclosure duties: general-purpose AI model providers must publish a summary of training data sources and respect Article 4 TDM opt-outs.

4. GDPR, CCPA, and personal data

This is the axis with the largest fines and the most common misconception.

"It was publicly available" is not a lawful basis under the GDPR. Nothing in the regulation exempts personal data because it was posted publicly. A name, email, profile URL, photo, or job history is personal data whether you found it behind a login or on an open page.

Clearview AI is the cautionary tale. It scraped only publicly accessible images, and has been fined €20M by France's CNIL (October 2022), €20M by Italy's Garante (February 2022), and €30.5M by the Dutch DPA (September 2024).

What the GDPR actually requires

In practice your only workable legal basis is legitimate interest under Article 6(1)(f) — consent is unobtainable at scale, and the EDPB has pushed back on treating it as a fig leaf. That means passing the three-part test in EDPB Guidelines 1/2024 (adopted 8 October 2024): a legitimate interest, necessity, and a balancing exercise against data subjects' rights.

Then there are the duties people forget: Article 14 transparency (you must generally tell people you hold their data), purpose limitation, data minimisation, retention limits, and — for special category data such as health, religion, or biometrics — a separate Article 9 condition on top of your Article 6 basis.

Regulatory guidance is moving fast right now:

• EDPB Opinion 28/2024 (17 December 2024) on data protection in AI models.
• CNIL guidance (19 June 2025) on scraping publicly accessible data to develop AI systems.
• EDPB Guidelines 03/2026 on web scraping in the context of generative AI, adopted 8 July 2026. These are draft guidelines, open for public consultation until 30 October 2026 — so expect the final text to shift. They emphasise reliable sourcing, collection timestamps, validation, minimisation measures, and case-by-case assessment for special category data.

If you are doing B2B lead generation or sales intelligence in Europe, this is your primary compliance surface, not the CFAA.

The US is more permissive — but no longer simple

The CCPA/CPRA excludes lawfully "publicly available" information from its definition of personal information, which is a genuinely more permissive starting point than the GDPR.

California's Delete Act then narrows it from a different direction. If you sell or share scraped personal data you may qualify as a data broker, with registration due to the CPPA by 31 January 2026, the Delete Request and Opt-Out Platform (DROP) live since 1 January 2026, and an obligation to process consumer deletion requests from 1 August 2026 — checking DROP at least every 45 days. Several other states have enacted comparable regimes.

The practical rule: if you can accomplish your goal without personal data, don't collect it. Discard rather than mask at the point of extraction — pulling only the fields you need is cheaper and far easier to defend than sanitising a hoard afterwards.

5. Trespass to chattels: does load matter?

The oldest theory, and now the weakest. In eBay v. Bidder's Edge (N.D. Cal. 2000) eBay won an injunction on the theory that crawling was an unauthorised intermeddling with its servers.

Intel v. Hamidi (California Supreme Court, 2003) then required actual harm — measurable impairment of the system's functioning. Pure economic harm or annoyance doesn't suffice. Courts have since largely declined to apply the doctrine to ordinary scraping of public sites.

It is not fully dead, and note that it appeared among the stipulated claims in the hiQ consent judgment. But it needs real degradation, which means the mitigation is engineering, not law: rate-limit yourself so there is no harm to allege. Concurrency in the low single digits, backoff on 429 and 503, and crawling off-peak eliminate this axis almost entirely.

A practical compliance checklist

Ordered by how much risk each item actually removes:

1. Stay logged out. No accounts, no credentials, no session tokens you didn't earn. This single decision defuses most of axes 1 and 2.
2. Don't defeat access controls. Paywalls, login walls, and token checks are gates. Anti-bot evasion is a live and unsettled question — see Reddit v. Perplexity.
3. Read robots.txt and honour TDM opt-outs. Little US legal force, real EU force, and it's evidence either way.
4. Rate-limit deliberately. Low concurrency and honest backoff remove axis 5.
5. Extract fields, don't hoard pages. Storing whole copyrighted pages creates copyright exposure you didn't need.
6. Treat personal data as a separate project. Document a lawful basis before collection, not after.
7. Keep provenance records. What you scraped, from where, when, and under what terms. The AI cases show acquisition history is what gets litigated.
8. Get real advice for anything commercial. Everything above is background, not a substitute for a lawyer who knows your facts.

How WebScraping.AI fits the legal picture

A scraping API doesn't change the law — the same five questions apply to our API, to a competitor's, and to a script you write yourself. But some of the checklist above is easier to hold to with the right tooling.

Requests through our API fetch pages the way an anonymous browser does. There's no credential store and no login automation, which keeps you on the right side of the gate-up/gate-down line by default.

For data minimisation, extracting only the fields you need beats downloading and storing whole pages:

curl -G https://api.webscraping.ai/ai/fields \
  -d api_key=YOUR_API_KEY \
  -d url=https://example.com/product/123 \
  -d 'fields[price]=Current product price in USD' \
  -d 'fields[availability]=In stock or out of stock'

import requests

resp = requests.get(
    "https://api.webscraping.ai/ai/fields",
    params={
        "api_key": "YOUR_API_KEY",
        "url": "https://example.com/product/123",
        "fields[price]": "Current product price in USD",
        "fields[availability]": "In stock or out of stock",
    },
    timeout=60,
)
print(resp.json())

You get two facts back — not a copyrighted page, and not the reviewer names and profile photos sitting next to them. That's data minimisation implemented at the fetch layer rather than promised in a policy document. The AI extraction endpoints work the same way for structured records, and the full parameter list is in the API docs.

On request rates, the timeout and concurrency limits on each plan cap how hard you can hit a target, and failed requests are never billed — so backing off politely costs you nothing. The free tier gives you 2,000 credits a month without a credit card if you want to test a compliance-conscious pipeline before committing.

So — is web scraping legal?

Yes, in the sense that no US or EU law prohibits automated collection of public web data, and the courts have repeatedly declined to criminalise it.

But that answer is useless on its own, because the question you actually need to answer is narrower: is this scrape, of this data, from this site, in this jurisdiction, for this purpose, legal? Run it through the five axes. Public pages, no login, polite rates, facts rather than expression, and no personal data is a posture that has held up well in court. Every step away from that profile adds a specific, identifiable risk — and now you know which one.

One more time, because it matters here more than on most pages: this is not legal advice. It's a map of the terrain, assembled from public sources and current to July 2026. If you're building something commercial on scraped data, take it to a lawyer.

If you want to understand the mechanics rather than the law, start with what web scraping is and our Python scraping guide.