Best Proxies for Web Scraping: 7 Providers Compared (2026)

Start with datacenter proxies. They cost roughly a tenth of residential bandwidth and work fine on most sites. Move to residential only for the specific targets that return 403s, CAPTCHAs, or silently empty HTML. Buying residential bandwidth for everything is the single most common way scraping budgets get wasted.

This guide covers what a proxy actually does for a scraper, the decision rule for picking a type, seven providers with pricing verified in July 2026, and working Python and Node code for rotation, sticky sessions, and retry-on-block.

Key Takeaways

• Datacenter first, residential on failure. Datacenter bandwidth runs around $0.03/proxy/month (Webshare, 100-proxy pack); residential runs $1.75–$7/GB. The gap is one to two orders of magnitude.
• Bandwidth, not requests, is what you pay for. At $3/GB, scraping 100,000 pages that average 500 KB costs about $150 — before you render any JavaScript, which multiplies page weight.
• Entry price and marginal price differ wildly. IPRoyal starts at $7.00/GB for 1 GB but drops to $1.75/GB in bulk; Bright Data's list residential rate is $8/GB with a promotional $4/GB.
• Advertised pool size is close to meaningless. Bright Data advertises 400M+ IPs and IPRoyal 64M+, but what matters is how many are clean for your target, which no vendor publishes.
• A raw proxy solves one problem out of four. You still own TLS fingerprinting, headers, JS rendering, and retries. A scraping API bundles all four for a per-request price.
• Never trust free proxy lists for production. They are shared, logged, and dead within hours — useful only for throwaway tests.

What you are actually paying for

A proxy forwards your scraper's request so the target sees its IP instead of yours. Concretely, you are buying three things: request distribution so per-IP rate limits never trip, geolocation so a German IP returns German prices, and survivability so one banned IP doesn't stop the crawl.

You are not buying anonymity from a modern anti-bot system. Cloudflare, DataDome, and PerimeterX fingerprint your TLS handshake, HTTP/2 frame ordering, and header casing. A residential IP paired with a default python-requests fingerprint is still trivially detectable — which is why the cheapest proxy that clears your target beats the most expensive one that doesn't. Proxies are one layer; user-agent and header rotation and a real browser engine are the others.

Which proxy type should you buy?

The decision rule: run datacenter until you measure a block rate above ~5% on a target, then switch that target — and only that target — to residential. Track block rate per domain, not globally, or one hostile site will push your whole crawl onto expensive bandwidth.

For the full taxonomy including SOCKS5, backconnect, and sticky vs. rotating sessions, see types of proxies for web scraping.

Proxy provider comparison (July 2026)

Prices below were checked against each vendor's public pricing page in July 2026. Proxy pricing changes often and most vendors run permanent "discounts," so treat these as an entry point, not a quote.

Disclosure: some provider links on this page are affiliate links, and we may earn a commission if you sign up through them. This never affects which providers we include or how we rank them — pricing and specs come from each provider's own public pages. WebScraping.AI is our own product.

Two notes on reading that table honestly:

• Bandwidth pricing is a step function. Decodo's $2.75/GB requires a $275/month commitment. If you burn 8 GB a month, you pay the pay-as-you-go rate, and the headline number is irrelevant to you.
• IPRoyal's non-expiring traffic is a real structural difference, not marketing. Most residential plans reset monthly and you forfeit unused GB. If your scraping is seasonal, that matters more than the per-GB rate.

Looking for a narrower list? We maintain dedicated rankings for residential proxy providers, datacenter proxy providers, and the cheapest residential proxies by price per GB. If you only need throwaway IPs for a test, free proxy lists explains why they break in production.

Raw proxies or a scraping API?

Buying bandwidth is the cheaper line item and the more expensive project. A raw proxy hands you an IP; you still build and maintain rotation, session pinning, header and TLS fingerprint management, headless browser infrastructure, CAPTCHA handling, and retry logic. That is a permanent maintenance surface, not a one-time build.

Rough decision rule:

• Raw proxies when you scrape a small number of predictable targets at high volume, you already run browser infrastructure, and bandwidth cost dominates your bill.
• A scraping API when you scrape many different sites, your targets change, or engineering time costs more than credits.
• Both is common: datacenter proxies for the easy 90% of your crawl, a scraping API for the sites that fight back.

Do the arithmetic before choosing. At 40,000 requests/month against a protected site, residential bandwidth at 500 KB/page is 20 GB — about $60–$80 at typical rates, plus whatever your browser infrastructure costs to run and maintain. The same 40,000 requests on WebScraping.AI's residential proxy with JavaScript rendering (25 credits each) is 1M credits, which is the $99/month plan, browsers included.

How to use a proxy in Python

Standard requests with an authenticated HTTP proxy:

import requests

PROXY = "http://USERNAME:PASSWORD@gate.example-proxy.com:7000"

response = requests.get(
    "https://example.com/product/123",
    proxies={"http": PROXY, "https": PROXY},
    timeout=30,
)
print(response.status_code, len(response.text))

httpx is the better choice for anything concurrent, since it speaks HTTP/2 — which matters, because an HTTP/1.1-only client is itself a bot signal on sites that expect browsers:

import httpx

PROXY = "http://USERNAME:PASSWORD@gate.example-proxy.com:7000"

with httpx.Client(proxy=PROXY, http2=True, timeout=30.0) as client:
    r = client.get("https://example.com/product/123")
    print(r.status_code, r.http_version)

Sticky sessions

Most residential gateways encode the session ID in the username, so the same IP is reused for a fixed window. The exact syntax is vendor-specific — check your provider's docs — but the shape is consistent:

import uuid, httpx

def sticky_client(session_id: str) -> httpx.Client:
    user = f"USERNAME-session-{session_id}"
    return httpx.Client(
        proxy=f"http://{user}:PASSWORD@gate.example-proxy.com:7000",
        timeout=30.0,
    )

# One IP for a whole multi-step flow: search -> listing -> detail page
with sticky_client(uuid.uuid4().hex[:8]) as client:
    client.get("https://example.com/search?q=laptops")
    client.get("https://example.com/category/laptops")
    client.get("https://example.com/product/123")

Use sticky sessions whenever the site carries state across requests — pagination, carts, anything behind a login. Use rotating IPs for independent page fetches. Mixing them up is a common cause of "it worked yesterday" breakage.

Retry on block, not just on error

The mistake most retry loops make is only catching exceptions. A soft block returns HTTP 200 with a challenge page in the body, so you have to inspect the response:

import random, time, httpx

BLOCK_MARKERS = ("captcha", "access denied", "unusual traffic")

def fetch(url: str, proxies: list[str], attempts: int = 4) -> httpx.Response | None:
    for attempt in range(attempts):
        proxy = random.choice(proxies)
        try:
            with httpx.Client(proxy=proxy, timeout=30.0) as client:
                r = client.get(url)
            blocked = r.status_code in (403, 429, 503) or any(
                m in r.text[:2000].lower() for m in BLOCK_MARKERS
            )
            if not blocked:
                return r
        except httpx.HTTPError:
            pass
        time.sleep(2 ** attempt + random.random())  # exponential backoff + jitter
    return None

The jitter matters. Fixed-interval retries from a pool of IPs produce a machine-regular traffic pattern that is easy to fingerprint even when every individual request looks fine.

How to use a proxy in Node.js

Node's built-in fetch ignores HTTP_PROXY environment variables. Use undici's ProxyAgent explicitly:

import { ProxyAgent } from 'undici';

const agent = new ProxyAgent('http://USERNAME:PASSWORD@gate.example-proxy.com:7000');

const res = await fetch('https://example.com/product/123', {
  dispatcher: agent,
  headers: { 'accept-language': 'en-US,en;q=0.9' },
});

console.log(res.status, (await res.text()).length);

For browser-based scraping, the proxy is set at launch and credentials are supplied separately — see the Playwright and Puppeteer guides for the full setup, including why per-context proxies beat relaunching the browser per IP.

Letting an API manage the proxies

If you would rather not run a proxy pool at all, WebScraping.AI takes a proxy parameter and handles rotation, geolocation, and the browser behind it:

import requests

response = requests.get(
    "https://api.webscraping.ai/html",
    params={
        "api_key": "YOUR_API_KEY",
        "url": "https://example.com/product/123",
        "proxy": "residential",   # datacenter (default) | residential | stealth
        "country": "de",          # geotargeting
        "js": "true",             # headless Chrome rendering
        "wait_for": ".price",     # wait for the element that matters
    },
    timeout=60,
)
print(response.status_code, len(response.text))

Switching a target from datacenter to residential is a one-word change, which is what makes the "datacenter until it breaks" rule cheap to follow. Credit costs are published: datacenter is 1 credit without JS and 5 with it; residential is 10 and 25; stealth is 50 either way. Failed requests cost nothing, so a target that blocks you doesn't also bill you.

If you want the parsed values rather than HTML, /ai/fields extracts them in the same call:

response = requests.get(
    "https://api.webscraping.ai/ai/fields",
    params={
        "api_key": "YOUR_API_KEY",
        "url": "https://example.com/product/123",
        "proxy": "residential",
        "fields[price]": "Current product price in USD",
        "fields[in_stock]": "Whether the product is in stock, true or false",
    },
    timeout=60,
)
print(response.json())

That pattern is the backbone of price monitoring and product data aggregation pipelines, where the schema is stable but every retailer's markup is different. Full parameter reference is in the docs, and SDKs exist for Ruby, Python, PHP, JavaScript, Go, Java, and C#.

What will proxies actually cost you?

Work it out before you commit to a plan:

monthly cost ≈ pages/month × avg page size (GB) × price per GB

Three worked examples, assuming 500 KB per rendered page:

The datacenter column looks like a free lunch and is not: those proxies are only worth anything on sites that accept them, and concurrency, not price, becomes your constraint. But it shows why the "datacenter first" rule pays. Cutting rendered page weight helps too — request the HTML endpoint rather than a full browser render when the data is in the initial response, and skip images.

Staying on the right side of the line

Proxies change which IP a site sees; they do not change what you are allowed to collect. Terms of service, personal data under GDPR/CCPA, and copyrighted content are all unaffected by your proxy setup. Rotating IPs specifically to evade a block you have been explicitly given is a materially different posture from distributing load politely. Our overview of whether web scraping is legal walks through where the lines actually sit.

Choosing, in one paragraph

If you scrape a handful of unprotected sites, buy datacenter proxies from Webshare and move on. If specific targets block you, add residential bandwidth from Decodo or IPRoyal and route only those domains through it. If you need enterprise contracts, invoicing, and compliance documentation, Bright Data and Oxylabs are the two that reliably have them. And if proxy plumbing isn't the problem you want to own, use a scraping API that bundles proxies, browsers, and retries into one request.

WebScraping.AI's free tier is 2,000 credits per month with no credit card — enough to test whether datacenter proxies clear your target before you buy a single gigabyte. Get an API key or compare us against the alternatives first.