---
name: webscraping-ai-api
description: Call the WebScraping.AI HTTP API to fetch rendered HTML, clean text, CSS-selected fragments, AI answers or structured JSON from any web page, parsed Google search results, or structured data for supported social sites. Use it when writing code that scrapes pages through WebScraping.AI, or when you need live page content and have an API key.
---

# WebScraping.AI API

WebScraping.AI fetches a web page for you, with a headless browser and rotating proxies, and returns it in the shape you ask for. Full docs in Markdown: https://webscraping.ai/llms.txt (index) and https://webscraping.ai/docs.md.

## Basics

- Base URL `https://api.webscraping.ai`. Every endpoint is a `GET`.
- Authenticate with the `api_key` query parameter. Keys come from https://webscraping.ai/dashboard after signing up; never hard-code one, read it from an environment variable such as `WEBSCRAPING_AI_API_KEY`.
- URL-encode every parameter value (with curl, use `-G` plus `--data-urlencode`).
- OpenAPI spec: https://webscraping.ai/openapi.yml (also `/openapi.json`). Official SDKs exist for Python, JavaScript, PHP, Ruby, Go, Java and C#: https://webscraping.ai/docs/sdks.md

## Pick the endpoint

| Need | Endpoint | Key parameters | Docs |
| --- | --- | --- | --- |
| A question answered about a page | `/ai/question` | `url`, `question` | https://webscraping.ai/docs/ai-question.md |
| Specific fields as JSON | `/ai/fields` | `url`, `fields[name]=description` | https://webscraping.ai/docs/ai-fields.md |
| Full rendered HTML | `/html` | `url` | https://webscraping.ai/docs/html.md |
| Readable page text for an LLM | `/text` | `url` | https://webscraping.ai/docs/text.md |
| HTML of one or several CSS selectors | `/selected`, `/selected-multiple` | `url`, `selector` / `selectors` | https://webscraping.ai/docs/selected.md |
| Google search results as JSON | `/serp` | `q` | https://webscraping.ai/docs/serp.md |
| JSON for a YouTube, TikTok, X, LinkedIn, Instagram or Reddit page | `/data` | `url` | https://webscraping.ai/docs/data.md |
| Remaining credits and concurrency | `/account` | none | https://webscraping.ai/docs/account.md |

Prefer `/text` or the AI endpoints over `/html` when the result goes into a model: they are far smaller. Use `/serp` for Google result pages instead of scraping google.com.

## Cost and options

Requests cost credits and failed requests are free. The main levers on page endpoints:

- `js` (default `true`) renders JavaScript in headless Chrome; `js=false` is cheaper and faster for static pages.
- `proxy`: `datacenter` (default), `residential`, `stealth`, or `auto` (tries each tier in one request and bills only the tier that worked).
- `country` picks the proxy location; `timeout` caps page retrieval time in ms (max 25000).

Per-option credit costs: https://webscraping.ai/docs/rules.md. All parameters: https://webscraping.ai/docs/parameters.md

## Errors

Error bodies are JSON with a `message`. `400` bad parameters, `402` out of credits, `403` wrong API key, `429` too many concurrent requests, `504` timeout. A `500` means the target page could not be scraped: read `error_code` and, when present, apply `next_step.params` (and drop `next_step.remove`) and retry; `next_step.cost` is the credit price of that retry. The proxy escalation ladder is datacenter → residential → stealth. Details: https://webscraping.ai/docs/errors.md

## Example

```bash
curl -G "https://api.webscraping.ai/ai/question" \
  --data-urlencode "api_key=$WEBSCRAPING_AI_API_KEY" \
  --data-urlencode "url=https://example.com" \
  --data-urlencode "question=What is this page about?"
```

For AI assistants that support MCP, the hosted server at `https://mcp.webscraping.ai/mcp` exposes the same endpoints as tools with OAuth sign-in and no API key: https://webscraping.ai/integrations/mcp-server.md