Rust Reqwest Guide: HTTP Requests, JSON, Timeouts, and Retries

reqwest is Rust's batteries-included HTTP client: built on hyper, async-first with an optional blocking API, and handling TLS, JSON, cookies, redirects, and connection pooling out of the box. It's the crate almost every Rust project doing HTTP reaches for first. This guide goes from cargo add reqwest to the production topics — timeouts, typed error handling, retries with middleware, streaming, mocking in tests, and how to see what your client is actually sending.

Key Takeaways

• Use one Client for your whole program — it holds the connection pool; creating a client per request throws pooling away
• reqwest is async by default (bring tokio); the blocking feature gives a synchronous API for CLIs and scripts
• JSON in and out is one .json() call each way with the json feature and serde derive
• reqwest doesn't fail on HTTP 404/500 — call .error_for_status() if you want statuses as errors
• Retries aren't built in: the reqwest-middleware + reqwest-retry crates add exponential backoff cleanly
• RUST_LOG=reqwest=debug plus .connection_verbose(true) answers most "what is it actually sending?" questions

Installing reqwest

cargo add reqwest --features json
cargo add tokio --features full        # async runtime
cargo add serde --features derive      # for typed JSON

Features you'll actually toggle:

rustls-tls deserves a note: it removes the OpenSSL system dependency, which is the standard fix for cross-compilation and slim Docker builds (default-features = false, features = ["rustls-tls"]).

GET requests and query parameters

use std::collections::HashMap;

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let client = reqwest::Client::new();

    let resp = client
        .get("https://httpbin.org/get")
        .query(&[("q", "web scraping"), ("page", "2")])   // ?q=web+scraping&page=2
        .header("Accept", "application/json")
        .send()
        .await?;

    println!("status: {}", resp.status());
    let body = resp.text().await?;
    println!("{body}");
    Ok(())
}

.query() takes anything serializable as key/value pairs — slices of tuples, HashMaps, or a serde struct — and percent-encodes for you. Values that need encoding (spaces, &, unicode) are handled; never format query strings by hand.

The one-liner reqwest::get(url) exists for quick experiments, but it builds a throwaway client each call — in real code, create one Client and reuse it everywhere (it's cheaply clonable; clones share the same pool).

Blocking client (no async required)

For CLIs, build scripts, and simple tools, skip tokio entirely with the blocking feature:

fn main() -> Result<(), reqwest::Error> {
    let client = reqwest::blocking::Client::new();
    let body = client.get("https://httpbin.org/get").send()?.text()?;
    println!("{body}");
    Ok(())
}

Same builder API, no .await. Two rules: don't call blocking reqwest inside an async runtime (it panics with "Cannot drop a runtime in a context where blocking is not allowed" — it spins up its own tokio internally), and prefer the async client anywhere you're already async.

JSON with serde

use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct NewPost { title: String, body: String }

#[derive(Deserialize, Debug)]
struct Post { id: u64, title: String }

let created: Post = client
    .post("https://api.example.com/posts")
    .json(&NewPost { title: "hi".into(), body: "…".into() })   // serializes + sets Content-Type
    .send()
    .await?
    .json()                                                    // deserializes the response
    .await?;

Deserialization failures surface as reqwest::Error with is_decode() == true — which is also what you get when a server returns an HTML error page where JSON was expected. When debugging "expected value at line 1", read the body as text first and look at what actually came back. For exploratory JSON without defining structs, deserialize into serde_json::Value.

Form posts are symmetric: .form(&[("user", "vlad")]) sends application/x-www-form-urlencoded. Multipart uploads (with the multipart feature):

let form = reqwest::multipart::Form::new()
    .text("description", "quarterly report")
    .part("file", reqwest::multipart::Part::bytes(std::fs::read("report.pdf")?)
        .file_name("report.pdf")
        .mime_str("application/pdf")?);
let resp = client.post(url).multipart(form).send().await?;

Configuring the client: timeouts, redirects, cookies, proxies

All the durable settings live on ClientBuilder:

use std::time::Duration;

let client = reqwest::Client::builder()
    .timeout(Duration::from_secs(30))            // total per-request deadline
    .connect_timeout(Duration::from_secs(5))     // TCP/TLS handshake budget
    .redirect(reqwest::redirect::Policy::limited(5))
    .cookie_store(true)                          // needs the `cookies` feature
    .user_agent("my-scraper/1.0")
    .proxy(reqwest::Proxy::all("http://user:pass@proxy.example.com:3128")?)
    .build()?;

• Timeouts: there is no default request timeout — a hung server hangs your task. Always set one; override per request with .timeout(...) on the request builder. Timeouts surface as errors where .is_timeout() is true.
• Redirects: followed by default up to 10 hops. Policy::none() lets you inspect 3xx responses yourself (resp.status().is_redirection() + the Location header) — the usual choice when scraping login flows, where a 302's Set-Cookie matters. Note: reqwest strips Authorization headers on cross-host redirects — that's by design, not a bug.
• Cookies: with cookie_store(true), Set-Cookie responses persist and replay automatically across requests to the same client, which is all that session-based logins need. For pre-seeding cookies, use reqwest::cookie::Jar and ClientBuilder::cookie_provider.
• Proxies: Proxy::all() routes everything; Proxy::http()/Proxy::https() split by scheme; socks5h:// URLs work with the socks feature (the h resolves DNS through the proxy). Without explicit configuration, reqwest also honors HTTP_PROXY/HTTPS_PROXY/NO_PROXY environment variables.

Connection pooling is on by default and per-client; tune it only when you have evidence:

let client = reqwest::Client::builder()
    .pool_max_idle_per_host(20)                          // default is unlimited idle
    .pool_idle_timeout(Duration::from_secs(90))
    .tcp_keepalive(Duration::from_secs(60))
    .build()?;

For high-concurrency scraping against one host, pool_max_idle_per_host roughly matching your concurrency keeps handshakes rare; servers that drop idle connections mid-reuse (sporadic "connection closed before message completed") are fixed by lowering pool_idle_timeout.

Error handling done properly

reqwest::Error is one type with classification methods, and — important — HTTP error statuses are not Rust errors:

match client.get(url).send().await {
    Ok(resp) if resp.status().is_success() => {
        let body = resp.text().await?;
        // …
    }
    Ok(resp) => eprintln!("HTTP {}: server answered, unhappily", resp.status()),
    Err(e) if e.is_timeout() => eprintln!("timed out: {e}"),
    Err(e) if e.is_connect() => eprintln!("connection failed (DNS/TCP/TLS): {e}"),
    Err(e) => eprintln!("other transport error: {e}"),
}

When "non-2xx should just be an error" is the right policy, resp.error_for_status()? converts 4xx/5xx into a reqwest::Error (with .status() populated) and passes success responses through. In application code, wrap transport errors into your own error enum with thiserror, or bubble everything as anyhow::Error in binaries — both compose fine with ? since reqwest::Error implements std::error::Error.

Retries with exponential backoff

reqwest has no built-in retry; the ecosystem answer is reqwest-middleware + reqwest-retry:

use reqwest_middleware::ClientBuilder;
use reqwest_retry::{RetryTransientMiddleware, policies::ExponentialBackoff};

let retry_policy = ExponentialBackoff::builder().build_with_max_retries(3);
let client = ClientBuilder::new(reqwest::Client::new())
    .with(RetryTransientMiddleware::new_with_policy(retry_policy))
    .build();

let resp = client.get(url).send().await?;   // now retries 5xx/connect errors with backoff

It retries transient failures (connection errors, 5xx, 429) and leaves deterministic ones alone. For custom rules — say, retrying only idempotent methods, or honoring Retry-After yourself — a hand-rolled loop is a dozen lines with tokio::time::sleep; keep jitter in the delay so a fleet of scrapers doesn't synchronize its hammering.

Streaming large downloads

Buffering a 2 GB file through .bytes() holds 2 GB in RAM. Stream instead (feature stream, plus the futures crate for StreamExt):

use futures_util::StreamExt;
use tokio::io::AsyncWriteExt;

let resp = client.get(file_url).send().await?.error_for_status()?;
let mut out = tokio::fs::File::create("dump.bin").await?;
let mut stream = resp.bytes_stream();
while let Some(chunk) = stream.next().await {
    out.write_all(&chunk?).await?;
}

The same idea powers progress bars (content_length() for the total), resumable downloads (Range headers), and bounded-memory scraping of huge exports. Uploads stream too: Body::wrap_stream() accepts any Stream of bytes.

Concurrency: fetching many URLs

The idiomatic bounded-concurrency fan-out:

use futures_util::{stream, StreamExt};

let client = reqwest::Client::new();
let results: Vec<_> = stream::iter(urls)
    .map(|url| {
        let client = client.clone();          // clones share the pool
        async move {
            let resp = client.get(&url).send().await?;
            Ok::<_, reqwest::Error>((url, resp.status()))
        }
    })
    .buffer_unordered(20)                     // at most 20 in flight
    .collect()
    .await;

buffer_unordered is the knob that keeps you fast and polite — unbounded join_all over ten thousand URLs is how you get banned and how you exhaust file descriptors, in that order.

Logging and debugging

reqwest logs through tracing/log. Wire up any subscriber and raise the filter:

RUST_LOG=reqwest=debug,hyper=debug cargo run

tracing_subscriber::fmt::init();     // or env_logger::init();

That shows request lifecycles, redirects, and pool activity. Two more tools when that's not enough:

• .connection_verbose(true) on ClientBuilder — logs read/write events on connections (trace level), close to a wire dump
• Point the client at a local debugging proxy (mitmproxy: Proxy::all("http://127.0.0.1:8080") + accepting its CA, or danger_accept_invalid_certs(true) strictly in a debug build) to inspect full requests and responses in a UI

Printing the almost-final request without sending it also works: client.get(url).build()? returns the Request, whose method/URL/headers you can log.

Testing: mock the server, not the client

Because reqwest talks real HTTP, the clean testing pattern is a local mock server — no traits to abstract, just a URL you control. With wiremock:

use wiremock::{MockServer, Mock, ResponseTemplate};
use wiremock::matchers::{method, path};

#[tokio::test]
async fn fetches_user() {
    let server = MockServer::start().await;
    Mock::given(method("GET")).and(path("/users/1"))
        .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
            "id": 1, "name": "Ada"
        })))
        .mount(&server)
        .await;

    let user: User = reqwest::get(format!("{}/users/1", server.uri()))
        .await.unwrap().json().await.unwrap();
    assert_eq!(user.name, "Ada");
}

mockito and httpmock fill the same role with different ergonomics. The structural requirement for all of them: your code must take the base URL as configuration rather than hardcoding it — which is a refactor worth doing anyway.

reqwest vs the other Rust HTTP clients

If you're deciding for a scraper: reqwest, plus the scraper crate for CSS-selector HTML parsing — that pairing is covered end-to-end in our Rust web scraping guide.

Scraping with reqwest, and when it's not enough

reqwest happily pulls static HTML at very high throughput. What it can't do is execute JavaScript or make a datacenter IP look residential — the two walls every serious scraper hits. When the response is an empty <div id="root"> or a bot-detection page, the fix isn't more headers; it's a rendering pipeline with rotating proxies. WebScraping.AI provides exactly that behind an HTTP API, so the Rust side stays plain reqwest:

#[derive(serde::Deserialize)]
struct Product { name: String, price: String }

let product: Product = client
    .get("https://api.webscraping.ai/ai/fields")
    .query(&[
        ("api_key", api_key.as_str()),
        ("url", "https://example.com/product/42"),
        ("fields[name]", "Product name"),
        ("fields[price]", "Price with currency"),
    ])
    .send().await?
    .error_for_status()?
    .json().await?;

The /html endpoint returns rendered pages for your own parsing; /ai/fields skips parsing entirely and returns the structured data — a good division of labor with Rust doing what it's best at: fast, safe processing of the results.

Frequently asked questions

Is reqwest async or blocking? Async-first: the primary API returns futures and expects a tokio runtime. The blocking feature adds reqwest::blocking::Client with the same builder API for synchronous code. Don't mix them in one call stack — blocking calls inside an async context panic; use the async client there instead.

Why does reqwest return Ok for a 404 response? Because HTTP worked: request out, response in. Transport failures (DNS, TCP, TLS, timeout) are Err; status codes are data on the Ok response. Call .error_for_status() on the response when you want 4xx/5xx converted into errors — it's a deliberate one-liner, not a missing feature.

How do I set headers on every request? ClientBuilder::default_headers with a HeaderMap for arbitrary headers, plus dedicated builders for common ones (user_agent(...)). Per-request .header(...) overrides defaults on collision. Headers with sensitive values can be marked set_sensitive(true) so logging middleware redacts them.

Does reqwest support HTTP/2? Yes — negotiated automatically via TLS ALPN when the server supports it, with no configuration. You can inspect the negotiated version with resp.version(). HTTP/3 exists behind an unstable feature flag; don't build on it yet.

How do I add a retry to reqwest? Use reqwest-middleware with reqwest-retry for policy-based exponential backoff on transient failures, or write a small loop with tokio::time::sleep for custom logic. Retry only idempotent requests blindly; for POSTs, make sure the server tolerates replays (idempotency keys) first.

Why is my reqwest client slow compared to curl? Almost always one of: creating a new Client per request (no connection reuse — hoist it), missing --release (debug builds are dramatically slower at TLS), or DNS — try hickory-dns via the corresponding feature. With a reused client and a release build, reqwest benchmarks at the same order as curl.