---
title: "C# HttpClient: The Complete Guide with Examples"
description: "Everything about HttpClient in C#/.NET with copy-paste examples: GET/POST, headers, auth, JSON, timeouts, proxies, file downloads, and IHttpClientFactory."
url: https://webscraping.ai/blog/csharp-httpclient-guide
markdown_index: https://webscraping.ai/llms.txt
---
# C# HttpClient: The Complete Guide with Examples

`HttpClient` is the standard way to make HTTP requests in C# and .NET — REST calls, file downloads, form submissions, and web scraping all go through it. It's also famously easy to misuse: dispose it per request and you exhaust sockets; make it a naive singleton and it ignores DNS changes. This guide is a complete, example-first reference for HttpClient in modern .NET: every common request type, headers, authentication, JSON, timeouts, proxies, cookies, file transfers, and the `IHttpClientFactory` patterns that make it production-safe.

## Key Takeaways

- Reuse one `HttpClient` (or use `IHttpClientFactory`) — creating a client per request causes socket exhaustion
- `GetAsync`/`PostAsync` cover simple calls; build an `HttpRequestMessage` and use `SendAsync` when a single request needs its own headers
- Set default headers once on `DefaultRequestHeaders`; set body-related headers on the `HttpContent` itself
- The default timeout is 100 seconds; pass a `CancellationToken` for per-request control
- Proxy, cookies, redirects, decompression, and SSL validation are all configured on `HttpClientHandler`, not on `HttpClient`
- HttpClient can't execute JavaScript — for dynamic pages, pair it with a rendering API

## What is HttpClient?

`HttpClient` (in `System.Net.Http`) is .NET's built-in HTTP API. One instance represents a reusable session: it holds default headers, a base address, a timeout, and — through its handler — a connection pool, cookies, and proxy settings.

It replaced two older APIs you'll still meet in legacy code:

| | HttpClient | WebClient | HttpWebRequest |
| --- | --- | --- | --- |
| Status | Current standard | Obsolete (warning since .NET 6) | Legacy, wraps poorly |
| Async | Native `async`/`await` | Bolted on | Callback-based |
| HTTP/2 and HTTP/3 | Yes | No | No |
| Testability | Mockable via `HttpMessageHandler` | Poor | Poor |
| Connection reuse | Pooled, configurable | Per call | Manual |

The difference between HttpClient and WebClient matters mostly in one direction: new code should always use `HttpClient` (or the higher-level abstractions built on it). Microsoft's own guidance is explicit about this.

## Creating and reusing an instance

The single most important HttpClient rule: **it's designed to be instantiated once and reused** , not created per request. `HttpClient` is thread-safe for concurrent requests, so sharing one instance is safe. Each new instance opens its own connection pool, and disposed instances leave sockets in `TIME_WAIT` — under load, a `new HttpClient()` per call exhausts ports and starts throwing `SocketException`:

```csharp
// ❌ Anti-pattern: socket exhaustion under load
using var client = new HttpClient();
var html = await client.GetStringAsync("https://example.com");

// ✅ Reuse a single static instance
private static readonly HttpClient Client = new()
{
    Timeout = TimeSpan.FromSeconds(30)
};
```

The static singleton has one caveat: it caches DNS resolution for the life of each connection. If your target's IP can change (load balancers, blue-green deploys), set a connection lifetime so the pool recycles connections periodically:

```csharp
private static readonly HttpClient Client = new(new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(2)
});
```

In ASP.NET Core apps, skip the manual singleton and use `IHttpClientFactory` — covered [at the end of this guide](https://webscraping.ai/blog/csharp-httpclient-guide#ihttpclientfactory-and-best-practices) — which handles pooling, DNS rotation, and named configuration for you.

## GET requests

The three shortcuts, in increasing order of control:

```csharp
// String — simplest, throws on network errors but NOT on 404/500
string html = await Client.GetStringAsync("https://example.com/products");

// Bytes and streams for binary content
byte[] bytes = await Client.GetByteArrayAsync("https://example.com/logo.png");
await using Stream stream = await Client.GetStreamAsync("https://example.com/feed.xml");

// Full response — inspect status, headers, and body separately
HttpResponseMessage response = await Client.GetAsync("https://example.com/products");
response.EnsureSuccessStatusCode(); // throws HttpRequestException on 4xx/5xx
string body = await response.Content.ReadAsStringAsync();
```

To read the HTTP status code, check `response.StatusCode` (an `HttpStatusCode` enum) or `response.IsSuccessStatusCode` before touching the body:

```csharp
var response = await Client.GetAsync(url);
Console.WriteLine((int)response.StatusCode); // 200
Console.WriteLine(response.StatusCode); // OK
if (!response.IsSuccessStatusCode)
{
    Console.WriteLine($"Request failed: {(int)response.StatusCode} {response.ReasonPhrase}");
}
```

Query strings are your job — build them safely with `Uri.EscapeDataString` rather than string concatenation:

```csharp
var query = $"q={Uri.EscapeDataString("web scraping api")}&page=2";
var response = await Client.GetAsync($"https://example.com/search?{query}");
```

## POST, PUT, PATCH, and DELETE

Each method takes an `HttpContent` describing the body:

```csharp
// POST JSON (System.Net.Http.Json — built into .NET since 5)
var response = await Client.PostAsJsonAsync("https://api.example.com/search",
    new { query = "laptops", limit = 20 });

// POST a form (application/x-www-form-urlencoded)
var form = new FormUrlEncodedContent(new Dictionary<string, string>
{
    ["username"] = "demo",
    ["password"] = "secret"
});
var login = await Client.PostAsync("https://example.com/login", form);

// POST raw JSON with explicit content type
var content = new StringContent("""{"query":"laptops"}""",
    Encoding.UTF8, "application/json");
await Client.PostAsync("https://api.example.com/search", content);

// PUT, PATCH, DELETE
await Client.PutAsJsonAsync("https://api.example.com/products/42",
    new { name = "Updated", price = 49.99 });
await Client.PatchAsync("https://api.example.com/products/42",
    new StringContent("""{"price":39.99}""", Encoding.UTF8, "application/json"));
await Client.DeleteAsync("https://api.example.com/products/42");
```

Note that `Content-Type` belongs to the _content_: set it via the `StringContent` constructor (or `content.Headers.ContentType`), not on the client's default headers — HttpClient rejects content headers set at request level with an `InvalidOperationException`.

## Setting request headers

Headers split into two scopes. **Default headers** apply to every request the client sends; **per-request headers** live on an `HttpRequestMessage`:

```csharp
// Once, at startup — sent with every request
Client.DefaultRequestHeaders.UserAgent.ParseAdd(
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36");
Client.DefaultRequestHeaders.Accept.ParseAdd("text/html,application/xhtml+xml");
Client.DefaultRequestHeaders.AcceptLanguage.ParseAdd("en-US,en;q=0.9");
Client.DefaultRequestHeaders.Add("X-API-Key", apiKey);
```

For one-off headers, build the request explicitly — this is exactly what `HttpRequestMessage` is for:

```csharp
var request = new HttpRequestMessage(HttpMethod.Get, "https://example.com/products");
request.Headers.Add("Referer", "https://www.google.com/");
request.Headers.IfModifiedSince = DateTimeOffset.UtcNow.AddHours(-1);

var response = await Client.SendAsync(request);
```

`GetAsync` vs `SendAsync` is a convenience-vs-control tradeoff: `GetAsync(url)` is literally `SendAsync(new HttpRequestMessage(HttpMethod.Get, url))` internally. Use the shortcuts until a request needs its own method, headers, version, or options — then use `SendAsync`. One rule to remember: an `HttpRequestMessage` can be sent exactly once; build a new one for each attempt (this bites people writing retry loops).

## Authentication

The `Authorization` header has a typed API — `AuthenticationHeaderValue` — with a scheme and a parameter:

```csharp
using System.Net.Http.Headers;

// Basic auth: base64-encode user:password yourself
var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes("apiuser:s3cret"));
Client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Basic", credentials);

// Bearer token (OAuth 2.0, JWTs)
Client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Bearer", accessToken);

// API key header — no scheme, just a custom header
Client.DefaultRequestHeaders.Add("X-API-Key", apiKey);
```

Per-request auth works the same way on `request.Headers.Authorization`. For OAuth flows, the token exchange itself is a plain POST — fetch the token with `PostAsync`, then attach it as a bearer.

Basic auth can also be configured on the handler with a `NetworkCredential`, which additionally answers server challenges (401 + `WWW-Authenticate`) and supports NTLM/Negotiate:

```csharp
var handler = new HttpClientHandler
{
    Credentials = new NetworkCredential("apiuser", "s3cret")
};
var client = new HttpClient(handler);
```

## Working with JSON

`System.Net.Http.Json` extensions handle serialization with `System.Text.Json` under the hood:

```csharp
public record Product(int Id, string Name, decimal Price);

// Deserialize a response directly
Product? product = await Client.GetFromJsonAsync<Product>(
    "https://api.example.com/products/42");

List<Product>? products = await Client.GetFromJsonAsync<List<Product>>(
    "https://api.example.com/products");

// From an HttpResponseMessage
var response = await Client.GetAsync("https://api.example.com/products/42");
var fromResponse = await response.Content.ReadFromJsonAsync<Product>();

// Case-insensitive matching and other options
var options = new JsonSerializerOptions { PropertyNameCaseInsensitive = true };
var data = await response.Content.ReadFromJsonAsync<Product>(options);
```

For large responses, deserialize from the stream instead of buffering the whole body into a string first — `GetFromJsonAsync` already does this internally.

## Timeouts and cancellation

HttpClient's default timeout is **100 seconds** , which is almost never what you want. `Timeout` is set once per client (changing it after the first request throws); per-request timeouts use a `CancellationTokenSource`:

```csharp
// Client-wide
var client = new HttpClient { Timeout = TimeSpan.FromSeconds(30) };

// Per request
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
try
{
    var response = await Client.GetAsync("https://slow.example.com", cts.Token);
}
catch (TaskCanceledException ex) when (ex.InnerException is TimeoutException)
{
    // .NET 5+: distinguishes timeout from explicit cancellation
    Console.WriteLine("Request timed out");
}
```

On timeout, HttpClient throws `TaskCanceledException` (wrapping a `TimeoutException` since .NET 5) — catch it separately from `HttpRequestException` (network/DNS/TLS failures) when you want distinct handling:

```csharp
try
{
    var html = await Client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
    Console.WriteLine($"Network or HTTP error: {ex.StatusCode} {ex.Message}");
}
catch (TaskCanceledException)
{
    Console.WriteLine("Timed out or canceled");
}
```

## HttpClientHandler: SSL, redirects, cookies, compression

Connection-level behavior is configured on the handler passed to the constructor. You can't change a handler after the first request — configure everything up front.

### Ignoring SSL certificate errors

For local development against self-signed certificates, override certificate validation. Never ship this to production — it disables the protection TLS exists for; trust the specific dev certificate instead where possible:

```csharp
var handler = new HttpClientHandler
{
    // Accepts ANY certificate — development only
    ServerCertificateCustomValidationCallback =
        HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
var client = new HttpClient(handler);
```

The callback also lets you pin a specific certificate thumbprint or log validation errors instead of blanket-accepting. Client certificates (mutual TLS) attach via `handler.ClientCertificates.Add(...)`.

### Redirects

Redirects are followed automatically (up to 50). Disable that when you need to observe the `Location` header yourself — for example, to log where scraped URLs actually lead:

```csharp
var handler = new HttpClientHandler { AllowAutoRedirect = false };
var client = new HttpClient(handler);

var response = await client.GetAsync("http://example.com");
Console.WriteLine((int)response.StatusCode); // 301
Console.WriteLine(response.Headers.Location); // https://example.com/
```

One security-driven gotcha: on a redirect from HTTPS to HTTP, or to a different host, .NET strips the `Authorization` header — re-authenticate on the new location explicitly if you need to.

### Cookies and sessions

`CookieContainer` gives you session persistence across requests — log in once, and subsequent requests carry the session cookie automatically:

```csharp
var cookies = new CookieContainer();
var handler = new HttpClientHandler { CookieContainer = cookies };
var client = new HttpClient(handler);

// Login stores the session cookie in the container
await client.PostAsync("https://example.com/login", new FormUrlEncodedContent(
    new Dictionary<string, string> { ["username"] = "demo", ["password"] = "secret" }));

// Sent with the session cookie automatically
var orders = await client.GetStringAsync("https://example.com/account/orders");

// Inspect or seed cookies manually
foreach (Cookie c in cookies.GetCookies(new Uri("https://example.com")))
    Console.WriteLine($"{c.Name}={c.Value}");
cookies.Add(new Uri("https://example.com"), new Cookie("currency", "USD"));
```

### Automatic decompression

Browsers always advertise gzip/brotli support; HttpClient doesn't unless you tell it to. Enabling `AutomaticDecompression` sets the `Accept-Encoding` header _and_ transparently decompresses responses:

```csharp
var handler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.All // gzip, deflate, brotli
};
```

This is worth enabling for scraping unconditionally: smaller transfers, and some servers treat clients without `Accept-Encoding` as suspicious.

## Using a proxy

Proxies are configured with `WebProxy` on the handler — with or without credentials:

```csharp
var handler = new HttpClientHandler
{
    Proxy = new WebProxy("http://proxy.example.com:8080")
    {
        Credentials = new NetworkCredential("proxyuser", "proxypass")
    },
    UseProxy = true
};
var client = new HttpClient(handler);
```

By default (`UseProxy = true`, `Proxy = null`), HttpClient uses the **system proxy** — on Windows the WinINet/WinHTTP settings, elsewhere the `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY` environment variables. That means you can route an unmodified app through a proxy just by setting environment variables before launch. Set `UseProxy = false` to bypass proxies entirely, which also skips proxy auto-detection — a common fix for slow first requests on Windows.

For scraping at scale, a single static proxy just moves your bot signature to a different IP. Rotating proxy pools are the standard answer — see our comparison of [proxy providers for web scraping](https://webscraping.ai/blog/best-proxy-providers-for-web-scraping) and the [types of proxies](https://webscraping.ai/blog/types-of-proxies-for-web-scraping) worth paying for.

## Downloading files and streaming

For small files, `GetByteArrayAsync` plus `File.WriteAllBytesAsync` is fine. For anything big, stream — `HttpCompletionOption.ResponseHeadersRead` makes HttpClient return as soon as headers arrive instead of buffering the entire body in memory:

```csharp
using var response = await Client.GetAsync(
    "https://example.com/dataset.zip",
    HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();

await using var source = await response.Content.ReadAsStreamAsync();
await using var target = File.Create("dataset.zip");
await source.CopyToAsync(target);
```

Download progress is a manual loop over the stream — read chunks, count bytes, report against `Content-Length`:

```csharp
var total = response.Content.Headers.ContentLength;
var buffer = new byte[81920];
long readSoFar = 0;
int read;
while ((read = await source.ReadAsync(buffer)) > 0)
{
    await target.WriteAsync(buffer.AsMemory(0, read));
    readSoFar += read;
    if (total.HasValue)
        Console.Write($"\r{readSoFar * 100 / total.Value}%");
}
```

## Uploading files: multipart form data

`MultipartFormDataContent` builds the `multipart/form-data` bodies that HTML file-upload forms send — mix text fields and files freely:

```csharp
using var form = new MultipartFormDataContent();
form.Add(new StringContent("Q3 Report"), "title");

var file = new StreamContent(File.OpenRead("report.pdf"));
file.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
form.Add(file, "document", "report.pdf"); // field name, file name

var response = await Client.PostAsync("https://example.com/upload", form);
```

## Retries and resilience

Transient failures — timeouts, 429 rate limits, 5xx responses — deserve retries with backoff, and that logic belongs in a message handler rather than around every call site. In modern .NET, `Microsoft.Extensions.Http.Resilience` packages the standard pipeline (retry + circuit breaker + per-attempt timeout) in one line:

```csharp
builder.Services.AddHttpClient("scraper", c =>
{
    c.Timeout = TimeSpan.FromSeconds(60);
})
.AddStandardResilienceHandler(); // retries, circuit breaker, timeouts
```

Rolling it by hand (console apps, no DI) is a loop with exponential backoff — remembering that `HttpRequestMessage` can't be reused across attempts:

```csharp
async Task<HttpResponseMessage> GetWithRetryAsync(string url, int attempts = 3)
{
    for (var i = 1; ; i++)
    {
        try
        {
            var response = await Client.GetAsync(url);
            if ((int)response.StatusCode < 500 && response.StatusCode != HttpStatusCode.TooManyRequests)
                return response;
            if (i == attempts) return response;
        }
        catch (HttpRequestException) when (i < attempts) { }
        await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, i))); // 2s, 4s, 8s
    }
}
```

Respect `Retry-After` on 429s when it's present — hammering a rate limiter earns IP bans, not data.

## IHttpClientFactory and best practices

In any app with dependency injection, `IHttpClientFactory` (from `Microsoft.Extensions.Http`) is the recommended way to get clients. It pools the underlying handlers (fixing both socket exhaustion _and_ the stale-DNS problem), centralizes configuration in named or typed clients, and composes with resilience handlers:

```csharp
// Program.cs
builder.Services.AddHttpClient<CatalogClient>(c =>
{
    c.BaseAddress = new Uri("https://api.example.com/");
    c.Timeout = TimeSpan.FromSeconds(30);
    c.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0");
});

// Typed client — inject anywhere
public class CatalogClient(HttpClient http)
{
    public Task<List<Product>?> GetProductsAsync() =>
        http.GetFromJsonAsync<List<Product>>("products");
}
```

The condensed best-practice list:

- **One client per logical destination** , reused for the app's lifetime — via factory or static instance
- **Don't subclass HttpClient** ; configure instances or write `DelegatingHandler` middleware instead
- **`SocketsHttpHandler.PooledConnectionLifetime`** (or the factory's handler rotation) if you run a long-lived singleton
- **Set a real timeout** — 100 seconds hides problems
- **`EnsureSuccessStatusCode` or explicit status checks** — HttpClient does not throw on 404s by itself
- **Dispose `HttpResponseMessage`** when streaming large bodies; it returns the connection to the pool

For high-load scraping specifically: raise `SocketsHttpHandler.MaxConnectionsPerServer` (default is effectively unlimited on modern .NET, but `HttpClientHandler` on .NET Framework capped at 2), enable decompression, and bound concurrency with a `SemaphoreSlim` rather than firing thousands of simultaneous requests.

## Web scraping with HttpClient

HttpClient fetches HTML; parsing it is a separate job. The standard C# pairing is [Html Agility Pack](https://webscraping.ai/blog/html-agility-pack-csharp) (or AngleSharp for CSS selectors):

```csharp
var html = await Client.GetStringAsync("https://example.com/products");

var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
foreach (var node in doc.DocumentNode.SelectNodes("//div[@class='product']//h2"))
    Console.WriteLine(node.InnerText.Trim());
```

Combined with everything above — browser-like default headers, decompression, cookies, a proxy, and retry handling — this covers static sites well, and it's enough for a lot of real work: server-rendered catalogs behind [price monitoring](https://webscraping.ai/use-cases/price-monitoring), directory pages behind [B2B lead generation](https://webscraping.ai/use-cases/b2b-lead-generation). Two problems it cannot solve:

1. **JavaScript rendering.** HttpClient receives the raw HTTP response. If products, prices, or listings are rendered client-side (React/Vue storefronts, infinite scroll), the HTML you download is an empty application shell. Rendering locally means running a headless browser like [PuppeteerSharp](https://webscraping.ai/blog/puppeteersharp-web-scraping) — a much heavier dependency.
2. **Anti-bot systems.** Cloudflare, DataDome, and friends fingerprint TLS handshakes and browser environments. No set of HttpClient headers makes a .NET TLS stack look like Chrome.

When you hit either wall, you don't have to abandon your HttpClient code — point it at a rendering API instead of the target site. [WebScraping.AI](https://webscraping.ai/) runs the page in real headless Chrome with rotating proxies and returns the final HTML through a plain HTTP endpoint:

```csharp
// Returns fully rendered HTML after JavaScript execution
var url = "https://api.webscraping.ai/html" +
          $"?api_key={apiKey}&url={Uri.EscapeDataString("https://example.com/spa-products")}";
var renderedHtml = await Client.GetStringAsync(url);

// Or extract structured fields with AI in one call
var fields = await Client.GetFromJsonAsync<Dictionary<string, string>>(
    "https://api.webscraping.ai/ai/fields" +
    $"?api_key={apiKey}&url={Uri.EscapeDataString(productUrl)}" +
    "&fields[name]=Product name&fields[price]=Price with currency");
```

Everything in this guide still applies — it's the same HttpClient, the same JSON handling, the same retry pipeline. See the [API documentation](https://webscraping.ai/docs) for the `/text`, `/selected`, and AI extraction endpoints, or the [AI web scraping overview](https://webscraping.ai/ai-web-scraping) for what the AI endpoints can pull from a page.

## Frequently asked questions

**Why am I getting SocketException / "Only one usage of each socket address" errors?** You're creating (and disposing) HttpClient instances per request. Each disposed client leaves its sockets in `TIME_WAIT` for up to four minutes. Switch to a single reused instance or `IHttpClientFactory`.

**Does HttpClient throw an exception on 404 or 500 responses?** No. `GetAsync`/`SendAsync` complete normally for any HTTP status; only `EnsureSuccessStatusCode()` (or `GetStringAsync`-style shortcuts, which call it internally) turn 4xx/5xx into `HttpRequestException`. Check `response.IsSuccessStatusCode` when an error page is a case you want to handle rather than throw on.

**Is HttpClient thread-safe?** Yes for sending: `GetAsync`, `PostAsync`, `SendAsync` and friends can be called concurrently from multiple threads on one instance — that's the intended usage. Configuration (`BaseAddress`, `Timeout`, `DefaultRequestHeaders`) is not thread-safe and must be set before the first request.

**How do I make a synchronous request?**.NET 5+ added `client.Send(request)` for the rare cases that truly can't be async (it has no shortcut overloads and buffers differently). Avoid `.Result`/`.Wait()` on the async methods — that's a deadlock recipe in UI and classic ASP.NET contexts.

**Can HttpClient scrape sites that require a login?** Yes, when the login is form- or token-based: attach a `CookieContainer`, POST the credentials, and the session cookie rides along on later requests. Logins behind JavaScript challenges or CAPTCHAs need a browser-based approach or a [rendering API](https://webscraping.ai/ai-web-scraping).

**Is web scraping with HttpClient legal?** Fetching publicly accessible pages is generally lawful, but terms of service, rate limits, and data-protection rules still apply — see our guide on [web scraping legality](https://webscraping.ai/blog/is-web-scraping-legal).