C# HttpClient: The Complete Guide with Examples

HttpClient is the standard way to make HTTP requests in C# and .NET — REST calls, file downloads, form submissions, and web scraping all go through it. It's also famously easy to misuse: dispose it per request and you exhaust sockets; make it a naive singleton and it ignores DNS changes. This guide is a complete, example-first reference for HttpClient in modern .NET: every common request type, headers, authentication, JSON, timeouts, proxies, cookies, file transfers, and the IHttpClientFactory patterns that make it production-safe.

Key Takeaways

• Reuse one HttpClient (or use IHttpClientFactory) — creating a client per request causes socket exhaustion
• GetAsync/PostAsync cover simple calls; build an HttpRequestMessage and use SendAsync when a single request needs its own headers
• Set default headers once on DefaultRequestHeaders; set body-related headers on the HttpContent itself
• The default timeout is 100 seconds; pass a CancellationToken for per-request control
• Proxy, cookies, redirects, decompression, and SSL validation are all configured on HttpClientHandler, not on HttpClient
• HttpClient can't execute JavaScript — for dynamic pages, pair it with a rendering API

What is HttpClient?

HttpClient (in System.Net.Http) is .NET's built-in HTTP API. One instance represents a reusable session: it holds default headers, a base address, a timeout, and — through its handler — a connection pool, cookies, and proxy settings.

It replaced two older APIs you'll still meet in legacy code:

The difference between HttpClient and WebClient matters mostly in one direction: new code should always use HttpClient (or the higher-level abstractions built on it). Microsoft's own guidance is explicit about this.

Creating and reusing an instance

The single most important HttpClient rule: it's designed to be instantiated once and reused, not created per request. HttpClient is thread-safe for concurrent requests, so sharing one instance is safe. Each new instance opens its own connection pool, and disposed instances leave sockets in TIME_WAIT — under load, a new HttpClient() per call exhausts ports and starts throwing SocketException:

// ❌ Anti-pattern: socket exhaustion under load
using var client = new HttpClient();
var html = await client.GetStringAsync("https://example.com");

// ✅ Reuse a single static instance
private static readonly HttpClient Client = new()
{
    Timeout = TimeSpan.FromSeconds(30)
};

The static singleton has one caveat: it caches DNS resolution for the life of each connection. If your target's IP can change (load balancers, blue-green deploys), set a connection lifetime so the pool recycles connections periodically:

private static readonly HttpClient Client = new(new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(2)
});

In ASP.NET Core apps, skip the manual singleton and use IHttpClientFactory — covered at the end of this guide — which handles pooling, DNS rotation, and named configuration for you.

GET requests

The three shortcuts, in increasing order of control:

// String — simplest, throws on network errors but NOT on 404/500
string html = await Client.GetStringAsync("https://example.com/products");

// Bytes and streams for binary content
byte[] bytes = await Client.GetByteArrayAsync("https://example.com/logo.png");
await using Stream stream = await Client.GetStreamAsync("https://example.com/feed.xml");

// Full response — inspect status, headers, and body separately
HttpResponseMessage response = await Client.GetAsync("https://example.com/products");
response.EnsureSuccessStatusCode(); // throws HttpRequestException on 4xx/5xx
string body = await response.Content.ReadAsStringAsync();

To read the HTTP status code, check response.StatusCode (an HttpStatusCode enum) or response.IsSuccessStatusCode before touching the body:

var response = await Client.GetAsync(url);
Console.WriteLine((int)response.StatusCode);        // 200
Console.WriteLine(response.StatusCode);             // OK
if (!response.IsSuccessStatusCode)
{
    Console.WriteLine($"Request failed: {(int)response.StatusCode} {response.ReasonPhrase}");
}

Query strings are your job — build them safely with Uri.EscapeDataString rather than string concatenation:

var query = $"q={Uri.EscapeDataString("web scraping api")}&page=2";
var response = await Client.GetAsync($"https://example.com/search?{query}");

POST, PUT, PATCH, and DELETE

Each method takes an HttpContent describing the body:

// POST JSON (System.Net.Http.Json — built into .NET since 5)
var response = await Client.PostAsJsonAsync("https://api.example.com/search",
    new { query = "laptops", limit = 20 });

// POST a form (application/x-www-form-urlencoded)
var form = new FormUrlEncodedContent(new Dictionary<string, string>
{
    ["username"] = "demo",
    ["password"] = "secret"
});
var login = await Client.PostAsync("https://example.com/login", form);

// POST raw JSON with explicit content type
var content = new StringContent("""{"query":"laptops"}""",
    Encoding.UTF8, "application/json");
await Client.PostAsync("https://api.example.com/search", content);

// PUT, PATCH, DELETE
await Client.PutAsJsonAsync("https://api.example.com/products/42",
    new { name = "Updated", price = 49.99 });
await Client.PatchAsync("https://api.example.com/products/42",
    new StringContent("""{"price":39.99}""", Encoding.UTF8, "application/json"));
await Client.DeleteAsync("https://api.example.com/products/42");

Note that Content-Type belongs to the content: set it via the StringContent constructor (or content.Headers.ContentType), not on the client's default headers — HttpClient rejects content headers set at request level with an InvalidOperationException.

Setting request headers

Headers split into two scopes. Default headers apply to every request the client sends; per-request headers live on an HttpRequestMessage:

// Once, at startup — sent with every request
Client.DefaultRequestHeaders.UserAgent.ParseAdd(
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36");
Client.DefaultRequestHeaders.Accept.ParseAdd("text/html,application/xhtml+xml");
Client.DefaultRequestHeaders.AcceptLanguage.ParseAdd("en-US,en;q=0.9");
Client.DefaultRequestHeaders.Add("X-API-Key", apiKey);

For one-off headers, build the request explicitly — this is exactly what HttpRequestMessage is for:

var request = new HttpRequestMessage(HttpMethod.Get, "https://example.com/products");
request.Headers.Add("Referer", "https://www.google.com/");
request.Headers.IfModifiedSince = DateTimeOffset.UtcNow.AddHours(-1);

var response = await Client.SendAsync(request);

GetAsync vs SendAsync is a convenience-vs-control tradeoff: GetAsync(url) is literally SendAsync(new HttpRequestMessage(HttpMethod.Get, url)) internally. Use the shortcuts until a request needs its own method, headers, version, or options — then use SendAsync. One rule to remember: an HttpRequestMessage can be sent exactly once; build a new one for each attempt (this bites people writing retry loops).

Authentication

The Authorization header has a typed API — AuthenticationHeaderValue — with a scheme and a parameter:

using System.Net.Http.Headers;

// Basic auth: base64-encode user:password yourself
var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes("apiuser:s3cret"));
Client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Basic", credentials);

// Bearer token (OAuth 2.0, JWTs)
Client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue("Bearer", accessToken);

// API key header — no scheme, just a custom header
Client.DefaultRequestHeaders.Add("X-API-Key", apiKey);

Per-request auth works the same way on request.Headers.Authorization. For OAuth flows, the token exchange itself is a plain POST — fetch the token with PostAsync, then attach it as a bearer.

Basic auth can also be configured on the handler with a NetworkCredential, which additionally answers server challenges (401 + WWW-Authenticate) and supports NTLM/Negotiate:

var handler = new HttpClientHandler
{
    Credentials = new NetworkCredential("apiuser", "s3cret")
};
var client = new HttpClient(handler);

Working with JSON

System.Net.Http.Json extensions handle serialization with System.Text.Json under the hood:

public record Product(int Id, string Name, decimal Price);

// Deserialize a response directly
Product? product = await Client.GetFromJsonAsync<Product>(
    "https://api.example.com/products/42");

List<Product>? products = await Client.GetFromJsonAsync<List<Product>>(
    "https://api.example.com/products");

// From an HttpResponseMessage
var response = await Client.GetAsync("https://api.example.com/products/42");
var fromResponse = await response.Content.ReadFromJsonAsync<Product>();

// Case-insensitive matching and other options
var options = new JsonSerializerOptions { PropertyNameCaseInsensitive = true };
var data = await response.Content.ReadFromJsonAsync<Product>(options);

For large responses, deserialize from the stream instead of buffering the whole body into a string first — GetFromJsonAsync already does this internally.

Timeouts and cancellation

HttpClient's default timeout is 100 seconds, which is almost never what you want. Timeout is set once per client (changing it after the first request throws); per-request timeouts use a CancellationTokenSource:

// Client-wide
var client = new HttpClient { Timeout = TimeSpan.FromSeconds(30) };

// Per request
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
try
{
    var response = await Client.GetAsync("https://slow.example.com", cts.Token);
}
catch (TaskCanceledException ex) when (ex.InnerException is TimeoutException)
{
    // .NET 5+: distinguishes timeout from explicit cancellation
    Console.WriteLine("Request timed out");
}

On timeout, HttpClient throws TaskCanceledException (wrapping a TimeoutException since .NET 5) — catch it separately from HttpRequestException (network/DNS/TLS failures) when you want distinct handling:

try
{
    var html = await Client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
    Console.WriteLine($"Network or HTTP error: {ex.StatusCode} {ex.Message}");
}
catch (TaskCanceledException)
{
    Console.WriteLine("Timed out or canceled");
}

HttpClientHandler: SSL, redirects, cookies, compression

Connection-level behavior is configured on the handler passed to the constructor. You can't change a handler after the first request — configure everything up front.

Ignoring SSL certificate errors

For local development against self-signed certificates, override certificate validation. Never ship this to production — it disables the protection TLS exists for; trust the specific dev certificate instead where possible:

var handler = new HttpClientHandler
{
    // Accepts ANY certificate — development only
    ServerCertificateCustomValidationCallback =
        HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
var client = new HttpClient(handler);

The callback also lets you pin a specific certificate thumbprint or log validation errors instead of blanket-accepting. Client certificates (mutual TLS) attach via handler.ClientCertificates.Add(...).

Redirects

Redirects are followed automatically (up to 50). Disable that when you need to observe the Location header yourself — for example, to log where scraped URLs actually lead:

var handler = new HttpClientHandler { AllowAutoRedirect = false };
var client = new HttpClient(handler);

var response = await client.GetAsync("http://example.com");
Console.WriteLine((int)response.StatusCode);              // 301
Console.WriteLine(response.Headers.Location);             // https://example.com/

One security-driven gotcha: on a redirect from HTTPS to HTTP, or to a different host, .NET strips the Authorization header — re-authenticate on the new location explicitly if you need to.

Cookies and sessions

CookieContainer gives you session persistence across requests — log in once, and subsequent requests carry the session cookie automatically:

var cookies = new CookieContainer();
var handler = new HttpClientHandler { CookieContainer = cookies };
var client = new HttpClient(handler);

// Login stores the session cookie in the container
await client.PostAsync("https://example.com/login", new FormUrlEncodedContent(
    new Dictionary<string, string> { ["username"] = "demo", ["password"] = "secret" }));

// Sent with the session cookie automatically
var orders = await client.GetStringAsync("https://example.com/account/orders");

// Inspect or seed cookies manually
foreach (Cookie c in cookies.GetCookies(new Uri("https://example.com")))
    Console.WriteLine($"{c.Name}={c.Value}");
cookies.Add(new Uri("https://example.com"), new Cookie("currency", "USD"));

Automatic decompression

Browsers always advertise gzip/brotli support; HttpClient doesn't unless you tell it to. Enabling AutomaticDecompression sets the Accept-Encoding header and transparently decompresses responses:

var handler = new HttpClientHandler
{
    AutomaticDecompression = DecompressionMethods.All // gzip, deflate, brotli
};

This is worth enabling for scraping unconditionally: smaller transfers, and some servers treat clients without Accept-Encoding as suspicious.

Using a proxy

Proxies are configured with WebProxy on the handler — with or without credentials:

var handler = new HttpClientHandler
{
    Proxy = new WebProxy("http://proxy.example.com:8080")
    {
        Credentials = new NetworkCredential("proxyuser", "proxypass")
    },
    UseProxy = true
};
var client = new HttpClient(handler);

By default (UseProxy = true, Proxy = null), HttpClient uses the system proxy — on Windows the WinINet/WinHTTP settings, elsewhere the HTTP_PROXY/HTTPS_PROXY/NO_PROXY environment variables. That means you can route an unmodified app through a proxy just by setting environment variables before launch. Set UseProxy = false to bypass proxies entirely, which also skips proxy auto-detection — a common fix for slow first requests on Windows.

For scraping at scale, a single static proxy just moves your bot signature to a different IP. Rotating proxy pools are the standard answer — see our comparison of proxy providers for web scraping and the types of proxies worth paying for.

Downloading files and streaming

For small files, GetByteArrayAsync plus File.WriteAllBytesAsync is fine. For anything big, stream — HttpCompletionOption.ResponseHeadersRead makes HttpClient return as soon as headers arrive instead of buffering the entire body in memory:

using var response = await Client.GetAsync(
    "https://example.com/dataset.zip",
    HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();

await using var source = await response.Content.ReadAsStreamAsync();
await using var target = File.Create("dataset.zip");
await source.CopyToAsync(target);

Download progress is a manual loop over the stream — read chunks, count bytes, report against Content-Length:

var total = response.Content.Headers.ContentLength;
var buffer = new byte[81920];
long readSoFar = 0;
int read;
while ((read = await source.ReadAsync(buffer)) > 0)
{
    await target.WriteAsync(buffer.AsMemory(0, read));
    readSoFar += read;
    if (total.HasValue)
        Console.Write($"\r{readSoFar * 100 / total.Value}%");
}

Uploading files: multipart form data

MultipartFormDataContent builds the multipart/form-data bodies that HTML file-upload forms send — mix text fields and files freely:

using var form = new MultipartFormDataContent();
form.Add(new StringContent("Q3 Report"), "title");

var file = new StreamContent(File.OpenRead("report.pdf"));
file.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
form.Add(file, "document", "report.pdf"); // field name, file name

var response = await Client.PostAsync("https://example.com/upload", form);

Retries and resilience

Transient failures — timeouts, 429 rate limits, 5xx responses — deserve retries with backoff, and that logic belongs in a message handler rather than around every call site. In modern .NET, Microsoft.Extensions.Http.Resilience packages the standard pipeline (retry + circuit breaker + per-attempt timeout) in one line:

builder.Services.AddHttpClient("scraper", c =>
{
    c.Timeout = TimeSpan.FromSeconds(60);
})
.AddStandardResilienceHandler(); // retries, circuit breaker, timeouts

Rolling it by hand (console apps, no DI) is a loop with exponential backoff — remembering that HttpRequestMessage can't be reused across attempts:

async Task<HttpResponseMessage> GetWithRetryAsync(string url, int attempts = 3)
{
    for (var i = 1; ; i++)
    {
        try
        {
            var response = await Client.GetAsync(url);
            if ((int)response.StatusCode < 500 && response.StatusCode != HttpStatusCode.TooManyRequests)
                return response;
            if (i == attempts) return response;
        }
        catch (HttpRequestException) when (i < attempts) { }
        await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, i))); // 2s, 4s, 8s
    }
}

Respect Retry-After on 429s when it's present — hammering a rate limiter earns IP bans, not data.

IHttpClientFactory and best practices

In any app with dependency injection, IHttpClientFactory (from Microsoft.Extensions.Http) is the recommended way to get clients. It pools the underlying handlers (fixing both socket exhaustion and the stale-DNS problem), centralizes configuration in named or typed clients, and composes with resilience handlers:

// Program.cs
builder.Services.AddHttpClient<CatalogClient>(c =>
{
    c.BaseAddress = new Uri("https://api.example.com/");
    c.Timeout = TimeSpan.FromSeconds(30);
    c.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0");
});

// Typed client — inject anywhere
public class CatalogClient(HttpClient http)
{
    public Task<List<Product>?> GetProductsAsync() =>
        http.GetFromJsonAsync<List<Product>>("products");
}

The condensed best-practice list:

• One client per logical destination, reused for the app's lifetime — via factory or static instance
• Don't subclass HttpClient; configure instances or write DelegatingHandler middleware instead
• SocketsHttpHandler.PooledConnectionLifetime (or the factory's handler rotation) if you run a long-lived singleton
• Set a real timeout — 100 seconds hides problems
• EnsureSuccessStatusCode or explicit status checks — HttpClient does not throw on 404s by itself
• Dispose HttpResponseMessage when streaming large bodies; it returns the connection to the pool

For high-load scraping specifically: raise SocketsHttpHandler.MaxConnectionsPerServer (default is effectively unlimited on modern .NET, but HttpClientHandler on .NET Framework capped at 2), enable decompression, and bound concurrency with a SemaphoreSlim rather than firing thousands of simultaneous requests.

Web scraping with HttpClient

HttpClient fetches HTML; parsing it is a separate job. The standard C# pairing is Html Agility Pack (or AngleSharp for CSS selectors):

var html = await Client.GetStringAsync("https://example.com/products");

var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
foreach (var node in doc.DocumentNode.SelectNodes("//div[@class='product']//h2"))
    Console.WriteLine(node.InnerText.Trim());

Combined with everything above — browser-like default headers, decompression, cookies, a proxy, and retry handling — this covers static sites well. Two problems it cannot solve:

1. JavaScript rendering. HttpClient receives the raw HTTP response. If products, prices, or listings are rendered client-side (React/Vue storefronts, infinite scroll), the HTML you download is an empty application shell. Rendering locally means running a headless browser like PuppeteerSharp — a much heavier dependency.
2. Anti-bot systems. Cloudflare, DataDome, and friends fingerprint TLS handshakes and browser environments. No set of HttpClient headers makes a .NET TLS stack look like Chrome.

When you hit either wall, you don't have to abandon your HttpClient code — point it at a rendering API instead of the target site. WebScraping.AI runs the page in real headless Chrome with rotating proxies and returns the final HTML through a plain HTTP endpoint:

// Returns fully rendered HTML after JavaScript execution
var url = "https://api.webscraping.ai/html" +
          $"?api_key={apiKey}&url={Uri.EscapeDataString("https://example.com/spa-products")}";
var renderedHtml = await Client.GetStringAsync(url);

// Or extract structured fields with AI in one call
var fields = await Client.GetFromJsonAsync<Dictionary<string, string>>(
    "https://api.webscraping.ai/ai/fields" +
    $"?api_key={apiKey}&url={Uri.EscapeDataString(productUrl)}" +
    "&fields[name]=Product name&fields[price]=Price with currency");

Everything in this guide still applies — it's the same HttpClient, the same JSON handling, the same retry pipeline. See the API documentation for the /text, /selected, and AI extraction endpoints, or the AI web scraping overview for what the AI endpoints can pull from a page.

Frequently asked questions

Why am I getting SocketException / "Only one usage of each socket address" errors? You're creating (and disposing) HttpClient instances per request. Each disposed client leaves its sockets in TIME_WAIT for up to four minutes. Switch to a single reused instance or IHttpClientFactory.

Does HttpClient throw an exception on 404 or 500 responses? No. GetAsync/SendAsync complete normally for any HTTP status; only EnsureSuccessStatusCode() (or GetStringAsync-style shortcuts, which call it internally) turn 4xx/5xx into HttpRequestException. Check response.IsSuccessStatusCode when an error page is a case you want to handle rather than throw on.

Is HttpClient thread-safe? Yes for sending: GetAsync, PostAsync, SendAsync and friends can be called concurrently from multiple threads on one instance — that's the intended usage. Configuration (BaseAddress, Timeout, DefaultRequestHeaders) is not thread-safe and must be set before the first request.

How do I make a synchronous request? .NET 5+ added client.Send(request) for the rare cases that truly can't be async (it has no shortcut overloads and buffers differently). Avoid .Result/.Wait() on the async methods — that's a deadlock recipe in UI and classic ASP.NET contexts.

Can HttpClient scrape sites that require a login? Yes, when the login is form- or token-based: attach a CookieContainer, POST the credentials, and the session cookie rides along on later requests. Logins behind JavaScript challenges or CAPTCHAs need a browser-based approach or a rendering API.

Is web scraping with HttpClient legal? Fetching publicly accessible pages is generally lawful, but terms of service, rate limits, and data-protection rules still apply — see our guide on web scraping legality.