Table of contents

How do I send custom headers with every request using HttpClient (C#)?

There are several ways to send custom headers with every HttpClient request in C#. The most common approach is using the DefaultRequestHeaders property, which automatically adds headers to all requests made by that HttpClient instance.

Method 1: Using DefaultRequestHeaders (Recommended)

The DefaultRequestHeaders property is the preferred way to add headers that should be sent with every request:

using System;
using System.Net.Http;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        using var httpClient = new HttpClient();

        // Add custom headers that will be sent with every request
        httpClient.DefaultRequestHeaders.Add("API-Key", "your-api-key-here");
        httpClient.DefaultRequestHeaders.Add("Custom-Client", "MyApp/1.0");

        // Set User-Agent (common requirement)
        httpClient.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0 (Windows NT 10.0; Win64; x64)");

        // Set Accept header for JSON responses
        httpClient.DefaultRequestHeaders.Accept.Clear();
        httpClient.DefaultRequestHeaders.Accept.Add(
            new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));

        // Add authorization header (Bearer token example)
        httpClient.DefaultRequestHeaders.Authorization = 
            new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", "your-token-here");

        // Now all requests will include these headers
        var response1 = await httpClient.GetAsync("https://api.example.com/users");
        var response2 = await httpClient.GetAsync("https://api.example.com/orders");

        // Both requests above will include all the default headers
    }
}

Method 2: Using HttpClientFactory with Named Clients

For production applications, use HttpClientFactory to configure headers:

// In Startup.cs or Program.cs
services.AddHttpClient("ApiClient", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
    client.DefaultRequestHeaders.Add("API-Key", "your-api-key");
    client.DefaultRequestHeaders.Add("User-Agent", "MyApp/1.0");
    client.Timeout = TimeSpan.FromSeconds(30);
});

// In your service class
public class ApiService
{
    private readonly HttpClient _httpClient;

    public ApiService(IHttpClientFactory httpClientFactory)
    {
        _httpClient = httpClientFactory.CreateClient("ApiClient");
    }

    public async Task<string> GetDataAsync()
    {
        // Headers are automatically included
        var response = await _httpClient.GetAsync("data");
        return await response.Content.ReadAsStringAsync();
    }
}

Method 3: Headers with Multiple Values

Some headers can have multiple values. Here's how to handle them:

using var httpClient = new HttpClient();

// Adding multiple Accept-Language values
httpClient.DefaultRequestHeaders.AcceptLanguage.Clear();
httpClient.DefaultRequestHeaders.AcceptLanguage.Add(
    new System.Net.Http.Headers.StringWithQualityHeaderValue("en-US", 0.9));
httpClient.DefaultRequestHeaders.AcceptLanguage.Add(
    new System.Net.Http.Headers.StringWithQualityHeaderValue("en", 0.8));

// Adding multiple Accept-Encoding values
httpClient.DefaultRequestHeaders.AcceptEncoding.Add(
    new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
httpClient.DefaultRequestHeaders.AcceptEncoding.Add(
    new System.Net.Http.Headers.StringWithQualityHeaderValue("deflate"));

Method 4: Handling Restricted Headers

Some headers are restricted and cannot be set via DefaultRequestHeaders. For these, use HttpRequestMessage:

using var httpClient = new HttpClient();

// Set default headers as usual
httpClient.DefaultRequestHeaders.Add("API-Key", "your-key");

// For restricted headers, use HttpRequestMessage
using var request = new HttpRequestMessage(HttpMethod.Get, "https://api.example.com/data");

// These headers must be set on individual requests
request.Headers.Add("Host", "api.example.com");
request.Headers.Add("Connection", "keep-alive");

var response = await httpClient.SendAsync(request);

Common Restricted Headers

The following headers cannot be set using DefaultRequestHeaders and must be set on individual HttpRequestMessage objects:

  • Connection
  • Content-Length
  • Date
  • Expect
  • Host
  • If-Modified-Since
  • Range
  • Referer
  • Transfer-Encoding
  • Upgrade

Best Practices

1. Reuse HttpClient Instances

// ✅ Good: Reuse HttpClient
public class ApiService
{
    private static readonly HttpClient _httpClient = new HttpClient();

    static ApiService()
    {
        _httpClient.DefaultRequestHeaders.Add("User-Agent", "MyApp/1.0");
    }
}

// ❌ Bad: Creating new HttpClient for each request
public async Task<string> GetDataAsync()
{
    using var client = new HttpClient(); // Don't do this!
    return await client.GetStringAsync("https://api.example.com/data");
}

2. Use HttpClientFactory in ASP.NET Core

// In Program.cs
builder.Services.AddHttpClient<ApiService>(client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
    client.DefaultRequestHeaders.Add("User-Agent", "MyApp/1.0");
});

3. Handle Headers Safely

// Check if header already exists before adding
if (!httpClient.DefaultRequestHeaders.Contains("Custom-Header"))
{
    httpClient.DefaultRequestHeaders.Add("Custom-Header", "value");
}

// Or remove and add to update
httpClient.DefaultRequestHeaders.Remove("Custom-Header");
httpClient.DefaultRequestHeaders.Add("Custom-Header", "new-value");

Troubleshooting Common Issues

Issue: InvalidOperationException when adding headers Solution: Some headers have specific properties. Use them instead of the generic Add method:

// ❌ Wrong
httpClient.DefaultRequestHeaders.Add("User-Agent", "MyApp/1.0");

// ✅ Correct
httpClient.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0");

Issue: Headers not appearing in requests Solution: Ensure you're not creating new HttpClient instances and verify the header name is correct.

By using these methods, you can ensure that custom headers are consistently sent with all HttpClient requests in your C# applications.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon