Table of contents

How do I set up HttpClient (C#) to use system proxy settings automatically?

By default, HttpClient in C# automatically uses the system proxy settings configured in the operating system. However, to ensure proper proxy usage and handle authentication scenarios, you should explicitly configure the HttpClientHandler.

Basic System Proxy Configuration

The most straightforward approach is to configure an HttpClientHandler with system proxy settings:

using System;
using System.Net.Http;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        var handler = new HttpClientHandler()
        {
            UseProxy = true,
            UseDefaultCredentials = true,
            Proxy = null // Use system proxy
        };

        using var client = new HttpClient(handler);

        try
        {
            string response = await client.GetStringAsync("https://api.example.com/data");
            Console.WriteLine($"Response received: {response.Length} characters");
        }
        catch (HttpRequestException ex)
        {
            Console.WriteLine($"Request failed: {ex.Message}");
        }
    }
}

Modern Approach with HttpClientFactory

For production applications, use HttpClientFactory with dependency injection:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using System.Net.Http;

// In Program.cs or Startup.cs
services.AddHttpClient("SystemProxyClient")
    .ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler()
    {
        UseProxy = true,
        UseDefaultCredentials = true,
        Proxy = null // Use system default
    });

// Usage in service class
public class ApiService
{
    private readonly HttpClient _httpClient;

    public ApiService(IHttpClientFactory httpClientFactory)
    {
        _httpClient = httpClientFactory.CreateClient("SystemProxyClient");
    }

    public async Task<string> GetDataAsync()
    {
        return await _httpClient.GetStringAsync("https://api.example.com/data");
    }
}

Configuration Options

Key HttpClientHandler Properties

| Property | Description | Default Value | |----------|-------------|---------------| | UseProxy | Enable/disable proxy usage | true | | UseDefaultCredentials | Use system credentials for proxy auth | false | | Proxy | Custom proxy settings (null = system default) | null | | PreAuthenticate | Send credentials with first request | false |

Advanced Configuration

var handler = new HttpClientHandler()
{
    UseProxy = true,
    UseDefaultCredentials = true,
    PreAuthenticate = true, // Send auth headers immediately
    // Proxy = WebRequest.DefaultWebProxy, // Explicit system proxy
};

// Configure additional settings
handler.ServerCertificateCustomValidationCallback = 
    HttpClientHandler.DangerousAcceptAnyServerCertificateValidator; // Only for dev!

using var client = new HttpClient(handler);

Handling Proxy Authentication

Windows Authentication

var handler = new HttpClientHandler()
{
    UseProxy = true,
    UseDefaultCredentials = true, // Uses current Windows user
    PreAuthenticate = true
};

Custom Credentials

var handler = new HttpClientHandler()
{
    UseProxy = true,
    Credentials = new NetworkCredential("username", "password", "domain")
};

Checking System Proxy Settings

You can inspect the current system proxy configuration:

using System.Net;

public static void DisplayProxyInfo()
{
    var systemProxy = WebRequest.DefaultWebProxy;
    var testUri = new Uri("https://example.com");

    if (systemProxy != null)
    {
        var proxyUri = systemProxy.GetProxy(testUri);
        bool isBypassed = systemProxy.IsBypassed(testUri);

        Console.WriteLine($"Proxy for {testUri}: {proxyUri}");
        Console.WriteLine($"Bypassed: {isBypassed}");

        // Check credentials
        var credentials = systemProxy.Credentials;
        Console.WriteLine($"Has credentials: {credentials != null}");
    }
    else
    {
        Console.WriteLine("No system proxy configured");
    }
}

Best Practices

1. Use HttpClientFactory in Production

// Register in DI container
services.AddHttpClient<ApiService>(client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
    client.Timeout = TimeSpan.FromSeconds(30);
})
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler()
{
    UseProxy = true,
    UseDefaultCredentials = true
});

2. Handle Proxy Failures Gracefully

public async Task<string> GetWithProxyFallback(string url)
{
    try
    {
        // Try with system proxy
        using var clientWithProxy = CreateClientWithSystemProxy();
        return await clientWithProxy.GetStringAsync(url);
    }
    catch (HttpRequestException ex) when (ex.Message.Contains("proxy"))
    {
        // Fallback to direct connection
        using var directClient = new HttpClient();
        return await directClient.GetStringAsync(url);
    }
}

private HttpClient CreateClientWithSystemProxy()
{
    var handler = new HttpClientHandler()
    {
        UseProxy = true,
        UseDefaultCredentials = true
    };

    return new HttpClient(handler);
}

3. Configuration-Based Proxy Settings

public class ProxyConfiguration
{
    public bool UseSystemProxy { get; set; } = true;
    public bool UseDefaultCredentials { get; set; } = true;
    public string? ProxyAddress { get; set; }
    public string? Username { get; set; }
    public string? Password { get; set; }
}

public HttpClient CreateConfiguredClient(ProxyConfiguration config)
{
    var handler = new HttpClientHandler();

    if (config.UseSystemProxy)
    {
        handler.UseProxy = true;
        handler.UseDefaultCredentials = config.UseDefaultCredentials;

        if (!string.IsNullOrEmpty(config.ProxyAddress))
        {
            handler.Proxy = new WebProxy(config.ProxyAddress);

            if (!string.IsNullOrEmpty(config.Username))
            {
                handler.Proxy.Credentials = new NetworkCredential(
                    config.Username, config.Password);
            }
        }
    }
    else
    {
        handler.UseProxy = false;
    }

    return new HttpClient(handler);
}

Troubleshooting Common Issues

Issue: Proxy Authentication Fails

Solution: Ensure UseDefaultCredentials = true and verify system credentials have proxy access.

Issue: Corporate Firewall Blocks Requests

Solution: Configure proxy bypass for specific domains:

var proxy = new WebProxy("http://proxy.company.com:8080")
{
    BypassProxyOnLocal = true,
    BypassList = new[] { "*.internal.com", "localhost" }
};

var handler = new HttpClientHandler()
{
    Proxy = proxy,
    UseProxy = true
};

Issue: SSL Certificate Errors Through Proxy

Solution: Handle certificate validation (use carefully):

var handler = new HttpClientHandler()
{
    UseProxy = true,
    UseDefaultCredentials = true,
    ServerCertificateCustomValidationCallback = (message, cert, chain, errors) =>
    {
        // Implement custom validation logic
        return errors == SslPolicyErrors.None;
    }
};

Summary

To set up HttpClient with system proxy settings:

  1. Use HttpClientHandler with UseProxy = true
  2. Enable default credentials with UseDefaultCredentials = true
  3. Leave Proxy = null to use system settings automatically
  4. Use HttpClientFactory for production applications
  5. Handle authentication scenarios appropriately
  6. Implement proper error handling for proxy failures

This configuration ensures your HttpClient respects the system's proxy configuration while providing flexibility for authentication and error handling.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon