Can I use urllib3 with a SOCKS proxy?

Yes, you can use urllib3 with a SOCKS proxy by installing the PySocks library. While urllib3 doesn't provide native SOCKS support, it includes a SOCKSProxyManager in its contrib module that enables SOCKS proxy functionality.

Installation

Install the required packages:

pip install urllib3[socks]
# Or install separately:
# pip install urllib3 pysocks

Basic SOCKS Proxy Usage

Method 1: Using SOCKSProxyManager (Recommended)

from urllib3.contrib.socks import SOCKSProxyManager

# SOCKS5 proxy
http = SOCKSProxyManager("socks5://localhost:9050/")

# SOCKS4 proxy
http = SOCKSProxyManager("socks4://localhost:1080/")

# Make requests through the proxy
response = http.request('GET', 'http://httpbin.org/ip')
print(response.data.decode())

Method 2: Global Socket Patching

import socks
import socket
from urllib3 import PoolManager

# Configure global SOCKS proxy
socks.set_default_proxy(socks.SOCKS5, "localhost", 9050)
socket.socket = socks.socksocket

# Use regular PoolManager - all connections will use SOCKS
http = PoolManager()
response = http.request('GET', 'http://httpbin.org/ip')
print(response.data.decode())

SOCKS Proxy with Authentication

from urllib3.contrib.socks import SOCKSProxyManager

# SOCKS5 with username/password authentication
http = SOCKSProxyManager("socks5://username:password@proxy.example.com:1080/")

response = http.request('GET', 'http://httpbin.org/ip')
print(response.data.decode())

HTTPS Support

For HTTPS requests through SOCKS proxy:

from urllib3.contrib.socks import SOCKSProxyManager
import urllib3

# Disable SSL warnings if needed (not recommended for production)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

# SOCKS proxy with SSL certificate verification
http = SOCKSProxyManager(
    "socks5://localhost:9050/",
    cert_reqs='CERT_REQUIRED',
    ca_certs='/path/to/ca-certificates.crt'
)

# For testing, you can disable SSL verification
http = SOCKSProxyManager(
    "socks5://localhost:9050/",
    cert_reqs='CERT_NONE'
)

response = http.request('GET', 'https://httpbin.org/ip')
print(response.data.decode())

Error Handling

from urllib3.contrib.socks import SOCKSProxyManager
from urllib3.exceptions import ProxyError, HTTPError
import socks

try:
    http = SOCKSProxyManager("socks5://localhost:9050/")
    response = http.request('GET', 'http://httpbin.org/ip', timeout=10)
    print(response.data.decode())

except socks.ProxyConnectionError:
    print("Failed to connect to SOCKS proxy")
except ProxyError as e:
    print(f"Proxy error: {e}")
except HTTPError as e:
    print(f"HTTP error: {e}")

Different SOCKS Versions

from urllib3.contrib.socks import SOCKSProxyManager

# SOCKS4
socks4_proxy = SOCKSProxyManager("socks4://localhost:1080/")

# SOCKS4A (supports domain names)
socks4a_proxy = SOCKSProxyManager("socks4a://localhost:1080/")

# SOCKS5 (most common, supports authentication)
socks5_proxy = SOCKSProxyManager("socks5://localhost:1080/")

Testing Your SOCKS Proxy

from urllib3.contrib.socks import SOCKSProxyManager
import json

def test_socks_proxy(proxy_url):
    try:
        http = SOCKSProxyManager(proxy_url)

        # Test with a service that returns your IP
        response = http.request('GET', 'http://httpbin.org/ip')
        ip_data = json.loads(response.data.decode())
        print(f"Your IP through proxy: {ip_data['origin']}")

        # Test HTTPS
        response = http.request('GET', 'https://httpbin.org/ip')
        ip_data = json.loads(response.data.decode())
        print(f"Your HTTPS IP through proxy: {ip_data['origin']}")

        return True
    except Exception as e:
        print(f"Proxy test failed: {e}")
        return False

# Test your SOCKS proxy
test_socks_proxy("socks5://localhost:9050/")

Important Notes

• SOCKSProxyManager is part of urllib3.contrib, indicating it's a contributed module
• SOCKS support requires the PySocks library to be installed
• SOCKS5 is recommended over SOCKS4 for better security and feature support
• Always handle proxy connection errors gracefully in production code
• Consider connection pooling and timeout settings for better performance

Common Use Cases

SOCKS proxies are commonly used with: - Tor network: SOCKS5 proxy typically runs on localhost:9050 - SSH tunneling: Create SOCKS proxy with ssh -D 1080 user@server - VPN services: Many VPN providers offer SOCKS proxy endpoints - Corporate networks: For accessing external resources through corporate proxies

Always ensure compliance with applicable laws and terms of service when using proxies for web scraping.