Managing SSL certificates properly in urllib3
is crucial for secure HTTPS communications and preventing man-in-the-middle attacks. This guide covers all aspects of SSL certificate handling in urllib3, from basic verification to advanced configurations.
Default SSL Certificate Verification
By default, urllib3
automatically verifies SSL certificates against the system's trusted Certificate Authority (CA) bundle:
import urllib3
# Default configuration with SSL verification enabled
http = urllib3.PoolManager()
response = http.request('GET', 'https://httpbin.org/get')
print(f"Status: {response.status}")
This provides secure connections out of the box and should be used in production environments.
SSL Certificate Verification Options
1. Default Verification (Recommended)
import urllib3
# Explicitly enable SSL verification (default behavior)
http = urllib3.PoolManager(
cert_reqs='CERT_REQUIRED', # Require valid certificate
ca_certs=None, # Use system CA bundle
assert_hostname=True # Verify hostname matches certificate
)
response = http.request('GET', 'https://httpbin.org/get')
2. Disabling SSL Verification (Not Recommended)
For development or testing with self-signed certificates:
import urllib3
# Disable SSL warnings
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# Disable SSL verification completely
http = urllib3.PoolManager(
cert_reqs='CERT_NONE',
assert_hostname=False
)
response = http.request('GET', 'https://self-signed.badssl.com/')
⚠️ Security Warning: Only use this in controlled environments. Never disable SSL verification in production.
Custom CA Certificates
Using a Custom CA Bundle
When working with private CAs or specific certificate authorities:
import urllib3
# Specify custom CA certificate file
http = urllib3.PoolManager(
ca_certs='/path/to/custom-ca-bundle.pem',
cert_reqs='CERT_REQUIRED'
)
response = http.request('GET', 'https://internal-api.company.com/')
Using System CA Bundle with Additional Certificates
import urllib3
import ssl
import certifi
# Get system CA bundle path
ca_bundle = certifi.where()
# Create SSL context with custom CA
ssl_context = ssl.create_default_context(cafile=ca_bundle)
ssl_context.load_verify_locations('/path/to/additional-ca.pem')
http = urllib3.PoolManager(
ssl_context=ssl_context
)
response = http.request('GET', 'https://example.com/')
Client Certificate Authentication
For mutual TLS authentication where the server requires client certificates:
Basic Client Certificate Setup
import urllib3
# Using separate certificate and key files
http = urllib3.PoolManager(
cert_file='/path/to/client-cert.pem',
key_file='/path/to/client-key.pem',
cert_reqs='CERT_REQUIRED'
)
response = http.request('GET', 'https://client-cert-required.example.com/')
Combined Certificate and Key File
import urllib3
# Using a single file containing both certificate and key
http = urllib3.PoolManager(
cert_file='/path/to/client-cert-and-key.pem',
cert_reqs='CERT_REQUIRED'
)
response = http.request('GET', 'https://api.example.com/')
Password-Protected Key Files
import urllib3
import ssl
# Create SSL context for password-protected keys
ssl_context = ssl.create_default_context()
ssl_context.load_cert_chain(
'/path/to/client-cert.pem',
'/path/to/encrypted-key.pem',
password='your-key-password'
)
http = urllib3.PoolManager(ssl_context=ssl_context)
response = http.request('GET', 'https://secure-api.example.com/')
Advanced SSL Configuration
Custom SSL Context
For fine-grained SSL control:
import urllib3
import ssl
# Create custom SSL context
ssl_context = ssl.create_default_context()
# Configure specific SSL/TLS versions
ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
ssl_context.maximum_version = ssl.TLSVersion.TLSv1_3
# Configure cipher suites (example)
ssl_context.set_ciphers('ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20:!aNULL:!MD5:!DSS')
http = urllib3.PoolManager(ssl_context=ssl_context)
response = http.request('GET', 'https://example.com/')
Hostname Verification Control
import urllib3
# Disable hostname verification while keeping certificate verification
http = urllib3.PoolManager(
cert_reqs='CERT_REQUIRED',
assert_hostname=False
)
# Useful when connecting via IP address to a server with domain-based certificate
response = http.request('GET', 'https://192.168.1.100/')
Error Handling and Debugging
Common SSL Errors and Solutions
import urllib3
from urllib3.exceptions import SSLError, MaxRetryError
def make_secure_request(url):
http = urllib3.PoolManager()
try:
response = http.request('GET', url)
return response
except SSLError as e:
print(f"SSL Error: {e}")
# Handle certificate verification failures
return None
except MaxRetryError as e:
print(f"Connection Error: {e}")
return None
# Test with various URLs
response = make_secure_request('https://expired.badssl.com/') # Expired certificate
response = make_secure_request('https://self-signed.badssl.com/') # Self-signed
response = make_secure_request('https://wrong.host.badssl.com/') # Wrong hostname
SSL Certificate Information
import urllib3
import ssl
import socket
def get_cert_info(hostname, port=443):
"""Get SSL certificate information for debugging"""
try:
# Get certificate info
cert = ssl.get_server_certificate((hostname, port))
cert_der = ssl.PEM_cert_to_DER_cert(cert)
cert_decoded = ssl.DER_cert_to_PEM_cert(cert_der)
# Parse certificate details
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
cert_info = ssock.getpeercert()
print(f"Subject: {cert_info['subject']}")
print(f"Issuer: {cert_info['issuer']}")
print(f"Version: {cert_info['version']}")
print(f"Serial Number: {cert_info['serialNumber']}")
print(f"Not Before: {cert_info['notBefore']}")
print(f"Not After: {cert_info['notAfter']}")
except Exception as e:
print(f"Error getting certificate info: {e}")
# Example usage
get_cert_info('httpbin.org')
Best Practices and Security Considerations
1. Production Security Guidelines
import urllib3
import certifi
# Production-ready configuration
def create_secure_pool_manager():
return urllib3.PoolManager(
cert_reqs='CERT_REQUIRED', # Always verify certificates
ca_certs=certifi.where(), # Use trusted CA bundle
assert_hostname=True, # Verify hostname matches
ssl_version=ssl.PROTOCOL_TLS, # Use latest TLS version
)
# Use in production
http = create_secure_pool_manager()
2. Development and Testing
import urllib3
import os
def create_pool_manager_for_env():
"""Create PoolManager based on environment"""
if os.getenv('ENVIRONMENT') == 'production':
# Strict SSL verification for production
return urllib3.PoolManager(
cert_reqs='CERT_REQUIRED',
assert_hostname=True
)
else:
# Relaxed settings for development (with warnings)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
return urllib3.PoolManager(
cert_reqs='CERT_NONE',
assert_hostname=False
)
http = create_pool_manager_for_env()
3. Certificate Pinning (Advanced)
import urllib3
import ssl
import hashlib
def verify_cert_fingerprint(cert_der, expected_fingerprint):
"""Verify certificate fingerprint for certificate pinning"""
cert_sha256 = hashlib.sha256(cert_der).hexdigest()
return cert_sha256 == expected_fingerprint
# This would require custom implementation with ssl module
# as urllib3 doesn't directly support certificate pinning
Common Use Cases
Corporate Networks with Proxy
import urllib3
# Configure for corporate environment with custom CA
http = urllib3.ProxyManager(
'http://proxy.company.com:8080',
ca_certs='/etc/ssl/certs/corporate-ca.pem',
cert_reqs='CERT_REQUIRED'
)
response = http.request('GET', 'https://external-api.com/')
Microservices with Self-Signed Certificates
import urllib3
# For internal microservices using self-signed certificates
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
http = urllib3.PoolManager(
cert_reqs='CERT_NONE',
assert_hostname=False
)
# Better approach: use custom CA for internal services
Summary
Proper SSL certificate management in urllib3 involves:
- Default verification for production environments
- Custom CA bundles for private certificate authorities
- Client certificates for mutual TLS authentication
- Error handling for certificate-related issues
- Environment-specific configurations for development vs. production
Always prioritize security by keeping SSL verification enabled in production environments and only disable it in controlled development scenarios when absolutely necessary.