urllib3 is a powerful, user-friendly HTTP client for Python. Logging requests and responses can be very useful for debugging and monitoring purposes.
To log requests and responses in urllib3, you can take advantage of Python's built-in logging module. Here's a step-by-step guide on how to set it up:
Step 1: Import the Required Modules
import logging
import urllib3
Step 2: Configure the Logger
You need to configure the logger to capture the desired level of logging detail. In this case, you'll want to capture DEBUG level logs, which typically include request and response information.
# Configure logger
logging.basicConfig(level=logging.DEBUG)
# Set up logging for urllib3
logging.getLogger('urllib3').setLevel(logging.DEBUG)
This will enable DEBUG logging for urllib3, which is typically very verbose and includes request and response headers and bodies.
Step 3: Make a Request Using urllib3
Now, you can make an HTTP request using urllib3, and the details of the request and response will be logged automatically.
# Create a PoolManager instance
http = urllib3.PoolManager()
# Make a request
response = http.request('GET', 'http://httpbin.org/get')
Step 4: Run Your Script
When you run your script, you should see detailed logs in your console, including the requests and responses.
Here's a complete example script:
import logging
import urllib3
# Configure logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger('urllib3').setLevel(logging.DEBUG)
# Initialize urllib3 PoolManager
http = urllib3.PoolManager()
# Make a request and log details
response = http.request('GET', 'http://httpbin.org/get')
# Optionally, print the response body
print(response.data.decode('utf-8'))
Custom Logging Handlers
If you want to log to a file or process the logs differently, you can add custom handlers to the logger:
# Create a custom logger
logger = logging.getLogger(__name__)
# Create a file handler
handler = logging.FileHandler('urllib3_logs.log')
handler.setLevel(logging.DEBUG)
# Create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# Add the handler to the logger
logger.addHandler(handler)
# Now, use `logger` to log custom messages
logger.info('This is a custom log message.')
You can also filter out logs from urllib3 that you don't care about by creating a custom filter class and adding it to the logger or handler.
Please note, logging sensitive information such as passwords or personal data should be avoided or handled with extreme caution to prevent security issues.
Keep in mind that enabling DEBUG level logging will result in a lot of output, which might include sensitive information, as noted above. It is generally not recommended to enable debug logging in production environments for this reason.
