Can I set a global timeout for all requests in the Requests library?

In the Python requests library, there is no built-in feature to set a global timeout that will be applied to all requests by default. However, you can define a global timeout value yourself and pass it to each request that you make. Here's how you can do this:

import requests

# Define a global timeout value in seconds
GLOBAL_TIMEOUT = 5  # Example: 5 seconds

# Use the global timeout value for each request
response = requests.get('https://example.com', timeout=GLOBAL_TIMEOUT)

If you need to apply the timeout to many requests and want to avoid passing the timeout argument each time, you can create a Session object and set the timeout for all requests made with that session.

Here's an example of how to define a session with a default timeout:

import requests
from requests.adapters import HTTPAdapter
from requests.exceptions import Timeout

class TimeoutHTTPAdapter(HTTPAdapter):
    def __init__(self, *args, **kwargs):
        self.timeout = kwargs.pop("timeout")
        super().__init__(*args, **kwargs)

    def send(self, request, **kwargs):
        timeout = kwargs.get("timeout")
        if timeout is None:
            kwargs["timeout"] = self.timeout
        return super().send(request, **kwargs)

# Define a global timeout value in seconds
GLOBAL_TIMEOUT = 5  # Example: 5 seconds

# Create a session with the custom adapter
session = requests.Session()
adapter = TimeoutHTTPAdapter(timeout=GLOBAL_TIMEOUT)
session.mount("http://", adapter)
session.mount("https://", adapter)

# Use the session to make requests
try:
    response = session.get('https://example.com')
    # Handle response here
except Timeout:
    print("The request timed out")

In this example, a custom HTTPAdapter is defined with an overridden send method that sets the timeout if one is not provided in the request. The session object then uses this adapter for both HTTP and HTTPS requests. This way, you only need to specify your timeout once when setting up the session, and it will apply to all requests made with that session.

Remember that setting a global timeout is a good practice to prevent your program from hanging indefinitely on a request. However, you should choose an appropriate timeout value based on the expected response time of the resources you are requesting and the network conditions.

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon