How do I handle timeouts in HTTParty when a website takes too long to respond?

When making HTTP requests with HTTParty in Ruby, you may encounter scenarios where a website takes too long to respond, potentially causing your application to hang indefinitely. HTTParty provides several ways to set timeout values to prevent this issue.

Setting Global Default Timeouts

HTTParty allows you to set default timeout values that apply to all requests made by your class using the default_timeout method:

require 'httparty'

class ApiClient
  include HTTParty
  base_uri 'https://api.example.com'
  default_timeout 10 # 10 seconds for all requests

  def self.fetch_data(endpoint)
    get(endpoint)
  rescue Net::OpenTimeout, Net::ReadTimeout => e
    { error: "Request timed out: #{e.message}" }
  end
end

# Usage
result = ApiClient.fetch_data('/slow-endpoint')

Setting Per-Request Timeouts

For more granular control, you can set timeout values for individual requests by passing the timeout option:

require 'httparty'

# Different timeout values for different endpoints
fast_response = HTTParty.get('https://api.example.com/quick', timeout: 5)
slow_response = HTTParty.get('https://api.example.com/heavy', timeout: 30)
file_upload = HTTParty.post('https://api.example.com/upload', 
                           body: { file: File.open('large_file.zip') },
                           timeout: 120) # 2 minutes for file uploads

Separate Connection and Read Timeouts

HTTParty also supports setting separate timeouts for connection establishment and data reading:

require 'httparty'

class WebScraper
  include HTTParty

  # Set different timeout values
  default_options.update(
    open_timeout: 5,    # Time to establish connection
    read_timeout: 15,   # Time to read response data
    write_timeout: 10   # Time to write request data (Ruby 2.6+)
  )
end

begin
  response = WebScraper.get('https://slow-website.com/data')
  puts response.body
rescue Net::OpenTimeout
  puts "Failed to establish connection within 5 seconds"
rescue Net::ReadTimeout
  puts "Server didn't send complete response within 15 seconds"
rescue Net::WriteTimeout
  puts "Failed to send request data within 10 seconds"
end

Advanced Timeout Handling with Retries

For robust applications, combine timeouts with retry logic:

require 'httparty'

class RobustHttpClient
  include HTTParty
  default_timeout 10

  def self.fetch_with_retry(url, max_retries: 3)
    retries = 0

    begin
      response = get(url)
      return response if response.success?

      raise "HTTP Error: #{response.code}"

    rescue Net::OpenTimeout, Net::ReadTimeout => e
      retries += 1

      if retries <= max_retries
        puts "Timeout occurred (attempt #{retries}/#{max_retries}). Retrying..."
        sleep(2 ** retries) # Exponential backoff
        retry
      else
        raise "Max retries exceeded: #{e.message}"
      end
    end
  end
end

# Usage
begin
  data = RobustHttpClient.fetch_with_retry('https://unreliable-api.com/data')
  puts data.body
rescue => e
  puts "Failed to fetch data: #{e.message}"
end

Timeout Configuration for Web Scraping

When scraping websites, different timeout strategies may be needed:

require 'httparty'

class WebScrapingClient
  include HTTParty

  # Conservative timeouts for general scraping
  default_options.update(
    open_timeout: 10,
    read_timeout: 30,
    headers: {
      'User-Agent' => 'Mozilla/5.0 (Compatible Scraper)',
      'Accept' => 'text/html,application/xhtml+xml'
    }
  )

  def self.scrape_page(url, custom_timeout: nil)
    options = custom_timeout ? { timeout: custom_timeout } : {}

    response = get(url, options)

    case response.code
    when 200
      response.body
    when 429
      raise "Rate limited - consider adding delays between requests"
    else
      raise "HTTP #{response.code}: #{response.message}"
    end

  rescue Net::OpenTimeout
    raise "Connection timeout - website may be down or blocking requests"
  rescue Net::ReadTimeout
    raise "Read timeout - website is responding slowly, try increasing timeout"
  end
end

# Scrape with default timeout
page_content = WebScrapingClient.scrape_page('https://example.com')

# Scrape with custom timeout for heavy pages
large_page = WebScrapingClient.scrape_page('https://example.com/heavy-page', custom_timeout: 60)

Best Practices

1. Choose appropriate timeout values: Too short may cause unnecessary failures, too long may hang your application
2. Use different timeouts for different operations: Quick API calls vs. file uploads vs. data processing endpoints
3. Always handle timeout exceptions: Graceful error handling improves user experience
4. Consider retry logic: Temporary network issues can often be resolved with retries
5. Monitor timeout patterns: Frequent timeouts may indicate server issues or need for timeout adjustments

Common Timeout Values

• API calls: 10-30 seconds
• File uploads: 60-300 seconds
  
• Data processing: 30-120 seconds
• Web scraping: 15-45 seconds
• Health checks: 5-10 seconds

Setting appropriate timeout values ensures your HTTParty-based applications remain responsive and handle slow or unresponsive servers gracefully.