Table of contents

Is there a built-in way to retry failed requests in Guzzle?

Yes, Guzzle provides built-in retry functionality through middleware. The Middleware::retry() method allows you to automatically retry failed requests with customizable conditions and backoff strategies.

Basic Retry Setup

Here's how to implement retry logic with Guzzle middleware:

<?php
require 'vendor/autoload.php';

use GuzzleHttp\Client;
use GuzzleHttp\HandlerStack;
use GuzzleHttp\Middleware;
use GuzzleHttp\Exception\RequestException;
use GuzzleHttp\Exception\ConnectException;

// Create handler stack
$stack = HandlerStack::create();

// Add retry middleware
$retryMiddleware = Middleware::retry(
    function ($retries, $request, $response, $exception) {
        // Stop retrying after 3 attempts
        if ($retries >= 3) {
            return false;
        }

        // Retry on server errors (5xx)
        if ($response && $response->getStatusCode() >= 500) {
            return true;
        }

        // Retry on connection errors
        if ($exception instanceof ConnectException) {
            return true;
        }

        // Retry on timeout errors
        if ($exception instanceof RequestException && 
            strpos($exception->getMessage(), 'timeout') !== false) {
            return true;
        }

        return false;
    },
    function ($retries) {
        // Exponential backoff: 1s, 2s, 4s
        return pow(2, $retries) * 1000;
    }
);

$stack->push($retryMiddleware);

// Create client with retry middleware
$client = new Client(['handler' => $stack]);

try {
    $response = $client->request('GET', 'https://httpbin.org/status/500');
    echo $response->getBody();
} catch (RequestException $e) {
    echo "Request failed after retries: " . $e->getMessage();
}

Advanced Retry Configuration

For more sophisticated retry logic, you can create custom retry conditions:

<?php
use GuzzleHttp\Exception\ClientException;
use GuzzleHttp\Exception\ServerException;
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;

$advancedRetryMiddleware = Middleware::retry(
    function ($retries, RequestInterface $request, ResponseInterface $response = null, $exception = null) {
        // Maximum retry attempts
        $maxRetries = 5;

        if ($retries >= $maxRetries) {
            return false;
        }

        // Retry on specific HTTP status codes
        if ($response) {
            $statusCode = $response->getStatusCode();
            $retryableCodes = [408, 429, 500, 502, 503, 504];

            if (in_array($statusCode, $retryableCodes)) {
                return true;
            }
        }

        // Retry on network errors
        if ($exception instanceof ConnectException) {
            return true;
        }

        // Don't retry client errors (4xx except 408, 429)
        if ($exception instanceof ClientException) {
            return false;
        }

        // Retry server errors
        if ($exception instanceof ServerException) {
            return true;
        }

        return false;
    },
    function ($retries) {
        // Exponential backoff with jitter
        $baseDelay = 1000; // 1 second
        $maxDelay = 30000; // 30 seconds

        $delay = min($baseDelay * pow(2, $retries), $maxDelay);

        // Add random jitter (±25%)
        $jitter = $delay * 0.25 * (rand() / getrandmax() * 2 - 1);

        return (int) ($delay + $jitter);
    }
);

Retry with Custom Options

You can also set retry options per request:

<?php
// Client with default retry behavior
$client = new Client(['handler' => $stack]);

// Override retry for specific request
$response = $client->request('GET', 'https://api.example.com/data', [
    'timeout' => 10,
    'retry' => [
        'max_attempts' => 2,
        'delay' => 500 // 500ms fixed delay
    ]
]);

Logging Retry Attempts

To monitor retry behavior, add logging to your retry middleware:

<?php
use Psr\Log\LoggerInterface;

function createRetryMiddleware(LoggerInterface $logger = null) {
    return Middleware::retry(
        function ($retries, $request, $response, $exception) use ($logger) {
            if ($retries >= 3) {
                return false;
            }

            $shouldRetry = false;
            $reason = '';

            if ($response && $response->getStatusCode() >= 500) {
                $shouldRetry = true;
                $reason = "HTTP {$response->getStatusCode()}";
            } elseif ($exception instanceof ConnectException) {
                $shouldRetry = true;
                $reason = 'Connection error';
            }

            if ($shouldRetry && $logger) {
                $logger->warning("Retrying request (attempt {$retries}): {$reason}", [
                    'uri' => (string) $request->getUri(),
                    'method' => $request->getMethod()
                ]);
            }

            return $shouldRetry;
        },
        function ($retries) {
            return pow(2, $retries) * 1000;
        }
    );
}

Key Features

  • Flexible retry conditions: Server errors, timeouts, connection issues
  • Configurable backoff strategies: Exponential, linear, or custom delays
  • Maximum retry limits: Prevent infinite retry loops
  • Per-request customization: Override retry behavior for specific requests
  • Exception handling: Proper handling of different error types

The retry middleware integrates seamlessly with Guzzle's request/response lifecycle, making your HTTP clients more resilient to transient failures.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon