How do I handle errors in Curl?

Curl is a powerful command-line tool for transferring data using various protocols. While robust, it can encounter errors during network requests, connection issues, or server problems. This guide covers comprehensive error handling techniques to make your Curl scripts more reliable.

Types of Curl Errors

Curl errors fall into two main categories:

1. HTTP status code errors: Server responses indicating request status
2. Curl exit code errors: Connection, protocol, or configuration issues

Method 1: Checking HTTP Status Codes

HTTP status codes indicate the outcome of your request: - 2xx: Success - 3xx: Redirection - 4xx: Client error (bad request, authentication, etc.) - 5xx: Server error

Basic Status Code Check

# Get only the HTTP status code
status_code=$(curl -s -o /dev/null -w "%{http_code}" https://example.com)

if [ "$status_code" -eq 200 ]; then
    echo "Success!"
elif [ "$status_code" -ge 400 ] && [ "$status_code" -lt 500 ]; then
    echo "Client error: $status_code"
elif [ "$status_code" -ge 500 ]; then
    echo "Server error: $status_code"
else
    echo "Unexpected status: $status_code"
fi

Advanced Status Code Handling

# Function to handle different status codes
handle_response() {
    local url="$1"
    local response=$(curl -s -o response.txt -w "%{http_code}" "$url")

    case "$response" in
        200) echo "Success! Response saved to response.txt" ;;
        301|302) echo "Redirect detected: $response" ;;
        400) echo "Bad request" ;;
        401) echo "Authentication required" ;;
        403) echo "Access forbidden" ;;
        404) echo "Page not found" ;;
        429) echo "Rate limit exceeded" ;;
        500) echo "Internal server error" ;;
        502) echo "Bad gateway" ;;
        503) echo "Service unavailable" ;;
        *) echo "Unexpected response: $response" ;;
    esac
}

handle_response "https://api.example.com/data"

Method 2: Checking Curl Exit Codes

Curl returns exit codes to indicate various error conditions:

Basic Exit Code Check

# Capture both output and exit code
if curl -s "https://example.com" > output.txt; then
    echo "Request successful"
else
    exit_code=$?
    echo "Curl failed with exit code: $exit_code"
fi

Common Curl Exit Codes

# Function to interpret curl exit codes
interpret_curl_error() {
    case $1 in
        0) echo "Success" ;;
        1) echo "Unsupported protocol" ;;
        2) echo "Failed to initialize" ;;
        3) echo "URL malformed" ;;
        5) echo "Couldn't resolve proxy" ;;
        6) echo "Couldn't resolve host" ;;
        7) echo "Failed to connect to host" ;;
        18) echo "Partial file transfer" ;;
        22) echo "HTTP response code said error" ;;
        23) echo "Write error" ;;
        26) echo "Read error" ;;
        28) echo "Operation timeout" ;;
        35) echo "SSL connect error" ;;
        51) echo "Peer certificate cannot be authenticated" ;;
        52) echo "Got nothing from server" ;;
        56) echo "Failure in receiving network data" ;;
        *) echo "Unknown error code: $1" ;;
    esac
}

# Usage
curl -s "https://example.com" || interpret_curl_error $?

Method 3: Comprehensive Error Handling

Combine both methods for robust error handling:

# Comprehensive curl error handling function
safe_curl() {
    local url="$1"
    local output_file="${2:-/dev/null}"
    local max_retries="${3:-3}"
    local retry_delay="${4:-5}"

    for ((i=1; i<=max_retries; i++)); do
        echo "Attempt $i of $max_retries..."

        # Capture both HTTP status and curl exit code
        http_code=$(curl -s -w "%{http_code}" -o "$output_file" \
                        --connect-timeout 10 \
                        --max-time 30 \
                        "$url")
        curl_exit_code=$?

        # Check curl exit code first
        if [ $curl_exit_code -ne 0 ]; then
            echo "Curl error (exit code $curl_exit_code): $(interpret_curl_error $curl_exit_code)"
            if [ $i -lt $max_retries ]; then
                echo "Retrying in $retry_delay seconds..."
                sleep $retry_delay
                continue
            else
                return $curl_exit_code
            fi
        fi

        # Check HTTP status code
        case "$http_code" in
            2*) echo "Success!"; return 0 ;;
            3*) echo "Redirect: $http_code"; return 0 ;;
            4*) echo "Client error: $http_code"; return 1 ;;
            5*) 
                echo "Server error: $http_code"
                if [ $i -lt $max_retries ]; then
                    echo "Retrying in $retry_delay seconds..."
                    sleep $retry_delay
                    continue
                else
                    return 1
                fi
                ;;
            *) echo "Unexpected status: $http_code"; return 1 ;;
        esac
    done
}

# Usage examples
safe_curl "https://api.example.com/data" "response.json" 3 5

Method 4: Using Curl's Built-in Error Options

Curl provides several options for better error handling:

Fail on HTTP Errors

# Fail on HTTP 4xx and 5xx errors
curl --fail "https://example.com" || echo "HTTP error occurred"

# Fail silently on errors
curl --fail --silent "https://example.com" || echo "Request failed"

Verbose Output for Debugging

# Enable verbose output for debugging
curl --verbose "https://example.com" 2> debug.log

# Show error details
curl --show-error --silent "https://example.com" || echo "Error occurred"

Using Write-Out for Detailed Information

# Get detailed request information
curl -w "HTTP Status: %{http_code}\nTotal Time: %{time_total}s\nSize: %{size_download} bytes\n" \
     -s -o /dev/null "https://example.com"

Best Practices for Error Handling

1. Always check both exit codes and HTTP status codes
2. Implement retry logic for transient failures
3. Use timeouts to prevent hanging requests
4. Log errors for debugging purposes
5. Handle different error types appropriately

Production-Ready Example

#!/bin/bash

# Production-ready curl wrapper
robust_curl() {
    local url="$1"
    local output_file="$2"
    local log_file="curl_errors.log"

    # Validate inputs
    if [[ -z "$url" ]]; then
        echo "Error: URL is required" >&2
        return 1
    fi

    # Default output to stdout if not specified
    output_file="${output_file:--}"

    # Execute curl with comprehensive error handling
    local temp_file=$(mktemp)
    local http_code

    http_code=$(curl \
        --silent \
        --show-error \
        --fail \
        --location \
        --connect-timeout 10 \
        --max-time 30 \
        --retry 3 \
        --retry-delay 5 \
        --write-out "%{http_code}" \
        --output "$temp_file" \
        "$url" 2>>"$log_file")

    local curl_exit_code=$?

    # Handle the response
    if [[ $curl_exit_code -eq 0 ]]; then
        # Success: move temp file to final destination
        if [[ "$output_file" == "-" ]]; then
            cat "$temp_file"
        else
            mv "$temp_file" "$output_file"
        fi
        echo "Success: HTTP $http_code" >&2
        return 0
    else
        # Error: log and cleanup
        echo "$(date): Failed to fetch $url (exit code: $curl_exit_code)" >>"$log_file"
        rm -f "$temp_file"
        return $curl_exit_code
    fi
}

# Usage
robust_curl "https://api.example.com/data" "output.json"

Conclusion

Effective error handling in Curl requires checking both HTTP status codes and Curl exit codes. By implementing comprehensive error handling with retries, timeouts, and proper logging, you can create robust scripts that gracefully handle network issues and server errors. Choose the method that best fits your use case, from simple status checks to production-ready wrappers with full error recovery.