How do I implement request signing with Reqwest?

Request signing is a critical security mechanism used to authenticate and verify the integrity of HTTP requests, particularly when working with APIs that require cryptographic authentication. Reqwest, Rust's popular HTTP client library, provides excellent support for implementing various request signing schemes through its flexible middleware and header systems.

Understanding Request Signing

Request signing involves creating a cryptographic signature for your HTTP request using a secret key or private key. The signature is typically included as a header in the request, allowing the receiving server to verify that the request came from an authorized source and hasn't been tampered with during transmission.

Common signing methods include: - HMAC (Hash-based Message Authentication Code) - Digital signatures using RSA or ECDSA - OAuth 1.0a signatures - AWS Signature Version 4

Basic HMAC Request Signing

Here's how to implement HMAC-SHA256 request signing with Reqwest:

use reqwest::Client;
use hmac::{Hmac, Mac};
use sha2::Sha256;
use base64::{Engine as _, engine::general_purpose};
use std::time::{SystemTime, UNIX_EPOCH};

type HmacSha256 = Hmac<Sha256>;

async fn sign_request_hmac(
    client: &Client,
    url: &str,
    method: &str,
    body: &str,
    secret_key: &str,
    api_key: &str,
) -> Result<reqwest::Response, Box<dyn std::error::Error>> {
    // Generate timestamp
    let timestamp = SystemTime::now()
        .duration_since(UNIX_EPOCH)?
        .as_secs()
        .to_string();

    // Create string to sign
    let string_to_sign = format!("{}\n{}\n{}\n{}", method, url, timestamp, body);

    // Create HMAC signature
    let mut mac = HmacSha256::new_from_slice(secret_key.as_bytes())?;
    mac.update(string_to_sign.as_bytes());
    let signature = mac.finalize().into_bytes();
    let signature_b64 = general_purpose::STANDARD.encode(signature);

    // Build and send request
    let response = client
        .request(method.parse()?, url)
        .header("X-API-Key", api_key)
        .header("X-Timestamp", timestamp)
        .header("X-Signature", signature_b64)
        .body(body.to_string())
        .send()
        .await?;

    Ok(response)
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new();
    let secret_key = "your-secret-key";
    let api_key = "your-api-key";

    let response = sign_request_hmac(
        &client,
        "https://api.example.com/data",
        "POST",
        r#"{"message": "Hello, World!"}"#,
        secret_key,
        api_key,
    ).await?;

    println!("Response status: {}", response.status());
    println!("Response body: {}", response.text().await?);

    Ok(())
}

Advanced RSA Digital Signatures

For more robust security, you can implement RSA digital signatures:

use reqwest::Client;
use rsa::{RsaPrivateKey, RsaPublicKey, Pkcs1v15Sign};
use rsa::signature::{Signer, Verifier};
use sha2::{Sha256, Digest};
use base64::{Engine as _, engine::general_purpose};

async fn sign_request_rsa(
    client: &Client,
    url: &str,
    method: &str,
    body: &str,
    private_key: &RsaPrivateKey,
    key_id: &str,
) -> Result<reqwest::Response, Box<dyn std::error::Error>> {
    let timestamp = std::time::SystemTime::now()
        .duration_since(std::time::UNIX_EPOCH)?
        .as_secs()
        .to_string();

    // Create canonical string
    let canonical_string = format!(
        "{}\n{}\n{}\n{}\n{}",
        method,
        url,
        timestamp,
        key_id,
        body
    );

    // Hash the canonical string
    let mut hasher = Sha256::new();
    hasher.update(canonical_string.as_bytes());
    let hash = hasher.finalize();

    // Sign the hash
    let signing_key = rsa::signature::SigningKey::<Sha256>::new(private_key.clone());
    let signature = signing_key.sign(&hash);
    let signature_b64 = general_purpose::STANDARD.encode(signature.to_bytes());

    // Send signed request
    let response = client
        .request(method.parse()?, url)
        .header("Authorization", format!("RSA-SHA256 KeyId={},Signature={}", key_id, signature_b64))
        .header("X-Timestamp", timestamp)
        .body(body.to_string())
        .send()
        .await?;

    Ok(response)
}

AWS Signature Version 4 Implementation

For AWS services, you'll need to implement Signature Version 4:

use reqwest::Client;
use hmac::{Hmac, Mac};
use sha2::{Sha256, Digest};
use hex;
use chrono::{DateTime, Utc};

type HmacSha256 = Hmac<Sha256>;

struct AwsSigner {
    access_key: String,
    secret_key: String,
    region: String,
    service: String,
}

impl AwsSigner {
    fn new(access_key: String, secret_key: String, region: String, service: String) -> Self {
        Self {
            access_key,
            secret_key,
            region,
            service,
        }
    }

    fn sign_request(&self, method: &str, url: &str, headers: &std::collections::HashMap<String, String>, payload: &str) -> Result<String, Box<dyn std::error::Error>> {
        let now = Utc::now();
        let amzdate = now.format("%Y%m%dT%H%M%SZ").to_string();
        let datestamp = now.format("%Y%m%d").to_string();

        // Create canonical request
        let canonical_headers = self.create_canonical_headers(headers, &amzdate);
        let signed_headers = self.create_signed_headers(headers);
        let payload_hash = hex::encode(Sha256::digest(payload.as_bytes()));

        let canonical_request = format!(
            "{}\n{}\n\n{}\n{}\n{}",
            method,
            "/", // Simplified - extract path from URL in production
            canonical_headers,
            signed_headers,
            payload_hash
        );

        // Create string to sign
        let algorithm = "AWS4-HMAC-SHA256";
        let credential_scope = format!("{}/{}/{}/aws4_request", datestamp, self.region, self.service);
        let string_to_sign = format!(
            "{}\n{}\n{}\n{}",
            algorithm,
            amzdate,
            credential_scope,
            hex::encode(Sha256::digest(canonical_request.as_bytes()))
        );

        // Calculate signature
        let signing_key = self.get_signature_key(&datestamp)?;
        let mut mac = HmacSha256::new_from_slice(&signing_key)?;
        mac.update(string_to_sign.as_bytes());
        let signature = hex::encode(mac.finalize().into_bytes());

        // Create authorization header
        let authorization = format!(
            "{} Credential={}/{}, SignedHeaders={}, Signature={}",
            algorithm,
            self.access_key,
            credential_scope,
            signed_headers,
            signature
        );

        Ok(authorization)
    }

    fn get_signature_key(&self, datestamp: &str) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
        let key = format!("AWS4{}", self.secret_key);
        let date_key = self.hmac_sha256(key.as_bytes(), datestamp.as_bytes())?;
        let date_region_key = self.hmac_sha256(&date_key, self.region.as_bytes())?;
        let date_region_service_key = self.hmac_sha256(&date_region_key, self.service.as_bytes())?;
        let signing_key = self.hmac_sha256(&date_region_service_key, b"aws4_request")?;
        Ok(signing_key)
    }

    fn hmac_sha256(&self, key: &[u8], data: &[u8]) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
        let mut mac = HmacSha256::new_from_slice(key)?;
        mac.update(data);
        Ok(mac.finalize().into_bytes().to_vec())
    }

    fn create_canonical_headers(&self, headers: &std::collections::HashMap<String, String>, amzdate: &str) -> String {
        let mut canonical_headers = headers
            .iter()
            .map(|(k, v)| format!("{}:{}", k.to_lowercase(), v.trim()))
            .collect::<Vec<_>>();
        canonical_headers.push(format!("x-amz-date:{}", amzdate));
        canonical_headers.sort();
        canonical_headers.join("\n") + "\n"
    }

    fn create_signed_headers(&self, headers: &std::collections::HashMap<String, String>) -> String {
        let mut signed_headers = headers
            .keys()
            .map(|k| k.to_lowercase())
            .collect::<Vec<_>>();
        signed_headers.push("x-amz-date".to_string());
        signed_headers.sort();
        signed_headers.join(";")
    }
}

Request Signing Middleware

Create reusable middleware for consistent request signing across your application:

use reqwest::{Client, Request, Response};
use reqwest_middleware::{Middleware, Next, Result};
use task_local_extensions::Extensions;

pub struct RequestSigningMiddleware {
    signer: Box<dyn RequestSigner + Send + Sync>,
}

#[async_trait::async_trait]
pub trait RequestSigner {
    async fn sign_request(&self, request: &mut Request) -> Result<(), Box<dyn std::error::Error + Send + Sync>>;
}

impl RequestSigningMiddleware {
    pub fn new(signer: Box<dyn RequestSigner + Send + Sync>) -> Self {
        Self { signer }
    }
}

#[async_trait::async_trait]
impl Middleware for RequestSigningMiddleware {
    async fn handle(
        &self,
        mut req: Request,
        extensions: &mut Extensions,
        next: Next<'_>,
    ) -> Result<Response> {
        // Sign the request
        if let Err(e) = self.signer.sign_request(&mut req).await {
            return Err(reqwest_middleware::Error::Middleware(e));
        }

        // Continue with the signed request
        next.run(req, extensions).await
    }
}

// Usage example
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let signing_middleware = RequestSigningMiddleware::new(
        Box::new(HmacSigner::new("your-secret-key".to_string()))
    );

    let client = reqwest_middleware::ClientBuilder::new(Client::new())
        .with(signing_middleware)
        .build();

    let response = client
        .post("https://api.example.com/secure-endpoint")
        .json(&serde_json::json!({"data": "sensitive information"}))
        .send()
        .await?;

    println!("Signed request completed: {}", response.status());
    Ok(())
}

Security Best Practices

When implementing request signing with Reqwest, follow these security guidelines:

1. Secure Key Management

use std::env;

fn get_signing_key() -> Result<String, Box<dyn std::error::Error>> {
    env::var("SIGNING_SECRET_KEY")
        .map_err(|_| "SIGNING_SECRET_KEY environment variable not set".into())
}

2. Timestamp Validation

use std::time::{SystemTime, UNIX_EPOCH, Duration};

fn validate_timestamp(timestamp_str: &str, max_age_seconds: u64) -> bool {
    if let Ok(timestamp) = timestamp_str.parse::<u64>() {
        if let Ok(now) = SystemTime::now().duration_since(UNIX_EPOCH) {
            let age = now.as_secs().saturating_sub(timestamp);
            return age <= max_age_seconds;
        }
    }
    false
}

3. Constant-Time Comparison

use subtle::ConstantTimeEq;

fn verify_signature(expected: &[u8], received: &[u8]) -> bool {
    expected.ct_eq(received).into()
}

Testing Request Signing

Implement comprehensive tests for your signing implementation:

#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_hmac_signing() {
        let client = Client::new();
        let secret_key = "test-secret-key";
        let api_key = "test-api-key";

        // This would typically test against a mock server
        // that verifies the signature
        let result = sign_request_hmac(
            &client,
            "https://httpbin.org/post",
            "POST",
            r#"{"test": "data"}"#,
            secret_key,
            api_key,
        ).await;

        assert!(result.is_ok());
    }

    #[test]
    fn test_signature_generation() {
        let secret = "test-secret";
        let message = "test-message";

        let mut mac = HmacSha256::new_from_slice(secret.as_bytes()).unwrap();
        mac.update(message.as_bytes());
        let signature = mac.finalize().into_bytes();

        // Test that signature is deterministic
        let mut mac2 = HmacSha256::new_from_slice(secret.as_bytes()).unwrap();
        mac2.update(message.as_bytes());
        let signature2 = mac2.finalize().into_bytes();

        assert_eq!(signature, signature2);
    }
}

Integration with Web Scraping

When implementing request signing for web scraping scenarios, similar to how to handle authentication in Puppeteer, you need to ensure consistent authentication across requests. For complex scenarios involving multiple authenticated requests, consider implementing connection pooling and session management similar to patterns used in how to handle browser sessions in Puppeteer.

OAuth 1.0a Signature Implementation

For APIs that use OAuth 1.0a, you can implement the signature method:

use reqwest::Client;
use hmac::{Hmac, Mac};
use sha1::Sha1;
use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
use std::collections::BTreeMap;

type HmacSha1 = Hmac<Sha1>;

struct OAuth1Signer {
    consumer_key: String,
    consumer_secret: String,
    access_token: String,
    access_token_secret: String,
}

impl OAuth1Signer {
    fn new(consumer_key: String, consumer_secret: String, access_token: String, access_token_secret: String) -> Self {
        Self {
            consumer_key,
            consumer_secret,
            access_token,
            access_token_secret,
        }
    }

    fn sign_request(&self, method: &str, url: &str, params: &BTreeMap<String, String>) -> Result<String, Box<dyn std::error::Error>> {
        let timestamp = std::time::SystemTime::now()
            .duration_since(std::time::UNIX_EPOCH)?
            .as_secs()
            .to_string();

        let nonce = format!("{:x}", rand::random::<u64>());

        // Collect OAuth parameters
        let mut oauth_params = BTreeMap::new();
        oauth_params.insert("oauth_consumer_key".to_string(), self.consumer_key.clone());
        oauth_params.insert("oauth_token".to_string(), self.access_token.clone());
        oauth_params.insert("oauth_signature_method".to_string(), "HMAC-SHA1".to_string());
        oauth_params.insert("oauth_timestamp".to_string(), timestamp);
        oauth_params.insert("oauth_nonce".to_string(), nonce);
        oauth_params.insert("oauth_version".to_string(), "1.0".to_string());

        // Merge with request parameters
        let mut all_params = oauth_params.clone();
        for (k, v) in params {
            all_params.insert(k.clone(), v.clone());
        }

        // Create parameter string
        let param_string = all_params
            .iter()
            .map(|(k, v)| format!("{}={}", 
                utf8_percent_encode(k, NON_ALPHANUMERIC),
                utf8_percent_encode(v, NON_ALPHANUMERIC)
            ))
            .collect::<Vec<_>>()
            .join("&");

        // Create signature base string
        let base_string = format!(
            "{}&{}&{}",
            method.to_uppercase(),
            utf8_percent_encode(url, NON_ALPHANUMERIC),
            utf8_percent_encode(&param_string, NON_ALPHANUMERIC)
        );

        // Create signing key
        let signing_key = format!(
            "{}&{}",
            utf8_percent_encode(&self.consumer_secret, NON_ALPHANUMERIC),
            utf8_percent_encode(&self.access_token_secret, NON_ALPHANUMERIC)
        );

        // Generate signature
        let mut mac = HmacSha1::new_from_slice(signing_key.as_bytes())?;
        mac.update(base_string.as_bytes());
        let signature = base64::encode(mac.finalize().into_bytes());

        oauth_params.insert("oauth_signature".to_string(), signature);

        // Create authorization header
        let auth_header = oauth_params
            .iter()
            .map(|(k, v)| format!("{}=\"{}\"", k, utf8_percent_encode(v, NON_ALPHANUMERIC)))
            .collect::<Vec<_>>()
            .join(", ");

        Ok(format!("OAuth {}", auth_header))
    }
}

Error Handling and Retry Logic

Implement robust error handling for signed requests:

use reqwest::{Client, StatusCode};
use tokio::time::{sleep, Duration};

async fn signed_request_with_retry(
    client: &Client,
    url: &str,
    method: &str,
    body: &str,
    secret_key: &str,
    api_key: &str,
    max_retries: u32,
) -> Result<reqwest::Response, Box<dyn std::error::Error>> {
    let mut attempts = 0;

    loop {
        match sign_request_hmac(client, url, method, body, secret_key, api_key).await {
            Ok(response) => {
                match response.status() {
                    StatusCode::OK | StatusCode::CREATED | StatusCode::ACCEPTED => {
                        return Ok(response);
                    }
                    StatusCode::UNAUTHORIZED => {
                        return Err("Authentication failed - check credentials".into());
                    }
                    StatusCode::TOO_MANY_REQUESTS => {
                        if attempts < max_retries {
                            attempts += 1;
                            let delay = Duration::from_secs(2_u64.pow(attempts));
                            sleep(delay).await;
                            continue;
                        } else {
                            return Err("Rate limit exceeded after retries".into());
                        }
                    }
                    _ => {
                        return Err(format!("Unexpected status code: {}", response.status()).into());
                    }
                }
            }
            Err(e) => {
                if attempts < max_retries {
                    attempts += 1;
                    sleep(Duration::from_secs(1)).await;
                    continue;
                } else {
                    return Err(e);
                }
            }
        }
    }
}

Performance Considerations

For high-throughput applications, consider these optimizations:

use std::sync::Arc;
use tokio::sync::RwLock;

// Cache computed signatures to avoid redundant calculations
struct SignatureCache {
    cache: Arc<RwLock<std::collections::HashMap<String, String>>>,
    max_entries: usize,
}

impl SignatureCache {
    fn new(max_entries: usize) -> Self {
        Self {
            cache: Arc::new(RwLock::new(std::collections::HashMap::new())),
            max_entries,
        }
    }

    async fn get_or_compute<F, Fut>(&self, key: &str, compute_fn: F) -> Result<String, Box<dyn std::error::Error>>
    where
        F: FnOnce() -> Fut,
        Fut: std::future::Future<Output = Result<String, Box<dyn std::error::Error>>>,
    {
        // Check cache first
        {
            let cache = self.cache.read().await;
            if let Some(signature) = cache.get(key) {
                return Ok(signature.clone());
            }
        }

        // Compute signature
        let signature = compute_fn().await?;

        // Store in cache
        {
            let mut cache = self.cache.write().await;
            if cache.len() >= self.max_entries {
                cache.clear(); // Simple eviction strategy
            }
            cache.insert(key.to_string(), signature.clone());
        }

        Ok(signature)
    }
}

Conclusion

Request signing with Reqwest provides robust security for API communications in Rust applications. Whether you're implementing HMAC signatures for simple authentication or complex AWS Signature Version 4 for cloud services, Reqwest's flexible architecture makes it straightforward to integrate cryptographic signing into your HTTP client workflow.

The key to successful implementation lies in proper key management, secure coding practices, and thorough testing. By following the patterns and examples provided above, you can build secure, reliable request signing systems that protect your API communications from tampering and unauthorized access.

Remember to always validate timestamps, use constant-time comparisons for signature verification, and implement proper error handling to maintain security even when things go wrong.