Table of contents

How do I handle multipart form uploads with Reqwest?

Handling multipart form uploads with Reqwest is essential for file uploads and complex form submissions in Rust applications. Reqwest provides built-in support for multipart forms through its multipart module, making it straightforward to upload files, send form data, and handle various content types.

Setting Up Reqwest for Multipart Forms

First, ensure you have Reqwest with the multipart feature enabled in your Cargo.toml:

[dependencies]
reqwest = { version = "0.11", features = ["json", "multipart"] }
tokio = { version = "1.0", features = ["full"] }

Basic File Upload

Here's how to upload a single file using Reqwest:

use reqwest::multipart;
use std::fs::File;
use std::io::Read;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();

    // Read file content
    let mut file = File::open("example.txt")?;
    let mut contents = Vec::new();
    file.read_to_end(&mut contents)?;

    // Create multipart form
    let form = multipart::Form::new()
        .part("file", multipart::Part::bytes(contents)
            .file_name("example.txt")
            .mime_str("text/plain")?);

    // Send the request
    let response = client
        .post("https://httpbin.org/post")
        .multipart(form)
        .send()
        .await?;

    println!("Status: {}", response.status());
    println!("Response: {}", response.text().await?);

    Ok(())
}

Uploading Multiple Files

When you need to upload multiple files, you can add multiple parts to the same form:

use reqwest::multipart;
use std::fs;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();

    // Create form with multiple files
    let mut form = multipart::Form::new();

    // Add first file
    let file1_content = fs::read("document1.pdf")?;
    form = form.part("document1", multipart::Part::bytes(file1_content)
        .file_name("document1.pdf")
        .mime_str("application/pdf")?);

    // Add second file
    let file2_content = fs::read("image.jpg")?;
    form = form.part("image", multipart::Part::bytes(file2_content)
        .file_name("image.jpg")
        .mime_str("image/jpeg")?);

    // Add text field
    form = form.text("description", "Multiple file upload example");

    let response = client
        .post("https://httpbin.org/post")
        .multipart(form)
        .send()
        .await?;

    println!("Upload status: {}", response.status());
    Ok(())
}

Mixed Form Data with Files

Often, you need to combine file uploads with regular form fields. Here's how to handle mixed multipart forms:

use reqwest::multipart;
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();

    // Read file
    let file_content = std::fs::read("avatar.png")?;

    // Create comprehensive form
    let form = multipart::Form::new()
        // Text fields
        .text("username", "john_doe")
        .text("email", "john@example.com")
        .text("age", "30")
        // JSON data as text
        .text("metadata", json!({
            "upload_date": "2024-01-15",
            "category": "profile"
        }).to_string())
        // File upload
        .part("avatar", multipart::Part::bytes(file_content)
            .file_name("avatar.png")
            .mime_str("image/png")?);

    let response = client
        .post("https://api.example.com/users")
        .header("Authorization", "Bearer your-token-here")
        .multipart(form)
        .send()
        .await?;

    match response.status().is_success() {
        true => println!("Upload successful!"),
        false => println!("Upload failed: {}", response.status()),
    }

    Ok(())
}

Streaming File Uploads

For large files, you might want to stream the content instead of loading it entirely into memory:

use reqwest::multipart;
use tokio::fs::File;
use tokio_util::codec::{BytesCodec, FramedRead};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();

    // Open file for streaming
    let file = File::open("large_file.zip").await?;
    let stream = FramedRead::new(file, BytesCodec::new());

    // Create streaming part
    let file_part = multipart::Part::stream(reqwest::Body::wrap_stream(stream))
        .file_name("large_file.zip")
        .mime_str("application/zip")?;

    let form = multipart::Form::new()
        .text("upload_type", "backup")
        .part("file", file_part);

    let response = client
        .post("https://api.example.com/upload")
        .multipart(form)
        .send()
        .await?;

    println!("Streaming upload completed: {}", response.status());
    Ok(())
}

Error Handling and Validation

Proper error handling is crucial when working with file uploads:

use reqwest::multipart;
use std::path::Path;

async fn upload_with_validation(
    file_path: &str,
    upload_url: &str
) -> Result<String, Box<dyn std::error::Error>> {
    // Validate file exists and size
    let path = Path::new(file_path);
    if !path.exists() {
        return Err("File does not exist".into());
    }

    let metadata = std::fs::metadata(path)?;
    let file_size = metadata.len();

    // Check file size limit (10MB)
    if file_size > 10 * 1024 * 1024 {
        return Err("File too large (max 10MB)".into());
    }

    // Determine MIME type based on extension
    let mime_type = match path.extension().and_then(|ext| ext.to_str()) {
        Some("jpg") | Some("jpeg") => "image/jpeg",
        Some("png") => "image/png",
        Some("pdf") => "application/pdf",
        Some("txt") => "text/plain",
        _ => "application/octet-stream",
    };

    let client = reqwest::Client::new();
    let file_content = std::fs::read(path)?;
    let file_name = path.file_name()
        .and_then(|name| name.to_str())
        .unwrap_or("unknown");

    let form = multipart::Form::new()
        .part("file", multipart::Part::bytes(file_content)
            .file_name(file_name)
            .mime_str(mime_type)?);

    let response = client
        .post(upload_url)
        .timeout(std::time::Duration::from_secs(30))
        .multipart(form)
        .send()
        .await?;

    if response.status().is_success() {
        Ok(response.text().await?)
    } else {
        Err(format!("Upload failed with status: {}", response.status()).into())
    }
}

#[tokio::main]
async fn main() {
    match upload_with_validation("document.pdf", "https://api.example.com/upload").await {
        Ok(response) => println!("Success: {}", response),
        Err(e) => eprintln!("Error: {}", e),
    }
}

Progress Tracking for Large Uploads

For better user experience with large file uploads, you can implement progress tracking:

use reqwest::multipart;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};

struct ProgressTracker {
    uploaded: Arc<AtomicU64>,
    total: u64,
}

impl ProgressTracker {
    fn new(total: u64) -> Self {
        Self {
            uploaded: Arc::new(AtomicU64::new(0)),
            total,
        }
    }

    fn progress(&self) -> f64 {
        let uploaded = self.uploaded.load(Ordering::Relaxed);
        (uploaded as f64 / self.total as f64) * 100.0
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();
    let file_content = std::fs::read("large_video.mp4")?;
    let total_size = file_content.len() as u64;

    let tracker = ProgressTracker::new(total_size);

    // In a real implementation, you'd wrap the content in a progress-tracking stream
    let form = multipart::Form::new()
        .text("title", "My Video Upload")
        .part("video", multipart::Part::bytes(file_content)
            .file_name("video.mp4")
            .mime_str("video/mp4")?);

    println!("Starting upload of {} bytes", total_size);

    let response = client
        .post("https://api.example.com/videos")
        .multipart(form)
        .send()
        .await?;

    println!("Upload completed with status: {}", response.status());
    Ok(())
}

Integration with Web Scraping Workflows

When building web scraping applications, you might need to upload scraped files or submit forms with attachments. Here's how multipart uploads can integrate with scraping workflows:

use reqwest::multipart;
use scraper::{Html, Selector};

async fn scrape_and_upload_images(
    source_url: &str,
    upload_endpoint: &str
) -> Result<(), Box<dyn std::error::Error>> {
    let client = reqwest::Client::new();

    // Scrape the source page
    let html_content = client.get(source_url).send().await?.text().await?;
    let document = Html::parse_document(&html_content);
    let img_selector = Selector::parse("img")?;

    // Process each image
    for img_element in document.select(&img_selector) {
        if let Some(src) = img_element.value().attr("src") {
            // Download image
            let img_response = client.get(src).send().await?;
            if img_response.status().is_success() {
                let img_bytes = img_response.bytes().await?;

                // Extract filename from URL
                let filename = src.split('/').last().unwrap_or("image.jpg");

                // Upload via multipart form
                let form = multipart::Form::new()
                    .text("source_url", source_url)
                    .text("original_src", src)
                    .part("image", multipart::Part::bytes(img_bytes.to_vec())
                        .file_name(filename)
                        .mime_str("image/jpeg")?);

                let upload_response = client
                    .post(upload_endpoint)
                    .multipart(form)
                    .send()
                    .await?;

                println!("Uploaded {}: {}", filename, upload_response.status());
            }
        }
    }

    Ok(())
}

Best Practices and Tips

Memory Management

  • For large files, prefer streaming uploads over loading entire files into memory
  • Consider chunked uploads for very large files
  • Monitor memory usage in production applications

Security Considerations

  • Always validate file types and sizes before uploading
  • Sanitize filenames to prevent path traversal attacks
  • Use proper authentication and authorization
  • Implement rate limiting for upload endpoints

Performance Optimization

Content-Type Headers

Reqwest automatically sets the correct Content-Type header for multipart forms, but you can customize individual parts:

let form = multipart::Form::new()
    .part("data", multipart::Part::text("custom data")
        .mime_str("application/json")?);

Troubleshooting Common Issues

Large File Timeouts: Increase the timeout duration for large file uploads:

let client = reqwest::Client::builder()
    .timeout(std::time::Duration::from_secs(300))
    .build()?;

Memory Issues: Use streaming for large files instead of loading into memory.

MIME Type Detection: Implement proper MIME type detection based on file content, not just extensions.

Advanced Scenarios

Custom Part Headers

You can add custom headers to individual parts of your multipart form:

let custom_part = multipart::Part::bytes(file_content)
    .file_name("data.json")
    .mime_str("application/json")?;

let form = multipart::Form::new()
    .part("data", custom_part);

Working with Different HTTP Methods

While most file uploads use POST, you can use multipart forms with other HTTP methods:

let response = client
    .put("https://api.example.com/update")
    .multipart(form)
    .send()
    .await?;

Handling Server Responses

Always check the server response for upload success and handle various status codes appropriately:

match response.status() {
    reqwest::StatusCode::OK | reqwest::StatusCode::CREATED => {
        println!("Upload successful");
    },
    reqwest::StatusCode::BAD_REQUEST => {
        eprintln!("Bad request - check file format");
    },
    reqwest::StatusCode::PAYLOAD_TOO_LARGE => {
        eprintln!("File too large");
    },
    _ => {
        eprintln!("Upload failed: {}", response.status());
    }
}

Multipart form uploads with Reqwest provide a robust foundation for file handling in Rust applications. Whether you're building web scrapers that need to upload collected data or APIs that handle file submissions, these patterns will help you implement reliable and efficient upload functionality. When handling timeouts in your applications, remember to account for larger files requiring longer upload times.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Can Reqwest automatically decompress Brotli-encoded responses?
How do I set custom DNS servers for Reqwest requests?
What is the difference between Reqwest's Client and ClientBuilder?

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon