How do I use Curl to upload files?

File uploads are a fundamental requirement in many web applications and APIs. Curl provides several powerful methods for uploading files, whether you're working with simple form uploads, RESTful APIs, or complex multipart requests. This comprehensive guide covers all the essential techniques for uploading files using Curl.

Basic File Upload with Multipart Forms

The most common method for uploading files is using multipart form data with the -F (or --form) option:

# Upload a single file
curl -X POST -F "file=@/path/to/your/file.txt" https://httpbin.org/post

# Upload with a custom field name
curl -X POST -F "document=@/path/to/document.pdf" https://api.example.com/upload

# Upload multiple files
curl -X POST \
  -F "file1=@/path/to/first.jpg" \
  -F "file2=@/path/to/second.png" \
  https://api.example.com/multiple-upload

The @ symbol tells Curl to read the file content from the specified path. Without it, Curl would treat the value as literal text.

Advanced Multipart Form Options

Specifying Content Type

You can explicitly set the content type for uploaded files:

# Upload with specific MIME type
curl -X POST \
  -F "image=@photo.jpg;type=image/jpeg" \
  https://api.example.com/upload

# Upload with custom filename
curl -X POST \
  -F "file=@document.pdf;filename=report.pdf" \
  https://api.example.com/upload

# Combine content type and filename
curl -X POST \
  -F "data=@data.json;type=application/json;filename=config.json" \
  https://api.example.com/upload

Adding Form Fields

Combine file uploads with additional form data:

curl -X POST \
  -F "file=@image.png" \
  -F "title=My Image" \
  -F "description=A beautiful landscape photo" \
  -F "category=nature" \
  https://api.example.com/upload

Binary File Upload

For APIs that expect raw binary data, use the --data-binary option:

# Upload raw binary data
curl -X POST \
  --data-binary @file.zip \
  -H "Content-Type: application/octet-stream" \
  https://api.example.com/binary-upload

# Upload image as binary
curl -X PUT \
  --data-binary @photo.jpg \
  -H "Content-Type: image/jpeg" \
  https://storage.example.com/images/photo.jpg

Upload with Authentication

Basic Authentication

# HTTP Basic Auth
curl -X POST \
  -u username:password \
  -F "file=@document.pdf" \
  https://secure.example.com/upload

# Alternative syntax
curl -X POST \
  --user username:password \
  -F "file=@document.pdf" \
  https://secure.example.com/upload

Bearer Token Authentication

# Upload with Bearer token
curl -X POST \
  -H "Authorization: Bearer your-jwt-token" \
  -F "file=@data.csv" \
  https://api.example.com/protected/upload

# Upload with API key
curl -X POST \
  -H "X-API-Key: your-api-key" \
  -F "attachment=@file.docx" \
  https://api.example.com/upload

Large File Uploads

For large files, consider these optimization techniques:

Progress Tracking

# Show upload progress
curl -X POST \
  --progress-bar \
  -F "largefile=@bigfile.zip" \
  https://api.example.com/upload

# Detailed progress information
curl -X POST \
  -# \
  -F "video=@movie.mp4" \
  https://upload.example.com/media

Resume Interrupted Uploads

# Resume upload from specific byte offset
curl -X PUT \
  --data-binary @largefile.bin \
  -H "Content-Range: bytes 1000000-2000000/5000000" \
  https://api.example.com/resume-upload/session-id

Upload from URLs

You can also upload content directly from URLs:

# Upload from URL (Curl fetches the content)
curl -X POST \
  -F "file=@https://example.com/remote-file.pdf" \
  https://api.example.com/upload-from-url

Common Upload Scenarios

JSON API with File Upload

# Upload file and JSON metadata
curl -X POST \
  -F 'metadata={"title":"Document","tags":["important","pdf"]};type=application/json' \
  -F "file=@document.pdf" \
  https://api.example.com/documents

Cloud Storage Upload

# Upload to cloud storage with signed URL
curl -X PUT \
  --data-binary @file.jpg \
  -H "Content-Type: image/jpeg" \
  "https://storage.googleapis.com/bucket/file.jpg?signed-url-params"

# S3-compatible upload
curl -X PUT \
  --data-binary @backup.tar.gz \
  -H "Content-Type: application/gzip" \
  -H "x-amz-acl: private" \
  https://s3.amazonaws.com/mybucket/backup.tar.gz

FTP Upload

# Upload via FTP
curl -T file.txt ftp://ftp.example.com/uploads/ --user username:password

# Upload to specific directory
curl -T document.pdf ftp://ftp.example.com/documents/document.pdf --user user:pass

Error Handling and Debugging

Verbose Output

# Debug upload issues
curl -v -X POST \
  -F "file=@document.pdf" \
  https://api.example.com/upload

# Save response headers and body
curl -X POST \
  -F "file=@image.png" \
  -D headers.txt \
  -o response.json \
  https://api.example.com/upload

Handle Upload Errors

# Check HTTP status and handle errors
curl -X POST \
  -F "file=@data.csv" \
  -w "HTTP Status: %{http_code}\nUpload Speed: %{speed_upload} bytes/sec\n" \
  https://api.example.com/upload

# Fail on HTTP errors
curl --fail -X POST \
  -F "file=@important.doc" \
  https://api.example.com/upload

Security Considerations

When uploading files with Curl, keep these security practices in mind:

Validate File Types

# Only upload specific file types
if [[ "$file" == *.pdf ]]; then
  curl -X POST -F "document=@$file" https://api.example.com/upload
else
  echo "Only PDF files are allowed"
fi

Use HTTPS

Always use HTTPS for sensitive file uploads:

# Secure upload with certificate verification
curl -X POST \
  --cacert ca-certificates.crt \
  -F "confidential=@secret.txt" \
  https://secure.example.com/upload

Integration with Scripts

Bash Script Example

#!/bin/bash

# Upload multiple files with error handling
upload_file() {
    local file="$1"
    local endpoint="$2"

    if [[ -f "$file" ]]; then
        response=$(curl -s -w "%{http_code}" -X POST \
            -F "file=@$file" \
            "$endpoint")

        http_code="${response: -3}"
        if [[ "$http_code" == "200" ]]; then
            echo "✓ Successfully uploaded: $file"
        else
            echo "✗ Failed to upload $file (HTTP $http_code)"
        fi
    else
        echo "✗ File not found: $file"
    fi
}

# Usage
for file in *.jpg; do
    upload_file "$file" "https://api.example.com/upload"
done

Best Practices

Use appropriate content types: Always specify the correct MIME type for your files
Handle large files carefully: Use progress tracking and consider chunked uploads for files over 100MB
Implement retry logic: Network issues can interrupt uploads, especially for large files
Validate responses: Check HTTP status codes and response content to ensure successful uploads
Secure credentials: Never hardcode passwords or API keys in scripts

When working with complex file upload scenarios in web scraping or automation workflows, you might also need to handle authentication flows or manage file downloads as part of your automation pipeline.

Understanding these Curl file upload techniques provides a solid foundation for integrating file transfer capabilities into your applications, APIs, and automation scripts. Whether you're building a simple file upload service or implementing complex data synchronization workflows, these examples will help you handle most file upload scenarios effectively.

Table of contents