How do I handle authentication with Curl?

How to Handle Authentication with Curl

Curl supports multiple authentication methods for accessing protected resources. Here's a comprehensive guide to the most common authentication types you'll encounter.

Basic Authentication

Basic Authentication sends credentials in the HTTP header as a base64-encoded string. It's simple but requires HTTPS for security.

Using the -u option (recommended):

curl -u username:password https://api.example.com/data

Using Authorization header manually:

curl -H "Authorization: Basic $(echo -n 'username:password' | base64)" https://api.example.com/data

Interactive password prompt (secure):

curl -u username https://api.example.com/data
# Curl will prompt for password without displaying it

Bearer Token Authentication

Most modern APIs use Bearer tokens (JWT, OAuth tokens, API keys). The token goes in the Authorization header.

Bearer token example:

curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." https://api.example.com/data

API key in header:

curl -H "X-API-Key: your-api-key-here" https://api.example.com/data

Multiple authentication headers:

curl -H "Authorization: Bearer your-token" \
     -H "X-API-Version: v2" \
     https://api.example.com/data

Digest Authentication

Digest Authentication is more secure than Basic Auth as it doesn't transmit passwords in plain text.

curl --digest -u username:password https://api.example.com/data

Let Curl negotiate authentication method:

curl --anyauth -u username:password https://api.example.com/data

OAuth 2.0 Authentication

For OAuth flows, you typically need to exchange credentials for an access token first.

Getting an access token:

curl -X POST https://oauth.example.com/token \
     -H "Content-Type: application/x-www-form-urlencoded" \
     -d "grant_type=client_credentials&client_id=your_client_id&client_secret=your_client_secret"

Using the access token:

curl -H "Authorization: Bearer ACCESS_TOKEN_HERE" https://api.example.com/data

Certificate-Based Authentication

For client certificate authentication:

curl --cert client.crt --key client.key https://api.example.com/data

With certificate password:

curl --cert client.p12:password https://api.example.com/data

Common Authentication Patterns

GitHub API:

curl -H "Authorization: token your_github_token" https://api.github.com/user

REST API with JSON response:

curl -u username:password \
     -H "Accept: application/json" \
     -H "Content-Type: application/json" \
     https://api.example.com/users

Form-based login (session cookies):

# Login and save cookies
curl -c cookies.txt -d "username=user&password=pass" https://example.com/login

# Use saved cookies for authenticated requests
curl -b cookies.txt https://example.com/protected-page

Security Best Practices

1. Always use HTTPS

# Good - encrypted connection
curl -u user:pass https://api.example.com/data

# Bad - credentials sent in plain text
curl -u user:pass http://api.example.com/data

2. Use environment variables for sensitive data

export API_TOKEN="your-secret-token"
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com/data

3. Use credential files with restricted permissions

# Create a .netrc file with 600 permissions
echo "machine api.example.com login username password secret" > ~/.netrc
chmod 600 ~/.netrc

# Curl will automatically use credentials from .netrc
curl https://api.example.com/data

4. Avoid logging sensitive data

# Use --silent to prevent credential exposure in logs
curl --silent -u username:password https://api.example.com/data

Troubleshooting Authentication

Debug authentication headers:

curl -v -u username:password https://api.example.com/data

Test different authentication methods:

# Try multiple methods automatically
curl --anyauth -u username:password https://api.example.com/data

Handle authentication failures:

curl -f -u username:password https://api.example.com/data || echo "Authentication failed"

Remember to replace placeholder values (username, password, your-token, etc.) with your actual credentials, and always use HTTPS when transmitting sensitive authentication information.

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon