How do I set a referer or user-agent for Guzzle requests?

When using Guzzle, an HTTP client for PHP, you may often need to customize the request headers such as Referer or User-Agent. These headers can be set for each request by passing an array of headers to the request method.

Here's how you can set the Referer and User-Agent headers for a Guzzle request:

require 'vendor/autoload.php';

use GuzzleHttp\Client;

$client = new Client();

$response = $client->request('GET', 'http://example.com', [
    'headers' => [
        'User-Agent' => 'MyApp/1.0',
        'Referer' => 'http://referrer.example.com'
    ]
]);

echo $response->getBody();

In the above example, the User-Agent header is set to MyApp/1.0, and the Referer header is set to http://referrer.example.com. These headers will be sent along with the GET request to http://example.com.

It's also possible to set default headers for all requests made by a Guzzle client by passing them to the client constructor:

$client = new Client([
    'base_uri' => 'http://example.com',
    'headers' => [
        'User-Agent' => 'MyApp/1.0',
        'Referer' => 'http://referrer.example.com'
    ]
]);

$response = $client->request('GET', '/some/path');
echo $response->getBody();

In this example, the User-Agent and Referer headers will be included in every request made by the $client instance.

It's important to note that while setting custom User-Agent and Referer headers can be useful for various reasons, you should always comply with the terms of service of the website you are accessing, and respect robots.txt files and other scraping etiquette guidelines.

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon