How can I set custom headers or cookies in Headless Chromium?

When using headless Chromium, you may want to customize the request headers or cookies for various reasons, such as simulating a logged-in user session, passing authentication tokens, or modifying user-agent strings. You can set custom headers or cookies in headless Chromium through different programming interfaces or tools. Below are examples of how to do this in both Python with Selenium and Puppeteer in JavaScript.

Python with Selenium and Chromedriver

To set custom headers or cookies in Selenium with headless Chromium, you'll need to use the add_argument method for headers and the add_cookie method for cookies. Here is an example in Python:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By

# Set up Chrome options
chrome_options = Options()
chrome_options.add_argument("--headless")

# To set custom headers, you need an extension or a workaround since Selenium does not support it directly.
# One common workaround is to use a browser extension or manipulate the network requests on the fly (which can be complex).

# To set custom cookies, you can navigate to the domain first and then add the cookie.
driver = webdriver.Chrome(options=chrome_options)

# Navigate to the domain before setting the cookie
driver.get("http://example.com")

# Set a custom cookie
cookie = {'name': 'my_cookie', 'value': 'cookie_value'}
driver.add_cookie(cookie)

# Now you can interact with the website and the custom cookie will be sent with the requests
driver.get("http://example.com")

# Don't forget to close the browser
driver.quit()

JavaScript with Puppeteer

Puppeteer provides a straightforward way to set custom headers and cookies. Here's how you can use Puppeteer to launch headless Chromium with custom headers and cookies:

const puppeteer = require('puppeteer');

(async () => {
  // Launch the browser
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--disable-setuid-sandbox'],
  });

  // Open a new page
  const page = await browser.newPage();

  // Set custom request headers
  await page.setExtraHTTPHeaders({
    'Custom-Header': 'header-value',
  });

  // Set cookies
  const cookie = {
    'name': 'my_cookie',
    'value': 'cookie_value',
    'domain': 'example.com'
  };
  await page.setCookie(cookie);

  // Navigate to the website
  await page.goto('http://example.com');

  // Close the browser
  await browser.close();
})();

Command Line with Chrome/Chromium

You can also set custom headers and cookies directly when launching headless Chromium from the command line. However, custom headers are not directly supported through command-line flags. For cookies, you might be able to use the --cookie-file flag to specify a file containing the cookies, although this is not a standard feature and might require a custom build or a third-party tool.

Generally, it's easier and more reliable to set custom headers and cookies programmatically using tools like Selenium or Puppeteer, as shown above.

Remember that when setting cookies, you must navigate to the domain of the cookie first, or the cookie won't be set properly. Also, your custom headers will only be included in the requests if you set them before making the actual request to the target URL.