How can I use Puppeteer with a proxy?

You can use Puppeteer with a proxy by passing the proxy server information to the puppeteer.launch() method as part of the options object. Here's how to do it in JavaScript:

First, you need to install Puppeteer. You can do this by running the following command in your console:

npm i puppeteer

Once Puppeteer is installed, you can use it with a proxy like so:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: ['--proxy-server=http://your-proxy:8080']
  });

  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

In the above example, replace 'http://your-proxy:8080' with the address of your proxy server. The --proxy-server argument tells Puppeteer to send all browsing traffic through a proxy.

Additionally, if your proxy requires authentication, you can use the page.authenticate() method like so:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: ['--proxy-server=http://your-proxy:8080']
  });

  const page = await browser.newPage();

  await page.authenticate({
    username: 'your-proxy-username',
    password: 'your-proxy-password'
  });

  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

Again, replace 'your-proxy-username' and 'your-proxy-password' with your proxy credentials. The page.authenticate() method will sign in to your proxy server before making any requests.

Remember to replace 'https://example.com' with the URL you want to scrape. The page is then navigated to this URL, a screenshot is taken and saved as example.png, and then the browser is closed.

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon