How to handle AJAX requests using Puppeteer?

Puppeteer is a Node.js library that provides a high-level API to control Chromium or Chrome over the DevTools Protocol. It can be used for website testing, scraping, taking screenshots, generating PDFs, etc. One of its powerful features is handling AJAX requests.

Here's a step-by-step guide on how to handle AJAX requests using Puppeteer:

1. Install Puppeteer

   Before you can use Puppeteer, you need to install it in your project. You can use the following npm command to install Puppeteer:

   npm i puppeteer
   

2. Setup Puppeteer

   Open the JavaScript file where you want to write your Puppeteer code and add the following lines at the top:

   const puppeteer = require('puppeteer');
   

3. Launch a New Browser Session

   To start interacting with a web page, you first need to launch a new browser session:

   const browser = await puppeteer.launch();
   

4. Open a New Page

   After launching a new browser session, you can open a new page:

   const page = await browser.newPage();
   

5. Handle AJAX Requests

   First, you need to enable request interception:

   await page.setRequestInterception(true);
   

   Then, you can handle AJAX requests using the request event. The following code will log all AJAX requests made to the console:

   page.on('request', (request) => {
       if (request.resourceType() === 'xhr') {
           console.log('AJAX request:', request.url());
       }
       request.continue();
   });
   

   The request event is emitted each time a page makes a request for a resource (such as an image, a script, an AJAX request, etc.). The request.resourceType() method returns the type of resource that the page is requesting. If this type is 'xhr', this means that the request is an AJAX request.

6. Navigate to a URL

   Finally, you can navigate to a URL to see how your code handles AJAX requests:

   await page.goto('https://example.com');
   

7. Close Browser After you've done with your tasks, you should close the browser.

   await browser.close();
   

So, the complete code will look like this:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    await page.setRequestInterception(true);

    page.on('request', (request) => {
        if (request.resourceType() === 'xhr') {
            console.log('AJAX request:', request.url());
        }
        request.continue();
    });

    await page.goto('https://example.com');

    await browser.close();
})();

This script will display all AJAX requests made by the page at the URL 'https://example.com'.