Table of contents

How to handle pop-ups and modals in Puppeteer?

How to Handle Pop-ups and Modals in Puppeteer

Puppeteer provides several methods to handle different types of pop-ups and modals that appear on web pages. This guide covers JavaScript dialogs, HTML modals, and popup windows.

JavaScript Dialogs (Alerts, Confirms, Prompts)

JavaScript dialogs like alert(), confirm(), and prompt() are handled using the dialog event listener.

Basic Dialog Handling

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // Set up dialog handler before navigating
  page.on('dialog', async dialog => {
    console.log(`Dialog type: ${dialog.type()}`);
    console.log(`Dialog message: ${dialog.message()}`);

    // Dismiss the dialog
    await dialog.dismiss();
  });

  await page.goto('https://example.com');

  // Trigger an alert
  await page.evaluate(() => alert('Hello World!'));

  await browser.close();
})();

Accepting Dialogs

page.on('dialog', async dialog => {
  if (dialog.type() === 'confirm') {
    // Accept confirm dialog
    await dialog.accept();
  } else if (dialog.type() === 'prompt') {
    // Accept prompt with input text
    await dialog.accept('My input text');
  } else {
    // Dismiss other dialogs
    await dialog.dismiss();
  }
});

Dialog Types and Methods

  • dialog.type(): Returns 'alert', 'confirm', 'prompt', or 'beforeunload'
  • dialog.message(): Gets the dialog message text
  • dialog.defaultValue(): Gets default value for prompts
  • dialog.accept(promptText?): Accepts the dialog
  • dialog.dismiss(): Dismisses the dialog

HTML Modals

HTML modals are regular DOM elements that can be interacted with using standard Puppeteer methods.

Waiting for and Closing Modals

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  await page.goto('https://example.com');

  // Click button that opens modal
  await page.click('#open-modal-btn');

  // Wait for modal to appear
  await page.waitForSelector('.modal', { visible: true });

  // Interact with modal content
  await page.type('.modal input[name="email"]', 'user@example.com');

  // Close modal by clicking close button
  await page.click('.modal .close-btn');

  // Wait for modal to disappear
  await page.waitForSelector('.modal', { hidden: true });

  await browser.close();
})();

Handling Bootstrap Modals

// Wait for Bootstrap modal to be fully shown
await page.waitForSelector('.modal.show', { visible: true });

// Close using backdrop click
await page.click('.modal-backdrop');

// Or close using close button
await page.click('.modal .btn-close');

// Wait for modal to be fully hidden
await page.waitForFunction(() => 
  !document.querySelector('.modal.show')
);

Popup Windows

Handle new browser windows/tabs that open via window.open() or target="_blank" links.

Detecting New Pages

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // Listen for new pages (popups)
  browser.on('targetcreated', async target => {
    if (target.type() === 'page') {
      const newPage = await target.page();
      console.log('New popup opened:', await newPage.url());

      // Interact with popup
      await newPage.waitForLoadState('networkidle');

      // Close popup
      await newPage.close();
    }
  });

  await page.goto('https://example.com');

  // Click link that opens popup
  await page.click('a[target="_blank"]');

  await browser.close();
})();

Working with Multiple Pages

// Get all open pages
const pages = await browser.pages();

// Click link and wait for new page
const [newPage] = await Promise.all([
  new Promise(resolve => browser.once('targetcreated', target => resolve(target.page()))),
  page.click('#popup-link')
]);

// Work with the new page
await newPage.waitForLoadState();
await newPage.screenshot({ path: 'popup.png' });
await newPage.close();

Advanced Techniques

Conditional Modal Handling

// Handle modals that may or may not appear
try {
  await page.waitForSelector('.modal', { timeout: 3000 });
  console.log('Modal appeared, closing it...');
  await page.click('.modal .close-btn');
} catch (error) {
  console.log('No modal appeared within 3 seconds');
}

Multiple Dialog Types

page.on('dialog', async dialog => {
  const type = dialog.type();
  const message = dialog.message();

  switch (type) {
    case 'alert':
      console.log(`Alert: ${message}`);
      await dialog.accept();
      break;
    case 'confirm':
      // Accept if message contains "proceed"
      if (message.toLowerCase().includes('proceed')) {
        await dialog.accept();
      } else {
        await dialog.dismiss();
      }
      break;
    case 'prompt':
      await dialog.accept('Default response');
      break;
    case 'beforeunload':
      await dialog.accept(); // Allow page navigation
      break;
  }
});

Handling Cookie Consent Modals

// Common cookie consent modal patterns
const cookieSelectors = [
  '#cookie-consent .accept-all',
  '.cookie-banner .accept',
  '[data-testid="cookie-accept"]',
  '.gdpr-consent .agree'
];

for (const selector of cookieSelectors) {
  try {
    await page.waitForSelector(selector, { timeout: 2000 });
    await page.click(selector);
    console.log(`Accepted cookies using selector: ${selector}`);
    break;
  } catch (error) {
    // Try next selector
  }
}

Best Practices

  1. Set up dialog handlers before navigation to avoid missing early dialogs
  2. Use proper timeouts when waiting for modals to prevent hanging
  3. Verify modal state before and after interactions
  4. Handle multiple dialog types appropriately
  5. Clean up popup windows to prevent resource leaks
  6. Use specific selectors rather than generic ones for reliability

Common Issues and Solutions

Dialog Not Handled

// Always set dialog handler before triggering events
page.on('dialog', dialog => dialog.dismiss());
await page.goto('https://example.com'); // Now dialogs will be handled

Modal Not Visible

// Wait for both presence and visibility
await page.waitForSelector('.modal', { visible: true });
// Or check if element is actually visible
await page.waitForFunction(() => {
  const modal = document.querySelector('.modal');
  return modal && modal.offsetParent !== null;
});

Popup Blocked

// Launch with popup blocking disabled
const browser = await puppeteer.launch({
  args: ['--disable-popup-blocking']
});

This comprehensive approach ensures reliable handling of all types of pop-ups and modals in your Puppeteer automation scripts.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon