Table of contents

What are the different types of waits available in Playwright?

Playwright provides several wait mechanisms to handle dynamic content and asynchronous operations in web automation. Understanding when and how to use each type is crucial for building reliable tests and scrapers.

Types of Waits

1. Time-Based Wait: waitForTimeout()

The simplest but least reliable wait. Use sparingly as fixed delays can make tests slow and flaky.

// Wait for 3 seconds
await page.waitForTimeout(3000);

When to use: Only as a last resort when other waits don't work.

2. Element-Based Wait: waitForSelector()

Waits for an element to appear in the DOM. Most commonly used wait in Playwright.

// Wait for element to be visible (default behavior)
await page.waitForSelector('#submit-button');

// Wait for element to be attached to DOM (even if hidden)
await page.waitForSelector('.loading-spinner', { state: 'attached' });

// Wait for element to be detached (removed from DOM)
await page.waitForSelector('.loading-spinner', { state: 'detached' });

// Wait for element to be hidden
await page.waitForSelector('#modal', { state: 'hidden' });

// With custom timeout (default is 30 seconds)
await page.waitForSelector('#slow-element', { timeout: 60000 });

States available: attached, detached, visible, hidden

3. Navigation Wait: waitForNavigation()

Note: Deprecated in favor of waitForURL() and waitForLoadState() in newer Playwright versions.

// Wait for any navigation
await page.waitForNavigation();

// Wait for specific URL pattern
await page.waitForURL('**/dashboard');
await page.waitForURL(/.*\/profile.*/);

// Wait for navigation with specific load state
await page.waitForNavigation({ waitUntil: 'networkidle' });

4. Network Response Wait: waitForResponse()

Waits for specific network requests to complete. Essential for API-dependent applications.

// Wait for specific URL response
await page.waitForResponse('https://api.example.com/data');

// Wait for response matching pattern
await page.waitForResponse(response => 
  response.url().includes('/api/users') && response.status() === 200
);

// Wait for response with timeout
await page.waitForResponse('**/api/data', { timeout: 10000 });

// Practical example: Wait for API call after button click
const [response] = await Promise.all([
  page.waitForResponse('**/api/submit'),
  page.click('#submit-button')
]);
console.log('API response:', await response.json());

5. Load State Wait: waitForLoadState()

Waits for page to reach specific loading states.

// Wait for 'load' event (default)
await page.waitForLoadState('load');

// Wait for DOM to be fully loaded
await page.waitForLoadState('domcontentloaded');

// Wait for no network activity for 500ms
await page.waitForLoadState('networkidle');

Load states: - load: Wait for the load event - domcontentloaded: Wait for the DOMContentLoaded event - networkidle: Wait until no network requests for 500ms

6. Function-Based Wait: waitForFunction()

Waits for a custom JavaScript condition to be true.

// Wait for window variable to be defined
await page.waitForFunction(() => window.myGlobalVar !== undefined);

// Wait for element count to reach specific number
await page.waitForFunction(
  selector => document.querySelectorAll(selector).length >= 5,
  '#list-item'
);

// Wait for custom condition with polling interval
await page.waitForFunction(
  () => document.querySelector('#status').textContent === 'Ready',
  {},
  { polling: 1000, timeout: 30000 }
);

Best Practices

1. Prefer Specific Waits Over Timeouts

// ❌ Bad: Fixed timeout
await page.waitForTimeout(5000);
await page.click('#submit');

// ✅ Good: Wait for element to be ready
await page.waitForSelector('#submit', { state: 'visible' });
await page.click('#submit');

2. Combine Waits for Complex Scenarios

// Wait for navigation AND API response
const [response] = await Promise.all([
  page.waitForResponse('**/api/save'),
  page.waitForNavigation(),
  page.click('#save-and-redirect')
]);

3. Handle Dynamic Content Loading

// Wait for initial load, then wait for dynamic content
await page.goto('https://example.com/dashboard');
await page.waitForLoadState('networkidle');
await page.waitForSelector('.dynamic-content[data-loaded="true"]');

4. Use Auto-Waiting Actions When Possible

Many Playwright actions have built-in waiting:

// These automatically wait for elements to be actionable
await page.click('#button');        // Waits for element to be visible and enabled
await page.fill('#input', 'text');  // Waits for element to be visible and enabled
await page.selectOption('#select', 'value'); // Waits for element to be visible

Common Patterns

SPA Navigation

// Single Page Application navigation
await page.click('#nav-link');
await page.waitForURL('**/new-route');
await page.waitForSelector('#page-content');

Form Submission with API Call

const [response] = await Promise.all([
  page.waitForResponse('**/api/users'),
  page.click('#create-user')
]);
const userData = await response.json();

Waiting for Multiple Elements

// Wait for all elements to be present
await Promise.all([
  page.waitForSelector('#header'),
  page.waitForSelector('#sidebar'),
  page.waitForSelector('#main-content')
]);

Understanding and properly using these wait mechanisms will make your Playwright automation more reliable and maintainable.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon