Table of contents

What are the ways to handle shadow DOM elements using Playwright?

Handling Shadow DOM Elements in Playwright

Shadow DOM allows developers to encapsulate HTML, CSS, and JavaScript within custom elements, but this can make testing challenging. Playwright provides several robust methods to interact with shadow DOM elements effectively.

What is Shadow DOM?

Shadow DOM creates an isolated DOM tree that's separate from the main document DOM. Elements inside the shadow DOM are not accessible through regular DOM queries, requiring special techniques to interact with them.

Modern Approach: Using the >>> Combinator

Playwright's recommended approach uses the CSS shadow combinator (>>>) to pierce through shadow DOM boundaries:

JavaScript Examples

// Basic shadow DOM selection
const shadowElement = await page.locator('my-custom-element >>> .inner-button');
await shadowElement.click();

// Chaining through multiple shadow DOM levels
const nestedElement = await page.locator('outer-component >>> inner-component >>> .target-element');
await nestedElement.fill('Hello World');

// Using with other actions
const shadowText = await page.locator('custom-card >>> .card-title').textContent();
console.log(shadowText);

Python Examples

from playwright.sync_api import sync_playwright

# Basic shadow DOM interaction
shadow_element = page.locator('my-custom-element >>> .inner-button')
shadow_element.click()

# Getting text content from shadow DOM
shadow_text = page.locator('custom-card >>> .card-title').text_content()
print(shadow_text)

# Filling form fields inside shadow DOM
page.locator('login-form >>> input[name="username"]').fill('user123')
page.locator('login-form >>> input[name="password"]').fill('password123')

Alternative Methods

1. Direct Element Querying

For more complex scenarios, you can use Playwright's evaluation methods:

// JavaScript - accessing shadow root directly
const shadowContent = await page.evaluate(() => {
    const host = document.querySelector('my-custom-element');
    const shadowRoot = host.shadowRoot;
    return shadowRoot.querySelector('.inner-element').textContent;
});

// Python equivalent
shadow_content = page.evaluate("""
    () => {
        const host = document.querySelector('my-custom-element');
        const shadowRoot = host.shadowRoot;
        return shadowRoot.querySelector('.inner-element').textContent;
    }
""")

2. Using $eval for Shadow DOM

// JavaScript - evaluating within shadow context
const shadowData = await page.$eval('my-custom-element', (element) => {
    const shadowRoot = element.shadowRoot;
    const innerElement = shadowRoot.querySelector('.data-element');
    return {
        text: innerElement.textContent,
        value: innerElement.getAttribute('data-value')
    };
});
# Python - equivalent shadow DOM evaluation
shadow_data = page.eval_on_selector('my-custom-element', """
    (element) => {
        const shadowRoot = element.shadowRoot;
        const innerElement = shadowRoot.querySelector('.data-element');
        return {
            text: innerElement.textContent,
            value: innerElement.getAttribute('data-value')
        };
    }
""")

Real-World Examples

Testing a Custom Web Component

// Testing a custom dropdown component
test('should interact with shadow DOM dropdown', async ({ page }) => {
    await page.goto('/custom-components');

    // Open dropdown
    await page.locator('custom-dropdown >>> .dropdown-trigger').click();

    // Wait for options to appear
    await page.locator('custom-dropdown >>> .dropdown-option').first().waitFor();

    // Select an option
    await page.locator('custom-dropdown >>> .dropdown-option:has-text("Option 2")').click();

    // Verify selection
    const selectedValue = await page.locator('custom-dropdown >>> .selected-value').textContent();
    expect(selectedValue).toBe('Option 2');
});

Handling Multiple Shadow DOM Levels

# Python - testing nested shadow DOM components
def test_nested_shadow_dom(page):
    page.goto('/nested-components')

    # Navigate through multiple shadow DOM levels
    outer_component = page.locator('outer-widget')
    inner_form = outer_component.locator('>>> inner-form')
    input_field = inner_form.locator('>>> input[type="text"]')

    # Interact with deeply nested element
    input_field.fill('Test data')
    inner_form.locator('>>> button[type="submit"]').click()

    # Verify result
    success_message = page.locator('outer-widget >>> .success-message')
    assert success_message.is_visible()

Best Practices

  1. Use the >>> combinator for most shadow DOM interactions - it's the most reliable method
  2. Test shadow DOM availability before interacting:
   const hasShadowRoot = await page.evaluate(() => {
       return !!document.querySelector('my-element').shadowRoot;
   });
  1. Handle timing issues with proper waits:
   await page.locator('custom-element >>> .dynamic-content').waitFor();
  1. Use descriptive selectors to make tests more maintainable
  2. Consider accessibility - ensure shadow DOM elements are properly labeled

Common Pitfalls

  • Closed shadow roots: Some components use closed shadow DOM which cannot be accessed
  • Timing issues: Shadow DOM content may load asynchronously
  • CSS specificity: Styles within shadow DOM are isolated
  • Event handling: Events may not bubble out of shadow DOM boundaries

Browser Support

The >>> combinator works across all browsers supported by Playwright: - Chromium-based browsers - Firefox
- WebKit (Safari)

This comprehensive approach to shadow DOM handling ensures your Playwright tests can interact with modern web components effectively, regardless of their implementation complexity.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon