Table of contents

How to handle mouse actions using Playwright?

Playwright provides comprehensive mouse action capabilities for automating browser interactions. This guide covers all essential mouse operations including clicks, movements, drag & drop, and hover events in both JavaScript and Python.

Basic Mouse Actions

Clicking Elements

The most common mouse action is clicking. Playwright offers several click methods:

JavaScript:

const { chromium } = require('playwright');

(async () => {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');

  // Click at specific coordinates
  await page.mouse.click(100, 100);

  // Click element by selector (recommended)
  await page.click('button#submit');

  // Right-click
  await page.mouse.click(100, 100, { button: 'right' });

  // Double-click
  await page.mouse.dblclick(100, 100);

  // Click with modifier keys
  await page.click('button', { modifiers: ['Shift'] });

  await browser.close();
})();

Python:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto('https://example.com')

    # Click at specific coordinates
    page.mouse.click(100, 100)

    # Click element by selector (recommended)
    page.click('button#submit')

    # Right-click
    page.mouse.click(100, 100, button='right')

    # Double-click
    page.mouse.dblclick(100, 100)

    # Click with modifier keys
    page.click('button', modifiers=['Shift'])

    browser.close()

Mouse Movement

JavaScript:

// Move mouse to coordinates
await page.mouse.move(100, 100);

// Move with steps (smoother animation)
await page.mouse.move(100, 100, { steps: 5 });

// Hover over element
await page.hover('button#submit');

Python:

# Move mouse to coordinates
page.mouse.move(100, 100)

# Move with steps (smoother animation)
page.mouse.move(100, 100, steps=5)

# Hover over element
page.hover('button#submit')

Advanced Mouse Actions

Drag and Drop

JavaScript:

// Method 1: Using mouse down/up
await page.mouse.move(100, 100);
await page.mouse.down();
await page.mouse.move(300, 300);
await page.mouse.up();

// Method 2: Using dragAndDrop (recommended)
await page.dragAndDrop('#source', '#target');

// Method 3: Using locator API
const source = page.locator('#source');
const target = page.locator('#target');
await source.dragTo(target);

Python:

# Method 1: Using mouse down/up
page.mouse.move(100, 100)
page.mouse.down()
page.mouse.move(300, 300)
page.mouse.up()

# Method 2: Using drag_and_drop (recommended)
page.drag_and_drop('#source', '#target')

# Method 3: Using locator API
source = page.locator('#source')
target = page.locator('#target')
source.drag_to(target)

Mouse Wheel Scrolling

JavaScript:

// Scroll at specific coordinates
await page.mouse.wheel(0, 100);  // Scroll down

// Scroll to element
await page.locator('#content').scroll_into_view_if_needed();

Python:

# Scroll at specific coordinates
page.mouse.wheel(0, 100)  # Scroll down

# Scroll to element
page.locator('#content').scroll_into_view_if_needed()

Practical Examples

Interacting with Canvas Elements

JavaScript:

// Drawing on canvas
const canvas = page.locator('#drawing-canvas');
const box = await canvas.bounding_box();

await page.mouse.move(box.x + 50, box.y + 50);
await page.mouse.down();
await page.mouse.move(box.x + 150, box.y + 150);
await page.mouse.up();

Handling Dropdown Menus

JavaScript:

// Open dropdown on hover
await page.hover('#dropdown-trigger');
await page.waitForSelector('#dropdown-menu', { state: 'visible' });
await page.click('#dropdown-option');

File Upload with Drag & Drop

JavaScript:

// Upload file by dropping it on drop zone
await page.setInputFiles('#file-input', 'path/to/file.txt');

// Or simulate drag and drop
const fileInput = page.locator('#file-input');
await fileInput.set_input_files('path/to/file.txt');

Mouse Button Options

All mouse actions support these button options:

  • 'left' (default)
  • 'right'
  • 'middle'

JavaScript:

await page.mouse.click(100, 100, { button: 'middle' });

Python:

page.mouse.click(100, 100, button='middle')

Best Practices

  1. Use element selectors instead of coordinates when possible - they're more reliable and maintainable.

  2. Wait for elements to be ready before interacting:

await page.waitForSelector('#button', { state: 'visible' });
await page.click('#button');
  1. Use hover for triggering tooltips or dropdowns:
await page.hover('#tooltip-trigger');
await page.waitForSelector('#tooltip');
  1. Handle dynamic content with proper waits:
await page.click('#load-more');
await page.waitForLoadState('networkidle');

Common Issues and Solutions

Problem: Mouse actions fail on hidden elements Solution: Ensure elements are visible before interaction:

await page.waitForSelector('#element', { state: 'visible' });

Problem: Coordinates are incorrect after page scroll Solution: Use element-based actions instead of fixed coordinates:

await page.click('#element');  // Better than page.mouse.click(x, y)

Problem: Drag and drop doesn't work Solution: Ensure proper timing and use the built-in method:

await page.dragAndDrop('#source', '#target');

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon