How do I handle keyboard and mouse events in Puppeteer-Sharp?

Puppeteer-Sharp provides comprehensive support for simulating keyboard and mouse interactions, making it an excellent choice for automated testing and web scraping scenarios that require user interaction simulation. This guide covers all aspects of handling keyboard and mouse events in your C# applications.

Understanding Input Events in Puppeteer-Sharp

Puppeteer-Sharp offers three main ways to handle user input:

1. Page-level methods - High-level methods for common interactions
2. Keyboard class - Direct keyboard control for complex key sequences
   
3. Mouse class - Precise mouse control for advanced interactions

Basic Mouse Operations

Clicking Elements

The most common mouse interaction is clicking elements on a page:

using PuppeteerSharp;

var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false });
var page = await browser.NewPageAsync();
await page.GoToAsync("https://example.com");

// Click by selector
await page.ClickAsync("#submit-button");

// Click with options
await page.ClickAsync(".menu-item", new ClickOptions
{
    Button = MouseButton.Left,
    ClickCount = 1,
    Delay = 100 // Delay between mousedown and mouseup
});

// Right-click for context menus
await page.ClickAsync("#context-target", new ClickOptions
{
    Button = MouseButton.Right
});

// Double-click
await page.ClickAsync("#double-click-target", new ClickOptions
{
    ClickCount = 2
});

Advanced Mouse Control

For more precise mouse control, use the Mouse property:

// Move mouse to specific coordinates
await page.Mouse.MoveAsync(100, 200);

// Click at current position
await page.Mouse.ClickAsync(100, 200);

// Mouse down and up separately
await page.Mouse.DownAsync();
await Task.Delay(500); // Hold for 500ms
await page.Mouse.UpAsync();

// Drag and drop
await page.Mouse.MoveAsync(100, 100);
await page.Mouse.DownAsync();
await page.Mouse.MoveAsync(200, 200);
await page.Mouse.UpAsync();

Hovering and Mouse Events

Simulate hover effects and mouse movements:

// Hover over an element
await page.HoverAsync("#hover-target");

// Move mouse with steps for smooth animation
await page.Mouse.MoveAsync(0, 0);
await page.Mouse.MoveAsync(100, 100, new MoveOptions { Steps = 10 });

// Simulate mouse wheel scrolling
await page.Mouse.WheelAsync(0, 500); // Scroll down 500 pixels

Keyboard Input Handling

Basic Text Input

For simple text input, use the high-level typing methods:

// Type into an input field
await page.TypeAsync("#username", "john.doe@example.com");

// Type with delay between characters
await page.TypeAsync("#password", "secretpassword", new TypeOptions
{
    Delay = 50 // 50ms delay between keystrokes
});

// Clear existing text and type new text
await page.FocusAsync("#search-input");
await page.Keyboard.DownAsync("Control");
await page.Keyboard.PressAsync("KeyA");
await page.Keyboard.UpAsync("Control");
await page.TypeAsync("#search-input", "new search term");

Advanced Keyboard Control

Use the Keyboard class for complex key combinations and sequences:

// Press individual keys
await page.Keyboard.PressAsync("Enter");
await page.Keyboard.PressAsync("Escape");
await page.Keyboard.PressAsync("Tab");

// Hold modifier keys
await page.Keyboard.DownAsync("Shift");
await page.Keyboard.PressAsync("Tab"); // Shift+Tab for reverse navigation
await page.Keyboard.UpAsync("Shift");

// Complex keyboard shortcuts
await page.Keyboard.DownAsync("Control");
await page.Keyboard.DownAsync("Shift");
await page.Keyboard.PressAsync("KeyI"); // Ctrl+Shift+I (Developer Tools)
await page.Keyboard.UpAsync("Shift");
await page.Keyboard.UpAsync("Control");

// Function keys and special keys
await page.Keyboard.PressAsync("F12");
await page.Keyboard.PressAsync("Home");
await page.Keyboard.PressAsync("End");
await page.Keyboard.PressAsync("PageDown");

Key Codes and Modifiers

Puppeteer-Sharp uses standard key codes. Here are common examples:

// Letter keys
await page.Keyboard.PressAsync("KeyA"); // 'a' or 'A' depending on shift state

// Number keys
await page.Keyboard.PressAsync("Digit1"); // '1' or '!' depending on shift state

// Arrow keys
await page.Keyboard.PressAsync("ArrowLeft");
await page.Keyboard.PressAsync("ArrowRight");
await page.Keyboard.PressAsync("ArrowUp");
await page.Keyboard.PressAsync("ArrowDown");

// Modifier combinations
var modifiers = new[] { "Control", "Alt" };
foreach (var modifier in modifiers)
{
    await page.Keyboard.DownAsync(modifier);
}
await page.Keyboard.PressAsync("Delete");
foreach (var modifier in modifiers.Reverse())
{
    await page.Keyboard.UpAsync(modifier);
}

Form Interaction Examples

Complete Form Filling

Here's a comprehensive example of filling out a complex form:

public async Task FillRegistrationForm()
{
    var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false });
    var page = await browser.NewPageAsync();

    await page.GoToAsync("https://example.com/register");

    // Fill text inputs
    await page.TypeAsync("#firstName", "John");
    await page.TypeAsync("#lastName", "Doe");
    await page.TypeAsync("#email", "john.doe@example.com");

    // Handle dropdown selection
    await page.ClickAsync("#country-dropdown");
    await page.ClickAsync("option[value='US']");

    // Handle checkboxes and radio buttons
    await page.ClickAsync("#terms-checkbox");
    await page.ClickAsync("input[name='gender'][value='male']");

    // Handle file upload
    var fileInput = await page.QuerySelectorAsync("#file-upload");
    await fileInput.UploadFileAsync("C:\\path\\to\\document.pdf");

    // Submit form
    await page.ClickAsync("#submit-button");

    // Wait for navigation or confirmation
    await page.WaitForNavigationAsync();

    await browser.CloseAsync();
}

Handling Dynamic Forms

For forms that change based on user input:

// Wait for element to appear before interacting
await page.TypeAsync("#zip-code", "12345");
await page.WaitForSelectorAsync("#city-field"); // City field appears after ZIP
await page.TypeAsync("#city-field", "New York");

// Handle autocomplete suggestions
await page.TypeAsync("#search-input", "puppeteer");
await page.WaitForSelectorAsync(".autocomplete-dropdown");
await page.ClickAsync(".autocomplete-item:first-child");

Event Coordination and Timing

Waiting for Events

Coordinate your interactions with page events:

// Wait for page to be ready before interacting
await page.WaitForLoadStateAsync(LoadState.NetworkIdle);

// Click and wait for navigation
var navigationTask = page.WaitForNavigationAsync();
await page.ClickAsync("#next-page-link");
await navigationTask;

// Wait for element state changes
await page.ClickAsync("#load-data-button");
await page.WaitForFunctionAsync(@"
    () => document.querySelector('#data-table').children.length > 0
");

Handling Animations and Transitions

When dealing with animated elements, you might need to account for timing:

// Wait for animations to complete
await page.HoverAsync("#animated-menu");
await Task.Delay(500); // Wait for animation
await page.ClickAsync("#menu-item");

// Use Puppeteer's built-in waiting
await page.ClickAsync("#modal-trigger");
await page.WaitForSelectorAsync("#modal", new WaitForSelectorOptions
{
    Visible = true
});

Error Handling and Best Practices

Robust Event Handling

Always include error handling for user interactions:

public async Task SafeInteraction(IPage page, string selector)
{
    try
    {
        // Wait for element to be available
        await page.WaitForSelectorAsync(selector, new WaitForSelectorOptions
        {
            Timeout = 5000
        });

        // Ensure element is clickable
        await page.WaitForFunctionAsync($@"
            () => {{
                const element = document.querySelector('{selector}');
                return element && !element.disabled && element.offsetParent !== null;
            }}
        ");

        await page.ClickAsync(selector);
    }
    catch (TimeoutException)
    {
        throw new InvalidOperationException($"Element {selector} not found or not clickable");
    }
    catch (Exception ex)
    {
        throw new InvalidOperationException($"Failed to interact with {selector}: {ex.Message}");
    }
}

Performance Considerations

Optimize your interactions for better performance:

// Use page.EvaluateExpressionAsync for bulk operations when possible
await page.EvaluateExpressionAsync(@"
    document.querySelectorAll('.checkbox').forEach(cb => cb.checked = true)
");

// Instead of individual clicks for many elements
// for (int i = 0; i < 100; i++)
// {
//     await page.ClickAsync($"#checkbox-{i}");
// }

Advanced Input Scenarios

Simulating Complex User Workflows

public async Task SimulateComplexWorkflow()
{
    var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false });
    var page = await browser.NewPageAsync();

    await page.GoToAsync("https://example.com/dashboard");

    // Multi-step interaction workflow
    await page.ClickAsync("#new-project-button");
    await page.WaitForSelectorAsync("#project-modal");

    // Fill form with validation
    await page.TypeAsync("#project-name", "My New Project");
    await page.Keyboard.PressAsync("Tab");
    await page.TypeAsync("#project-description", "This is a test project");

    // Handle custom dropdown
    await page.ClickAsync("#category-dropdown");
    await page.Keyboard.PressAsync("ArrowDown");
    await page.Keyboard.PressAsync("ArrowDown");
    await page.Keyboard.PressAsync("Enter");

    // Submit and handle confirmation
    await page.ClickAsync("#save-project");
    await page.WaitForSelectorAsync(".success-message");

    await browser.CloseAsync();
}

Handling Keyboard Shortcuts and Hotkeys

// Common application shortcuts
public async Task HandleApplicationShortcuts(IPage page)
{
    // Copy (Ctrl+C)
    await page.Keyboard.DownAsync("Control");
    await page.Keyboard.PressAsync("KeyC");
    await page.Keyboard.UpAsync("Control");

    // Paste (Ctrl+V)
    await page.Keyboard.DownAsync("Control");
    await page.Keyboard.PressAsync("KeyV");
    await page.Keyboard.UpAsync("Control");

    // Undo (Ctrl+Z)
    await page.Keyboard.DownAsync("Control");
    await page.Keyboard.PressAsync("KeyZ");
    await page.Keyboard.UpAsync("Control");

    // Select all (Ctrl+A)
    await page.Keyboard.DownAsync("Control");
    await page.Keyboard.PressAsync("KeyA");
    await page.Keyboard.UpAsync("Control");
}

Integration with Web Scraping Workflows

Keyboard and mouse events are essential components of comprehensive web scraping workflows. When working with complex web applications, you often need to handle form submissions and input interactions in Puppeteer-Sharp as part of your data extraction process.

Additionally, these input capabilities work seamlessly with dynamic content handling in Puppeteer-Sharp, allowing you to trigger content loading through user interactions and then wait for the appropriate elements to appear before extracting data.

Testing and Debugging Input Events

Debugging Input Interactions

public async Task DebugInputEvents(IPage page)
{
    // Enable console logging
    page.Console += (sender, e) => Console.WriteLine($"Console: {e.Message.Text}");

    // Log mouse movements
    await page.EvaluateExpressionAsync(@"
        document.addEventListener('mousemove', (e) => {
            console.log(`Mouse moved to: ${e.clientX}, ${e.clientY}`);
        });

        document.addEventListener('click', (e) => {
            console.log(`Clicked at: ${e.clientX}, ${e.clientY} on element: ${e.target.tagName}`);
        });

        document.addEventListener('keydown', (e) => {
            console.log(`Key pressed: ${e.key}`);
        });
    ");
}

Input Event Validation

public async Task ValidateInputEvents(IPage page)
{
    // Validate form submission
    await page.TypeAsync("#email", "test@example.com");
    var emailValue = await page.EvaluateExpressionAsync<string>(
        "document.querySelector('#email').value"
    );

    if (emailValue != "test@example.com")
    {
        throw new InvalidOperationException("Email input failed");
    }

    // Validate checkbox state
    await page.ClickAsync("#terms-checkbox");
    var isChecked = await page.EvaluateExpressionAsync<bool>(
        "document.querySelector('#terms-checkbox').checked"
    );

    if (!isChecked)
    {
        throw new InvalidOperationException("Checkbox click failed");
    }
}

Conclusion

Puppeteer-Sharp provides comprehensive support for keyboard and mouse event simulation, enabling sophisticated automation scenarios. Whether you're building automated tests, web scrapers, or user interaction bots, understanding these input handling capabilities is crucial for creating robust applications.

Remember to always handle errors gracefully, wait for elements to be ready before interacting with them, and consider the timing of animations and dynamic content. With these techniques, you can create reliable automation scripts that effectively simulate real user behavior.

The key to successful automation with Puppeteer-Sharp is combining these low-level input controls with proper waiting strategies and error handling to create maintainable and reliable automation solutions.