---
title: "PuppeteerSharp Web Scraping: Headless Chrome in C#"
description: "How to scrape websites with PuppeteerSharp: NuGet setup, selectors, timeouts, logins, downloads, PDFs, Docker, and PuppeteerSharp vs Playwright for .NET."
url: https://webscraping.ai/blog/puppeteersharp-web-scraping
markdown_index: https://webscraping.ai/llms.txt
---
# PuppeteerSharp Web Scraping: Headless Chrome in C#

PuppeteerSharp brings headless Chrome automation to .NET: it's a C# port of Google's [Puppeteer](https://webscraping.ai/blog/puppeteer-web-scraping) that drives a real browser over the DevTools Protocol, so your scraper sees pages the way Chrome renders them — JavaScript executed, XHR data applied, DOM complete. This guide covers the whole scraping workflow in C#: installation, selectors, waiting and timeouts, forms and logins, file downloads, screenshots and PDFs, running on Linux and Docker, and an honest comparison with Playwright for .NET and Selenium.

## Key Takeaways

- `dotnet add package PuppeteerSharp` installs the library from NuGet; `new BrowserFetcher().DownloadAsync()` fetches a matching Chrome build on first run
- The API mirrors Node.js Puppeteer with .NET conventions: `page.goto()` becomes `page.GoToAsync()`, everything is `async`/`await` over Tasks
- Prefer `WaitForSelectorAsync`/`WaitForNetworkIdleAsync` over `Task.Delay` — fixed sleeps are the number-one cause of flaky scrapers
- Set `page.DefaultTimeout` and `page.DefaultNavigationTimeout` once instead of passing timeouts to every call
- Reuse one browser across many pages and wrap everything in `await using` — leaked Chrome processes are the classic PuppeteerSharp memory bug
- A real browser doesn't beat anti-bot walls: Cloudflare and DataDome fingerprint headless Chrome regardless of the language driving it — plan for proxies or a scraping API on protected targets

## What is PuppeteerSharp?

PuppeteerSharp (published on NuGet as `PuppeteerSharp`) is an open-source .NET port of Puppeteer, created and maintained by Darío Kondratiuk. Like the original, it controls Chrome or Chromium over the DevTools Protocol — and recent versions can also drive Firefox via WebDriver BiDi. The API tracks upstream Puppeteer closely, which means two useful things: almost any Puppeteer recipe you find translates to C# nearly line-for-line, and the [Puppeteer documentation](https://pptr.dev) doubles as a reference for the .NET API.

The differences are exactly what you'd expect from idiomatic .NET:

| | Puppeteer (Node.js) | PuppeteerSharp (C#) |
| --- | --- | --- |
| Install | `npm i puppeteer` | `dotnet add package PuppeteerSharp` |
| Naming | camelCase: `goto()`, `newPage()` | PascalCase + Async suffix: `GoToAsync()`, `NewPageAsync()` |
| Async model | Promises | Task-based `async`/`await` |
| Browser download | during `npm install` | explicit `BrowserFetcher` call at runtime |
| Types | TypeScript optional | strongly typed options objects (`LaunchOptions`, `ViewPortOptions`) |

Use PuppeteerSharp when your stack is .NET and the target site renders content with JavaScript. For static HTML, a plain [HttpClient](https://webscraping.ai/blog/csharp-httpclient-guide) with [Html Agility Pack](https://webscraping.ai/blog/html-agility-pack-csharp) is 10–50× cheaper per page — don't launch Chrome for markup that's already in the HTTP response. Our [headless browser guide](https://webscraping.ai/blog/headless-browser-guide) covers when a real browser is and isn't worth the overhead.

## Installation and first scrape

Install the NuGet package into any modern .NET project:

```bash
dotnet add package PuppeteerSharp
```

Unlike the Node version, PuppeteerSharp doesn't bundle a browser with the package — you download one explicitly (once) with `BrowserFetcher`. The download lands in the user profile by default and is skipped if already present:

```csharp
using PuppeteerSharp;

// Download a compatible Chrome build on first run (no-op afterwards)
await new BrowserFetcher().DownloadAsync();

await using var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
    Headless = true
});
await using var page = await browser.NewPageAsync();

await page.GoToAsync("https://example.com");
Console.WriteLine(await page.GetTitleAsync());

var html = await page.GetContentAsync(); // rendered HTML, after JavaScript
```

Two setup tips that save debugging time later:

- **Watch it work while developing.** `Headless = false` with `SlowMo = 50` in `LaunchOptions` shows the browser executing your steps — the fastest way to see why a selector misses.
- **Use an existing Chrome instead of downloading.** Set `ExecutablePath` in `LaunchOptions` to a system Chrome/Chromium — the standard move in locked-down CI images (more in the Docker section).

## Selectors and extracting data

PuppeteerSharp queries the live DOM with CSS selectors (the syntax itself is covered in our [CSS selectors cheat sheet](https://webscraping.ai/blog/css-selectors-cheat-sheet)):

```csharp
// One element / all elements
var heading = await page.QuerySelectorAsync("h1");
var rows = await page.QuerySelectorAllAsync("table tr");

// Evaluate JavaScript in the page and get plain data back — the workhorse
var title = await page.EvaluateFunctionAsync<string>(
    "() => document.querySelector('h1').innerText.trim()");

var products = await page.EvaluateFunctionAsync<Product[]>(@"
    () => [...document.querySelectorAll('.product')].map(el => ({
        name: el.querySelector('.name')?.innerText.trim(),
        price: el.querySelector('.price')?.innerText.trim(),
        url: el.querySelector('a')?.href
    }))");

public record Product(string Name, string Price, string Url);
```

`EvaluateFunctionAsync<T>` runs your JavaScript inside the browser and deserializes the JSON result straight into a C# type — for list pages it's usually one call instead of a loop of element handles. On individual handles, `element.EvaluateFunctionAsync<string>("el => el.innerText")` extracts values the same way.

XPath still works through the `xpath/` selector prefix (`page.QuerySelectorAllAsync("xpath///div[@class='price']")`), but CSS covers nearly every scraping case — see the [XPath cheat sheet](https://webscraping.ai/blog/xpath-cheat-sheet) if you're migrating old XPath-based scrapers.

## Waiting, timeouts, and delays

Most "PuppeteerSharp is flaky" reports are waiting bugs: the code queried the DOM before the site's JavaScript finished building it. The fix is never `Task.Delay` — it's waiting for an observable condition:

```csharp
// Wait until the element exists (default timeout: 30s)
await page.WaitForSelectorAsync(".results .product");

// Wait for an arbitrary page condition
await page.WaitForFunctionAsync(
    "() => document.querySelectorAll('.product').length >= 20");

// Wait for network traffic to settle (good for AJAX-heavy pages)
await page.WaitForNetworkIdleAsync();

// Click that triggers navigation: start waiting BEFORE clicking
var nav = page.WaitForNavigationAsync();
await page.ClickAsync("a.next-page");
await nav;
```

Navigation itself takes a `WaitUntil` option: `Load` (default), `DOMContentLoaded` (fastest), or `Networkidle0`/`Networkidle2` for pages that keep loading data after the load event:

```csharp
await page.GoToAsync("https://example.com/spa", new NavigationOptions
{
    WaitUntil = new[] { WaitUntilNavigation.Networkidle2 },
    Timeout = 60_000
});
```

Rather than passing timeouts everywhere, set the defaults once per page — `SetDefaultNavigationTimeoutAsync` covers `GoToAsync`/`WaitForNavigationAsync`, and `DefaultTimeout` covers everything else:

```csharp
page.DefaultTimeout = 30_000; // waits, selectors
page.DefaultNavigationTimeout = 60_000; // page loads
```

Every timeout throws a `WaitTaskTimeoutException`/`NavigationException` subclass of `PuppeteerException` — catch it, log the URL, and retry with backoff rather than letting one slow page kill a crawl.

## Clicking, typing, and multiple pages

Interaction methods dispatch real input events, so client-side frameworks react exactly as they do for a human:

```csharp
await page.ClickAsync("button.load-more");
await page.TypeAsync("input[name='q']", "wireless headphones", new TypeOptions { Delay = 50 });
await page.Keyboard.PressAsync("Enter");
await page.SelectAsync("select#sort", "price_asc");
```

One browser can run many pages — that's the unit of concurrency. Throttle with a `SemaphoreSlim` so you don't open fifty tabs at once:

```csharp
var throttle = new SemaphoreSlim(4); // 4 concurrent tabs
var tasks = urls.Select(async url =>
{
    await throttle.WaitAsync();
    try
    {
        await using var p = await browser.NewPageAsync();
        await p.GoToAsync(url);
        return await p.EvaluateFunctionAsync<string>("() => document.title");
    }
    finally { throttle.Release(); }
});
var titles = await Task.WhenAll(tasks);
```

Around 100–400 MB of RAM per concurrent page is a realistic budget; on long crawls, restart the browser every few hundred pages to keep Chrome's memory growth in check. That budget is the thing that decides whether a .NET service can run a scrape like [price monitoring](https://webscraping.ai/use-cases/price-monitoring) or [sales intelligence](https://webscraping.ai/use-cases/sales-intelligence) in-process, or whether the browser layer needs to live somewhere else.

## Viewport, mobile emulation, and user agents

Sites serve different markup to different screens, so pin the environment your scraper sees. `SetViewportAsync` controls size and pixel density, `SetUserAgentAsync` the UA string, and `SetExtraHttpHeadersAsync` any additional headers:

```csharp
// Desktop at standard density
await page.SetViewportAsync(new ViewPortOptions { Width = 1920, Height = 1080 });

// Mobile emulation: small viewport, touch, 3x scale factor, mobile UA
await page.SetViewportAsync(new ViewPortOptions
{
    Width = 390, Height = 844,
    DeviceScaleFactor = 3,
    IsMobile = true,
    HasTouch = true
});
await page.SetUserAgentAsync(
    "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 " +
    "(KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1");

await page.SetExtraHttpHeadersAsync(new Dictionary<string, string>
{
    ["Accept-Language"] = "en-US,en;q=0.9"
});
```

`DeviceScaleFactor` matters for screenshots too: `2` or `3` produces retina-quality captures without changing layout. Keep the user agent, viewport, and `Accept-Language` consistent with each other — a desktop UA on a 390px viewport is exactly the kind of mismatch anti-bot heuristics flag.

## Authentication, cookies, and sessions

For form-based logins, drive the form like a user, then persist the session cookies so subsequent runs skip the login page:

```csharp
// Log in once
await page.GoToAsync("https://example.com/login");
await page.TypeAsync("#email", Environment.GetEnvironmentVariable("SCRAPE_USER"));
await page.TypeAsync("#password", Environment.GetEnvironmentVariable("SCRAPE_PASS"));
var loggedIn = page.WaitForNavigationAsync();
await page.ClickAsync("button[type='submit']");
await loggedIn;

// Save the session
var cookies = await page.GetCookiesAsync();
await File.WriteAllTextAsync("cookies.json", JsonSerializer.Serialize(cookies));

// Later run: restore instead of logging in again
var saved = JsonSerializer.Deserialize<CookieParam[]>(
    await File.ReadAllTextAsync("cookies.json"));
await page.SetCookieAsync(saved);
await page.GoToAsync("https://example.com/account");
```

HTTP Basic auth and authenticated proxies both go through `page.AuthenticateAsync(new Credentials { Username = ..., Password = ... })`. Two cautions: store credentials in configuration, not code, and only automate logins on accounts you own or have permission to use — scraping behind a login is also where terms-of-service and legal exposure concentrate (see our [legality guide](https://webscraping.ai/blog/is-web-scraping-legal)).

## File downloads

Chrome won't download files in headless mode until you tell it where to put them. PuppeteerSharp exposes the DevTools Protocol directly for this:

```csharp
var downloadPath = Path.Combine(Environment.CurrentDirectory, "downloads");
Directory.CreateDirectory(downloadPath);

await page.Client.SendAsync("Page.setDownloadBehavior", new
{
    behavior = "allow",
    downloadPath
});

await page.ClickAsync("a.download-csv");
```

There's no completion event, so poll the directory until the file stops growing and Chrome's `.crdownload` temp file disappears. For plain file URLs, skip the browser entirely — once you have the session cookies, passing them to [HttpClient](https://webscraping.ai/blog/csharp-httpclient-guide) downloads the file with a fraction of the overhead.

## Screenshots and PDFs

```csharp
// Full-page screenshot
await page.ScreenshotAsync("page.png", new ScreenshotOptions { FullPage = true });

// Single element
var chart = await page.QuerySelectorAsync(".chart");
await chart.ScreenshotAsync("chart.png");

// PDF (headless mode only)
await page.PdfAsync("report.pdf", new PdfOptions { PrintBackground = true });
```

`PdfAsync` renders the print stylesheet, so pages can look different than on screen — `PrintBackground = true` and an explicit `Format` fix the most common surprises. This trio (rendered HTML, screenshots, PDFs) is why PuppeteerSharp shows up in .NET reporting stacks, not just scrapers.

## Running on Linux, macOS, and Docker

PuppeteerSharp is cross-platform — the usual friction is Chrome's system dependencies on slim Linux images. In containers, the standard launch flags are:

```csharp
var browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
    Headless = true,
    ExecutablePath = "/usr/bin/chromium", // system package instead of BrowserFetcher
    Args = new[] { "--no-sandbox", "--disable-dev-shm-usage" }
});
```

A minimal Dockerfile installs Chromium from the distro so all shared libraries come with it:

```dockerfile
FROM mcr.microsoft.com/dotnet/runtime:8.0
RUN apt-get update && apt-get install -y chromium fonts-liberation \
    && rm -rf /var/lib/apt/lists/*
COPY bin/Release/net8.0/publish/ /app/
WORKDIR /app
ENTRYPOINT ["dotnet", "Scraper.dll"]
```

`--disable-dev-shm-usage` avoids crashes from Docker's default 64 MB `/dev/shm`; `--no-sandbox` is required when the container runs as root (prefer a non-root user where you can). On developer macOS/Windows machines, the plain `BrowserFetcher` download works with no extra flags.

## PuppeteerSharp vs Playwright vs Selenium in .NET

All three drive real browsers from C#; the differences are ergonomics and ecosystem:

| | PuppeteerSharp | Playwright for .NET | Selenium WebDriver |
| --- | --- | --- | --- |
| Maintainer | community (Darío Kondratiuk) | Microsoft | Selenium project |
| Browsers | Chrome/Chromium, Firefox (BiDi) | Chromium, Firefox, WebKit | all major, via drivers |
| Auto-waiting | manual `WaitFor*` calls | built into every action | explicit/implicit waits |
| Protocol | DevTools (CDP) | own driver protocol | WebDriver |
| Best fit | porting Puppeteer knowledge, CDP-level control | new .NET automation projects | existing test infrastructure, grid setups |

Honest guidance: for a brand-new .NET scraping or testing project, [Playwright](https://webscraping.ai/blog/playwright-web-scraping) is the safer default — Microsoft backing, auto-waiting, and WebKit coverage. PuppeteerSharp wins when you're translating existing Puppeteer code or recipes, want the DevTools Protocol exposed directly, or prefer its smaller API surface. [Selenium](https://webscraping.ai/blog/python-selenium) remains the choice for cross-browser test grids. None of the three is better at avoiding bot detection — that fight is decided by IP reputation and fingerprinting, not the automation library.

## When PuppeteerSharp isn't enough

PuppeteerSharp solves JavaScript rendering. It doesn't solve blocked IPs, Cloudflare challenges, CAPTCHAs, or the ops cost of running a Chrome fleet from your .NET services. When those become the problem, point your code at a rendering API instead of the target site — [WebScraping.AI](https://webscraping.ai/) runs the page in real Chrome with rotating datacenter or residential proxies and returns the result over plain HTTP:

```csharp
using var client = new HttpClient();

// Fully rendered HTML through a managed browser + residential proxies
var html = await client.GetStringAsync(
    "https://api.webscraping.ai/html" +
    $"?api_key={apiKey}&url={Uri.EscapeDataString("https://example.com/product/42")}" +
    "&js=true&proxy=residential");

// Or skip parsing entirely — structured fields from any page
var fields = await client.GetFromJsonAsync<Dictionary<string, string>>(
    "https://api.webscraping.ai/ai/fields" +
    $"?api_key={apiKey}&url={Uri.EscapeDataString("https://example.com/product/42")}" +
    "&fields[name]=Product name&fields[price]=Price with currency");
```

A common hybrid: prototype locally with PuppeteerSharp, then swap `GoToAsync` + `GetContentAsync` for the [/html endpoint](https://webscraping.ai/docs) in production and keep your parsing logic — whether that's `EvaluateFunctionAsync` ported to [Html Agility Pack](https://webscraping.ai/blog/html-agility-pack-csharp) or the [AI extraction endpoints](https://webscraping.ai/ai-web-scraping) doing the parsing for you.

## Frequently asked questions

**Is PuppeteerSharp free, and is it still maintained?** Yes — MIT-licensed and actively maintained, with releases tracking upstream Puppeteer closely. It's one of the most-downloaded browser automation packages on NuGet.

**How is PuppeteerSharp different from Puppeteer?** Same architecture and near-identical API, different runtime: C#/.NET Tasks instead of Node.js Promises, PascalCase `*Async` methods instead of camelCase, NuGet instead of npm, and an explicit `BrowserFetcher` browser download instead of one bundled at install time. Puppeteer recipes translate almost mechanically — our [Puppeteer guide](https://webscraping.ai/blog/puppeteer-web-scraping) is effectively a companion to this one.

**Should I use PuppeteerSharp or Playwright for .NET?** Starting fresh, Playwright — official Microsoft support, auto-waiting, three browser engines. Choose PuppeteerSharp when you're porting Puppeteer code, need raw DevTools Protocol access, or want a lighter dependency. Both scrape JavaScript sites equally well.

**Does PuppeteerSharp run on Linux and in Docker?** Yes — it's .NET-cross-platform. In Docker, install Chromium from the distro (so its dependencies come along), set `ExecutablePath`, and launch with `--no-sandbox --disable-dev-shm-usage`. See the Docker section above for a working Dockerfile.

**Why does my scraper work locally but time out on the server?** Usually one of three things: missing Chrome dependencies on the slim server image, a smaller `/dev/shm` in containers (add `--disable-dev-shm-usage`), or slower network making default 30-second timeouts too tight — raise `page.DefaultNavigationTimeout` and wait on `Networkidle2` for AJAX-heavy pages.

**Can PuppeteerSharp bypass Cloudflare or CAPTCHAs?** Not reliably. Headless Chrome is fingerprintable regardless of whether Node.js or .NET drives it, and the C# ecosystem doesn't have a maintained equivalent of the stealth plugins. For protected targets, a scraping API that owns the challenge layer — which is what [WebScraping.AI](https://webscraping.ai/) does — is usually cheaper than building evasion into your scraper.