How do I filter nodes by their attributes with Html Agility Pack?

The Html Agility Pack (HAP) is a .NET library used to manipulate HTML documents. It is particularly useful for web scraping because it allows you to navigate the DOM tree and select nodes using XPath or LINQ queries.

If you want to filter nodes by their attributes, you can use XPath expressions. XPath can be used to navigate through elements and attributes in an HTML document.

Here is an example of how you can filter nodes by their attributes using Html Agility Pack in C#:

First, make sure you have installed the Html Agility Pack via NuGet:

Install-Package HtmlAgilityPack

Now, let's say you want to find all the a elements with an href attribute that contains the word "example":

using HtmlAgilityPack;
using System;
using System.Linq;

class Program
{
    static void Main()
    {
        // Load the HTML document
        var htmlDoc = new HtmlDocument();
        htmlDoc.Load("yourHtmlFile.html"); // or use LoadHtml method if you have the HTML as a string

        // Use XPath to select nodes with an 'a' tag and an 'href' attribute containing the word 'example'
        var nodes = htmlDoc.DocumentNode.SelectNodes("//a[contains(@href, 'example')]");

        if (nodes != null)
        {
            foreach (var node in nodes)
            {
                Console.WriteLine(node.OuterHtml);
            }
        }
        else
        {
            Console.WriteLine("No matching nodes found.");
        }
    }
}

If you want to perform more complex filtering, such as selecting nodes based on multiple attributes, you can use more complex XPath expressions:

// Use XPath to select nodes with an 'a' tag that have both 'href' and 'title' attributes
var nodes = htmlDoc.DocumentNode.SelectNodes("//a[@href and @title]");

// Use XPath to select nodes with an 'a' tag where 'href' attribute contains the word 'example' and 'title' attribute starts with 'link'
var nodes = htmlDoc.DocumentNode.SelectNodes("//a[contains(@href, 'example') and starts-with(@title, 'link')]");

XPath is a powerful language for selecting nodes, and it allows for sophisticated queries. You can also filter nodes by their position, value, or any other characteristic that can be accessed via XPath.

Remember that the SelectNodes method returns null if no matching nodes are found. Always check for null before iterating over the nodes to avoid a NullReferenceException.

With LINQ, you can achieve similar results. Here's an example using LINQ to filter nodes:

using HtmlAgilityPack;
using System;
using System.Linq;

class Program
{
    static void Main()
    {
        var htmlDoc = new HtmlDocument();
        htmlDoc.Load("yourHtmlFile.html");

        var nodes = htmlDoc.DocumentNode.Descendants("a")
                                        .Where(a => a.Attributes["href"] != null && a.Attributes["href"].Value.Contains("example"))
                                        .ToList();

        foreach (var node in nodes)
        {
            Console.WriteLine(node.OuterHtml);
        }
    }
}

LINQ queries can be more readable and can leverage the full power of .NET's LINQ to Objects, making it a strong alternative to XPath for many use cases.