Can Html Agility Pack be extended with custom functionality?

Yes, the Html Agility Pack (HAP), which is a powerful and flexible .NET library used for parsing and manipulating HTML documents, can indeed be extended with custom functionality. This can be achieved by creating custom classes and extension methods that leverage the capabilities of the HAP. Below, I will demonstrate how you can extend HAP with custom functionality using C#.

1. Creating Custom Extension Methods

One way to extend HAP is by creating extension methods for the HtmlNode class or HtmlDocument class. Extension methods allow you to add new methods to existing types without creating a new derived type or modifying the original type.

using HtmlAgilityPack;
using System;
using System.Linq;

public static class HtmlAgilityPackExtensions
{
    // Example extension method to get the inner text of an HTML element, trimmed and with normalized spaces
    public static string GetCleanInnerText(this HtmlNode node)
    {
        if (node == null)
            throw new ArgumentNullException(nameof(node));

        string innerText = node.InnerText;
        innerText = System.Net.WebUtility.HtmlDecode(innerText); // Decode HTML entities
        innerText = innerText.Trim();
        innerText = System.Text.RegularExpressions.Regex.Replace(innerText, @"\s+", " "); // Replace multiple whitespaces with a single space
        return innerText;
    }
}

2. Creating Custom Wrapper Classes

Another approach to extending the functionality of HAP is by creating wrapper classes that encapsulate the HtmlDocument or HtmlNode objects and provide additional functionality.

public class MyHtmlDocument
{
    private HtmlDocument _htmlDocument;

    public MyHtmlDocument(HtmlDocument htmlDocument)
    {
        _htmlDocument = htmlDocument ?? throw new ArgumentNullException(nameof(htmlDocument));
    }

    // Custom functionality to retrieve all images with alt text
    public IEnumerable<HtmlNode> GetAllImagesWithAltText()
    {
        return _htmlDocument.DocumentNode.SelectNodes("//img[@alt]") ?? Enumerable.Empty<HtmlNode>();
    }

    // You can add more custom methods here
}

3. Custom Query Methods

You might want to create methods that perform custom queries on an HtmlDocument, such as retrieving elements by a specific attribute value or applying complex filtering logic.

public static class HtmlQueryExtensions
{
    public static IEnumerable<HtmlNode> GetElementsWithAttribute(this HtmlDocument document, string tagName, string attributeName)
    {
        if (document == null)
            throw new ArgumentNullException(nameof(document));

        if (string.IsNullOrEmpty(tagName))
            throw new ArgumentException("Tag name cannot be null or empty.", nameof(tagName));

        if (string.IsNullOrEmpty(attributeName))
            throw new ArgumentException("Attribute name cannot be null or empty.", nameof(attributeName));

        var query = $"//{tagName}[@{attributeName}]";
        var nodes = document.DocumentNode.SelectNodes(query);
        return nodes ?? Enumerable.Empty<HtmlNode>();
    }
}

To use these extensions, you would first need to include the namespace in which the extension methods are defined in your code file:

using HtmlAgilityPack;
using YourNamespaceWithExtensions; // Replace with the actual namespace where your extensions are defined

// ...

HtmlDocument doc = new HtmlDocument();
// Load your HTML data into the document...

// Use your custom extension methods
string cleanText = doc.DocumentNode.GetCleanInnerText();
IEnumerable<HtmlNode> imagesWithAlt = new MyHtmlDocument(doc).GetAllImagesWithAltText();

// Use your custom query methods
var elementsWithAttribute = doc.GetElementsWithAttribute("div", "class");

Remember that these are just examples, and you can create any kind of custom functionality based on your requirements. The Html Agility Pack is quite versatile, and when combined with the power of .NET's language features like LINQ and extension methods, it can be tailored to fit a wide range of use cases.