Nokogiri is a popular Ruby library for parsing HTML, XML, and other markup languages. It is based on libxml2 and libxslt libraries, which are known for their performance and compliance with web standards.
Nokogiri itself does not specifically target HTML5 parsing. However, it can parse HTML5 documents because the underlying libxml2 library treats HTML5 as an XML application. This means that Nokogiri can handle HTML5 elements and attributes, but with some limitations:
Parsing Mode: Nokogiri has two parsing modes, one for XML and one for HTML. When parsing HTML5, you should use the HTML mode to ensure that the parser can handle typical features of "tag soup" HTML, such as self-closing tags, optional closing tags, and incorrectly nested elements.
HTML5 Elements: HTML5 introduces a number of new elements (
<article>,<section>,<nav>,<header>,<footer>,<aside>, etc.). Nokogiri will parse these elements correctly as part of the DOM.HTML5 Attributes: New global attributes like
data-*,hidden, andcontenteditable, along with new element-specific attributes, are also parsed correctly by Nokogiri.Doctype: Nokogiri correctly parses the HTML5 doctype (
<!DOCTYPE html>).Void Elements: HTML5 defines a set of void elements (elements that do not have a closing tag, like
<img>,<br>,<input>, etc.). Nokogiri correctly handles these elements in HTML parsing mode.Character Encoding: HTML5 places a strong emphasis on character encoding. Nokogiri does a good job of handling character encoding, but you should ensure that the document encoding is specified correctly for accurate parsing results.
Limitations: One limitation is that Nokogiri does not perform HTML5-specific validations. It will parse the document into a tree structure, but it won't validate if the HTML5 elements are being used according to the specification. Another limitation is that libxml2, which Nokogiri relies on, does not implement all the HTML5 parsing rules as defined by the HTML Living Standard, so in some edge cases, the parsed document tree might not exactly match the expected DOM as it would be constructed by a browser.
Here's a basic example of how you can use Nokogiri to parse an HTML5 document in Ruby:
require 'nokogiri'
html_content = <<-HTML
<!DOCTYPE html>
<html>
<head>
<title>HTML5 Document</title>
</head>
<body>
<header>
<nav>
<a href="#">Home</a>
<a href="#">About</a>
</nav>
</header>
<article data-article-id="1">
<h1>Introduction to HTML5</h1>
<p>HTML5 is the latest evolution of the standard that defines HTML.</p>
</article>
<footer>
<p>Copyright 2023</p>
</footer>
</body>
</html>
HTML
doc = Nokogiri::HTML(html_content)
article = doc.at_css('article')
puts "Article ID: #{article['data-article-id']}"
Output:
Article ID: 1
In this example, Nokogiri parses the HTML5 document, and we extract the data-article-id attribute from the <article> element.
Remember that Nokogiri's ability to parse HTML5 is largely dependent on the robustness of libxml2's HTML parser. If you run into issues parsing certain HTML5-specific features, you might need to preprocess the HTML content or use a different tool that is specifically designed for HTML5 parsing, such as HTML5lib in Python.
