How do I debug XPath expressions in Nokogiri?

Debugging XPath expressions in Nokogiri, a Ruby gem for parsing HTML and XML, can sometimes be a challenge, especially if you are new to XPath or the structure of the document you are working with. Here are some strategies to debug XPath expressions in Nokogiri:

1. Use irb or pry: Interactive Ruby (irb) or an alternative like pry is a great way to test out your XPath expressions incrementally. Start by loading your document in Nokogiri and then try out your expressions in real-time.

   require 'nokogiri'
   require 'open-uri'
   
   doc = Nokogiri::HTML(open("http://www.example.com"))
   
   nodes = doc.xpath('your/xpath/expression')
   puts nodes.to_xml
   

2. Start with simple expressions: If your XPath expression is not giving you the expected results, simplify it. Start with a very basic expression that you know should work, and then build it up piece by piece.

3. Use .// to search anywhere: If you're not sure where the content is located within the document hierarchy, use .// at the beginning of your XPath to search for any matching nodes within the document.

4. Check for namespaces: XML documents often have namespaces that can trip you up. If you're querying an XML document and not getting expected results, it might be due to namespaces. You can either include the namespace in your XPath queries or ignore the namespaces:

   # Ignoring namespaces
   doc.remove_namespaces!
   nodes = doc.xpath('//namespace:node') # Replace with actual namespace and node.
   

5. Use XPath functions: XPath has a variety of functions that can be used to check node names, string values, etc. Use these functions to get more information about the nodes you are selecting.

   # Check the name of the first node
   puts doc.xpath('your/xpath/expression').first.name
   

6. Inspect the nodes: Nokogiri nodes have methods like .name, .attributes, .text, etc., which can be used in irb or pry to inspect the nodes you've selected. This can help you confirm that you're getting the right elements.

7. Print out the context: Sometimes it's helpful to get some context around the nodes you've selected. You can print out surrounding HTML/XML to see if you're in the right area of the document.

   nodes = doc.xpath('your/xpath/expression')
   nodes.each do |node|
     puts node.parent.to_xml
   end
   

8. Logging: Create a log of the nodes you're getting at each step of your XPath expression. This can help you pinpoint where the expression is going wrong.

9. Online tools: Use online XPath testers like "FreeFormatter XPath Tester" or "XPather" to experiment with your XPath expressions against sample HTML/XML content.

10. Read the documentation: Make sure you understand the XPath syntax and functions properly. The W3C XPath documentation is a good place to start.

11. Error handling: Use Nokogiri error handling to capture and inspect any parsing errors.

Here's a simple debugging session using irb:

require 'nokogiri'
require 'open-uri'

# Load the document
doc = Nokogiri::HTML(open("http://www.example.com"))

# Start with a simple XPath expression
simple_nodes = doc.xpath('//body')
puts simple_nodes.to_xml

# Add more complexity to the XPath expression step by step
more_complex_nodes = doc.xpath('//body//div[@class="content"]')
puts more_complex_nodes.to_xml

# If you're not getting expected results, check each step
more_complex_nodes.each do |node|
  puts node.name
  puts node.text
end

Remember that debugging is often an iterative process. Take small steps, and verify your results at each point. The flexibility of irb or pry makes them invaluable tools for this kind of work.