How do I export scraped data into CSV or Excel formats using IronWebScraper?

IronWebScraper is a C# library that provides a fast, simple way to parse and process web content into your applications. When you're using IronWebScraper, you may want to export the scraped data into more portable and accessible formats like CSV (Comma-Separated Values) or Excel for further analysis or reporting.

Here's a step-by-step guide on how to achieve this:

Step 1: Install IronWebScraper

Make sure you have IronWebScraper installed in your C# project. You can install it using NuGet Package Manager:

Install-Package IronWebScraper

Or use the .NET CLI:

dotnet add package IronWebScraper

Step 2: Scrape the Web Content

First, you will need to scrape the data using IronWebScraper. Here's a simple example of how to use it to scrape a website:

using IronWebScraper;
using System.Collections.Generic;

public class BlogScraper : WebScraper
{
    public override void Init()
    {
        this.Request("http://example.com/blog", Parse);
    }

    public override void Parse(Response response)
    {
        foreach (var title_link in response.Css("h2.entry-title a"))
        {
            string title = title_link.TextContentClean;
            string link = title_link.Attributes["href"];
            // Store the title and link for later use
            // Your logic to store the scraped data goes here
        }
        // If there are more pages, you can follow them as well:
        // If(response.CssExists("a.next")) Follow(response.Css("a.next").First().Attributes["href"]);
    }
}

Step 3: Export Data to CSV

Once you have scraped the data, you can export it to CSV. IronWebScraper does not have a built-in CSV exporter, but you can easily create a CSV file with C#:

using System.IO;
using System.Text;

// Assuming you have a list of titles and links
List<KeyValuePair<string, string>> scrapedData = new List<KeyValuePair<string, string>>();

// ... Populate your list with scraped data ...

string csvFilePath = @"path_to_your_output_file.csv";
StringBuilder csvContent = new StringBuilder();

// Add CSV headers
csvContent.AppendLine("Title,Link");

// Add data rows
foreach (var item in scrapedData)
{
    string title = item.Key.Replace(",", ";"); // Replace commas to avoid CSV conflicts
    string link = item.Value;
    csvContent.AppendLine($"{title},{link}");
}

// Write the content to a CSV file
File.WriteAllText(csvFilePath, csvContent.ToString());

Step 4: Export Data to Excel

Exporting to Excel is a bit more complex since you need to write to a binary file format (XLSX). You can use a library like ClosedXML to make this easier:

First, install ClosedXML via NuGet:

Install-Package ClosedXML

Then, you can use the following code to export your data to an Excel file:

using ClosedXML.Excel;

// ... Assuming you have your scraped data ...

using (var workbook = new XLWorkbook())
{
    var worksheet = workbook.Worksheets.Add("ScrapedData");
    worksheet.Cell("A1").Value = "Title";
    worksheet.Cell("B1").Value = "Link";

    int currentRow = 2;
    foreach (var item in scrapedData)
    {
        worksheet.Cell(currentRow, 1).Value = item.Key;
        worksheet.Cell(currentRow, 2).Value = item.Value;
        currentRow++;
    }

    workbook.SaveAs("scraped_data.xlsx");
}

Make sure to replace "path_to_your_output_file.csv" and "scraped_data.xlsx" with the actual paths where you want to save your files.

IronWebScraper is a powerful tool, but you'll often need to supplement it with other libraries or custom code to handle tasks such as exporting data. Always ensure that you're respecting the terms of service and robot.txt files of websites you scrape and that you're handling the data ethically and legally.