How do I use DiDOM to scrape data from a list of URLs?

DiDOM is a simple and fast HTML parser written in PHP, which allows you to navigate the DOM and pick out the pieces of information you need from web pages. To use DiDOM to scrape data from a list of URLs, you'll typically follow these steps:

1. Install DiDOM.
2. Create a PHP script that loads HTML content from a URL.
3. Parse the HTML content to extract the data you're interested in.
4. Iterate over a list of URLs and apply the extraction logic to each one.

First, ensure that you have DiDOM installed. You can install it via composer with the following command:

composer require Imangazaliev/DiDOM

Now, let's write a PHP script to scrape data from a list of URLs using DiDOM:

<?php

require_once 'vendor/autoload.php';

use DiDom\Document;

function scrapeDataFromUrl($url) {
    try {
        // Load the web page's content into DiDOM
        $document = new Document($url, true);

        // Use CSS selectors to find the elements you want to scrape
        // For example, this might be a list of items:
        $items = $document->find('.item-class');

        $data = [];

        // Extract the data from each item
        foreach ($items as $item) {
            $data[] = $item->text(); // Assuming you want the text content
            // You can also access attributes like this:
            // $data[] = $item->attr('href');
        }

        return $data;
    } catch (\Exception $e) {
        // Handle exceptions (e.g., network problems, invalid selector)
        echo "Error scraping $url: " . $e->getMessage() . PHP_EOL;
        return null;
    }
}

// Define the list of URLs you want to scrape
$urls = [
    'http://example.com/page1',
    'http://example.com/page2',
    // ... more URLs
];

// Iterate over the URLs and scrape data from each one
foreach ($urls as $url) {
    $scrapedData = scrapeDataFromUrl($url);
    if ($scrapedData !== null) {
        // Do something with the scraped data, like saving it to a database or a file
        print_r($scrapedData);
    }
}

In the example above, we define a function scrapeDataFromUrl, which takes a URL as its argument and uses DiDOM to load the HTML content of that URL. It then finds elements with the class .item-class and extracts their text content. You would need to modify the CSS selector to match the elements you want to scrape.

The function is then called for each URL in the $urls array, and the returned data is printed out. You can modify this script to handle the scraped data as needed, for example by storing it in a database or writing it to a CSV file.

Please note that web scraping can have legal and ethical implications. You should always ensure that you're allowed to scrape the website you're targeting and that you comply with its robots.txt file and terms of service. Additionally, be respectful of the website's resources and don't overload their servers with too many requests in a short period of time.