Table of contents

How to Implement Data-Driven Testing with Selenium WebDriver

Data-driven testing is a powerful approach that allows you to execute the same test script with multiple sets of test data. This methodology separates test logic from test data, making your Selenium WebDriver tests more maintainable, scalable, and comprehensive. Instead of hardcoding test values, you can feed different data sets to your tests from external sources like CSV files, JSON files, or databases.

Understanding Data-Driven Testing

Data-driven testing involves: - Test Data: External data sources containing input values and expected results - Test Script: The automation code that remains constant across different data sets - Test Framework: The structure that reads data and executes tests iteratively

This approach is particularly valuable when you need to test the same functionality with various input combinations, such as login scenarios with different usernames and passwords, or form validation with multiple data sets.

Setting Up Data-Driven Tests in Python

Using CSV Files

CSV (Comma-Separated Values) files are one of the most common data sources for data-driven testing. Here's how to implement it:

import csv
import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class TestDataDriven:
    def setup_method(self):
        self.driver = webdriver.Chrome()
        self.driver.implicitly_wait(10)

    def teardown_method(self):
        self.driver.quit()

    def read_csv_data(self, filename):
        """Read test data from CSV file"""
        test_data = []
        with open(filename, 'r', newline='', encoding='utf-8') as file:
            reader = csv.DictReader(file)
            for row in reader:
                test_data.append(row)
        return test_data

    @pytest.mark.parametrize("test_data", 
                            lambda: TestDataDriven().read_csv_data('login_data.csv'))
    def test_login_with_csv_data(self, test_data):
        """Test login functionality with CSV data"""
        self.driver.get("https://example.com/login")

        # Find and interact with elements
        username_field = self.driver.find_element(By.ID, "username")
        password_field = self.driver.find_element(By.ID, "password")
        login_button = self.driver.find_element(By.ID, "login-btn")

        # Input test data
        username_field.clear()
        username_field.send_keys(test_data['username'])
        password_field.clear()
        password_field.send_keys(test_data['password'])
        login_button.click()

        # Verify expected result
        if test_data['expected_result'] == 'success':
            WebDriverWait(self.driver, 10).until(
                EC.presence_of_element_located((By.CLASS_NAME, "dashboard"))
            )
        else:
            error_message = self.driver.find_element(By.CLASS_NAME, "error-message")
            assert error_message.is_displayed()

Create a corresponding CSV file (login_data.csv):

username,password,expected_result
valid_user,valid_pass,success
invalid_user,invalid_pass,failure
valid_user,invalid_pass,failure
,valid_pass,failure
valid_user,,failure

Using JSON Files

JSON files provide more flexibility for complex data structures:

import json
import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By

class TestJSONDataDriven:
    def setup_method(self):
        self.driver = webdriver.Chrome()

    def teardown_method(self):
        self.driver.quit()

    def load_json_data(self, filename):
        """Load test data from JSON file"""
        with open(filename, 'r', encoding='utf-8') as file:
            return json.load(file)

    @pytest.mark.parametrize("test_case", 
                            lambda: TestJSONDataDriven().load_json_data('form_data.json'))
    def test_form_submission(self, test_case):
        """Test form submission with JSON data"""
        self.driver.get("https://example.com/register")

        # Fill form fields
        for field_name, field_value in test_case['input_data'].items():
            element = self.driver.find_element(By.NAME, field_name)
            element.clear()
            element.send_keys(field_value)

        # Submit form
        submit_button = self.driver.find_element(By.ID, "submit-btn")
        submit_button.click()

        # Validate results
        if test_case['expected_result']['success']:
            success_message = self.driver.find_element(By.CLASS_NAME, "success")
            assert success_message.is_displayed()
        else:
            error_elements = self.driver.find_elements(By.CLASS_NAME, "error")
            assert len(error_elements) > 0

Corresponding JSON file (form_data.json):

[
  {
    "test_name": "Valid registration",
    "input_data": {
      "first_name": "John",
      "last_name": "Doe",
      "email": "john.doe@example.com",
      "phone": "1234567890"
    },
    "expected_result": {
      "success": true,
      "message": "Registration successful"
    }
  },
  {
    "test_name": "Invalid email format",
    "input_data": {
      "first_name": "Jane",
      "last_name": "Smith",
      "email": "invalid-email",
      "phone": "0987654321"
    },
    "expected_result": {
      "success": false,
      "error_fields": ["email"]
    }
  }
]

Data-Driven Testing in Java

Using TestNG with DataProvider

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import org.testng.Assert;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class DataDrivenTest {
    private WebDriver driver;

    @DataProvider(name = "loginData")
    public Object[][] getLoginData() throws IOException {
        return readCSVData("src/test/resources/login_data.csv");
    }

    @DataProvider(name = "searchData")
    public Object[][] getSearchData() {
        return new Object[][] {
            {"Selenium WebDriver", "automation tool", true},
            {"Java programming", "programming language", true},
            {"InvalidSearchTerm123", "no results", false}
        };
    }

    @Test(dataProvider = "loginData")
    public void testLogin(String username, String password, String expectedResult) {
        driver = new ChromeDriver();
        driver.get("https://example.com/login");

        WebElement usernameField = driver.findElement(By.id("username"));
        WebElement passwordField = driver.findElement(By.id("password"));
        WebElement loginButton = driver.findElement(By.id("login-btn"));

        usernameField.sendKeys(username);
        passwordField.sendKeys(password);
        loginButton.click();

        if ("success".equals(expectedResult)) {
            Assert.assertTrue(driver.findElement(By.className("dashboard")).isDisplayed());
        } else {
            Assert.assertTrue(driver.findElement(By.className("error-message")).isDisplayed());
        }

        driver.quit();
    }

    @Test(dataProvider = "searchData")
    public void testSearch(String searchTerm, String expectedContent, boolean shouldFind) {
        driver = new ChromeDriver();
        driver.get("https://example.com/search");

        WebElement searchBox = driver.findElement(By.id("search-input"));
        WebElement searchButton = driver.findElement(By.id("search-btn"));

        searchBox.sendKeys(searchTerm);
        searchButton.click();

        List<WebElement> results = driver.findElements(By.className("search-result"));

        if (shouldFind) {
            Assert.assertTrue(results.size() > 0, "Expected to find search results");
        } else {
            Assert.assertTrue(results.size() == 0, "Expected no search results");
        }

        driver.quit();
    }

    private Object[][] readCSVData(String filePath) throws IOException {
        List<Object[]> data = new ArrayList<>();
        BufferedReader reader = new BufferedReader(new FileReader(filePath));
        String line;

        // Skip header
        reader.readLine();

        while ((line = reader.readLine()) != null) {
            String[] values = line.split(",");
            data.add(values);
        }

        reader.close();
        return data.toArray(new Object[0][]);
    }
}

Using JUnit 5 with ParameterizedTest

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvFileSource;
import org.junit.jupiter.params.provider.ValueSource;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.By;
import static org.junit.jupiter.api.Assertions.*;

public class JUnitDataDrivenTest {
    private WebDriver driver;

    @BeforeEach
    void setUp() {
        driver = new ChromeDriver();
    }

    @AfterEach
    void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }

    @ParameterizedTest
    @CsvFileSource(resources = "/login_data.csv", numLinesToSkip = 1)
    void testLoginWithCSV(String username, String password, String expectedResult) {
        driver.get("https://example.com/login");

        driver.findElement(By.id("username")).sendKeys(username);
        driver.findElement(By.id("password")).sendKeys(password);
        driver.findElement(By.id("login-btn")).click();

        if ("success".equals(expectedResult)) {
            assertTrue(driver.findElement(By.className("dashboard")).isDisplayed());
        } else {
            assertTrue(driver.findElement(By.className("error-message")).isDisplayed());
        }
    }

    @ParameterizedTest
    @ValueSource(strings = {"chrome", "firefox", "edge"})
    void testCrossBrowser(String browserName) {
        // Implementation would vary based on browser
        // This demonstrates testing across different browsers
        assertNotNull(browserName);
    }
}

Data-Driven Testing in JavaScript/Node.js

Using Mocha with External Data

const { Builder, By, until } = require('selenium-webdriver');
const fs = require('fs');
const csv = require('csv-parser');
const assert = require('assert');

describe('Data-Driven Tests', function() {
    let driver;

    beforeEach(async function() {
        driver = await new Builder().forBrowser('chrome').build();
    });

    afterEach(async function() {
        await driver.quit();
    });

    // Read CSV data and create dynamic tests
    const testData = [];

    before(function(done) {
        fs.createReadStream('test_data.csv')
            .pipe(csv())
            .on('data', (row) => {
                testData.push(row);
            })
            .on('end', () => {
                done();
            });
    });

    it('should run login tests with CSV data', async function() {
        for (const data of testData) {
            await driver.get('https://example.com/login');

            const usernameField = await driver.findElement(By.id('username'));
            const passwordField = await driver.findElement(By.id('password'));
            const loginButton = await driver.findElement(By.id('login-btn'));

            await usernameField.clear();
            await usernameField.sendKeys(data.username);
            await passwordField.clear();
            await passwordField.sendKeys(data.password);
            await loginButton.click();

            if (data.expected_result === 'success') {
                await driver.wait(until.elementLocated(By.className('dashboard')), 5000);
            } else {
                const errorMessage = await driver.findElement(By.className('error-message'));
                assert(await errorMessage.isDisplayed());
            }
        }
    });
});

Using Jest with JSON Data

const { Builder, By, until } = require('selenium-webdriver');
const testData = require('./test-data.json');

describe('Data-Driven Login Tests', () => {
    let driver;

    beforeEach(async () => {
        driver = await new Builder().forBrowser('chrome').build();
    });

    afterEach(async () => {
        await driver.quit();
    });

    test.each(testData)('Login test with %s', async (testCase) => {
        await driver.get('https://example.com/login');

        const usernameField = await driver.findElement(By.id('username'));
        const passwordField = await driver.findElement(By.id('password'));
        const loginButton = await driver.findElement(By.id('login-btn'));

        await usernameField.sendKeys(testCase.username);
        await passwordField.sendKeys(testCase.password);
        await loginButton.click();

        if (testCase.shouldSucceed) {
            await driver.wait(until.elementLocated(By.className('dashboard')), 5000);
        } else {
            const errorElement = await driver.findElement(By.className('error-message'));
            expect(await errorElement.isDisplayed()).toBe(true);
        }
    });
});

Advanced Data-Driven Testing Techniques

Using Excel Files (Java Example)

import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class ExcelDataProvider {

    public static Object[][] readExcelData(String filePath, String sheetName) throws IOException {
        List<Object[]> data = new ArrayList<>();

        try (FileInputStream fis = new FileInputStream(filePath);
             Workbook workbook = new XSSFWorkbook(fis)) {

            Sheet sheet = workbook.getSheet(sheetName);

            for (int i = 1; i <= sheet.getLastRowNum(); i++) { // Skip header
                Row row = sheet.getRow(i);
                if (row != null) {
                    Object[] rowData = new Object[row.getLastCellNum()];
                    for (int j = 0; j < row.getLastCellNum(); j++) {
                        Cell cell = row.getCell(j);
                        rowData[j] = getCellValue(cell);
                    }
                    data.add(rowData);
                }
            }
        }

        return data.toArray(new Object[0][]);
    }

    private static Object getCellValue(Cell cell) {
        if (cell == null) return "";

        switch (cell.getCellType()) {
            case STRING:
                return cell.getStringCellValue();
            case NUMERIC:
                return cell.getNumericCellValue();
            case BOOLEAN:
                return cell.getBooleanCellValue();
            default:
                return "";
        }
    }
}

Database-Driven Testing (Python Example)

import sqlite3
import pytest
from selenium import webdriver
from selenium.webdriver.common.by import By

class TestDatabaseDriven:
    def setup_method(self):
        self.driver = webdriver.Chrome()

    def teardown_method(self):
        self.driver.quit()

    def get_test_data_from_database(self):
        """Fetch test data from database"""
        conn = sqlite3.connect('test_data.db')
        cursor = conn.cursor()

        cursor.execute("SELECT username, password, expected_result FROM login_tests")
        test_data = cursor.fetchall()

        conn.close()
        return test_data

    @pytest.mark.parametrize("username,password,expected_result", 
                            lambda: TestDatabaseDriven().get_test_data_from_database())
    def test_login_from_database(self, username, password, expected_result):
        """Test login with database-driven data"""
        self.driver.get("https://example.com/login")

        self.driver.find_element(By.ID, "username").send_keys(username)
        self.driver.find_element(By.ID, "password").send_keys(password)
        self.driver.find_element(By.ID, "login-btn").click()

        if expected_result == 'success':
            assert self.driver.find_element(By.CLASS_NAME, "dashboard").is_displayed()
        else:
            assert self.driver.find_element(By.CLASS_NAME, "error-message").is_displayed()

Best Practices for Data-Driven Testing

1. Data Organization and Management

  • Separate test data by functionality: Create different data files for login, registration, search, etc.
  • Use meaningful file names: login_test_data.csv, product_search_data.json
  • Include test case descriptions: Add columns for test case names and descriptions
  • Validate data integrity: Ensure data consistency and completeness before test execution

2. Test Data Design

  • Cover edge cases: Include boundary values, special characters, and null/empty values
  • Include negative test cases: Test with invalid data to ensure proper error handling
  • Use realistic data: Mirror production-like data patterns when possible
  • Keep data maintainable: Avoid hardcoding values that change frequently

3. Framework Integration

When implementing data-driven testing alongside other Selenium practices, consider how it integrates with comprehensive element waiting strategies and proper error handling techniques.

4. Performance Considerations

  • Optimize data loading: Load data once and reuse across tests
  • Use parallel execution: Run data-driven tests in parallel when possible
  • Implement proper cleanup: Ensure browser instances are properly closed
  • Monitor resource usage: Keep track of memory and CPU usage during large data sets

5. Reporting and Debugging

import logging
from datetime import datetime

class DataDrivenReporter:
    def __init__(self):
        self.setup_logging()
        self.results = []

    def setup_logging(self):
        logging.basicConfig(
            level=logging.INFO,
            format='%(asctime)s - %(levelname)s - %(message)s',
            handlers=[
                logging.FileHandler('data_driven_tests.log'),
                logging.StreamHandler()
            ]
        )

    def log_test_result(self, test_data, result, execution_time):
        """Log individual test results"""
        log_entry = {
            'timestamp': datetime.now().isoformat(),
            'test_data': test_data,
            'result': result,
            'execution_time': execution_time
        }

        self.results.append(log_entry)
        logging.info(f"Test completed: {test_data} - {result} - {execution_time}s")

    def generate_report(self):
        """Generate comprehensive test report"""
        total_tests = len(self.results)
        passed_tests = len([r for r in self.results if r['result'] == 'PASS'])
        failed_tests = total_tests - passed_tests

        report = f"""
        Data-Driven Test Report
        ======================
        Total Tests: {total_tests}
        Passed: {passed_tests}
        Failed: {failed_tests}
        Pass Rate: {(passed_tests/total_tests)*100:.2f}%
        """

        return report

Data-driven testing with Selenium WebDriver provides a scalable approach to test automation that improves test coverage while reducing maintenance overhead. By separating test logic from test data, you can easily add new test scenarios without modifying your automation code. Whether you choose CSV files, JSON, Excel, or database sources, the key is to maintain clean, organized data that supports your testing objectives and integrates well with your overall test automation strategy.

Try WebScraping.AI for Your Web Scraping Needs

Looking for a powerful web scraping solution? WebScraping.AI provides an LLM-powered API that combines Chromium JavaScript rendering with rotating proxies for reliable data extraction.

Key Features:

  • AI-powered extraction: Ask questions about web pages or extract structured data fields
  • JavaScript rendering: Full Chromium browser support for dynamic content
  • Rotating proxies: Datacenter and residential proxies from multiple countries
  • Easy integration: Simple REST API with SDKs for Python, Ruby, PHP, and more
  • Reliable & scalable: Built for developers who need consistent results

Getting Started:

Get page content with AI analysis:

curl "https://api.webscraping.ai/ai/question?url=https://example.com&question=What is the main topic?&api_key=YOUR_API_KEY"

Extract structured data:

curl "https://api.webscraping.ai/ai/fields?url=https://example.com&fields[title]=Page title&fields[price]=Product price&api_key=YOUR_API_KEY"

Try in request builder

Related Questions

What are the common anti-bot detection techniques and how does Selenium WebDriver handle them?
How do I optimize Selenium WebDriver for large-scale web scraping operations?
How can I handle geolocation and permissions with Selenium WebDriver?

Get Started Now

WebScraping.AI provides rotating proxies, Chromium rendering and built-in HTML parser for web scraping
Icon