How does Headless Chromium handle website cookies and local storage?

Headless Chromium handles website cookies and local storage identically to a regular GUI Chrome browser. It automatically receives, stores, and sends cookies while supporting JavaScript-based local storage operations. The key difference lies in data persistence across sessions.

Cookie Management in Headless Chromium

Automatic Cookie Handling

Headless Chromium processes cookies following standard HTTP protocols:

• Receiving Cookies: Automatically stores cookies from Set-Cookie headers
• Sending Cookies: Includes relevant cookies in subsequent requests to the same domain
• Cookie Policies: Respects domain, path, secure, and SameSite attributes
• Default Behavior: Accepts all cookies unless explicitly configured otherwise

Session vs. Persistent Cookies

Without User Data Directory: - All cookies are treated as session cookies - Data is lost when the browser instance closes - Each new session starts fresh with no existing cookies

With User Data Directory: - Persistent cookies are saved to disk - Session cookies are preserved during the browser lifetime - Cookies persist across multiple headless sessions

Cookie Security and Privacy

# Launch with enhanced cookie security
google-chrome --headless --disable-web-security=false --user-data-dir=/tmp/chrome-data

Local Storage Behavior

Local storage in headless Chromium works exactly like in regular Chrome:

• JavaScript Access: Full support for localStorage and sessionStorage APIs
• Domain Isolation: Storage is isolated per origin (protocol + domain + port)
• Capacity: Default 5-10MB limit per origin
• Persistence: Requires --user-data-dir flag for data to survive browser restarts

Storage Types Comparison

| Storage Type | Persistence | Capacity | Accessibility | |--------------|-------------|----------|---------------| | localStorage | Permanent* | ~5-10MB | Same origin only | | sessionStorage | Session only | ~5-10MB | Same origin + tab | | Cookies | Configurable | ~4KB each | Cross-request |

*Requires user data directory

Practical Implementation Examples

Python with Selenium WebDriver

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
import json
import os

class HeadlessBrowser:
    def __init__(self, user_data_dir=None):
        self.options = Options()
        self.options.add_argument("--headless=new")  # Use new headless mode
        self.options.add_argument("--no-sandbox")
        self.options.add_argument("--disable-dev-shm-usage")

        if user_data_dir:
            self.options.add_argument(f"--user-data-dir={user_data_dir}")

        self.driver = webdriver.Chrome(options=self.options)

    def manage_cookies(self, url):
        self.driver.get(url)

        # Add custom cookie
        self.driver.add_cookie({
            'name': 'session_id',
            'value': 'abc123',
            'domain': 'example.com',
            'path': '/',
            'secure': True,
            'httpOnly': False
        })

        # Get all cookies
        cookies = self.driver.get_cookies()
        print(f"Current cookies: {json.dumps(cookies, indent=2)}")

        # Delete specific cookie
        self.driver.delete_cookie('session_id')

        # Clear all cookies
        self.driver.delete_all_cookies()

    def manage_local_storage(self):
        # Set local storage data
        self.driver.execute_script("""
            localStorage.setItem('user_preferences', JSON.stringify({
                theme: 'dark',
                language: 'en'
            }));
        """)

        # Get local storage data
        preferences = self.driver.execute_script("""
            return JSON.parse(localStorage.getItem('user_preferences') || '{}');
        """)
        print(f"User preferences: {preferences}")

        # Get all local storage keys
        all_keys = self.driver.execute_script("""
            return Object.keys(localStorage);
        """)
        print(f"Local storage keys: {all_keys}")

        # Clear local storage
        self.driver.execute_script("localStorage.clear();")

    def close(self):
        self.driver.quit()

# Usage example
browser = HeadlessBrowser(user_data_dir="/tmp/chrome-profile")
browser.manage_cookies("https://example.com")
browser.manage_local_storage()
browser.close()

JavaScript with Puppeteer

const puppeteer = require('puppeteer');
const path = require('path');

class HeadlessBrowser {
    constructor(userDataDir = null) {
        this.userDataDir = userDataDir;
        this.browser = null;
        this.page = null;
    }

    async launch() {
        const launchOptions = {
            headless: 'new',
            args: [
                '--no-sandbox',
                '--disable-setuid-sandbox',
                '--disable-dev-shm-usage'
            ]
        };

        if (this.userDataDir) {
            launchOptions.userDataDir = this.userDataDir;
        }

        this.browser = await puppeteer.launch(launchOptions);
        this.page = await this.browser.newPage();
    }

    async manageCookies(url) {
        await this.page.goto(url);

        // Set cookies before navigation
        await this.page.setCookie(
            {
                name: 'auth_token',
                value: 'xyz789',
                domain: 'example.com',
                path: '/',
                httpOnly: true,
                secure: true,
                sameSite: 'Lax'
            },
            {
                name: 'user_id',
                value: '12345',
                domain: 'example.com'
            }
        );

        // Get all cookies for current page
        const cookies = await this.page.cookies();
        console.log('Current cookies:', cookies);

        // Get cookies for specific URL
        const urlCookies = await this.page.cookies('https://example.com');
        console.log('URL-specific cookies:', urlCookies);

        // Delete specific cookie
        await this.page.deleteCookie({name: 'user_id'});
    }

    async manageLocalStorage() {
        // Set multiple local storage items
        await this.page.evaluate(() => {
            localStorage.setItem('app_state', JSON.stringify({
                currentPage: 'dashboard',
                sidebar: 'collapsed'
            }));
            localStorage.setItem('api_key', 'secret-key-123');
            localStorage.setItem('last_login', new Date().toISOString());
        });

        // Get local storage contents
        const localStorageData = await this.page.evaluate(() => {
            const data = {};
            for (let i = 0; i < localStorage.length; i++) {
                const key = localStorage.key(i);
                data[key] = localStorage.getItem(key);
            }
            return data;
        });
        console.log('Local storage data:', localStorageData);

        // Check if specific key exists
        const hasApiKey = await this.page.evaluate(() => {
            return localStorage.getItem('api_key') !== null;
        });
        console.log('Has API key:', hasApiKey);

        // Remove specific item
        await this.page.evaluate(() => {
            localStorage.removeItem('last_login');
        });

        // Get storage quota info
        const storageEstimate = await this.page.evaluate(async () => {
            if ('storage' in navigator && 'estimate' in navigator.storage) {
                return await navigator.storage.estimate();
            }
            return null;
        });
        console.log('Storage estimate:', storageEstimate);
    }

    async close() {
        if (this.browser) {
            await this.browser.close();
        }
    }
}

// Usage example
(async () => {
    const browser = new HeadlessBrowser('/tmp/puppeteer-profile');
    await browser.launch();
    await browser.manageCookies('https://example.com');
    await browser.manageLocalStorage();
    await browser.close();
})();

Direct Command Line Usage

# Launch with persistent user data
google-chrome --headless --user-data-dir=/tmp/chrome-data --remote-debugging-port=9222

# Launch with specific profile
google-chrome --headless --profile-directory="Profile 1" --user-data-dir=/home/user/.config/google-chrome

# Launch in incognito mode (no persistence)
google-chrome --headless --incognito

Data Persistence Best Practices

1. Choose Appropriate Storage Location

import tempfile
import os

# Temporary directory (cleaned up automatically)
temp_dir = tempfile.mkdtemp(prefix='chrome_profile_')

# Persistent directory
persistent_dir = os.path.expanduser('~/.chrome_headless_profiles/my_app')
os.makedirs(persistent_dir, exist_ok=True)

2. Handle Profile Cleanup

const fs = require('fs').promises;

class ProfileManager {
    static async createTempProfile() {
        const tempDir = await fs.mkdtemp('/tmp/chrome-');
        return tempDir;
    }

    static async cleanupProfile(profilePath) {
        try {
            await fs.rmdir(profilePath, { recursive: true });
            console.log(`Cleaned up profile: ${profilePath}`);
        } catch (error) {
            console.error(`Failed to cleanup profile: ${error.message}`);
        }
    }
}

3. Session Management Patterns

class SessionManager:
    def __init__(self, base_profile_dir):
        self.base_profile_dir = base_profile_dir
        self.active_sessions = {}

    def create_session(self, session_id):
        profile_dir = os.path.join(self.base_profile_dir, session_id)
        os.makedirs(profile_dir, exist_ok=True)

        options = Options()
        options.add_argument("--headless=new")
        options.add_argument(f"--user-data-dir={profile_dir}")

        driver = webdriver.Chrome(options=options)
        self.active_sessions[session_id] = {
            'driver': driver,
            'profile_dir': profile_dir
        }
        return driver

    def get_session(self, session_id):
        return self.active_sessions.get(session_id, {}).get('driver')

    def close_session(self, session_id):
        if session_id in self.active_sessions:
            self.active_sessions[session_id]['driver'].quit()
            del self.active_sessions[session_id]

Common Troubleshooting

Storage Not Persisting

• Ensure --user-data-dir is specified and writable
• Check that the directory has sufficient disk space
• Verify permissions on the profile directory

Cookies Not Working

• Confirm the domain and path settings are correct
• Check if the site requires secure cookies (HTTPS)
• Verify SameSite attribute compatibility

Local Storage Quota Exceeded

// Check available storage
const checkStorageQuota = async (page) => {
    return await page.evaluate(async () => {
        if ('storage' in navigator && 'estimate' in navigator.storage) {
            const estimate = await navigator.storage.estimate();
            return {
                quota: estimate.quota,
                usage: estimate.usage,
                available: estimate.quota - estimate.usage
            };
        }
        return null;
    });
};

The key to successful cookie and local storage management in headless Chromium is proper configuration of the user data directory and understanding the persistence model. Without this configuration, each session starts fresh, which may or may not be the desired behavior depending on your use case.