How do I handle cookies while using MechanicalSoup?

MechanicalSoup is a Python library for automating interaction with websites. It provides a simple API for navigating and submitting forms, akin to that of a web browser. Handling cookies is essential when interacting with websites that require sessions or user authentication.

When you use MechanicalSoup, cookie handling is done automatically in the background by the http.cookiejar module. MechanicalSoup uses a StatefulBrowser object, which maintains the state across the requests, including cookies.

Here's an example of how to use MechanicalSoup with cookies:

import mechanicalsoup

# Create a browser object. This will hold cookies, among other things.
browser = mechanicalsoup.StatefulBrowser()

# Go to the example website
browser.open("http://example.com")

# Cookies are automatically handled. Any cookies set by the server will be stored in the
# browser session and sent with subsequent requests.

# You can inspect the cookies if you need to
cookies = browser.get_cookiejar()
print(cookies)

# Perform actions like submitting forms, clicking links, etc.
# The browser will continue to handle cookies for you.

Sometimes, you may need to manually add or modify cookies. This can be done by directly interacting with the cookie jar:

# Add a new cookie to the jar
browser.set_cookie('cookie_name', 'cookie_value', domain='example.com')

# You can also set a cookie directly from a dictionary
cookie_dict = {'name': 'cookie_name', 'value': 'cookie_value', 'domain': 'example.com'}
browser.get_cookiejar().set_cookie(cookie_dict)

# Now, when you make a new request, MechanicalSoup will send the custom cookie along
browser.open("http://example.com/another_page")

If you need to clear the cookies, you can do it like this:

# Clear all cookies
browser.get_cookiejar().clear()

# Clear cookies for a specific domain
browser.get_cookiejar().clear(domain='example.com')

# Clear a specific cookie by name
browser.get_cookiejar().clear(name='cookie_name', domain='example.com')

Remember that when using MechanicalSoup or any other web scraping tool, you should always comply with the website's terms of service and use the tool responsibly to avoid overloading the website with requests.