Is it possible to scrape dynamic content with jsoup?

No, jsoup cannot scrape dynamic content that is loaded through JavaScript. Jsoup is a Java library for working with real-world HTML. It parses HTML to the same DOM as modern browsers do, but it does not execute JavaScript. It is designed to deal with static HTML content.

Dynamic content is typically loaded through AJAX or JavaScript after the initial page has been loaded. To scrape such content, you would need a tool that can execute JavaScript and mimic a browser environment.

One way to scrape dynamic content is by using a headless browser such as Puppeteer for Node.js, Selenium with a driver for Chrome or Firefox, or Playwright. These tools can programmatically control a web browser, allowing them to wait for JavaScript to execute and for content to load before scraping the resulting HTML.

Here's a simple example of scraping dynamic content using Selenium with Python:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Setup the webdriver for Chrome
driver = webdriver.Chrome(executable_path='/path/to/chromedriver')

# Open a page with dynamic content
driver.get('http://example.com/dynamic-content')

# Wait for the dynamic content to load
element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, 'dynamic-element'))
)

# Now you can parse the page_source with bs4 or just extract with Selenium
content = element.get_attribute('innerHTML')

# Do something with the content...
print(content)

# Clean up and close the browser
driver.quit()

For JavaScript, you could use Puppeteer to scrape dynamic content:

const puppeteer = require('puppeteer');

(async () => {
  // Launch a new browser session
  const browser = await puppeteer.launch();

  // Open a new page
  const page = await browser.newPage();

  // Go to the webpage with dynamic content
  await page.goto('http://example.com/dynamic-content');

  // Wait for a specific element to be loaded
  await page.waitForSelector('#dynamic-element');

  // Extract content from the page
  const content = await page.$eval('#dynamic-element', el => el.innerHTML);

  // Output the content
  console.log(content);

  // Close the browser
  await browser.close();
})();

Remember that scraping websites should be done responsibly and ethically. Always check the website's robots.txt file and terms of service to see if scraping is allowed, and do not overload their servers with frequent or concurrent requests.