User-agent: *
Disallow: /api/
Disallow: /api$
Disallow: /api?
Disallow: /description.md
Disallow: /email-confirmation
Sitemap: https://webscraping.ai/sitemap.xml.gz