User-Agent: * Allow: / Disallow: /dashboard Disallow: /api/ Disallow: /portal Disallow: /admin Disallow: /internal Disallow: /auth Disallow: /top/ User-Agent: GPTBot User-Agent: ChatGPT-User User-Agent: Google-Extended User-Agent: PerplexityBot User-Agent: ClaudeBot User-Agent: Applebot-Extended Allow: / Allow: /llms.txt Allow: /llms-full.txt Sitemap: https://simporter.com/sitemap.xml Sitemap: https://simporter.com/ai-picks/sitemap.xml