tonybark.com/static/robots.txt

24 lines
560 B
Text
Raw Normal View History

2024-12-07 15:49:29 -05:00
User-Agent: GPTBot
User-Agent: ChatGPT-User
User-Agent: ClaudeBot
User-Agent: anthropic-ai
User-Agent: CCBot
User-Agent: FacebookBot
User-Agent: Twitterbot
User-Agent: PerplexityBot
User-Agent: OmigiliBot
User-Agent: AISearchBot
User-Agent: mlbot
User-Agent: PetalBot
User-Agent: Amazonbot
User-Agent: DuckAssistBot
User-Agent: OAI-SearchBot
User-Agent: OAI-SearchBot
User-Agent: omgili
User-Agent: omgilibot
User-Agent: PerplexityBot
User-Agent: Webzio-Extended
User-Agent: Meta-ExternalAgent
User-Agent: Meta-ExternalFetcher
User-Agent: cohere-ai
Disallow: /