# # AI Scrapers Rules # # Block Open AI User-agent: GPTBot Disallow: / # Block Google (Gemini) User-agent: Google-Extended Disallow: / # Block Claude User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Claudebot Disallow: / # Block CommonCrawl User-agent: CCBot Disallow: / # Block Diffbot User-agent: Diffbot Disallow: / # Block Meta (Facebook) # User-agent: FacebookBot # Disallow: / # Block ByteDance User-agent: Bytespider Disallow: / # Block Webz.io User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / # Block ImagesiftBot User-agent: ImagesiftBot Disallow: / # Block Meltwater User-agent: Meltwater Disallow: / # Block Seekr User-agent: Seekr Disallow: / # # AI Assistants & AI Crawler Rules # # Block Open AI User-agent: ChatGPT-User Disallow: / # Block Perplexity User-agent: PerplexityBot Disallow: / # Block Amazon User-agent: Amazonbot Disallow: / # Block Apple # User-agent: Applebot # Disallow: / # Block Cohere User-agent: cohere-ai Disallow: / # Block You.com User-agent: YouBot Disallow: / # # Others # # Ahrefs User-agent: AhrefsBot Disallow: /