AI Crawler Robots.txt Audit
Analyzes a site's robots.txt specifically for AI crawler access policies.
Complements /seo-technical (which does a broad robots.txt check) with
deep AI-specific analysis.
@skills/seo/references/ai-crawlers-guide.md
AI Crawler Registry
| Bot Name | Owner | Purpose |
|---|---|---|
| GPTBot | OpenAI | Training data + ChatGPT web search |
| OAI-SearchBot | OpenAI | ChatGPT search only (not training) |
| ChatGPT-User | OpenAI | ChatGPT browsing (real-time) |
| C |
[Description truncada. Veja o README completo no GitHub.]