AI crawlers are how content reaches AI models in the first place. A site that blocks GPTBot in robots.txt will not be cited inside ChatGPT, even if its content is otherwise excellent. The technical AEO checklist starts here: verify which AI crawlers your robots.txt allows, and confirm your site renders correctly when those crawlers fetch it.
AI visibility glossary
AI crawler
An AI crawler is a web crawler operated by an AI company, GPTBot, ClaudeBot, PerplexityBot, Google-Extended, that collects training data or real-time grounding for AI-generated answers.