AI crawler: definition
An AI crawler is a program that an AI company sends to read web pages, either to train its models or to answer a question.
Why does it matter for a company?
Tests published by Vercel found that the crawlers of OpenAI and Anthropic do not execute JavaScript. A site must therefore put its content in the HTML it sends.
Where are the other terms?
All the terms are in the glossary.