DiscoverMyCompany.ai

Glossary

Glossary

AI crawler: definition

An AI crawler is a program that an AI company sends to read web pages, either to train its models or to answer a question.

Why does it matter for a company?

Tests published by Vercel found that the crawlers of OpenAI and Anthropic do not execute JavaScript. A site must therefore put its content in the HTML it sends.

Where are the other terms?

All the terms are in the glossary.

Sources

Author: Franck Rosset · Published · Updated