Glossary

GPTBot

GPTBot is OpenAI's web crawler for gathering training data for its models. It is distinct from ChatGPT-User, the agent that fetches pages live when ChatGPT answers a question with web search. A site can allow one and block the other in robots.txt, and the two choices have very different consequences.

Blocking GPTBot keeps your content out of future training runs — a legitimate editorial choice with little effect on whether ChatGPT recommends your business today. Blocking ChatGPT-User is the consequential one: it prevents the assistant from reading your site at answer time, so when a buyer asks who to hire, your own pages cannot serve as evidence.

OpenAI operates three agents with three jobs, and the names are close enough to cause exactly the mistake described above. GPTBot collects training data. ChatGPT-User fetches a specific page because a user's question required it. OAI-SearchBot builds the index that ChatGPT's search feature queries. Blocking the last two removes a business from the channel; blocking the first only removes it from future model memory.

Publishers with genuinely valuable archives have a real argument for blocking training. A local service business almost never does. Your service page is not being monetised by appearing in a training corpus, and the reputational asset you are protecting — the description of what you do and where — is precisely the thing you want repeated. The calculus that makes sense for a newspaper does not transfer to a roofing company.

Checking is a thirty-second job most businesses have never done. Request your own robots.txt and read the user-agent blocks. Then verify at the server layer, because robots.txt is only half the story: a Cloudflare bot-fighting rule or a security plugin can return 403 to these agents regardless of what robots.txt permits, and that block is invisible in the file everyone inspects.

Many businesses block all three without realizing they are different, usually via a CMS toggle or a copied robots.txt. Checking takes seconds with the AI readability checker, and AI crawler covers the equivalent agents from Anthropic, Perplexity, and Google.

Related terms

See where your business stands.

The free AI visibility report shows what ChatGPT, Gemini, and Perplexity say when buyers ask about your market — with the sources behind every answer.

No charge and no call required — the report arrives by email.