Can ChatGPT see my website? How to check in ten seconds (2026)
ChatGPT can see your website if two things are true: its crawlers are allowed to fetch your pages, and the words on those pages exist in the HTML your server sends, not only after JavaScript runs. OpenAI uses three crawlers with three jobs. GPTBot collects pages for training, OAI-SearchBot indexes pages so ChatGPT search can link to them, and ChatGPT-User fetches a page live when someone asks about it. Each one reads your robots.txt separately, and none of them executes JavaScript. The quickest check is the free Is it agentic? scan, which tests all of this in about ten seconds. The manual version takes five minutes and is below.
What "seeing" your site means
People ask this question meaning three different things, and the answer differs for each.
| What you want | Which crawler | What has to be true |
|---|---|---|
| ChatGPT can cite your page in an answer with a link | OAI-SearchBot | Allowed in robots.txt, page in the index, text readable without JavaScript |
| ChatGPT can open your page when a user pastes the link or asks about you | ChatGPT-User | Allowed in robots.txt, not blocked by a bot firewall or login wall |
| Your content is in the training data of future models | GPTBot | Allowed in robots.txt |
If you sell something, the first two rows are the ones that bring visitors. The third one is a policy decision that does not change whether ChatGPT can recommend you this month.
The five things that make a site invisible
- A Disallow line in robots.txt. The most common cause, and often unintentional: a rule written to block one training bot in 2023 that also blocks the search and user-fetch bots. Some hosting platforms and WordPress security plugins add these by default.
- A bot firewall. Cloudflare's AI bot blocking, Bot Fight Mode, and similar settings at other CDNs return a challenge page or a 403 to crawlers. Your robots.txt can say Allow while the firewall says no.
- Text that only appears after JavaScript runs. A joint study by Vercel and MERJ looked at more than 500 million GPTBot fetches and found no evidence of JavaScript execution. ClaudeBot downloaded script files in about a quarter of its requests and never ran them. If your product page is an empty shell filled in by a framework in the browser, the crawler reads the shell.
- A login, paywall or cookie wall in front of the content. Crawlers do not click Accept.
- No path to the page. A page linked from nowhere, missing from the sitemap and noindexed by mistake is a page no crawler finds.
How to check by hand
Step one: read your robots.txt. Open yoursite.com/robots.txt. Look for these user agents and for any Disallow: / under User-agent: *.
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
A rule only applies to the agent named above it. If none of the three is named, the wildcard rules apply to them. An Allow: / under each name removes all doubt.
Step two: view the source, not the page. In your browser, open your most important page and press View Source. Search the source for a sentence from the middle of the page. If it is there, the crawler can read it. If the source is a few hundred bytes of script tags and a root element, the crawler cannot.
Step three: fetch the page as the bot does. From a terminal:
curl -A "OAI-SearchBot" -I https://yoursite.com/
A 200 is good. A 403, a 503 or a redirect to a challenge page means a firewall is in the way.
Step four: read the server logs. If you can, filter the last month of access logs by user agent for the three names above. Pages the crawlers fetched are pages ChatGPT can talk about. If the list is empty and the checks above pass, the site may be too new or too unlinked to have been discovered yet.
The ten-second version
The Is it agentic? scan runs seven checks against your homepage: robots.txt rules for GPTBot, ClaudeBot, PerplexityBot and the other assistant crawlers, how many words are readable without JavaScript, the title and meta description, llms.txt, structured data, sitemap.xml and the main heading. It returns a score out of 100 with the fixes in order. It is free and needs no account. We built it, so weigh that. It does not read your server logs, so it tells you whether ChatGPT can see your site, not whether it has.
What to do if the answer is no
The fixes are in how to make your website readable by AI crawlers. In short: name the crawlers you want in robots.txt with an Allow, turn off AI bot blocking at your CDN for the crawlers you want, and make sure your pages arrive with their text in the HTML. Then give it a few weeks and look for utm_source=chatgpt.com in your analytics. How to track ChatGPT traffic to your website explains what those visits look like.
Frequently asked questions
Does blocking GPTBot stop ChatGPT from linking to my site?
No. GPTBot is the training crawler. Citations in ChatGPT search come from OAI-SearchBot, and live page reads come from ChatGPT-User. You can block training and still be cited, as long as the other two are allowed.
My robots.txt allows everything. Why does ChatGPT still not know my site?
Two usual reasons. A CDN or firewall setting is blocking the crawlers before robots.txt matters, or the page text only exists after JavaScript runs. View the page source and fetch it with a crawler user agent to tell which.
Can I see ChatGPT's crawlers in Google Analytics or Clerion?
No. Crawlers do not run JavaScript, so no browser-side analytics tool records them. Use server logs or your CDN's bot reports. Clicks from ChatGPT by real people do show up, tagged with utm_source=chatgpt.com.
How long after fixing robots.txt does ChatGPT see the site?
There is no published schedule. Crawlers commonly return within days once a page is linked and in the sitemap, but indexing for search citations can take longer. Keep the pages linked and in your sitemap so they are found.