What ChatGPT Can (and Cannot) See on Your Website: A Comprehensive Audit

Technical Research

What ChatGPT can see on your website. And what it can't.

We tested five controlled sites to see what ChatGPT could retrieve, miss, or misread. The short version: static HTML and explicit links win; dynamic, image-only, or vague content often disappears.

Ben Tannenbaum · January 10, 2025 · 15 min read

We built five controlled test websites to check what ChatGPT could retrieve from static pages, links, JavaScript-driven sections, images, and hidden text. The highest-value finding was blunt: content in static HTML with descriptive links was retrievable; content that depended on user interaction, vague link text, or images was much easier to miss.

With the rise of AI-powered search and retrieval systems like ChatGPT's Search and Bing-powered search features, the way users discover and interact with web content is fundamentally changing. Instead of browsing through search results and visiting websites directly, users increasingly ask AI assistants to find and summarize information for them.

Why This Matters

If ChatGPT can't "see" certain content on your website, that content effectively becomes invisible to users who rely on AI assistants to find information. Understanding these limitations is crucial for ensuring your content remains discoverable in an AI-first world.

Critical Questions We Set Out to Answer

Can AI systems like ChatGPT effectively "see" and retrieve all the content on my website?

What types of content structures or technical implementations might prevent AI systems from accessing my content?

How can I optimize my website to ensure it's fully accessible to AI retrieval systems?

Research Methodology

We conducted a methodical audit of ChatGPT's web retrieval capabilities across a series of controlled test websites with varying levels of complexity:

  1. Built five test websites with controlled content structures
  2. Each site contained both general content and specific "marker content" with unique identifiers
  3. Deployed all sites using Vercel connected to GitHub repositories
  4. Tested ChatGPT's ability to retrieve specific information using "Browse with Bing" feature
  5. Documented results to understand what ChatGPT could and couldn't access

🧪 Test Sites Overview

Site 1: Text-Only Website

Purpose: Baseline test for simple content retrieval
Description: Basic static HTML with plain text content to establish retrieval baseline

Site 2: Blog Article Website

Purpose: Test link following with generic titles
Description: Main page with links to blog articles using non-descriptive titles

Site 3: Blog Title Hint Website

Purpose: Test link following with descriptive titles
Description: Blog articles with explicit, keyword-rich titles in link text

Site 4: Blog Content Hint Website

Purpose: Test partial title matching
Description: Articles with titles containing partial query matches

Site 5: JavaScript-Hidden Content

Purpose: Test JavaScript execution capabilities
Description: Content hidden in tabs, accordions, and dynamically loaded sections

Site 6: Image-Based Content (OCR)

Purpose: Test text extraction from images
Description: Critical information embedded only in image files

Site 7: White-on-White Text

Purpose: Test HTML parsing vs visual rendering
Description: Text hidden with CSS styling but present in HTML

Key Findings

Static Content Accessibility

High Impact
ChatGPT reliably retrieves static HTML content on main pages Action Required: Ensure critical information is in static HTML on primary pages

Link Following Behavior

Critical Impact
Only follows links with explicit, relevant titles - ignores generic titles Action Required: Use descriptive, keyword-rich anchor text for internal links

JavaScript Limitations

High Impact
Cannot execute JavaScript for dynamic content loading Action Required: Make critical content available without user interactions

Search Engine Workaround for JavaScript Content

Medium Impact
OpenAI uses Bing and Google searches as a workaround to access JavaScript-rendered pages that have been indexed by search engines Action Required: Ensure JavaScript pages are indexed by Bing and Google; move critical content (especially FAQ answers) above the JavaScript fold

CSS vs HTML Processing

Medium Impact
Processes HTML content regardless of CSS styling/visibility Action Required: Text in HTML is accessible even when visually hidden

Image Text Extraction

High Impact
Cannot extract text from images, may hallucinate responses Action Required: Provide text alternatives for image-based content

6 Key Actions to Optimize Your Website for AI Retrieval

1. Use Explicit, Descriptive Link Text

Critical
Ensure links to important content contain explicit keywords that clearly indicate what information can be found on the linked page

Implementation:

2. Optimize Page Titles

High
Explicit page titles significantly improve content discovery, while vague titles result in content being missed entirely

Implementation:

3. Keep Critical Content in Static HTML

High
ChatGPT can access hidden content in tabs and accordions, but content loaded dynamically after user interactions is invisible

Implementation:

4. Place Important Information on Main Pages

Medium
Content on main pages is reliably accessed, while secondary pages are only discovered under specific conditions

Implementation:

5. Avoid Image-Only Content for Critical Information

High
ChatGPT cannot reliably extract text from images, while it processes HTML content regardless of styling

Implementation:

6. Ensure Search Engine Indexing

Medium
Even JavaScript-dependent content can be discoverable if it's properly indexed by search engines

Implementation:

Conclusion: Preparing for an AI-First World

Our comprehensive audit of ChatGPT's web retrieval capabilities has revealed both strengths and limitations in how AI systems access and retrieve web content. While ChatGPT can successfully retrieve static content and execute JavaScript to access hidden content in tabs and accordions, it struggles with following links without explicit relevance signals and cannot access content that requires user interactions to load dynamically.

These findings have significant implications for website owners, content creators, and marketers who want to ensure their content remains discoverable in an AI-first world. By implementing the actionable insights from our research, you can optimize your website for AI retrieval and ensure that your valuable content doesn't become invisible to users who rely on AI assistants to find information.

Final Thought

The rise of AI assistants represents a fundamental shift in how users discover and consume web content. Just as websites had to adapt to mobile devices and search engine algorithms in the past, they must now adapt to AI retrieval systems to remain visible and relevant in the evolving digital landscape.