Skip to main content
← Browse GTM Playbooks

How to Optimize Your Website for AI and AEO

Overview​

This is an interview with Kevin White, a B2B tech marketing veteran who has led marketing at Segment, Retool, Common Room, and now Scrunch AI. The discussion focuses on transforming websites to serve both human visitors and AI bots effectively.

Key Themes​

The Shift in Marketing​

AI is fundamentally changing how customers research and evaluate products. Rather than visiting websites directly, prospects now use LLMs like ChatGPT, Claude, and Gemini to conduct research. As Kevin notes, "the world of creating white papers and ebooks...is so commoditized."

Bot Traffic as a Growth Channel​

Traffic Composition:

  • Currently, 10-20% of website traffic originates from AI bots
  • This percentage is projected to grow toward 50-90% as AI adoption increases
  • This traffic often represents genuine buying intent

"Bot traffic is real traffic" because users prompt AI assistants to retrieve specific information, meaning actual humans receive the results.

Retrieval Bots vs. Other Bot Types​

  • Retrieval bots: Perform real-time data retrieval in response to user prompts
  • Training bots: Crawl sites to train LLM models
  • Standard crawlers: Traditional search engine indexing

Practical Implementation Strategy​

1. Establish Baseline Measurements​

Begin by measuring AI-driven traffic through tools like Google Analytics or Scrunch AI to understand:

  • Volume of bot traffic
  • Which pages attract bot visits
  • Intent patterns

2. Structure Content for AI Consumption​

Technical optimizations:

  • Use Markdown and JSON formats
  • Implement schema markup for better parsing
  • Minimize reliance on heavy JavaScript (difficult for AI to parse)
  • Ensure pricing pages render clearly with aligned backend data

3. Create Parallel AI-Optimized Sites​

Maintain two versions:

  • Human-facing: Polished, visually appealing design
  • AI-focused: Clean formatting, well-structured data, FAQs, and machine-readable content

4. Measurement and Iteration​

Track prompts, citations, and conversion signals from retrieval bot traffic to optimize continuously.

Results and Cautions​

Performance Improvements​

One customer reportedly achieved "multi-fold gains" after optimization, though Kevin emphasizes results vary based on adoption and market conditions.

Warning Against AI Content Arbitrage​

"Avoid generic AI-content arbitrage—quickly generated pages without real value risk Panda-like penalties." Focus instead on authentic topics with credible data addressing genuine user problems.

Career Insights​

Kevin attributes his success to curiosity and a growth mindset. He advocates for hands-on work over premature management roles, particularly in flat AI startups where IC (individual contributor) roles remain valuable.

He cautions: "be careful what you wish for" regarding leadership tracks, noting that team management often reduces engagement with innovative work.

Call to Action​

  1. Start measuring AI traffic now
  2. Experiment with AI-friendly content formats
  3. Consider building an AI-optimized parallel site
  4. Focus on authentic, user-centric content rather than rapid scaling