# Vercel Knowledge Base - Canonical HTML: https://growth.iangoh.com/sources/vercel-knowledge-base-vercel-com/ - Original source: https://vercel.com/kb/guide/make-your-documentation-readable-by-ai-agents - Domain: vercel.com - Latest check: 2026-05-27 - Channels: AI Search, Content, SEO, Website ## Cited GrowthDex tactics - [Explicit AI-bot allowlist in robots.txt](/growth-ideas/explicit-ai-bot-allowlist-in-robots-txt/) - Name the major AI crawlers in `robots.txt` and explicitly allow them instead of relying on a generic wildcard and hoping the agent interprets it the way you intended. - [AI-agent auto-detected markdown fallback](/growth-ideas/ai-agent-auto-detected-markdown-fallback/) - Detect likely agent requests and return markdown automatically, even when the client does not explicitly ask for it. - [Content-negotiated markdown on canonical URLs](/growth-ideas/content-negotiated-markdown-on-canonical-urls/) - Serve markdown from the same canonical URL when an agent requests `Accept: text/markdown`, instead of forcing a separate docs-only experience. - [Markdown shadow routes for direct agent retrieval](/growth-ideas/markdown-shadow-routes-for-direct-agent-retrieval/) - Generate a `.md` version for every important content URL so agents, IDEs, and operators can fetch clean text without special headers. - [sitemap.md semantic discovery map](/growth-ideas/sitemap-md-semantic-discovery-map/) - Publish a Markdown sitemap with section labels and page descriptions so agents can understand the site structure before they start fetching individual pages. ## Related reading - [GrowthDex Blog](/blog/) - [All growth ideas](/growth-ideas/index.md)