agent-docs

# Agent Docs Write documentation that AI agents can efficiently consume. Based on Vercel benchmarks and industry standards (AGENTS.md, llms.txt, CLAUDE.md). ## The Hybrid Context Hierarchy Three-layer architecture for optimal agent performance: ### Layer 1: Constitution (Inline) **Always in context.** 2,000–4,000 tokens max. ```markdown # AGENTS.md > Context: Next.js 16 | Tailwind | Supabase ## 🚨 CRITICAL - NO SECRETS in output - Use `app/` directory ONLY ## 📚 DOCS INDEX (use read_file) - Auth: `docs/auth/llms.txt` - DB: `docs/db/schema.md` ``` **Include:** - Security rules, architecture constraints - Build/test/lint commands (top for primacy bias) - Documentation map (where to find more) ### Layer 2: Reference Library (Local Retrieval) **Fetched on demand.** 1K–5K token chunks. - Framework-specific guides - Detailed style guides - API schemas ### Layer 3: Research Assistant (External) **Gated by allow-lists.** Edge cases only. - Latest library updates - Stack Overflow for obscure errors - Third-party llms.txt ## Why This Works **Vercel Benchmark (2026):** | Approach | Pass Rate | |----------|-----------| | Tool-based retrieval | 53% | | Retrieval + prompting | 79% | | **Inline AGENTS.md** | **100%** | **Root cause:** Meta-cognitive failure. Agents don't know what they don't know—they assume training data is sufficient. Inline docs bypass this entirely. ## Core Principles ### 1. Compressed Index > Full Docs An 8KB compressed index outperforms a 40KB full dump. **Compress to:** - File paths (where code lives) - Function signatures (names + types only) - Negative constraints ("Do NOT use X") ### 2. Structure for Chunking RAG systems split at headers. Each section must be self-contained: ```markdown ## Database Setup ← Chunk boundary Prerequisites: PostgreSQL 14+ 1. Create database... ``` **Rules:** - Front-load key info (chunkers truncate) - Descriptive headers (agents search by header text) ### 3. Inline Over Links Agents can't autonomously browse. Each link = tool call + latency + potential failure. | Approach | Token Load | Agent Success | |----------|------------|---------------| | Full inline | ~12K | ✅ High | | Links only | ~2K | ❌ Requires fetching | | Hybrid | ~4K base | ✅ Best of both | ### 4. The "Lost in the Middle" Problem LLMs have U-shaped attention: - **Strong:** Start of context (primacy) - **Strong:** End of context (recency) - **Weak:** Middle of context **Solution:** Put critical rules at TOP of AGENTS.md. Governance first, details later. ### 5. Signal-to-Noise Ratio Strip everything that isn't essential: - No "Welcome to..." preambles - No marketing text - No changelogs in core docs Formats like llms.txt and AGENTS.md mechanically increase SNR. ## llms.txt Standard Machine-readable doc index for agents: ```markdown # Project Name > One-line project description. ## Authentication - [Setup](docs/auth/setup.md): Environment vars and init - [Server](docs/auth/server.md): Cookie handling ## Database - [Schema](docs/db/schema.md): Full Prisma schema ``` **Location:** `/llms.txt` at domain root **Companion:** `/llms-full.txt` — full concatenated docs, HTML stripped ## Security Considerations ### Inline = Trusted AGENTS.md is part of your codebase. Controlled, version-pinned. ### External = Attack Surface - Indirect prompt injection via hidden text - SSRF risks if agents can browse freely - Dependency on external uptime **Mitigation:** Domain allow-lists, human-in-the-loop for external retrieval. ## Anti-Patterns 1. **Pasting 50 pages** — triggers "Lost in the Middle" 2. **"See external docs"** — agents can't browse autonomously 3. **Generic advice** — "Write clean code" (use specific constraints) 4. **TOC-only docs** — indexes without content 5. **Trusting retrieval alone** — 53% vs 100% pass rate ## Advanced Patterns For detailed guidance on RAG optimization, multi-framework docs, and API templates, see [references/advanced-patterns.md](references/advanced-patterns.md). ## Validation Checklist - [ ] Critical governance at TOP of doc - [ ] Total inline context under 4K tokens - [ ] Each H2 section self-contained - [ ] No external links without inline summary - [ ] Negative constraints explicit ("Do NOT...") - [ ] File paths and signatures, not full code