Before diving into the details, let's explore the key benefits of Karpathy's method:

• Persistent Knowledge Base: Unlike stateless RAG systems, Karpathy's approach builds a lasting repository of information.
• Enhanced Cross-Document Synthesis: Facilitates comprehensive insights by connecting related concepts across documents.
• Contradiction Detection: Identifies and resolves conflicting information through a structured linting process.
• Scalable and Efficient: Optimizes knowledge management without the need for complex vector databases.

Retrieval-Augmented Generation (RAG) is a widely used method for enabling LLMs to access document data. It involves segmenting documents into chunks, embedding these into vectors, storing them in a database, and retrieving relevant chunks during queries. However, RAG has several limitations:

• Stateless Operation: Each query starts anew, forcing the LLM to rediscover knowledge repeatedly, which is inefficient.
• Limited Cross-Document Connections: Queries are confined to a few retrieved chunks, often missing broader insights.
• Inability to Detect Contradictions: Without a persistent layer, conflicting information can go unnoticed.
• Poor Scalability for Deep Analysis: Complex queries may require retrieving numerous chunks, often exceeding context limits.

Karpathy's technique addresses these issues by enabling the LLM to compile and maintain knowledge over time, creating a persistent, evolving knowledge base. This approach not only improves efficiency but also enhances the quality of insights derived from the data.

This layer contains the original documents, such as articles, papers, PDFs, and data files. These documents are read by the LLM but remain unaltered, providing an immutable source of truth and an audit trail.

raw/
├── articles/
│   ├── 2026-04-01-transformer-scaling.md
│   ├── 2026-03-28-llm-reasoning-survey.pdf
│   └── 2026-04-05-karpathy-interview.md
├── papers/
│   └── attention-is-all-you-need.pdf
└── data/
    └── benchmark-results-2026.csv

This layer is the LLM-generated knowledge base, consisting of structured markdown files. It includes summaries, entity pages, concept pages, comparisons, and synthesis documents. The LLM autonomously creates and updates this layer, ensuring it evolves with new information.

wiki/
├── index.md              # Content catalog with one-line summaries
├── log.md                # Chronological ingest log
├── concepts/
│   ├── attention-mechanism.md
│   ├── scaling-laws.md
│   └── chain-of-thought.md
├── entities/
│   ├── karpathy-andrej.md
│   └── openai.md
└── comparisons/
    └── rag-vs-compilation.md

The schema is a configuration document that guides the LLM in maintaining the wiki. It outlines the directory structure, naming conventions, page templates, and workflows, acting as a style guide and operating manual. This layer ensures the system is reusable across different sessions and topics.

```markdown

Wiki Schema

You are maintaining a personal knowledge base. Follow these rules exactly.

• raw/ — Immutable source documents. Never modify these.
• wiki/ — Your maintained knowledge layer. You own this entirely.

- wiki/index.md — Master catalog of all pages with one-line summaries

- wiki/log.md — Append-only chronological log of all ingests

- wiki/concepts/ — Concept pages (one per concept)

- wiki/entities/ — Entity pages (people, orgs, products)

- wiki/comparisons/ — Side-by-side comparisons

Every wiki page must include:

• Title as H1
• One-paragraph summary
• Key facts as bullet points
• Related pages as links
• Sources with citations to files in raw/

Ingest

When I say "ingest [source]":

1. Read the source document in raw/
2. Write a summary page in wiki/
3. Update or create relevant concept and entity pages
4. Add cross-references between related pages
5. Update wiki/index.md with the new page
6. Append an entry to wiki/log.md

Query

When I ask a question:

1. Search relevant wiki pages (use index.md for discovery)
2. Synthesize an answer with citations to wiki pages
3. If the answer reveals a gap, note it for future investigation

Lint

When I say "lint":

1. Check for contradictions between pages
2. Find orphan pages (not linked from anywhere)
3. Identify stale claims (based on source dates)
4. List pages with missing cross-references
5. Suggest investigation opportunities

```

Karpathy's approach fundamentally differs from RAG in several key aspects:

| Aspect | RAG (Retrieval) | LLM Wiki (Compilation) |

|--------|-----------------|------------------------|

| Knowledge State | Stateless — rediscovered each query | Persistent — accumulates over time |

| Cross-Document Synthesis | Limited to retrieved chunks | Built into wiki pages |

| Contradiction Detection | None | Lint workflow catches conflicts |

| Query Quality Over Time | Flat — same retrieval each time | Improves — wiki gets richer |

| Setup Complexity | Vector DB, embeddings, chunking | Markdown files, any LLM agent |

| Source Count Sweet Spot | Hundreds to thousands | ~100 sources producing ~400K words of wiki output |

The wiki acts as a persistent, compounding artifact, continuously enriching itself with each new source. While RAG remains effective for large-scale retrieval, the LLM wiki pattern excels in focused research domains where depth and synthesis are paramount.

Open your terminal and set up the three-layer structure:

mkdir -p ~/wiki/raw ~/wiki/wiki ~/wiki/wiki/concepts ~/wiki/wiki/entities ~/wiki/wiki/comparisons

This structure will look like:

~/wiki/
├── raw/          # Your immutable source documents
└── wiki/         # LLM-maintained knowledge layer
    ├── concepts/
    ├── entities/
    └── comparisons/

Create the schema file that instructs the LLM on maintaining your wiki. This is the cornerstone of your system.

Create ~/wiki/CLAUDE.md with the content provided in the previous section.

Save an article as a markdown file in your raw directory. Use tools like Obsidian's Web Clipper to convert web articles to markdown, or copy-paste the text manually.

For example, save an article as raw/my-first-article.md.

Then, open Claude Code (or any LLM agent with file access) in your wiki directory:

cd ~/wiki
claude

Instruct the agent to ingest the source:

Ingest raw/my-first-article.md

The LLM will process the article, create wiki pages, update the index, and log the ingest. Typically, a single source impacts 10-15 pages as the LLM generates concept pages, entity pages, and cross-references.

Pose a question about the ingested content:

What are the key takeaways from the article I just added?

The LLM will search your wiki pages and synthesize an answer grounded in the compiled knowledge. Unlike RAG, this response is based on structured wiki pages with existing cross-references and synthesis.

Try a more analytical question:

How does this relate to other concepts in the wiki?

As you add more sources, these cross-cutting questions become increasingly insightful due to the wiki's accumulated connections.

The Lint Workflow

As your wiki expands, periodically run lint passes to maintain quality. Simply instruct the agent:

Lint the wiki

The LLM will check for:

• Contradictions: Conflicting claims between pages.
• Orphan Pages: Pages not linked from anywhere.
• Stale Claims: Facts tied to outdated sources.
• Missing Cross-References: Related pages lacking links.
• Data Gaps: Topics mentioned but not covered by any wiki page.

Address issues interactively or batch them for later. Treat linting like code review to keep your knowledge base robust.

Adding Sources Over Time

The true strength of Karpathy's technique emerges over time. Each new source not only adds information but refines existing knowledge. When ingesting a new source, the LLM:

1. Reads the new document.
2. Compares it with existing wiki content.
3. Updates pages with new insights or stronger evidence.
4. Flags contradictions with previous sources.
5. Creates new pages for novel concepts.
6. Enhances cross-references across the knowledge network.

At moderate scale (~100 sources producing ~400K words of structured wiki output), simple index.md-based retrieval is effective. The LLM uses the master index to locate relevant pages without requiring vector embeddings.

Evolving the Schema

Your CLAUDE.md schema is dynamic. As your wiki grows, adapt it:

• Add New Page Categories: When patterns emerge that don't fit existing categories, introduce new directories and update the schema.
• Refine Workflows: If specific analyses are frequently requested, incorporate them as named workflows in the schema.
• Tighten Conventions: Codify successful page formats for your domain in the template section.

The schema's adaptability makes this technique transferable. To start a new wiki on a different topic, copy your CLAUDE.md, adjust the domain-specific sections, and the LLM will build the new wiki from scratch.

Andrej Karpathy's LLM knowledge base technique offers a powerful alternative to traditional RAG systems, providing a persistent, evolving repository of knowledge that enhances both efficiency and insight. By implementing this method, you can transform how you manage and synthesize information, leveraging the power of LLMs to create a dynamic, interconnected knowledge ecosystem. As you continue to add sources and refine your schema, your knowledge base will grow richer and more insightful, empowering you to stay ahead in the fast-paced world of artificial intelligence.