Technical Implementation of LLM Wiki: A Cloud-Native Knowledge Base

I. Introduction: Karpathy's LLM Wiki Philosophy

"Obsidian is the IDE, the LLM is the programmer, and the wiki is the codebase." — Karpathy's core metaphor captures a radical rethinking of how humans and AI should divide knowledge work.

Traditional RAG asks the LLM to "retrieve"—pose a question, then on-the-fly pull fragments from documents and stitch an answer. Karpathy calls this equivalent to re-reading all source code every time you run the program. Instead, he argues: you compile once and amortize the cost across the entire lifecycle. Knowledge should work the same way.

Three-Layer Structure

Two auxiliary files make the system inspectable: index.md (a one-line-per-page directory) and log.md (an append-only log of operations).

Three Operations

• Ingest: Read a new source → create summary page → update index → incidentally update 10–15 related pages → auto-generate cross-references.
• Query: The result of answering a question is written back as a new page—every question grows the system, compounding.
• Lint: Periodically check for contradictions, stale claims, orphan pages, and gaps.

Karpathy traces the intellectual lineage to Vannevar Bush's 1945 essay As We May Think and the Memex—the hypothetical "knowledge machine." The Memex failed because manual cross-referencing is unsustainable; LLM Wiki reduces maintenance cost to nearly zero, making Memex viable for the first time in 80 years.

But this philosophy carries two implicit assumptions: single-machine, single-user. When a product must become a cloud-based multi-tenant service, every element of the local convention must be redesigned. This article is about that redesign.

II. System Overview: From Local Conventions to Cloud Services

Our llm-wiki is a plugin-style capability of the 2Ryun platform—a full-stack SaaS for document management, knowledge base, and site building. Frontend in Nuxt, backend in Express + Mongoose, AI layer orchestrated by LangGraph via DeerFlow.

Document Library (raw sources)
   │  Save/setting change → notify-update
   ▼
wiki-service (:8002) ──calls──> DeerFlow (:8001, LangGraph + LLM)
   │  llm-wiki-extract / llm-wiki-organize / llm-wiki-maintain
   ▼
MongoDB (wiki_entries / wiki_links / wiki_clusters / insights / operation_log)
   │
   ├── Semantic retrieval (embedding cosine similarity + keyword fallback)
   └── Graph builder (nodes / edges / clusters)
        │
        ▼
Frontend plugin @dave/llm-wiki (WikiMain / WikiGraph / WikiEntryCard / Insights)

Local → Cloud Mapping

Key difference: The local convention relies on humans "manually dropping new articles into /raw and letting the Agent process them"; the cloud relies on automatic triggering on change—the moment a document is saved, the backend asynchronously notifies wiki-service to extract. This is the server-side manifestation of "reducing maintenance cost to zero."

III. Extraction Pipeline: How the LLM "Compiles" Documents

3.1 Triggering: Extract-on-Change

After a document is saved, the main backend calls wiki-service's notify-update. Extraction is asynchronous—the user gets an immediate response, and knowledge "grows" in the background. Each extraction carries deduplication semantics: skip_if_extracted ensures idempotency.

3.2 DeerFlow: Three Skills for Three Operations

wiki-service never touches the LLM directly; it calls DeerFlow's stateless run interface via HTTP. Three skills map to Karpathy's three operations:

llm-wiki-extract   → Ingest: extract entries from document content
llm-wiki-organize  → Lint: merge, link, cluster, flag stale, generate insights
llm-wiki-maintain  → Incrementally update existing entries when new evidence appears

This decoupling is critical: wiki-service doesn't care which LLM model is used or how the prompt is assembled—it only sends content and receives structured JSON. Swapping models has zero impact on the storage layer.

3.3 Extraction Prompt: "Trustworthy" Content by Design

Extraction is not "throw the document at the LLM and let it summarize"—it is structured extraction with strict constraints. The prompt enforces three pillars of trustworthiness:

1. Content comes only from the source + text_spans provenance. Every piece of content is constrained to "what the document explicitly states," with source snippets as evidence. This directly blocks hallucination contamination at the source.
2. Confidence tiering. confidence_tier distinguishes three types: extracted (0.9+), inferred (0.7–0.8), and ambiguous (0.5–0.6). Karpathy's local convention relied on human eyes; the cloud version internalizes these judgments as first-class fields.
3. Separation of enrichments from content. The LLM's out-of-document knowledge—background, corrections, timeliness notes—goes into a separate enrichments array as pending-review supplements, never written into the content body. This is the clear line between "LLM as compiler" and "LLM as creator."

These three pillars crystallize Karpathy's schema (conventions in agents.md) into hard rules in the prompt and fields in the data model.

IV. Organization and Linking: Letting Knowledge Grow Structure

Extraction alone produces a flat pile of entries. The "meaning" of a knowledge base lies in relationships—this is precisely where Memex died. This project hands "cross-referencing" to the LLM and vectors.

4.1 Organize: Deduplication, Clustering, Stale Detection

A single run of llm-wiki-organize does four things:

• Duplicate merge: Same entity extracted multiple times → canonical selection + intelligent content merge.
• Clustering: Community detection produces wiki_clusters, each with a central entry and member list—the "skeleton" of the knowledge graph.
• Stale flagging: Detects outdated entries and tags them stale—Karpathy's Lint operation.
• Insight generation: At cluster/root level, the LLM distills patterns, themes, and gaps—insights. A layer of synthesis above "compilation."

4.2 Three Types of Edges

This provenance-aware three-type edge system goes further than Karpathy's [[links]]: in the cloud, structured relationship metadata allows the graph to be directly rendered, filtered, and colored by type.

4.3 Graph

graph_builder assembles entries + links + clusters into GraphData (nodes / edges / clusters / insights), directly consumable by the frontend. Each node carries category, confidence_tier, stale; each edge carries link_source, link_type, strength. The frontend builds 2D force-directed and 3D spherical views on top of this.

V. Retrieval: Semantic Search

In Karpathy's local solution, the Agent reads index.md first. At cloud scale, "stuffing the directory into context" is not feasible. This project uses embeddings for semantic retrieval:

async def search_entries(self, user_id, query, max_results=10):
    # Semantic-first: vectorize query, compute cosine similarity
    query_vec = await compute_embedding(query)
    if query_vec:
        results = await self._semantic_search(user_id, query_vec, max_results)
        if results:
            return results
    # Fallback: keyword full-text search
    return await self._keyword_search(user_id, query, max_results)

Embeddings default to Qwen text-embedding-v3 (OpenAI-compatible), with graceful degradation to keyword vectors and full-text search. This is the cloud-scale replacement for index.md.

VI. Frontend and Quality Guardrails

The frontend is a plugin (@dave/llm-wiki), deeply integrated with the document editor:

• WikiSidebar: Document sidebar showing extraction status with one-click initialization and re-extraction.
• WikiMain: Knowledge base main interface with Google-style search + entries / graph tabs.
• WikiEntryCard: Entry detail view with confidence badges, category, tags, sources, relations, and pending enrichments.
• WikiGraph: 2D/3D knowledge graph, colored by type/degree/confidence, with neighbor highlighting.
• InsightsPanel / PendingEnrichmentsList: Insight list + human review queue for AI-generated supplements.

Quality guardrails run through the entire chain: confidence badges make "is this from the source or inferred?" visible at a glance; text_spans let every claim click back to the source; AI supplements always go to pending review, never auto-written into content. Together, these answer the most critical question: "On what basis should I trust what's in here?"

VII. Validation and Reflection

Operations Mapping

What cloud-ification solved: "Near-zero maintenance cost" went from human discipline to system automation—documents are extracted the moment they're saved, the graph is viewable in real time, multi-tenant data is naturally isolated via user_id, and embedding indices are generated at ingestion.

What was sacrificed: Karpathy's output is pure markdown—humans can read, edit, and diff it directly. After moving to the cloud, knowledge becomes entries + vectors in a database; humans lose direct editability of the compilation result. Controllability shifts to "how well the extraction prompts are written." The bottleneck is those three skill prompts and the JSON schema they produce.

Limitations (Shared with Karpathy's Critics)

• Error compounding: A false "fact" at extraction propagates through organize, insights, and graph—and the checker is the same fallible LLM. Mitigation is confidence tiering and pending-review, but no fundamental cure.
• Scale: Still follows "extract a batch → manage entries" model, not the long-term compounding organism Karpathy envisions. At true million-scale, graph and retrieval need new engineering.
• "Compilation" ≠ understanding: A well-maintained knowledge base doesn't mean the user has actually internalized the knowledge.

VIII. Conclusion

Karpathy's LLM Wiki transforms "knowledge maintenance" from labor into a convention: humans do curation (deciding what to read, what to ask), LLMs do compilation (building pages, cross-referencing, linting), and the output is a readable, traceable, compounding body of knowledge.

This project's llm-wiki brings that convention to the cloud: manual ingest → automatic triggering, free-form markdown → structured entries + provenance + confidence, index.md → embedding retrieval, hand-written links → a provenance-aware graph.