# Technical Implementation of LLM Wiki: A Cloud-Native Knowledge Base ### — From Karpathy's Vision to Multi-Tenant Services > In April 2026, Andrej Karpathy published a pattern he called "LLM Wiki" (or "LLM Knowledge Bases"). It is not a piece of software, but a set of **conventions** for how humans and LLM agents should divide the labor of maintaining knowledge. This article first explains the philosophy, then uses our own cloud implementation (llm-wiki, deployed on the multi-tenant SaaS platform 2Ryun) as the main thread to show what engineering challenges arise when upgrading from a "local markdown folder" to a "cloud-native multi-tenant knowledge base service"—and how we solved them. --- ## I. Introduction: Karpathy's LLM Wiki Philosophy Karpathy's core metaphor in one sentence: **"Obsidian is the IDE, the LLM is the programmer, and the wiki is the codebase."** Traditionally, we ask LLMs to "retrieve" knowledge—pose a question, then on-the-fly pull relevant fragments from a pile of documents and stitch together an answer (i.e., RAG). Karpathy argues this is equivalent to **re-reading all source code every time you run the program**. He says: you don't re-read all your code before every run; instead, you **compile once** and amortize the cost across the entire lifecycle. Knowledge should work the same way: **at ingestion time, compile messy source material into a structured, cross-linked wiki; then retrieve on demand, rather than computing from scratch every query.** This pattern has a three-layer structure and three operations: **Three-Layer Structure** | | | Layer | | | | | Content | | | | | Who Writes | | | | | | `/raw` | | | | | Immutable source documents, articles, papers | | | | | Humans (read-only; Agent never modifies) | | | | | | `/wiki` | | | | | LLM-maintained markdown pages: entity pages, concept pages, synthesis pages | | | | | Agent only | | | | | | Schema (e.g., | `agents.md` | / | `CLAUDE.md` | ) | | | | | Natural-language conventions for structure, norms, workflows | | | | | Human-designed | | | Two auxiliary files make the system inspectable: `index.md` (a one-line-per-page directory that the Agent reads first) and `log.md` (an append-only log of ingest/query/lint operations). **Three Operations** - **Ingest**: Read a new source → create summary page → update index → incidentally update 10-15 related pages → auto-generate cross-references. - **Query**: The result of answering a question is **written back** as a new page—every question grows the system, compounding. - **Lint**: Periodically check for contradictions, stale claims, orphan pages, and gaps. Karpathy explicitly traces the intellectual lineage to Vannevar Bush's 1945 essay *As We May Think* and the **Memex**—the hypothetical "knowledge machine." The Memex failed because **manual cross-referencing is unsustainable**; LLM Wiki reduces maintenance cost to nearly zero, making Memex viable for the first time in 80 years. This philosophy is beautiful, but it carries two implicit assumptions: **single-machine, single-user**. It assumes one markdown folder, one Obsidian vault, one Agent. When a product must become a **cloud-based multi-tenant service**—multiple users, shared backend, documents changing in real time, real-time graph requirements—every element of the local convention must be redesigned. This article is precisely about the engineering practice of **productizing the idea of "LLM as compiler."** ## II. System Overview: From Local Conventions to Cloud Services Our llm-wiki is a plugin-style capability of the 2Ryun platform. 2Ryun is a full-stack SaaS for document management + knowledge base + site building: frontend in Nuxt, backend in Express + Mongoose, AI layer is a LangGraph-orchestrated DeerFlow service. All knowledge extraction in llm-wiki goes through DeerFlow. ``` Document Library (raw sources) │ Save/setting change → notify-update ▼ wiki-service (:8002) ──calls──> DeerFlow (:8001, LangGraph + LLM) │ llm-wiki-extract / llm-wiki-organize / llm-wiki-maintain ▼ MongoDB (wiki_entries / wiki_links / wiki_clusters / insights / operation_log) │ ├── Semantic retrieval (embedding cosine similarity + keyword fallback) └── Graph builder (nodes / edges / clusters) │ ▼ Frontend plugin @dave/llm-wiki (WikiMain / WikiGraph / WikiEntryCard / Insights) ``` Translating Karpathy's local conventions to cloud services, one-to-one: | | | Karpathy's Local Convention | | | | | Cloud Service Equivalent | | | | | | `/raw` | source documents | | | | | Main platform's document library (immutable source) | | | | | | `/wiki` | markdown pages | | | | | `wiki_entries` | collection (structured entries with provenance and confidence) | | | | | | Schema / | `agents.md` | | | | | Extract & organize | **skill prompts** | (sole source of structure, classification, quality rules) | | | | | | `index.md` | directory | | | | | Semantic retrieval (embedding + cosine similarity) | | | | | | `log.md` | ledger | | | | | `operation_log` | (every create/merge/link/stale is recorded) | | | | | | Ingest / Query / Lint | | | | | `llm-wiki-extract` | / semantic search / | `llm-wiki-organize` | | | **Key difference**: The local convention relies on humans "manually dropping new articles into /raw and letting the Agent process them"; the cloud relies on **automatic triggering on change**—the moment a document is saved, the backend asynchronously notifies wiki-service to extract. This is the server-side manifestation of "reducing maintenance cost to zero": it's not a person remembering to ingest, but the system perceiving it itself. ## III. Extraction Pipeline: How the LLM "Compiles" Documents ### 3.1 Triggering: Extract-on-Change After a document is saved in the editor, the main backend calls wiki-service's `notify-update`. Extraction is **asynchronous**—the document-saving request never blocks on the LLM call; the user gets an immediate response after saving, and knowledge "grows" in the background. Each extraction also has deduplication semantics: `skip_if_extracted` skips already-processed documents, making it idempotent. ### 3.2 DeerFlow: Three Skills for Three Operations wiki-service itself never touches the LLM directly; it calls DeerFlow's **stateless run interface** via HTTP, and DeerFlow (orchestrated by LangGraph) executes the LLM. There are three skills in total, covering Karpathy's three operations: ``` llm-wiki-extract → Ingest: extract entries from document content llm-wiki-organize → Lint: merge, link, cluster, flag stale, generate insights llm-wiki-maintain → incrementally update existing entries when new evidence appears ``` This decoupling of "orchestration layer + stateless interface" is critical: wiki-service doesn't care which LLM model is used, whether streaming is available, or how the prompt is assembled—it only sends content and receives structured JSON. Swapping models or skills has zero impact on the storage layer. ### 3.3 Extraction Prompt: "Trustworthy" Content Is by Design Extraction is not "throw the document at the LLM and let it summarize"—it is **structured extraction with strict constraints**. Looking at the core conventions in `_build_extract_prompt`: ```text For each entity (concept, person, project, event, location, organization, technology, etc.) found, output: - title: Concise descriptive title - content: Well-structured markdown — ONLY use info explicitly stated in the document - summary: One-sentence description - category: concept|person|project|event|location|organization|technology|tool|... - confidence_tier: extracted|inferred|ambiguous - confidence_score: 0.9+ for explicit, 0.7-0.8 for implied, 0.5-0.6 for tentative - text_spans: exact text snippet(s) from the document that this entry is based on - tags: 2-5 keywords - related_titles: titles of existing entries this relates to (if provided) - related_labels: VERY SHORT label (≤15 chars) describing HOW they relate CRITICAL — Content Rules: - The 'content' field must ONLY contain facts, descriptions, and claims - that APPEAR IN THE DOCUMENT - Do NOT inject your own knowledge, definitions, background, or examples - If the doc mentions 'coronary bypass' without explaining it, do NOT add an explanation ``` Here lie the three pillars of the entire knowledge base's trustworthiness: - **Content comes only from the source + **`text_spans`** provenance.** Every piece of content is constrained to "what the document explicitly states," with source snippets provided as evidence. This directly addresses the concern most raised by critics of Karpathy's vision—that "compilation" could mix the LLM's own hallucinations into the knowledge base. This project uses prompt constraints + provenance fields to block such contamination at the source. - **Confidence tiering.** `confidence_tier` distinguishes three types of knowledge: `extracted` (directly stated in the source, 0.9+), `inferred` (model reasoning, 0.7-0.8), and `ambiguous` (uncertain, 0.5-0.6). Karpathy's local convention relied on human eyes to judge page credibility; the cloud version **internalizes these judgments as first-class fields** of each entry—frontend graphs, cards, and legends can all visualize by confidence. - **Separation of enrichments from content.** During extraction, the LLM inevitably has "out-of-document" knowledge—background, term explanations, even challenges to the document. If stuffed into content, it contaminates the source; this project places them in a separate `enrichments` array (`detail_supplement`, `correction_challenge`, `timeliness_note`, `data_discrepancy`, `source_gap`, etc.) as **pending-review supplements**, never written into the content body. This is the clear dividing line between "LLM as compiler" and "LLM as creator": **compiled output is faithful to the source; creative artifacts are quarantined separately.** These three pillars essentially take Karpathy's `schema` (conventions like "every person entry must include motivation, limitations, achievements" in agents.md) and **crystallize it into hard rules in the prompt + fields in the data model.** Schema goes from a human-readable convention to a structure enforced by the extraction pipeline. ## IV. Organization and Linking: Letting Knowledge Grow Structure Extraction alone produces a flat pile of entries. The "meaning" of a knowledge base lies in **relationships**—this is precisely the cross-referencing Karpathy spoke of, and also where Memex died in the manual workshop. This project hands "cross-referencing" to the LLM and vectors. ### 4.1 Organize: Deduplication, Clustering, Stale Detection A single run of `llm-wiki-organize` does four things: - **Duplicate merge**: When the same entity is extracted multiple times into different entries, canonical selection is performed—first candidates are picked by rules, then the LLM selects a canonical name if needed, and content is intelligently merged (`merge_entries` preserves all information without loss). - **Clustering**: Community detection on entries produces `wiki_clusters`, each with a central entry and member list. Clusters form the "skeleton" of the knowledge graph and the basis for frontend grouped views. - **Stale flagging**: Detects outdated entries and tags them `stale`. This corresponds to "stale claims" in Karpathy's Lint. - **Insight generation**: At the cluster/root level, the LLM distills patterns, themes, and gaps—`insights`. This is a layer of **synthesis** above "compilation," making the knowledge base not just a collection of facts, but observations about facts. organize also supports `organize-by-root`: scoping by document tree, writing `root_id` back to entries, enabling the knowledge graph to be organized by document tree. ### 4.2 Three Types of Edges: system / llm / semantic Relationships between entries come from three sources, distinguished by color/line style in the frontend graph: - **system** (structural): Same document, parent-child, same root, same board—derived from the document's own structure, no LLM needed. - **llm** (semantic): Semantic relationships assigned by the LLM during extraction (`related_titles` + `related_labels`, e.g., `extends`, `contradicts`, `depends_on`, `supersedes`, `works_at`, `member_of`...), with labels ≤15 chars, human-readable. - **semantic** (vector): Auto-linked edges based on embedding vector similarity. This "provenance-aware three-type edge system" goes further than Karpathy's local cross-referencing: locally, the Agent manually writes `[[links]]` in markdown; in the cloud, **structured relationship metadata** allows the graph to be directly rendered, filtered, and colored by type. ### 4.3 Graph `graph_builder` assembles entries + links + clusters into `GraphData` (nodes / edges / clusters / insights) directly consumable by the frontend. Each node carries `category`, `confidence_tier`, `confidence_score`, `stale`; each edge carries `link_source`, `link_type`, `strength`. The frontend builds a full graph interaction on top of this (2D force-directed + 3D spherical view, d3-force-3d + three.js). ## V. Retrieval: Semantic Search In Karpathy's local solution, the Agent reads `index.md` first to find relevant pages. At cloud scale, "stuffing the directory into context" is not feasible; this project uses **embeddings** for semantic retrieval: ```python async def search_entries(self, user_id, query, max_results=10): # Semantic-first: vectorize query, compute cosine similarity with entry vectors query_vec = await compute_embedding(query) if query_vec: results = await self._semantic_search(user_id, query_vec, max_results) if results: return results # Fallback: keyword full-text search return await self._keyword_search(user_id, query, max_results) ``` Embeddings default to Qwen `text-embedding-v3` (OpenAI-compatible interface), degrading to "keyword overlap vectors" and then to full-text search when the embedding service is unavailable—**graceful degradation at each level: semantic-first, availability fallback.** This is the scaled replacement for `index.md`: the index is no longer a human-readable directory, but a vector written at the time each entry is generated. An honest difference worth calling out: Karpathy's **Query "writes back"**—every question grows into a new page, the system compounds. This project's retrieval is currently **read-only**: search hits, but doesn't settle "this question + answer" back into the knowledge base. This is the most visible gap between vision and implementation, and the next thing worth adding beyond "maintain." ## VI. Frontend and Quality Guardrails The frontend is a plugin (`@dave/llm-wiki`), mounted on 2Ryun's plugin system. It is not an isolated page, but deeply integrated with the document editor: - **WikiSidebar**: Document sidebar showing extraction status (not extracted / extracting / extracted), with one-click initialization and re-extraction. - **WikiMain**: Knowledge base main interface, Google-style search + two tabs (entries / graph). - **WikiEntryCard**: Entry detail view, showing confidence badges, category, tags, sources, relations, and pending enrichments. - **WikiGraph**: 2D/3D knowledge graph, colored by type/degree/confidence, with neighbor highlighting and non-related fading on focus. - **InsightsPanel / PendingEnrichmentsList**: Insight list + **human review queue** for AI-generated supplements. Quality guardrails run through the entire chain: confidence badges make "is this from the source or inferred by the model?" visible at a glance; `text_spans` let every piece of content click back to the source; `source_gap`-type enrichments proactively flag "this has no provenance"; and AI supplements always go to **pending review**, never auto-written into content. Together, these guardrails answer the most critical question for a knowledge base product: **"On what basis should I trust what's in here?"** ## VII. Validation and Reflection Mapping Karpathy's three operations to this project, most align: | | | Karpathy's Operation | | | | | This Project's Implementation | | | | | | Ingest (new source → build page → cross-reference) | | | | | `notify-update` | auto-trigger + | `llm-wiki-extract` | | | | | | Lint (contradiction/stale/orphan/gap) | | | | | `llm-wiki-organize` | (cluster/dedup/stale/insights) | | | | | | Query (retrieve + write-back) | | | | | Semantic retrieval ( | **missing write-back** | , see above) | | | | | | `/raw` | immutable | | | | | Document library + | `text_spans` | provenance | | | | | | `log.md` | | | | | `operation_log` | | | **What cloud-ification solved**: "Near-zero maintenance cost" went from human discipline to system automation—documents are extracted the moment they're saved, the graph is viewable in real time, multi-tenant data is naturally isolated (all queries carry `user_id`), and embedding indices are generated at ingestion. **What was sacrificed**: The most charming aspect of Karpathy's model is that **the output is pure markdown**—humans can read it, edit it, and diff it directly. After moving to the cloud, knowledge becomes entries + embedding vectors in a database; **humans lose direct editability of the "compilation result"**; controllability shifts to "how well the extraction prompts are written." This precisely confirms the critics' claim that "Schema is the bottleneck"—in this project, the bottleneck is those three skill prompts and the JSON schema they produce. **Limitations (shared with Karpathy's critics)**: - **Error compounding**: If the extraction stage produces a false "fact," subsequent organize, insights, and graph all build on top of it—and the one checking it is the same fallible LLM. This project's mitigation is confidence tiering and pending-review enrichments, but no fundamental cure. - **Scale**: This project is a multi-tenant entry-level knowledge base, but still follows the "extract a batch of documents → manage entries" model, not the long-term compounding growth organism that Karpathy envisions for personal wikis. At true million-scale, the graph and retrieval would need new engineering (node budgets, backend layout, WASM physics, etc.). - **"Compilation" ≠ understanding**: A well-maintained knowledge base doesn't mean the user has actually internalized the knowledge. Karpathy himself admits that someone who outsources the organizing work to an LLM may end up with a wiki they haven't digested. ## VIII. Conclusion Karpathy's LLM Wiki transforms "knowledge maintenance" from a form of labor into a **convention**: humans do curation (deciding what to read, what to ask), LLMs do compilation (building pages, cross-referencing, linting), and the output is a readable, traceable, compounding body of knowledge. This article's llm-wiki brings this convention to the cloud: replacing manual ingest with automatic triggering, free-form markdown with structured entries + provenance + confidence, `index.md` with embedding retrieval, and hand-written links with a graph. The lesson woven through these two threads: **the vision is "compile once," the engineering is "make the compilation trustworthy."** Karpathy gave the direction; turning it into a product others dare to use relies on hard constraints in prompts, confidence fields in the data model, and the discipline of "content is faithful to source, supplements quarantined separately"—treating the LLM as a compiler, not an author. #