The AI Memory Stack: Notes, Graphs, and Agent Memory in Layers


An AI knowledge base answers the question of what your material contains. A memory stack answers a different one: what gets remembered — by you, about your thinking, and by the agents that work in your base — and where each kind of memory should live so that none of them contaminates the others.

The methodology on this page arranges that memory in five layers. Each has a different author, a different store, and a different purpose. They compose because they share the same entity names — the concepts in your notes are the nodes in your graphs, the subjects of your recorded insights, and the anchors of what your agents learn — so any layer can be queried against any other. And they stay separate for a reason that matters more than tidiness: memory that is loaded into every session steers reasoning invisibly, so what may be ambient and what must stay contestable have to live in different places.

Set It Up

1
Create an Account

Sign up for a free InfraNodus account to get access to the knowledge graph tools and your API key.

Create a Free Account
2
Get InfraNodus MCP
https://mcp.infranodus.com

Add this URL as a connector via Settings > Connectors in Claude Web / Desktop and authenticate with your InfraNodus API key via OAuth.

3
Install the Skill
/plugin marketplace add infranodus/skills
/plugin install infranodus@infranodus-skills

Run these two commands in the Claude Code prompt to install the skill as a plugin (use infranodus-all@infranodus-skills to install all our skills at once). Or use npx in your terminal: 

npx skills add infranodus/skills --skill infranodus -g -a claude-code -y

npx comes bundled with Node.js. For ChatGPT and OpenClaw, see the full installation instructions.

Didn't work? You can simply download infranodus.zip from our GitHub repository and add it to your LLM client manually — see the step-by-step instructions.



A knowledge base and the memory layers around it represented as a knowledge graph of concepts, topics, and gaps
The base layer: your notes, and the graph of what is inside them.

 

Layer 1 — Your Notes: What You Write


The ground layer is the one you already have: your own markdown notes in a vault or repository. Ideas, research, drafts, project notes. Everything else in the stack derives from this layer or points back into it, and it remains the source of truth — versioned with git, readable without any tooling, entirely yours.

One convention is worth setting on day one: keep the boundary between what you wrote and what a model wrote visible. If AI-generated notes enter the vault at all, they go into a dedicated folder that you curate — you decide what graduates into the main structure. The same principle returns at every layer above this one, because the entire stack depends on always knowing who authored what.


 

Layer 2 — Content Graphs: What the Notes Contain


The second layer maps the first. With the InfraNodus skill, your notes are extracted into knowledge graphs — note content becomes a concept graph, [[wikilinks]] become a link-structure graph, and for the themes that span many folders, a digest: a hundred or more one-line statements distilling how a whole area of the vault thinks, uploaded as its own graph. The extraction is deterministic — nothing is written into your files, and the graphs mirror the notes rather than interpreting them.

A manifest.json records which graph holds what, and instructions in CLAUDE.md tell the model to route questions through it: structure questions go to graph analysis, content questions to GraphRAG retrieval, direction questions to gap detection. When notes change, only the changed files' statements are replaced in the graphs — the layer stays current without rebuilds.

This is the layer that can do what neither retrieval nor reading can: measure the shape of your base. Which topics have formed, which concepts hold them together, and — most usefully — which pairs of topics both exist without being connected. Those structural gaps are where the next idea tends to be hiding, and they are the input to the loop described below.

The GraphRAG pipeline: knowledge base to knowledge graph to enriched context to response
The content graphs sit between your notes and the model.

 

Layer 3 — Insight Memory: What You Realized in Conversations


Working with a model over your own material produces something neither the notes nor the graphs capture: realizations. A connection between two of your concepts you had never made. A contradiction in your own thinking that a structural analysis surfaced. These arrive mid-conversation and, by default, evaporate when the session ends.

The third layer records them. At the end of a substantive session, the model distills the new connections you arrived at — not its own mechanics, not vault content — into short relational statements, each linking at least two of your concepts and carrying the session date. It shows them to you, and saves them to a dedicated memory graph only on your approval, using the memory tools of the MCP server. The convention lives in CLAUDE.md — the InfraNodus skill offers to install it during setup (--register-project --with-memory), so any session knows to propose it.

Because every statement is timestamped, this graph becomes a timeline of your thinking: you can ask what you previously concluded about a concept, watch which ideas recur across months, and run gap analysis on your own trail of realizations. An insight that matures graduates downward — you write it into a note, where the content graphs pick it up — which keeps this layer a staging area rather than a second vault.


 

Layer 4 — Agent Learnings: What the Model Learned Operating Here


An agent that works in your base keeps rediscovering the same things: which folder is excluded from the graphs, which server holds which graph, what chunk size an upload tolerates, which API call fails right after a graph is created. The fourth layer — the Project Learnings skill — turns those rediscoveries into an append-only learnings graph that the next session reads before it starts.

The entries are typed — traps (what went wrong first and why), conventions, workflows, open questions, and the agent's own self-assessment of what worked — and every batch is shown to you before it is saved. The admission bar is deliberately high: only knowledge that is not derivable from the files in a few reads, would have saved time up front, and survived verification. Nothing about you, ever — this layer is about the project, and the distinction is enforced, not just recommended.

Because the learnings link to the same entities as everything else — file paths, graph names, concepts — this layer composes with the others: comparing it against the content graphs shows the parts of your base no agent has learned anything about yet, which is unexplored territory in an operational sense rather than a conceptual one.


 

Layer 5 — LLM Wiki: Curated Pages That Compound


The optional fifth layer is authored rather than extracted: an LLM wiki — machine-written, human-curated pages that give recurring themes a permanent home. Where the content graphs map what exists and the insight memory records what occurred to you, the wiki consolidates: a maintained page absorbs each session's additions instead of leaving them as one-shot notes.

Adopt it when a theme has outgrown scattered notes — when you notice the same concept accumulating fragments across the vault and the insight graph. Run it in its own folder, per the rule from Layer 1, and the content graphs will map the wiki pages too, so the fifth layer feeds back into the second. The LLM Wiki skill scaffolds the structure and generates the graphs alongside the pages.

An LLM wiki of markdown pages read as a knowledge graph with topical clusters and gaps
The wiki layer, read back through the graph layer: pages as nodes, topics in colour.

 

Why the Layers Stay Separate


It would be simpler to pour everything into one store, and it would be a mistake. Anything a session loads automatically becomes part of the model's frame before reasoning begins — it shapes interpretation without ever being examined. That is exactly what you want for operational facts: the agent should silently know which folder is excluded and which upload size fails. It is exactly what you do not want for ideas: half-finished conceptual positions injected as ambient context read as settled conclusions, and your thinking quietly stops being challengeable — the second brain starts thinking for you instead of with you.

The design principle

Operational knowledge may be ambient; intellectual knowledge must stay contestable. Agent learnings are read automatically at the start of a task. Insights are proposed, reviewed, and approved — and reach the model's working context only when you query them or write them into a note. The separation of stores is what makes both behaviours possible at once.

The same principle governs authorship at every layer: extraction is deterministic and writes nothing into your files; anything machine-written is kept in its own store or its own folder; and every write to a memory graph is shown to you first. The stack augments your perspective on your own material — it does not replace your judgment about it, which is the point of building one at all rather than letting an undocumented retrieval harness decide what your AI remembers.


 

The Loop


Arranged this way, the layers form a cycle rather than a pile:

A conversation produces an insight (Layer 3) → you write it into a note (Layer 1) → the graphs fold it in (Layer 2) → gap analysis on the updated graphs seeds the next conversation — which produces the next insight.

Agent learnings (Layer 4) make every pass through the loop faster, because the operational mistakes of one session are not repeated in the next. The wiki (Layer 5) consolidates what keeps coming around. The vault stays the source of truth; the graphs make it measurable; the memories make it cumulative. Each session starts where the last one ended instead of starting over — which is the difference between using an AI on your knowledge base and building one that develops it with you.


 

The Five Layers Compared


Layer Author Store When it grows What it is for
1 — Notes You Your vault (git) Whenever you think Source of truth; everything derives from here
2 — Content graphs Deterministic extraction + digests Graphs in your account, routed by a manifest On note changes, updated in place GraphRAG retrieval, topic structure, gap detection
3 — Insight memory Model distills, you approve A timestamped memory graph End of substantive sessions A timeline of your realizations, queryable by concept
4 — Agent learnings The agent, approved per batch An append-only learnings graph End of substantive agent tasks Traps, conventions, workflows — the next session starts smarter
5 — LLM wiki Model writes, you curate Pages in their own folder (graphed by Layer 2) When themes recur Consolidated understanding that compounds over time

 

Set It Up


  • InfraNodus skill — run /infranodus in your vault to build Layer 2: the content graphs, the manifest, and the CLAUDE.md routing rules.
  • MCP server — carries the memory tools for Layer 3 (add and retrieve timestamped insight relations) alongside graph analysis and GraphRAG retrieval; the InfraNodus skill installs the end-of-session convention into CLAUDE.md when you opt in at its register step.
  • Project Learnings skill — Layer 4: enable it per project, and every substantive agent task reads the learnings on start and proposes new ones on finish.
  • LLM Wiki skill — Layer 5, when you are ready for it: scaffolds the wiki structure and generates its graphs in one pass.
  • RAG vs LLM Wiki vs Knowledge Graph — how the retrieval-facing layers compare on the same question, if you are choosing where to start.

 

FAQ

What is an AI memory stack?

A layered arrangement of the different kinds of memory that accumulate around a knowledge base: the notes you write, the graphs generated from them, the insights you reach in conversations with the model, the operational lessons agents learn while working in your base, and — optionally — a machine-written wiki. Each layer has a different author, store, and purpose.

Why not keep everything in one memory store?

Because memory that is injected into every session steers the model's reasoning invisibly. Operational facts can safely be ambient; ideas cannot — if your half-finished positions are silently loaded as context, the model treats them as settled and your thinking stops being contestable. Separate stores make operational knowledge automatic while keeping intellectual knowledge explicit and reviewable.

What is the difference between agent learnings and insight memory?

Agent learnings record what the model learned about operating in your project — traps, conventions, workflows. Insight memory records what you realized through the conversation — new connections between your own concepts. The first makes the next session faster; the second makes your thinking cumulative.

Do I need all five layers?

No. Notes plus content graphs are the working minimum. Add insight memory when you want realizations to survive between sessions, agent learnings when the same agents work in the base repeatedly, and a wiki when recurring themes outgrow scattered notes.

How is this different from RAG?

RAG re-derives context on every query and forgets the result. The memory stack is what persists between queries — structure, insights, and accumulated agent knowledge — so each session starts further ahead than the last. Retrieval still runs underneath; see the RAG vs LLM Wiki comparison for that layer on its own.

Where does the memory live?

Notes and wiki pages live in your vault as markdown, versioned with git. The graphs — content, insights, learnings — live in your InfraNodus account, reachable from any client through the MCP server, deletable at any time. The manifest in your vault records which graph holds what.


 

Build Your Memory Stack


Start with the layer that makes the rest possible — a structural view of your knowledge base: the topics inside it, the concepts that connect them, and the gaps between them:


Sign Up     Log In