Map: Continuous memory organization for context-graph (streamline ai-memory.md)

[antejavor]

Destination

A specced, decided design for the context-graph change(s) that let ai-memory.md-style demos regain Docker-only simplicity while staying backed by real sessions-graph/actions-graph/skills-graph code — specifically: a continuous background "memory organizer" mechanism (generalizing the reconciliation pattern already used by sessions-graph's reconcile_session/reconcile --pending, itself built on unstructured2graph) fed by adaptable per-harness capture hooks (generalizing agent-context-graph's existing Runtime Adapter layer), producing all three memory types (semantic/episodic/procedural) automatically and continuously — not via explicit write-tool calls. A narrow, read-only query surface (tentatively MCP) lets any LLM/harness "peek" at organized memory; it is explicitly not the write path.

End state: a design doc/ADR naming the organizing mechanism(s), their trigger/cadence, their schema impact on sessions-graph/actions-graph/skills-graph, how per-harness capture hooks are authored, and what ai-memory.md would look like once this exists — ready to implement, not yet built.

Notes

Domain: context-graph in memgraph/ai-toolkit — sessions-graph (semantic: Memory nodes, entities via reconciliation; episodic: Episode nodes via HAS_EPISODE, implemented in PR #271), skills-graph (procedural: new Procedure nodes, mined — distinct from curated, spec-conformant Skill nodes, which stay as-is), agent-context-graph (hook/adapter capture layer). See context-graph/CONTEXT-MAP.md.

Skills every session should consult: /grilling + /domain-modeling for HITL tickets; a /research subagent for research tickets.

Standing preferences for this effort:

• Writes happen via hooks + background reconciliation-style jobs. The read-only surface (MCP or otherwise) is never a write path — confirmed explicitly by the driving user after an earlier line of questioning assumed otherwise.
• Collection vs. Memory, as two distinct tiers (established resolving Grilling: what does continuous episodic-memory organization mean? #261, applies across all three packages): actions-graph's raw Action nodes, and the raw text sources sessions-graph/skills-graph draw from, are collection-tier — ephemeral observability data, deleted/compressed after some retention period (see Grilling: retention/compression policy for raw collection-tier data #267). The clean, distilled output of processing that collection data — Session.summary, Memory nodes, mined Procedure nodes — is memory-tier: durable, and what "memory" actually means in this project. Every remaining organizing-mechanism ticket should respect this split rather than treat collection-tier data itself as memory.
• The relationship between the future read-only surface and the existing agent-context-graph hook plugin is a real open question but explicitly deferred — ticketed, not resolved, until the organizing-mechanism tickets land.
• Longer-term goal (stated by the driving user, informs but does not resolve tickets): support most harnesses via different hook mechanisms — i.e. agent-context-graph's adapter layer keeps growing, rather than being displaced by the read-only surface.

Prior art to read before designing (do this before opening any ticket below):

• context-graph/sessions-graph/src/sessions_graph/{core.py,cli.py,reconciliation.py} — today's only "background organizing" mechanism (semantic-ish, LLM-backed, triggered by SESSION_END → reconciliation_status='pending' → manual/scheduled reconcile --pending sweep).
• context-graph/actions-graph/CONTEXT.md — actions-graph's own stated mission is agent-session observability/analytics, not memory recall; the collection-vs-memory split (above) is how that's reconciled with using its data as episodic memory.
• context-graph/skills-graph/CONTEXT.md — Skill is explicitly locked to the Agent Skills specification ("'skill' should not be a local invention"); this is why mined procedural output is a new, separate node type (Procedure), not a loosened Skill.
• context-graph/agent-context-graph/README.md — the Runtime Adapter Protocol (Runtime Adapter -> Event Protocol -> Graph Connector(s)), already designed for multiple harnesses (Claude Code, Codex, OpenAI SDK today).
• memgraph/memgraph-platform#110 — the just-merged ai-memory.md PR: the current hand-seeded baseline (ai-memory.py) this map aims to replace with something automatic.
• Research: what does sessions-graph's existing reconciliation actually write today? findings, in full: docs/research/2026-08-reconciliation-today.md on the research/reconciliation-today branch.
• PR #253 (merged 2026-08-11) — resolves LLM API keys via the same config-file-only mechanism (ADR 0002) already used for Memgraph credentials, consistently whether reconciliation is hook-spawned or standalone/cron-invoked. Grilling: what does ai-memory.md look like once background organizing exists? #265's design depended on this; it's now landed, so that dependency is clear.

Decisions so far

• Research: what does sessions-graph's existing reconciliation actually write today? — Reconciliation never writes (:Memory); it only writes Chunk+LightRAG entity nodes (semantic-ish, not sessions-graph's own Memory type). Trigger is synchronous SESSION_END → reconciliation_status='pending', with actual extraction either strictly manual (reconcile --pending) or via an existing-but-off-by-default auto_reconcile detached-subprocess flag — no CI/cron ever runs it today. Source text is both Action content (truncated 8000 chars) and Memory content (untruncated), deduped by hash. No repeated-pattern/skill-mining detection exists anywhere in the codebase today — a genuine gap, not a stretch-fit.
• Grilling: what does continuous episodic-memory organization mean? — A background job, reusing sessions-graph's reconciliation mechanism and trigger (SESSION_END), produces the episodic memory once a session completes. Episode = exactly one Session, no cross-session stitching in scope. Surfaced the collection-vs-memory split (now in Notes above) and spun out raw-data retention/compression as its own ticket, Grilling: retention/compression policy for raw collection-tier data #267. Shape reversed during implementation: originally decided as a summary property directly on (:Session) ("Episode" rejected as a redundant synonym for Session); reversed to a real (:Episode {summary, summarized_at}) node linked via HAS_EPISODE, matching this codebase's existing pattern of dedicated node types for owned data. Implemented in PR #271.
• Grilling: what does continuous procedural-memory organization (skill mining) mean? — Two separate mining processes (sequence-mining over actions-graph's chronological Action data, no LLM; content-mining over files touched during sessions, LLM-backed via the same unstructured2graph-style pipeline). Both write a new node type, Procedure, kept deliberately distinct from Skill (which stays spec-locked, untouched). Auto-write, no approval gate — separateness from Skill is the safety mechanism. Promotion from Procedure to a real Skill is explicit future work, out of scope here.
• Grilling: what triggers each background organizing job, and at what cadence? — No new daemon: a cron-style periodic sweep over existing/extended CLI commands, on the user's own ops infra. Episodic+semantic share one combined reconcile_session() pass (one entity-extraction LLM call plus one dedicated summarization LLM call, producing both Chunk/entity nodes and the Episode), triggered by the existing per-session reconciliation_status='pending' flag, short interval (minutes-scale). Procedural mining is separate: watermark/cursor over Action history (not per-item pending), no-op if nothing new since the cursor, longer interval (hourly/daily-scale).
• Grilling: how are custom per-harness capture hooks authored and registered? — Audited the code first: RuntimeAdapter is already a public, documented extension point for in-process/SDK-style runtimes (no fork needed). The real gap was command-hook runtimes (Claude Code/Codex-style) having no documented pattern despite two real implementations to copy from. Decision: document that pattern now (done — PR #270, via Task: document the command-hook runtime adapter pattern #268), defer the separate CLI plugin-discovery gap (bootstrap/doctor/setup hardcode a closed --runtime enum) to Grilling (deferred): CLI plugin-discovery for custom runtime adapters #269.
• Grilling: what does ai-memory.md look like once background organizing exists? — The core destination ticket. New baseline: requires an LLM API key ("no API keys" is explicitly retired, not kept as a fallback tier). Key injection via PR context-graph: propagate resolved Memgraph/LLM config to the reconcile subprocess #253 (see Notes). Trigger: demo invokes the same organizing CLI commands directly/on-demand rather than waiting on a real cron timer. Narrative shows only the end state (distilled memory), not a two-stage raw-vs-organized reveal. At the time this ticket resolved, Episode/Procedure/extended reconcile_session() didn't exist in code — episodic organizing has since landed (PR #271); procedural mining and the ai-memory.md rewrite itself remain future implementation.
• Grilling: retention/compression policy for raw collection-tier data — Confirmation-gated deletion (not time-based alone): only once the relevant memory-tier output exists, plus a grace-period buffer. Where raw data feeds multiple consumers (actions-graph's raw Actions feed both episodic summarization and procedural sequence-mining), ALL consumers must confirm before deletion — data survives as long as the slowest consumer's cadence. Compression means full node deletion, not partial field-stripping. Deletion is final — no tombstone/archive step.

This closes out the map's core destination — every organizing-mechanism ticket (episodic, procedural, trigger/cadence, hook authoring, ai-memory.md's shape, retention/compression) is now resolved.

Implementation progress: episodic memory (Episode nodes) is built and merged — PR #271. CLI plugin-discovery is built — PR #273. Procedural mining's implementation is deferred — see #272 (the sequence-mining matching algorithm and file-touch tracking for content-mining are still open, on top of the already-decided design in #262).

• Grilling: how are custom per-harness capture hooks authored and registered? — CLI plugin-discovery — Picked back up and resolved: Python entry_points (group agent_context_graph.runtimes), one bundled plugin object per runtime (not per-facet groups) exposing adapter_class/response_for_payload/build_hooks_config/init, with a generic runner in agent-context-graph owning the shared stdin/argv/AgentLink plumbing for hook run <runtime> — collapsing codex_main/claude_code_main's near-duplicate logic (resolving a pre-existing # TODO in codex.py). Codex and Claude Code become the first two registered plugins rather than special-cased code paths. Implemented in PR #273.

Two tickets remain open, explicitly deferred/parked: #266, #272.

Not yet specified

(empty — every fog item sketched at charting time immediately graduated into a ticket below; see the map's child issues.)

Out of scope

• The read-only surface (MCP or otherwise) acting as a write path for any memory type — ruled out explicitly; all writes go through hooks + background organizing.
• Promotion from a mined Procedure node into a real curated Skill node — future work (surfaced resolving Grilling: what does continuous procedural-memory organization (skill mining) mean? #262).