Cambium Memory — Design

Date: 2026-04-11 Status: Approved for implementation planning Package: @cambium/memory

Purpose

Give Cambium a persistent memory substrate so Obadiah can recall what it knows about Joshua across sessions, across agent lifecycles, and across time. The first concrete goal: Joshua writes in today’s daily log at 11pm “Arnold recommended cooking vegetables 20 min at 180°C,” and the next morning Obadiah can recall that observation, cite its evidence, and blend it with what Joshua’s long-term notes say about food preferences.

More broadly: bridge Joshua’s existing file-based memory at /Users/joshua/Projects/Obadiah/memory/ into Cambium as a participating substrate, and add an active layer of agent-generated epigenetic marks that records how the environment is expressing Joshua’s written identity — without ever editing that identity automatically.

Framing: Memory has layers, and each layer has its own write rules

This is not a single-store “remember things” system. It is a four-layer memory modeled on how nature actually keeps memory. Each layer has its own write discipline, its own decay behavior, and its own backend. The layers compose at query time.

The four layers

1. Substrate (DNA-like). Joshua’s handwritten long-term notes at /Obadiah/memory/long-term/*.md — boundaries, decisions, identity, lessons, workflow. Nearly immutable within a day or week. Changes through a deliberate molting ritual — Joshua sits down, rewrites a section, commits. Nobody else writes here. Not agents, not a propose-edit flow, not an append. Agents can only plant seeds (see below) that influence the next molt. Your identity is a gift you give yourself, not a collaborative document.

2. Epigenetic marks. New Postgres-backed layer. Experience writes marks on the substrate without editing the substrate. When an agent observes “Joshua declined three lunch meetings this month after 2pm,” that observation leaves a typed, timestamped, decaying mark — attributed to the observing agent and citing the evidence that produced it. Future agents read these marks alongside the substrate. The substrate says “I protect focus time in the afternoons”; the marks record how the environment is expressing that right now. Marks have decay (short-lived ephemeral notes wither; reinforced patterns persist). Marks never edit the substrate.

3. Daily (neural-like). /Obadiah/memory/daily/YYYY-MM-DD.md. Working memory for today. Fast, plastic, both Joshua and agents write freely — Joshua logs meetings, thoughts, todos; Obadiah appends observations and completed actions. The file is canonical only for that day. Each night, a consolidation rhythm runs, extracts salient observations as epigenetic marks, and archives the raw file. Like sleep. Like dreams.

4. Compost (intergenerational). Already exists in @cambium/genesis. When an agent dies, composting extracts its learnings into perennial template knowledge. The memory package reads this layer — it doesn’t write to it — and surfaces relevant composted learnings in query results.

The query invariant

An agent querying “what do you know about Joshua’s morning preferences?” reads all four layers at once, blended, weighted by recency / reinforcement / source-authority. The caller doesn’t say “check the substrate, then the marks, then the daily.” It asks once and receives a structured response with fields per layer. The layers are internal architecture; the query surface is one.

Core Concepts

Subject namespace

Every mark, seed, and query scope uses a dot-path subject: joshua.sleep.bedtime, joshua.food.breakfast, joshua.focus.afternoon, joshua.people.arnold. Queries match by prefix — joshua.sleep.* returns every mark whose subject starts with joshua.sleep.. Trees form naturally without requiring a predesigned ontology; new subjects are just new paths. This is stringly-typed on purpose — the type system doesn’t know the valid subjects, because the valid subjects are an open set that grows as Obadiah’s understanding of Joshua grows.

Epigenetic mark

A mark is a typed, timestamped observation about something (the subject), citing the evidence that produced it, with a strength that decays over time and can be reinforced by subsequent observations.

Evidence

Every mark points back at the specific signals, messages, daily-file line numbers, or composted learnings that produced it. This is how a mark stays honest. An agent can’t invent observations — it must cite what it saw. Evidence is queryable: a caller inspecting a mark can surface the raw trace.

Seed (proposal for substrate change)

When a mark reaches high reinforcement on a subject that the substrate doesn’t mention yet, an agent can plant a seed — a proposal that Joshua consider adding that topic to one of his long-term files. Seeds never commit anything. They sit in a pool that Joshua reviews during a molting ritual. Joshua either edits the substrate file himself (seed marked accepted) or dismisses (seed marked rejected). Automatic acceptance is never a thing — not in v1, not in v2, not ever. The only thing the system does automatically is notice a pattern worth suggesting.

Consolidation

The nightly process that turns daily-log writing into persistent marks. Runs as a rhythm (memory.consolidate signal fires at 03:00 Berlin), processes today’s daily/YYYY-MM-DD.md file, extracts candidate observations via a single LLM call, creates marks with evidence pointers back to the raw file, reinforces existing marks where today confirmed them, applies decay to all marks, and archives the daily file. The LLM call is the one non-deterministic step; it is acceptable because consolidation runs once per night with a single file as input.

Molting

Joshua’s seasonal ritual of rewriting substrate files. Not automated. Not scheduled. Triggered by Joshua sitting down to revise. The memory package supports molting by surfacing pending seeds and relevant marks when Joshua opens a substrate file for edit — but the editing itself is always Joshua’s hands on the keyboard.

Data Model

Files (layer 1 and 3 — Substrate, Daily)

No schema change. The memory package reads substrate and daily paths from spore.paths.substrate (a list) and spore.paths.daily (a single directory) — it does not hardcode any path. For Joshua’s Obadiah deployment specifically, the spore resolves those values to:

Substrate — a list of directories and/or individual files. Each entry is walked uniformly: directories contribute all their top-level *.md files; individual .md file entries contribute themselves. The Substrate layer reader doesn’t care whether an entry is a directory or a file — the spore loader’s stat.isDirectory() check determines how each entry is walked.

Substrate entries for Obadiah (from spore.paths.substrate):

/Users/joshua/Projects/Obadiah/memory/long-term/       (directory)
├── boundaries.md
├── decisions.md
├── identity.md
├── lessons.md
└── workflow.md

/Users/joshua/Projects/Obadiah/personal-context/       (directory — 10 numbered files)
├── 01-identity.md
├── 02-role-and-responsibilities.md
├── 03-current-projects.md
├── 04-team-and-relationships.md
├── 05-tools-and-systems.md
├── 06-communication-style.md
├── 07-goals-and-priorities.md
├── 08-preferences-and-constraints.md
├── 09-domain-knowledge.md
└── 10-decision-log.md

/Users/joshua/Projects/Obadiah/SOUL.md                 (single file — Obadiah's identity DNA)
/Users/joshua/Projects/Obadiah/USER.md                 (single file — operational context)

Daily — one directory at spore.paths.daily, containing:

/Users/joshua/Projects/Obadiah/memory/daily/
├── 2026-04-11.md      ← today (read/write for Joshua and agents)
└── archive/           ← post-consolidation files go here
    ├── 2026-03-20.md
    └── ...

The memory package watches all substrate entries + the daily directory and invalidates its parsed cache on file change. Joshua editing a substrate file in his own editor is picked up on the next query — regardless of whether the file lives in memory/long-term/, personal-context/, or as a single top-level file like SOUL.md.

Postgres tables (new)

memory_marks
  id                     uuid pk
  workspace_id           fk
  subject                text        -- dot-path, e.g. "joshua.sleep.bedtime"
  observation            text        -- short natural-language summary
  evidence               jsonb       -- references: { signal_ids?, daily_file?, line_range?, compost_ids? }
  initial_strength       real        -- 0-1, value at first_observed_at
  decay_rate             real        -- half-life in days
  observed_by            text        -- agent id or "joshua" for manual marks
  first_observed_at      timestamptz
  last_reinforced_at     timestamptz
  reinforcement_count    int
  affinity               jsonb       -- string[] tags for secondary indexing
  substrate_refs         jsonb       -- optional: long-term/*.md files this mark relates to
  state                  text        -- "active" | "withered" | "promoted"
  created_at             timestamptz
  updated_at             timestamptz

  indexes:
    - workspace_id
    - (workspace_id, subject)  -- for prefix queries via LIKE/gin
    - state
    - first_observed_at

memory_mark_reinforcements
  id                     uuid pk
  mark_id                fk -> memory_marks
  reinforced_at          timestamptz
  reinforced_by          text        -- agent id
  evidence               jsonb       -- what confirmed this
  strength_delta         real        -- how much this reinforcement adds to the mark's strength

  indexes:
    - mark_id
    - reinforced_at

memory_seeds
  id                     uuid pk
  workspace_id           fk
  subject                text
  proposed_change        text        -- natural-language: "add to long-term/boundaries.md: ..."
  target_file            text        -- which substrate file this seed targets
  based_on_marks         jsonb       -- mark IDs that produced this seed
  planted_at             timestamptz
  planted_by             text        -- agent id
  state                  text        -- "pending" | "accepted" | "rejected" | "withered"
  reviewed_at            timestamptz
  created_at             timestamptz
  updated_at             timestamptz

  indexes:
    - workspace_id
    - state
    - planted_at

Strength computation: current_strength = initial_strength * 2^(-days_since_first_observed / decay_rate) + sum(reinforcement_deltas). Computed on query, not stored. A mark with strength below a threshold (default 0.1) is automatically marked withered during consolidation or query-time prune.

The Query API

interface RecallInput {
  about: string;                                            // subject prefix, e.g. "joshua.sleep"
  horizon?: '1d' | '7d' | '30d' | 'all' | Date;             // time scope; default 'all'
  layers?: Array<'substrate' | 'marks' | 'daily' | 'compost'>;  // which layers to include; default all
  min_strength?: number;                                    // skip marks below this current_strength; default 0.1
  workspace_id: WorkspaceId;
}

interface RecallResult {
  substrate: Array<{ file: string; excerpt: string; reason: string }>;
  marks: Array<{
    id: MarkId;
    subject: string;
    observation: string;
    current_strength: number;
    observed_by: string;
    first_observed_at: string;
    last_reinforced_at: string;
    evidence: EvidenceRefs;
  }>;
  daily: Array<{ date: string; excerpt: string; line_range: [number, number] }>;
  compost: Array<{ template_id: string; learning: string; source_agent_count: number }>;
  summary: null;  // v1 stub — LLM blend is deferred
}

recall(input: RecallInput): Promise<RecallResult>

The return is always structured; the summary field is a stub in v1 that returns null. Later versions add LLM summarization.

Writing marks

Agents don’t write marks directly in production — they fire signals with an observation payload, and a memory receptor writes the mark. This keeps the audit trail in the signal bus (persist-before-deliver still applies) and means marks inherit Cambium’s existing tracking for free.

// Signal shape (new type)
signal: {
  type: 'memory.observation',
  class: 'knowledge',
  source_layer: '<observing agent layer>',
  source_id: '<observing agent id>',
  workspace_id: '<workspace>',
  intensity: 0.0–1.0,   // becomes the mark's initial_strength
  decay_rate: <half-life in days — but signal decay_rate is in seconds, so a converter runs>
  payload: {
    subject: 'joshua.sleep.bedtime',
    observation: 'Joshua went to bed after midnight 4 times in the last 2 weeks',
    evidence: { signal_ids: [...], daily_file: '...', line_range: [...] },
    affinity: ['sleep', 'routine'],
    substrate_refs: ['long-term/workflow.md']
  }
}

A memory.observation receptor in @cambium/memory catches these, writes the mark, and optionally emits a memory.mark_created signal for downstream tracking.

For debugging and manual Joshua-driven marks, there’s also a direct CLI path (obadiah memory mark <subject> <observation>) that bypasses the signal layer.

Planting seeds

Similar pattern — seeds are planted via memory.seed_planted signal, caught by a receptor, stored in memory_seeds. Agents decide when to plant a seed (logic lives in the observing agent, not in the memory package). The memory package provides the plumbing.

Package Boundaries

@cambium/memory depends on: - @cambium/core — branded IDs, contracts - @cambium/db — Drizzle schemas, migrations - @cambium/signals — consume observation signals, emit mark-created signals - @cambium/genesis — read compost_records and perennial_learnings (read-only)

@cambium/memory does NOT depend on: - @cambium/stigmergy — different timescale, different purpose. Marks are not pheromones. - @cambium/hypha — memory doesn’t know about workflows. Workflows can be receptors that observe, but the memory package itself doesn’t invoke them. - @cambium/lifecycle — memory doesn’t need to hook into the lifecycle engine’s tick loop. Consolidation runs as a rhythm (see @cambium/rhythms), which is the right layer for scheduled work.

What Ships in v1

1. @cambium/memory package scaffolded matching the @cambium/rhythms v1 convention
2. Drizzle migration creating memory_marks, memory_mark_reinforcements, and memory_seeds tables
3. Substrate reader — watches long-term/*.md, parses markdown, returns raw excerpts on query
4. Daily reader/appender — watches daily/YYYY-MM-DD.md, reads current and archived files, appends new lines on demand
5. Marks store — Drizzle-backed CRUD for marks and reinforcements, including strength computation and prefix query
6. Seeds store — Drizzle-backed CRUD for seeds
7. Compost reader — thin wrapper that queries @cambium/genesis composting records by subject affinity
8. recall() query function — blends all four layers, returns structured RecallResult (with summary: null in v1)
9. memory.observation receptor — registered with a SignalBus, turns signals into marks
10. memory.seed_planted receptor — registered with a SignalBus, turns signals into seeds
11. Consolidation rhythm — new deployments/obadiah/rhythms/consolidation.yaml firing daily at 03:00 Berlin, emitting memory.consolidate signal. A receptor in @cambium/memory handles it: reads today’s daily file, extracts observations via one LLM call, writes marks with evidence pointers, applies decay, archives the daily file
12. CLI commands (in deployments/obadiah/bridge):
    • obadiah memory recall <subject> [--horizon 7d] [--layers substrate,marks]
    • obadiah memory mark <subject> "<observation>" — manual mark for debugging
    • obadiah memory seeds [--state pending]
    • obadiah memory substrate — prints parsed substrate content
13. Integration test: Write a line to today’s daily file → fire memory.consolidate signal manually → verify marks created with correct evidence → call recall() with matching subject → marks appear in result with correct strength

Deferred (intentional)

Each item has an intentional reason and a trigger for pickup. Captured separately in GSD seed/backlog.

Non-Goals

• Automated substrate editing. The memory package never writes to long-term/*.md. Not in v1, not in v2, not by a propose-edit flow. Joshua’s handwritten identity is his hands on the keyboard. Seeds exist to suggest, never to commit.
• Cross-deployment memory sharing. Obadiah’s memory is Obadiah’s. BLO’s memory is BLO’s. Kit’s memory is Kit’s. Sharing memory across deployments is out of scope — if Joshua wants cross-deployment context, he writes something in his own substrate files.
• Replacement of composting. @cambium/genesis composting stays exactly as it is. Memory reads from it; it doesn’t rewrite the composting model.
• Arbitrary-file backend outside of spore paths. The memory package reads only from paths declared in spore.paths.substrate and spore.paths.daily. A generic “Obsidian vault adapter” that watches arbitrary user-chosen folders at runtime is out of scope — the substrate list is the extensibility point. A new customer with a different folder layout points their spore at wherever their substrate lives.
• Observability dashboards. Loom page is deferred. v1 is CLI + programmatic.

Open Questions

None blocking. The LLM consolidation step has one open implementation question (which LLM provider, which model, what’s the prompt) that gets answered during planning, not design.

Success Criteria

1. pnpm --filter @cambium/memory test passes
2. Migration applies cleanly to dev DB
3. recall() returns structured multi-layer results with correct substrate excerpts, marks by prefix, daily excerpts, and compost entries
4. End-to-end: Joshua writes “Arnold recommended cooking vegetables 20 min at 180°C” in today’s daily/YYYY-MM-DD.md. A manual invocation of the consolidation receptor processes the file and creates a mark with subject joshua.food.vegetables.* (or similar), observation matching the quote, and evidence pointing at the correct line in the daily file. A subsequent recall({about: "joshua.food"}) returns the mark with a plausible strength value.
5. A daily file that gets consolidated is moved to daily/archive/ and no longer appears in routine recall() queries (unless horizon: 'all' is specified).
6. A mark’s current_strength decays correctly over simulated time — writing a mark with initial_strength: 1.0, decay_rate: 7 (half-life 7 days), then querying 14 days later, returns current_strength ≈ 0.25.
7. Planting a seed via the signal path creates a memory_seeds row with state: 'pending'. obadiah memory seeds lists it. Marking it accepted via CLI updates the row. The substrate file is not touched.