Local-first agent infrastructure

MemoryAndSoul

A durable memory, requirements, AI-docs, soul, and whiteboard system for Claude and Codex agents that need to remember what happened, why it mattered, and how to resume work without rebuilding context from scratch.

Problem

Long-running AI work fails when the next agent cannot tell which requirements are current, which notes are historical, or which lessons were learned the hard way. Chat history is not a database, and keyword search alone is not enough when the useful memory uses different words than the current problem.

Solution

Meaning-based recall

MAS semantic search retrieves memories by meaning instead of only exact keywords, so agents can find prior decisions even when the exact wording is forgotten. Use memory_semantic_search for open-ended recall; memory_related starts from one known memory and discovers related work that would otherwise stay buried.

Private local embeddings

Embeddings stay local and private with fastembed, BAAI/bge-base, and pgvector. There is no external embedding API in the recall path; when the semantic stack is unavailable, MAS gracefully falls back to keyword/FTS search or recency-based related-memory suggestions.

Agent-readable operating memory

MemoryAndSoul stores project facts, requirements, plans, AI documentation, first-person soul lessons, and whiteboard coordination so Claude and Codex can work from the same durable record.

Durability model

Immutable memory

Memory entries preserve what was true when written. Corrections supersede older heads; additions append. Timestamped supersede and append chains make point-in-time reconstruction and rollback practical instead of aspirational.

Replayable events

Events are not just audit decoration. The event/replay contract treats them as operational facts that can be replayed, exported, inspected, and used to rebuild state after failure.

JSONL recovery spine

JSONL export acts as the external recovery and replication spine. It gives the system a simple, append-friendly path for recovery, migration, and cross-machine replication.

Tetracom replication

The Tetracom replication path can catch up after outages, dedupe repeated events, and recover from cloud database wipes by replaying JSONL into a fresh store.

Coordination for real agent work

The system is designed around day-to-day agent operations, not a demo-only notebook. Per-harness MCP services let Claude and Codex connect through their own tool surfaces while sharing the same PostgreSQL-backed state. The whiteboard supports inter-agent proposals, reviews, decisions, and status updates without forcing every collaboration into a chat log.

Requirements

Numbered deliverables move through explicit lifecycle states instead of living as loose TODO text.

AI docs

Component references capture contracts, invariants, algorithms, and gotchas for future agents.

Soul

First-person lessons preserve the scars and tradeoffs that do not fit into a normal README.

Stack and implementation notes

Core stack

  • PostgreSQL for canonical durable state.
  • FastAPI for the local service surface.
  • MCP tools for Claude, Codex, and worker agents.
  • fastembed plus BAAI/bge-base embeddings and pgvector for private semantic recall.
  • memory_semantic_search and memory_related keep a documented fallback path when semantic search is unavailable.

Honest current edge

Addendum-only text is not yet folded into parent embeddings, so semantic search can miss meaning that lives only in appended content. This requirement 488 limitation is tracked as re-embedding parent heads from combined content.

Open source repository

The source lives on GitHub. The GitHub Pages overview is linked as public documentation, though the exact public Pages URL depends on repository Pages settings.