What is MemInject
MemInject mines anyone's chat and JSON/MD backups, synthesizes the signal, and injects role-aware memory into per-agent Hindsight banks. Rust parser core handles ~300k records from ~2,900 files in seconds. Python LLM smart-sort does attribution. The result is wrapped in an MCP/API so every agent in the fleet starts each session knowing its own history.
- Source: chat backups, JSON/MD exports, any text corpus
- Rust parser: ~300k records, ~2,900 files, ~2.8x faster than Python
- LLM smart-sort: attributes each item to the right agent
- Hindsight banks: per-agent memory on Hetzner
- MCP/API wrapper: agents recall their history at session start
mem-inject.exe parse <backups>The Layered Memory Stack
Agent memory is built in three layers. Layer 1 gives every agent its identity and role pack. Layer 2 loads shared project knowledge scoped by role. Layer 3 (the hard one) injects personal chat history so each agent recalls its own prior work. Auto-create spins up new agent banks on first mention.
- Layer 1: role and identity packs for all 31 agents
- Layer 2: project knowledge, each agent sees what is relevant to its role
- Layer 3: chat-history injection, the hard layer, what MemInject solves
- Auto-create: new agent banks provisioned on first mention in the roster
- 31-agent roster: ava oliver carlos queen petey dan merlin cody einstein ghost tommy sherlock raven knox frankie willie picasso shakespeare spielberg gino freddy hawkeye vox linx impy memz automa architect matteo-jr keymaker schemabrain
The Bug
The first injection run looked complete but was badly broken. 66,602 of 80,548 chat items landed in one agent (cody). Non-roster bank names were created. The per-agent spread that should look like a healthy distribution looked like one giant bar and everything else near zero.
- 66,602 of 80,548 chat items dumped into cody alone (83% of all items)
- Non-roster banks created (case variants, runtime names)
- All other agents severely under-loaded
- The spread that should be distributed looked like one dominant spike
- Cody was not cody: it was the entire unattributed corpus
Two Compounding Bugs
The attribution logic had two bugs working together. First: the default bucket was hardcoded to cody, so every record with no detected roster name fell into cody (~93% of chat records name no agent explicitly). Second: a stale roster still listed runtime names (Oliver, Herm) that are not in the 31-agent roster.
- Bug 1: default bucket = cody (should be unattributed)
- ~93% of chat records contain no explicit agent name
- All no-match records silently routed to cody
- Bug 2: stale roster listed runtime aliases (Oliver -> oliver, Herm -> not in roster)
- These aliases created non-roster bank rows
The Fix
Three surgical changes to the smart-sort pipeline. Roster-gate every detected name against the 31-agent roster. Normalize case. Route all non-roster and no-name records to an explicit unattributed sink, never to cody. Hard-refuse any write to a non-roster bank at the write boundary.
- Roster-gate: every name match checked against the 31-agent roster list
- Case normalization: cody, Cody, CODY all resolve to cody
- Unattributed sink: no-match and no-name records go to unattributed, never cody
- Hard-refuse: write boundary rejects non-roster bank names before any write
- Purge command: reversible purge of the bad run before re-injection
python stages/smart_sort_async.py records.json --no-dry-run --out items.jsonl --concurrency 12 --cost-cap 8.0 --hindsight http://localhost:8888/v1/defaultpython stages/purge_bad_run.py --bank cody --tag chat-historyThe Injection Fix
The first injection attempt failed with ReadTimeout: 0 items written. It posted each agent's entire bucket as one synchronous mega-batch (up to 78,134 items), hit the 30-second timeout, and gave up. Fix: chunk to ~200 items per POST, use async queue, cap concurrency with asyncio.Semaphore(8), extend timeout, make resumable.
- Original: one synchronous POST per agent, up to 78,134 items, 30s timeout
- Result: ReadTimeout, 0 items written to any bank
- Fix: chunk each bucket into ~200-item batches
- async:true queue on Hindsight so large banks do not block
- asyncio.Semaphore(8): 8 concurrent chunk uploads, not 1,027
- Resumable: completed chunks tracked so a restart skips already-done chunks
Proof: The Corrected Corpus
After the fix and re-injection, the corpus is live in the Hindsight banks. The unattributed sink holds the honestly-unattributed bulk. Cody is clean at its real count. Real agents gained facts where authorship existed. Zero non-roster banks. Grand total ~875,170 items across 31 banks.
- oliver: 120,773 items (largest attributed agent)
- dan: 61,659 | queen: 30,747 | einstein: 28,615 | tommy: 24,871
- cody: 1,081 (clean, not the 66k bug dump)
- unattributed sink: 48,839 (honest, non-roster bulk)
- Zero non-roster or case-variant banks created
- Grand total across 31 banks: ~875,170 items
python stages/verify_spread.py