How a machine pulls
from what it stored.
This started as a question about ranking algorithms and became a question about how a system can work well and still be wrong about how it is working. That second version is where the comparison to human memory stops being decorative and starts doing real work.
What was supposed to happen
The design is unremarkable, which is the point, because it is roughly what any retrieval-augmented system does. Text gets embedded into 384-dimensional vectors, a query gets embedded the same way, and cosine similarity ranks the stored records against that query, blended with a keyword score and a recency weighting. The top results get injected into the model's prompt.
4 kinds of record were stored and meant to be searchable: journal entries, chunks of uploaded documents, summaries of past conversations, and claims, which are small structured facts pulled out of everything else. "Prefers morning work." "The Henderson project ships in March."
What actually happened
3 of the 4 never reached the ranker at all, and the fourth reached it without any vectors attached.
load_entries() and load_uploads() returned an
empty list for every workspace, always. load_conversations()
returned an empty list for every workspace, always. Claims arrived with
no embedding at all and scored 0.0 on every semantic
comparison.
The ranker therefore operated on zero embedded records and quietly degraded to keyword matching, on every installation, for roughly 4 months.
The causes were 4 separate instances of the same class of bug, which is what creates the interesting part rather than the merely embarrassing part:
- Entries and uploads: the loader read a field called
textwhile the writers stored onlytext_preview, a 200-character excerpt. The loader's first action wasif not text: continue, and a repo-wide grep confirmed that no writer anywhere had ever writtentext. - Uploads, again: the loader probed
chunk_idxwhile both writers storedchunk_index. It also readfilenameanduploaded_at, neither of which was ever written, so even past the first gate the hits would have carried blank titles and permanently neutral recency. - Conversations: the summaries were embedded correctly, but the loader fed itself from a lightweight sidecar index that deliberately left the vectors out, on the grounds that they would bloat the file. The nested summary it went looking for was always absent.
- Claims: the
embeddingfield existed, the storage layer persisted it, and the adapter round-tripped it, but no production writer ever populated it. The schema comment read "lazy-populated by future embedding pipeline"; that pipeline was never built.
Each side was individually correct and individually tested. The bug lived in the seam between them, where nothing was tested, because a seam is not a unit and unit tests test units.
Why nothing caught it
3 independent safeguards should have caught this and each one failed in a way worth naming, because none of them were missing. They were all present and ineffective, which is considerably harder to notice than an absence.
-
The fallback was too graceful
When the embedder was unavailable the ranker zero-filled the semantic scores and carried on with keyword matching, which is correct behavior. The availability check swallowed every failure with no logging at all, and everything else on that path logged at
DEBUG. A system that degrades silently is indistinguishable from a system that works. -
The shipped builds contained no embedder
The application bundle contained exactly 5 Python packages. The embedding stack had been excluded deliberately because it would have added roughly 590 MB, and the opt-in installer that was supposed to replace it never got built. Every customer therefore ran keyword-only regardless of the schema bugs, while the settings screen advised running
pip install sentence-transformers, which targets the system Python rather than the interpreter inside the bundle. The advice could not have worked for anybody. -
The regression test certified the broken state
There was a recall baseline of 25 queries with expected scores, checked on every run, and it passed throughout. It had been captured after the bugs were introduced, so it was pinning the broken behavior in place as correct. Its own docstring had warned about exactly this in advance, saying that if today's recall has a bug the baseline locks that bug in, and it was also workspace-bound, so it skipped in CI entirely.
Where the human-memory comparison holds
I was wary of this comparison, because it usually gets reached for as decoration. While most of it is decoration, 2 specific correspondences turned out to be load-bearing.
A gap the system cannot perceive
The failure mode was never "I don't know." Asked about something it had stored but could not reach by meaning, the system answered from whatever keyword matches happened to surface, and it did not experience the difference as a gap at all. The answer was usually still useful, which is the part that matters here, because a useless answer would have been reported within a week.
That structure is what confabulation shows in humans with certain memory impairments: not a gap the person perceives, but a smooth account filling a gap they cannot perceive. The similarity is real and it is specifically about perception rather than quality, because in both cases the retrieval failure is invisible to the system doing the retrieving, while the machinery creating the output sits downstream of the break and keeps working perfectly.
Consolidation is a separate process from storage
The architecture ended up treating raw records and derived structure as different things governed by different rules. I did not arrive there through the analogy; I arrived there because nothing else worked, and noticed the analogy afterwards.
| Layer | What it holds | Mutability |
|---|---|---|
| L0 | Memories and claims as written, the canonical log | Immutable, never rewritten |
| L1 | Typed associative edges between records | Derived, rebuildable |
| L2 | Threads, patterns, clusters | Derived, rebuildable |
| L3 | Insights, stored as ordinary claims so noticing compounds | Feeds back into L0 |
| L4 | Abstractions: traits, habits, values, failure modes | Derived, hypothesis-gated |
The whole derived layer can be deleted without losing anything the user wrote, because rebuilding it is just replaying the log. Retrieval across it works through activation spreading, following edges outward from a matched record rather than scoring every record independently, which sits closer to how associative recall is usually modelled than a flat vector search does.
Where the comparison misleads
There are 2 places, and I think both of them are where popular writing about AI memory goes wrong.
Human forgetting is adaptive; machine forgetting is a bug. People shed detail and keep gist, and that compression creates most of what makes recall useful in the first place. This system forgets only through failure, whether that is a dropped record or an overflowed context window. Every attempt I made at building deliberate decay created something that felt like data loss rather than abstraction, because the system has no equivalent of knowing what mattered.
Confidence is computed, not felt. Human memory arrives with a sense of certainty attached, which is unreliable, but which is at least generated by the same process that produced the memory. A language model will assert a confidence score on request and that number is worth nothing, because it is produced by the text generator rather than by the retrieval mechanism.
That second one had a direct consequence in the design. Model-asserted confidence values are discarded at parse time, and confidence is computed instead out of evidence counts and whether earlier predictions were later borne out. Before any abstraction about a person can be promoted from hypothesis to provisional, the system has to run and log an explicit counter-evidence search and come back with nothing unresolved.
No search, no ladder. A belief cannot be promoted on accumulated supporting evidence alone; the system has to go looking for the opposite first and record that it did. Confirmation bias is the default behavior of anything that only counts supporting evidence, so it has to be designed against explicitly rather than assumed away.
What retrieval does now
After the repair every source reaches the ranker carrying real vectors. The measurements below run end-to-end through the real writers and a real embedding model, using paraphrased queries chosen so that keyword matching scores zero, which is a test a keyword system cannot pass.
| Source | Query (paraphrase) | Semantic | Keyword |
|---|---|---|---|
| Conversation | "that chat about moving our SQL storage engine" | 0.723 | 0.00 |
| Entry | "experiments with bread fermentation and baking" | 0.649 | n/a |
| Upload | "what does my rental contract say about renting to someone else" | 0.585 | 0.00 |
| Claim | "the user's weekly endurance jogging habit" | 0.519 | 0.00 |
The sourdough entry contained no word that appears in its query, and the lease clause was found by a question sharing no vocabulary with it at all. The gap between the semantic column and the keyword column is the entire value of the mechanism, and it is exactly what was missing for 4 months without creating a single visible error.