Why LLMs Forget Everything: A Developer's Guide to Context Windows, RAG, and Memory Architectures

Great point. I think that's one of the biggest challenges with persistent memory systems deciding what deserves to become memory versus what should remain transient context.

My view is that not everything should be stored verbatim. Stable facts (user preferences, long-term goals, project context, etc.) are good candidates for persistence, while temporary conversational details are often better summarized or discarded. Compression, however, introduces the risk of losing information that later becomes important, so retrieval quality becomes just as critical as storage.

That's also why I find work reuse interesting. Rather than replaying or compressing entire histories, systems can sometimes reuse previously computed work when the new request is semantically similar, avoiding unnecessary context reconstruction altogether. There isn't a single best approach, it's often a combination of retrieval, summarization, and selective reuse depending on the application.

Thanks for bringing this up!