conversation-memory

Conversation Memory

Persistent memory systems for LLM conversations including short-term, long-term, and entity-based memory

Capabilities

• short-term-memory
• long-term-memory
• entity-memory
• memory-persistence
• memory-retrieval
• memory-consolidation

Prerequisites

• Knowledge: LLM conversation patterns, Database basics, Key-value stores
• Skills_recommended: context-window-management, rag-implementation

Scope

• Does_not_cover: Knowledge graph construction, Semantic search implementation, Database administration
• Boundaries: Focus is memory patterns for LLMs, Covers storage and retrieval strategies

Ecosystem

Primary_tools

• Mem0 - Memory layer for AI applications
• LangChain Memory - Memory utilities in LangChain
• Redis - In-memory data store for session memory

Patterns

Tiered Memory System

Different memory tiers for different purposes

When to use: Building any conversational AI

interface MemorySystem { // Buffer: Current conversation (in context) buffer: ConversationBuffer;

// Short-term: Recent interactions (session) shortTerm: ShortTermMemory;

// Long-term: Persistent across sessions longTerm: LongTermMemory;

// Entity: Facts about people, places, things entity: EntityMemory; }

class TieredMemory implements MemorySystem { async addMessage(message: Message): Promise<void> { // Always add to buffer this.buffer.add(message);

// Extract entities const entities = await extractEntities(message); for (const entity of entities) { await this.entity.upsert(entity); }

// Check for memorable content if (await isMemoryWorthy(message)) { await this.shortTerm.add({ content: message.content, timestamp: Date.now(), importance: await scoreImportance(message) }); } }

async consolidate(): Promise<void> { // Move important short-term to long-term const memories = await this.shortTerm.getOld(24 60 60 * 1000); for (const memory of memories) { if (memory.importance > 0.7 || memory.referenced > 2) { await this.longTerm.add(memory); } await this.shortTerm.remove(memory.id); } }

async buildContext(query: string): Promise<string> { const parts: string[] = [];

// Relevant long-term memories const longTermRelevant = await this.longTerm.search(query, 3); if (longTermRelevant.length) { parts.push('## Relevant Memories\n' + longTermRelevant.map(m => - ${m.content}).join('\n')); }

// Relevant entities const entities = await this.entity.getRelevant(query); if (entities.length) { parts.push('## Known Entities\n' + entities.map(e => - ${e.name}: ${e.facts.join(', ')}).join('\n')); }

// Recent conversation const recent = this.buffer.getRecent(10); parts.push('## Recent Conversation\n' + formatMessages(recent));

return parts.join('\n\n'); } }

Entity Memory

Store and update facts about entities

When to use: Need to remember details about people, places, things

interface Entity { id: string; name: string; type: 'person' | 'place' | 'thing' | 'concept'; facts: Fact[]; lastMentioned: number; mentionCount: number; }

interface Fact { content: string; confidence: number; source: string; // Which message this came from timestamp: number; }

class EntityMemory { async extractAndStore(message: Message): Promise<void> { // Use LLM to extract entities and facts const extraction = await llm.complete(` Extract entities and facts from this message. Return JSON: { "entities": [ { "name": "...", "type": "...", "facts": ["..."] } ]}

Message: "${message.content}" `);

const { entities } = JSON.parse(extraction); for (const entity of entities) { await this.upsert(entity, message.id); } }

async upsert(entity: ExtractedEntity, sourceId: string): Promise<void> { const existing = await this.store.get(entity.name.toLowerCase());

if (existing) { // Merge facts, avoiding duplicates for (const fact of entity.facts) { if (!this.hasSimilarFact(existing.facts, fact)) { existing.facts.push({ content: fact, confidence: 0.9, source: sourceId, timestamp: Date.now() }); } } existing.lastMentioned = Date.now(); existing.mentionCount++; await this.store.set(existing.id, existing); } else { // Create new entity await this.store.set(entity.name.toLowerCase(), { id: generateId(), name: entity.name, type: entity.type, facts: entity.facts.map(f => ({ content: f, confidence: 0.9, source: sourceId, timestamp: Date.now() })), lastMentioned: Date.now(), mentionCount: 1 }); } } }

Memory-Aware Prompting

Include relevant memories in prompts

When to use: Making LLM calls with memory context

async function promptWithMemory( query: string, memory: MemorySystem, systemPrompt: string ): Promise<string> { // Retrieve relevant memories const relevantMemories = await memory.longTerm.search(query, 5); const entities = await memory.entity.getRelevant(query); const recentContext = memory.buffer.getRecent(5);

// Build memory-augmented prompt const prompt = ` ${systemPrompt}

User Context

${entities.length ? Known about user:\n${entities.map(e => - ${e.name}: ${e.facts.map(f => f.content).join('; ')} ).join('\n')} : ''}

${relevantMemories.length ? Relevant past interactions:\n${relevantMemories.map(m => - [${formatDate(m.timestamp)}] ${m.content} ).join('\n')} : ''}

Recent Conversation

${formatMessages(recentContext)}

Current Query

${query} `.trim();

const response = await llm.complete(prompt);

// Extract any new memories from response await memory.addMessage({ role: 'assistant', content: response });

return response; }

Sharp Edges

Memory store grows unbounded, system slows

Severity: HIGH

Situation: System slows over time, costs increase

Symptoms:

• Slow memory retrieval
• High storage costs
• Increasing latency over time

Why this breaks: Every message stored as memory. No cleanup or consolidation. Retrieval over millions of items.

Recommended fix:

// Implement memory lifecycle management

class ManagedMemory { // Limits private readonly SHORT_TERM_MAX = 100; private readonly LONG_TERM_MAX = 10000; private readonly CONSOLIDATION_INTERVAL = 24 60 60 * 1000;

async add(memory: Memory): Promise<void> { // Score importance before storing const score = await this.scoreImportance(memory); if (score < 0.3) return; // Don't store low-importance

memory.importance = score; await this.shortTerm.add(memory);

// Check limits await this.enforceShortTermLimit(); }

async enforceShortTermLimit(): Promise<void> { const count = await this.shortTerm.count(); if (count > this.SHORT_TERM_MAX) { // Consolidate: move important to long-term, delete rest const memories = await this.shortTerm.getAll(); memories.sort((a, b) => b.importance - a.importance);

const toKeep = memories.slice(0, this.SHORT_TERM_MA