--- title: "Persistent Agent with Memory" description: "Memory/state handling is what separates a stateless chatbot demo from a real assistant product — directly relevant to AI Engineer roles building user-facing..." source: "https://confidentprep.com/paths/ai-engineering-fundamentals/persistent-agent/" path: "AI Engineering Fundamentals" repository: "https://github.com/confidentprep14-web/ai-engineering-projects/tree/main/persistent-agent" token_estimate: 3359 --- # Persistent Agent with Memory ## Overview A CLI chatbot that stores conversation history and key facts in SQLite and recalls them across sessions — your name, preferences, and past context are available every time you return. Combined with the tool-calling loop from the last project, this is the point where your agent stops being a single-session toy and starts behaving like a real product. ## What to do **Path:** 1 **Position:** 7 of 12 **Difficulty:** 🟢 Local — one API key, no Docker, no cloud **Estimated time:** 3–4 hours **AWS cost:** None --- ### Agent Pickup Instructions **This spec is self-contained. Build this project without reading any other spec file.** ```bash mkdir -p path-1/p1-07-persistent-agent cd path-1/p1-07-persistent-agent pytest tests/ -v python src/main.py # start a session, say "My name is Alex" # Press Ctrl-C to end, re-run to verify Alex is remembered python src/main.py --show-memory # inspect stored facts python src/main.py --clear-memory # wipe the slate ``` **Done when:** - [ ] `pytest tests/ -v` → minimum 5 tests, all green - [ ] Messages persist to SQLite between runs - [ ] Agent recalls a name or preference mentioned in a previous session - [ ] `--show-memory` prints all stored facts as a table - [ ] `--clear-memory` wipes history and facts with a confirmation prompt - [ ] Memory hit rate logged on exit - [ ] No file in `src/` exceeds 200 lines --- ### What this project is An agent that remembers you across sessions. It stores conversation history and extracted facts in SQLite. When you restart it, it loads prior history and injects relevant facts into the system prompt. The agent extracts memorable facts automatically ("the user's name is Alex", "prefers concise answers") and uses them to personalise subsequent responses. This is the bridge from a stateless CLI tool to something worth serving over HTTP — because an agent that remembers users is genuinely useful. --- ### What the learner achieves "I built a persistent agent that stores conversation history and extracts key facts to SQLite, recalls them across sessions, and logs a memory hit rate — the fraction of responses where prior memory was actually relevant." --- ### Folder structure ``` p1-07-persistent-agent/ ├── README.md ├── GUIDE.md ├── .env.example ├── requirements.txt ├── src/ │ ├── main.py ← CLI entry point │ ├── llm.py ← provider-agnostic LLM wrapper │ ├── memory.py ← SQLite memory store │ └── agent.py ← agent loop with memory injection └── tests/ └── test_memory.py ``` --- ### .env.example ```bash LLM_PROVIDER=anthropic LLM_API_KEY= LLM_MODEL= # SQLite database path MEMORY_DB_PATH=./agent_memory.db # Number of recent messages to load as active context RECENT_MESSAGES_LIMIT=20 # Number of top facts to inject into system prompt TOP_FACTS_LIMIT=10 ``` --- ### requirements.txt ``` anthropic==0.40.0 openai==1.58.0 ollama==0.4.4 python-dotenv==1.0.1 ``` --- ### src/ — what to implement #### src/llm.py **`get_completion(messages: list[dict], system: str = "") -> str`** - Standard router: anthropic / openai / ollama - Raises `ValueError` for unknown provider #### src/memory.py Handles all SQLite I/O. Database path from `MEMORY_DB_PATH` env var. **Schema (create on first run if not exists):** ```sql CREATE TABLE IF NOT EXISTS messages ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT NOT NULL, role TEXT NOT NULL, -- 'user' or 'assistant' content TEXT NOT NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); CREATE TABLE IF NOT EXISTS facts ( id INTEGER PRIMARY KEY AUTOINCREMENT, fact_key TEXT NOT NULL, -- e.g. "user_name", "prefers_language" fact_value TEXT NOT NULL, -- e.g. "Alex", "Python" confidence REAL NOT NULL, -- 0.0–1.0, LLM-assessed confidence source_message_id INTEGER, -- which message this was extracted from created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, UNIQUE(fact_key) -- one value per key; INSERT OR REPLACE ); ``` **`MemoryStore` class:** `__init__(self, db_path: str)` — opens SQLite connection, creates tables if needed `save_message(self, session_id: str, role: str, content: str) -> int` — insert, return rowid `load_recent_messages(self, session_id: str, limit: int) -> list[dict]` — return `[{"role": ..., "content": ...}, ...]` ordered by `created_at` ASC `save_fact(self, key: str, value: str, confidence: float, source_id: int = None)` — INSERT OR REPLACE `load_all_facts(self) -> list[dict]` — return `[{"key": ..., "value": ..., "confidence": ...}, ...]` `load_top_facts(self, limit: int) -> list[dict]` — return highest-confidence facts, limit N `clear_all(self)` — DELETE FROM messages; DELETE FROM facts `get_message_count(self) -> int` — total stored messages `get_fact_count(self) -> int` — total stored facts #### src/agent.py **`extract_facts(message_content: str, role: str) -> list[dict]`** - Only extract from `role == "user"` messages - System prompt: "Extract memorable facts from this user message. Return JSON array of objects: `[{\"key\": \"fact_category\", \"value\": \"fact_value\", \"confidence\": 0.0-1.0}]`. Examples: user_name, user_role, preferred_language, project_name. Return empty array `[]` if no memorable facts present. Return ONLY JSON." - Parse result; return list of `{"key": str, "value": str, "confidence": float}` - On parse failure: return `[]` (never raise) **`build_system_prompt(facts: list[dict]) -> str`** - Base: "You are a helpful assistant with memory of past conversations." - If facts non-empty: append "Known facts about the user:\n" + each fact as `- {key}: {value} (confidence: {confidence:.0%})` - Return complete system prompt **`run_session(memory: MemoryStore, session_id: str) -> dict`** - Returns `{"turns": N, "facts_extracted": N, "memory_hits": N}` - Load `RECENT_MESSAGES_LIMIT` messages from memory - Load `TOP_FACTS_LIMIT` facts from memory - Build system prompt with facts - Print memory summary: `📚 Loaded {message_count} prior messages | {fact_count} known facts` - Conversation loop: 1. `user_input = input("You: ")` 2. `msg_id = memory.save_message(session_id, "user", user_input)` 3. Extract facts from user_input; save each to memory 4. Build full message list (loaded history + current user message) 5. Call LLM with system prompt + messages 6. Print `Assistant: {response}` 7. `memory.save_message(session_id, "assistant", response)` 8. Track "memory hit": if any fact appears in the response (simple substring check), increment hit counter - On `quit` / Ctrl-C: print stats and return result #### src/main.py CLI: `python src/main.py [--show-memory] [--clear-memory] [--session-id ]` - Default session_id: `"default"` (single-user local tool) - `--show-memory`: print facts table and message count, exit - `--clear-memory`: prompt "Clear all memory? (yes/no): ", require exact "yes", call `memory.clear_all()`, exit - Normal mode: call `run_session()` - On exit: print `Session ended: {turns} turns | {facts_extracted} new facts extracted | Memory hits: {memory_hits}/{turns} ({pct:.0%})` --- ### tests/ — what to test **File:** `tests/test_memory.py` — use a temporary SQLite file (tempfile.mktemp(suffix='.db')). **Test 1 — save and load messages round-trips:** Save 3 messages. Load with limit=10. Assert all 3 returned in order. **Test 2 — load respects limit:** Save 10 messages. Load with limit=3. Assert only 3 returned (most recent 3). **Test 3 — save_fact and load_top_facts:** Save fact `("user_name", "Alice", 0.95)` and `("user_role", "engineer", 0.7)`. Load top facts with limit=2. Assert user_name fact is in results. **Test 4 — UNIQUE constraint replaces existing fact:** Save `("user_name", "Alice", 0.9)` then `("user_name", "Bob", 0.95)`. Load facts. Assert only one user_name fact exists and its value is "Bob". **Test 5 — clear_all removes all records:** Save 3 messages and 2 facts. Call `clear_all()`. Assert `get_message_count() == 0` and `get_fact_count() == 0`. **Test 6 — extract_facts returns empty list on non-factual message:** Mock `llm.get_completion` to return `"[]"`. Call `extract_facts("What time is it?", "user")`. Assert result is `[]`. --- ### README.md content ```markdown # Persistent Agent with Memory A CLI chatbot that remembers you. It stores conversation history and key facts in SQLite and recalls them across sessions — your name, preferences, and past context are available every time you return. ## Setup ```bash cd p1-07-persistent-agent cp .env.example .env pip install -r requirements.txt ``` ## Run ```bash ## Start chatting (creates agent_memory.db on first run) python src/main.py ## View stored facts python src/main.py --show-memory ## Clear all memory python src/main.py --clear-memory ``` Expected output: ``` 📚 Loaded 0 prior messages | 0 known facts You: My name is Alex and I work in Python Assistant: Nice to meet you, Alex! Python is a great language... You: quit Session ended: 1 turns | 1 new facts extracted | Memory hits: 0/1 (0%) ``` Next session: ``` 📚 Loaded 2 prior messages | 1 known facts - user_name: Alex (confidence: 95%) You: What's my name? Assistant: Your name is Alex — you mentioned it in our last conversation. Session ended: 1 turns | 0 new facts extracted | Memory hits: 1/1 (100%) ``` ## Tests ```bash pytest tests/ -v ``` Tests verify SQLite persistence, fact deduplication, and message limit enforcement. No API key required. ## What to try next - Add a `--session-id` flag so you can run multiple independent memory sessions - Extract facts from assistant responses too (the agent learns from its own answers) - Add a fact expiry: facts older than 30 days are downweighted in the system prompt ``` --- ### GUIDE.md content ```markdown # Build guide: Persistent Agent with Memory ## What you're building and why it matters Memory is what separates a chatbot from an assistant. Every commercial AI assistant — ChatGPT, Claude.ai, Gemini — has some form of memory. The fundamental pattern is simple: extract facts from conversations, store them, inject them into future system prompts. The engineering challenge is deciding what to remember, how long to keep it, and how to inject it without blowing up the context window. This project gives you that foundation with a real database, not an in-memory dict that disappears on restart. ## The decision that matters in this build **Where to inject memory: system prompt or user message?** Facts injected into the system prompt arrive with higher authority — the model treats them as persistent context about who it is talking to. Facts injected as a prior user message can be "argued with" by subsequent conversation. Put persistent facts (name, role, preferences) in the system prompt. Put recent context (last conversation summary) in the message history. ## What will break **Fact extraction will hallucinate.** If the user says "I think Python is overrated," the extractor might extract `preferred_language: Python`. Add a confidence score and only inject facts with confidence >= 0.8. Low-confidence "facts" are noise. **Loading too many messages blows up the context window.** `RECENT_MESSAGES_LIMIT=20` is a safe default. Set it to 100 and you'll eventually hit token limits. For long-running sessions, consider summarising old messages (the pattern from Project 4) before injection. ## How to talk about this in an interview "I built a persistent agent that extracts named facts from user messages — name, preferences, context — stores them in SQLite with a confidence score, and injects the top-confidence facts into every future system prompt. I measured memory hit rate as the fraction of responses where a prior fact appeared in the answer. It's the pattern behind every 'personalised' AI assistant." ``` --- ### Failure modes to handle | Failure | Where | How to handle | |---------|-------|---------------| | SQLite DB locked | `memory.py` | Catch `sqlite3.OperationalError`, retry once, then raise with message | | Fact extraction JSON parse failure | `agent.extract_facts()` | Return `[]` — never raise, never block the conversation | | Empty user input | `agent.run_session()` | Skip save and LLM call, re-prompt | | `--clear-memory` without "yes" | `main.py` | Print "Cancelled" and exit cleanly | | MEMORY_DB_PATH directory missing | `memory.py` | Create parent directory if it doesn't exist | --- ### The metric this project measures **Memory hit rate** — fraction of assistant responses where a stored fact was used. Logged on exit: `Memory hits: {N}/{total} ({pct:.0%})` Target: >50% in sessions after the first (once facts are stored). Low rate suggests facts are not being extracted or injected correctly. **Facts extracted** — how many new facts were extracted in the current session. ### Model source currency - Anthropic model list: https://docs.anthropic.com/en/docs/about-claude/models/overview - OpenAI model list: https://platform.openai.com/docs/models - verifiedOn: 2026-08-07 - `LLM_MODEL` is required and has no implicit default; choose a supported model for the configured provider at setup time. ## Source code A reference implementation of Persistent Agent with Memory — README, guide, source and tests — is at https://github.com/confidentprep14-web/ai-engineering-projects/tree/main/persistent-agent.