Building an AI Agent with Persistent Memory

Most AI assistants start fresh every conversation. Hermes Agent doesn't.

The Memory Stack

Hermes uses a three-layer memory system:

1. SQLite FTS5 (Full-Text Search) — Fast keyword search across all past conversations. Every session is indexed and searchable within milliseconds.

2. Vector Embeddings — Semantic similarity search. When you mention a past project, Hermes finds related context even if you don't use the exact same words.

3. Knowledge Graph (gbrain) — Structured relationship storage. Pages, tags, links, and timeline entries form a connected graph of knowledge that persists across sessions.

Implementation

The memory system is built on SQLite with the sqlite-vec extension for vector search:

# Memory is injected automatically at conversation start
memory_context = session_search("relevant past context")
system_prompt += f"\n\nRelevant context:\n{memory_context}"

No external services. No cloud dependencies. Your data stays on your machine.

Why This Matters

• Continuity: Reference past decisions without repeating yourself
• Learning: The agent gets smarter about your preferences over time
• Autonomy: Make complex multi-session plans that actually execute

Try It

pip install hermes-agent
hermes setup

Open source on GitHub. MIT licensed.