Hermes Agent Persistent Memory System: Making AI Actually Remember You and Your Projects

After more than three months of working with AI tools, you've almost certainly hit this wall: open a new session, re-explain everything from scratch. "This project uses TypeScript, not JavaScript," "My SSH port is 2222 not 22," "Don't run Docker with sudo, I'm in the docker group" — the same sentences, repeated who knows how many times.

This isn't the model being dumb. It's an architecture problem. Most AI tools treat "current conversation" and "cross-session memory" as two separate things — the former exists, the latter doesn't, or requires you to maintain it manually.

Hermes Agent's memory system was designed from the start to connect these two things.

The Two Files of the Memory System

Hermes's persistent memory lives in two documents:

Both files are stored in ~/.hermes/memories/ and injected into the system prompt at the start of every session — as a "frozen snapshot" that won't update during the session (though the agent can modify the on-disk content mid-session; changes take effect in the next session).

The frozen snapshot design is intentional: it preserves the LLM's prefix cache, keeping token costs predictable — the context doesn't change with every turn because memory updated in real time.

How the Agent Sees Its Own Memory

At session start, memory appears in the system prompt in this format:

══════════════════════════════════════════════
MEMORY (your personal notes) [67% — 1,474/2,200 chars]
══════════════════════════════════════════════
User's project is a Rust web service at ~/code/myapi using Axum + SQLx§
This machine runs Ubuntu 22.04, has Docker and Podman installed§
SSH port is 2222, key at ~/.ssh/staging_ed25519

The agent sees this and knows "I remember these facts, no need to ask the user again." The 67% capacity usage also tells it "getting full — consolidate before adding next time."

Agent-Managed Memory: The memory Tool

Hermes agents use the memory tool to operate on memory — not because you reminded them, but because they autonomously decide "this is worth saving":

• add: Add a new memory entry
• replace: Update an existing memory entry (uses substring matching, no need for exact original text)
• remove: Remove memory that's no longer relevant

# After agent learns your SSH configuration
memory(action="add", target="memory",
       content="SSH port 2222, key at ~/.ssh/staging_ed25519")
 
# You tell it "actually I switched to port 3000"
memory(action="replace", target="memory",
       old_text="SSH port 2222",
       content="SSH port 3000, key at ~/.ssh/staging_ed25519")

The substring matching design means the agent doesn't need to remember the complete original text — just a unique fragment to locate that memory entry.

What to Store and What Not to Store

This is the key to whether the memory system actually delivers value — capacity is limited, and every memory entry consumes resources.

What the Agent Should Proactively Save

Environment facts: "This machine runs macOS 14, Homebrew installed, Docker Desktop running, shell is zsh + oh-my-zsh" — one entry covers a lot of background knowledge that might otherwise get asked repeatedly.

Project conventions: "~/code/api uses Go 1.22, sqlc for DB queries, chi router, run tests with make test" — the agent won't need to re-explore the next time it touches this project.

Corrected behaviors: "Don't run Docker with sudo, user is in the docker group" — don't repeat past mistakes; this is one of the most valuable scenarios for the memory system.

Completed significant work: "Migrated database from MySQL to PostgreSQL on 2026-04-15, migration scripts at ~/code/api/migrations/" — lets the agent know the historical state.

What Not to Store

Overly vague information: "User is working on a project" — no actionable value, takes up space and remembers nothing useful.

Easily re-queryable facts: "Python 3.12 supports f-string nesting" — the agent can find this with a search; not worth occupying memory capacity.

Large raw data: Log files, large code blocks — too space-intensive; use session_search to find historical conversations instead.

Content already in context files: If .hermes.md or AGENTS.md already contains project documentation, don't store a copy in MEMORY.md too.

Capacity Management: What Happens When It's Full

When memory is full, the agent receives a tool response like this:

{
  "success": false,
  "error": "Memory at 2,100/2,200 chars. Adding this entry (250 chars) would exceed the limit.",
  "current_entries": ["..."]
}

The agent should then: read existing memory → find entries to consolidate or delete → reorganize, then add the new entry.

What a good memory entry looks like:

# Good: high information density, one entry covers multiple facts
User runs macOS 14, uses Homebrew, Docker Desktop and Podman installed. Shell: zsh with oh-my-zsh. Editor: VS Code with Vim keybindings.

# Good: specific and actionable
Project ~/code/api uses Go 1.22, sqlc for DB queries, chi router. Run tests with 'make test'. CI via GitHub Actions.

# Bad: too vague
User has a project they're working on.

# Bad: too verbose
On April 15th, the user mentioned that they have a project located at ~/code/api. I discovered it uses Go version 1.22 and...

Session Search: The Long Tail Beyond Memory

The memory system has a capacity limit, but all your conversations with Hermes are stored in ~/.hermes/state.db (SQLite + FTS5 full-text search).

The session_search tool lets the agent query historical conversations:

hermes sessions list   # Browse past sessions

When the agent internally calls session_search, it finds relevant past conversations and uses Gemini Flash to summarize them — letting it answer questions like "last week we discussed that auth migration plan, what did we decide in the end?"

The two are complementary: memory stores "common knowledge," session search finds "history."

The Real Effect of the Memory System

After using the memory system, opening a new Hermes session feels different — not "Hello, I'm an AI, how can I help you?" but "I remember your API project is using Go 1.22, chi router, with tests run via make test."

This difference becomes very noticeable with long-term use: your conversations with the agent increasingly feel like talking to someone who genuinely understands your work environment, rather than onboarding a new hire to your system every single time.

The design trade-off is: capacity is limited. You can't stuff everything in — you actually need to develop the judgment of "is this worth occupying memory space" — which is consistent with Hermes's overall design philosophy: agents should have constraints and discipline, not expand without limits.

Further Reading

• Hermes Agent Beginner Course: Series Overview and Reading Map
• The Skills System: Getting AI Agents to Do the Right Thing at the Right Time
• Automation: Cron Scheduling, Parallel Sub-Agents, and Code Execution