memory-mcp

memory-mcp-server

Unified semantic memory + time-series intelligence layer for OpenHome abilities.

Architecture: memory tiers

Tier 1    — Semantic memory       entities, memories, relations, vectors
Tier 1.5  — Episodic memory       conversation sessions, turn-by-turn transcripts
Tier 1.75 — Working memory        short-lived task scratchpads, promote-on-close
Tier 2    — Time-series store     readings (numeric/categorical/composite), rollups, schedule
Tier 3    — Pattern engine        background task: promotes stable trends → Tier 1 memories
Tier 4    — Prospective memory    intentions: trigger_text → action_text, checked on each turn
Tier 5    — Spatial memory        last-known object locations with confidence decay

The pattern engine closes the loop: raw sensor data (Tier 2) automatically becomes searchable, natural-language memory ("Brian's temperature preference is consistently 68°F") that any ability can recall semantically (Tier 1).

Files

| File | Purpose | |-------------------|------------------------------------------------------| | server.py | MCP server (stdio transport, all 35 tools) | | api.py | FastAPI HTTP wrapper + admin UI mount | | admin.py | Admin UI router (served at /admin) | | voice_routes.py | Speaker identity API (/voices/* — enroll, merge, update voiceprints) | | graph_routes.py | Entity graph API (/graph SPA + /api/graph data endpoint) | | exporters/markdown.py | Markdown two-way sync — export and import of Obsidian-compatible .md files | | reembed.py | Utility to re-embed all memories when swapping models| | templates/admin | Jinja2 HTML templates for the admin UI | | templates/graph | vis.js entity graph SPA template | | integrations/ | Standalone tools that connect external systems to memory-mcp via HTTP |

Documentation

| Doc | Contents | |---|---| | docs/overview.md | What it is, motivation, integration patterns (OpenHome, HA, MQTT, IoT) | | docs/installation.md | Requirements, step-by-step setup, first run, verification | | docs/quickstart.md | First entity, memory, reading — common operations with curl examples | | docs/api-reference.md | Full HTTP API — every endpoint, request/response shapes, examples | | docs/admin-ui.md | Admin dashboard guide, pages, reading confidence, prune, security | | docs/ai-backend.md | AI backend config, provider examples, model swap guide | | docs/pattern-engine.md | How detectors work, all 5 detector types, how to add new ones | | docs/retention.md | Retention policy config, what gets deleted, storage estimates | | docs/deployment.md | systemd service, Docker Compose, reverse proxy, environment config | | docs/maintenance.md | Keeping it healthy — backups, upgrades, model swaps, reembed.py walkthrough | | docs/testing.md | Running tests, fixture design, what is and isn't covered | | docs/troubleshooting.md | Common errors, what they mean, how to fix them | | integrations/README.md | Integration index — MQTT bridge, HA state poller, OpenHome, Cloudflare | | integrations/background_example.py | Background worker template — health data, environment sensors, weather | | integrations/ha_state_poller.py | Pull-based HA state poller — polls HA REST API, pushes to memory-mcp | | integrations/homeassistant/README.md | HA package setup — rest_commands, automations, scripts | | integrations/openhome/README.md | OpenHome ability setup — background daemon + recall skill | | integrations/cloudflare/README.md | Cloudflare Tunnel setup — safe internet exposure for cloud callers |

Setup

# Install deps
pip install -r requirements.txt

# Pull embedding and LLM models (Ollama default)
ollama pull nomic-embed-text
ollama pull llama3.2

# Run MCP server (for OpenHome abilities)
python server.py

# Run HTTP API + admin UI (for HA webhooks, Node-RED, scripts)
python api.py          # listens on :8900
                       # admin UI at http://localhost:8900/admin/

AI Backend

Uses the OpenAI-compatible API (/v1/embeddings + /v1/chat/completions). Works with Ollama, OpenAI, LM Studio, Together AI, or any compatible provider. Configure via environment variables — no code changes needed:

# Default (local Ollama)
export MEMORY_AI_BASE_URL=http://localhost:11434/v1
export MEMORY_EMBED_MODEL=nomic-embed-text   # 768-dim
export MEMORY_LLM_MODEL=llama3.2

# OpenAI
export MEMORY_AI_BASE_URL=https://api.openai.com/v1
export MEMORY_AI_API_KEY=sk-...
export MEMORY_EMBED_MODEL=text-embedding-3-small
export MEMORY_EMBED_DIM=1536
export MEMORY_LLM_MODEL=gpt-4o-mini

Split backends are supported — embed and LLM can run on different hosts:

# nomic-embed-text on a Raspberry Pi 4, LLM on a GPU machine
export MEMORY_AI_BASE_URL=http://pi4.local:11434/v1
export MEMORY_LLM_BASE_URL=http://gpu-host.local:11434/v1

See docs/ai-backend.md for full configuration guide, provider examples, and split backend setup.

Source trust tiers

Every memory carries a trust tier that controls conflict resolution. When a new fact is written, it can only supersede an existing contradicting memory if its trust is equal or higher. Lower-trust sources cannot overwrite what you explicitly told the system.

| Tier | Label | When to use | |---|---|---| | 5 | user | Direct user statements, manual entries via admin UI | | 4 | hardware | Verified sensors, signed device data | | 3 | system | Pattern engine promotions, LLM-extracted facts | | 2 | inferred | Working memory promotions, low-confidence extractions | | 1 | external | Third-party imports, unverified webhooks |

Example: a sensor (tier 4) recording "bedroom temperature is 68°F" will not overwrite an explicit user statement (tier 5) "I keep my bedroom at 66°F at night."

Set via the source_trust parameter on remember / POST /remember. Defaults to MEMORY_TRUST_DEFAULT_REMEMBER (env var, default: 5=user).

Confidence decay

Memory confidence decays automatically over time so stale facts fade gracefully instead of accumulating indefinitely. Decay runs every hour in the pattern engine.

Formula: confidence = confidence × 2^(−days / halflife)

| Category | Default half-life | Meaning | |---|---|---| | preference, habit, routine | 90 days | Stable — takes months to fade | | insight, general | 90 days | Same default | | relationship | 90 days | Same default | | Location records | 24 hours | Unconfirmed location drops to 50% overnight |

Configure per-category overrides: ``bash export MEMORY_DECAY_HALFLIFE_DAYS=90 # global default export MEMORY_DECAY_CATEGORY_HALFLIFE='{"preference": 180, "insight": 30}' export MEMORY_LOCATION_DECAY_HALFLIFE_HOURS=24 ``

Use GET /fading (or get_fading_memories) to surface memories whose confidence has dropped below a threshold — a prompt to confirm or update them.

AI call timeout

export MEMORY_AI_TIMEOUT=30    # seconds; applies to both embed() and LLM calls

Increase if using a slow local model. LLM calls use max(MEMORY_AI_TIMEOUT, 60) to guarantee at least 60 seconds for generation.

Testing

pip install -r requirements.txt
python -m pytest                     # full suite (722 tests, no Ollama needed)
python -m pytest tests/test_tools.py # just tool tests
python -m pytest tests/test_spatial.py # just spatial/location tests

See docs/testing.md for fixture design and conventions.

OpenHome SDK config

{
  "mcpServers": {
    "memory": {
      "command": "python",
      "args": ["/path/to/memory-mcp/server.py"]
    }
  }
}

Schema

TIER 1
  entities          id, name*, type, meta(JSON), created, updated
  memories          id, entity_id, fact, category, confidence, source, created, updated,
                    last_accessed, access_count, superseded_by
  relations         id, entity_a, entity_b, rel_type, meta(JSON), created,
                    valid_from, valid_until
  memory_vectors    rowid=memories.id, embedding FLOAT[768]   ← sqlite-vec

TIER 1.5
  sessions          id, entity_id, started_at, ended_at, summary, meta
  session_turns     id, session_id, role, content, ts

TIER 1.75
  working_memory_tasks   id, name, entity_id, status, ttl_ts, created, closed_at
  working_memory_slots   id, task_id, key, value(JSON), created, updated

TIER 2
  readings          id, entity_id, metric, unit, value_type, value_num,
                    value_cat, value_json, source, ts
                    (composite readings also decomposed into {metric}.{key} child rows)
  reading_rollups   id, entity_id, metric, bucket_type, bucket_ts,
                    count, avg_num, min_num, max_num, p10_num, p90_num, mode_cat
  rollup_watermarks entity_id, metric, last_ts   ← incremental build tracking
  schedule_events   id, entity_id, title, start_ts, end_ts, recurrence, meta, created

TIER 3
  promoted_patterns id, entity_id, metric, pattern_key, memory_id, detected

TIER 5
  locations         id, entity_id, container_id, container_name, confidence,
                    last_confirmed_ts, active, source, note, created
                    (active=1 → current location; active=0 → archived sighting)

Entity types (open — add any string)

person | house | room | device

Memory categories

preference | habit | routine | relationship | insight | general

Value types for readings

| value_type | field | example | |---------------|---------------|-------------------------------------------| | numeric | value_num | temperature=71.4, heart_rate=62 | | categorical | value_cat | mood="calm", presence="home" | | composite | value_json | {"mood":"calm","confidence":0.91} |

MCP Tools

Tier 1 — Semantic memory

| Tool | Description | |----------------|-------------------------------------------------------| | remember | Store a fact about any entity (embeds + indexes it) | | recall | Semantic search — multi-factor: cosine × recency × confidence | | get_context | Relevance-filtered context snapshot (preferred for ability use) | | get_profile | Full profile: memories + relationships + readings | | relate | Create directed relationship between entities | | unrelate | Soft-delete a relationship (sets valid_until, preserves history) | | forget | Delete a memory or entire entity | | extract_and_remember | LLM-powered fact extraction from conversation text |

Tier 2 — Time-series

| Tool | Description | |----------------|-------------------------------------------------------| | record | Ingest a reading (numeric/categorical/composite) | | query_stream | Query readings: raw or hour/day/week rollups | | get_trends | Natural-language trend summary for a metric | | schedule | Add a schedule event (one-off or recurring) |

Episodic memory

| Tool | Description | |------------------|-----------------------------------------------------| | open_session | Open a conversation session for an entity | | log_turn | Append a turn (user/assistant/system) to a session | | close_session | Close a session with optional summary | | get_session | Retrieve full session transcript |

Working memory (Tier 1.75)

| Tool | Description | |------------|----------------------------------------------------------------------| | wm_open | Open a task-scoped scratchpad; optional TTL and entity association | | wm_set | Write a key/value slot into an open task | | wm_get | Read one slot by key, or all slots with task metadata | | wm_list | List tasks filtered by status (open/closed/expired/all) and entity | | wm_close | Close a task; optionally promote slots to long-term memory |

FTS keyword recall + session search

| Tool | Description | |--------------------|------------------------------------------------------------------| | recall (mode=) | Add mode="keyword" or mode="hybrid" for FTS5/BM25 recall — no embedding model needed | | search_sessions | Full-text search across episodic session turns (FTS5/BM25) |

Token-budget context assembly

| Tool | Description | |-----------------------|---------------------------------------------------------------| | get_context_budget | Greedily fills a token budget with ranked memories + readings; recall_mode="keyword" for Pi/no-Ollama |

Prospective / intention memory (Tier 4)

| Tool | Description | |----------------------|----------------------------------------------------------------| | intend | Set a condition → action intention for an entity | | check_intentions | Check if current text triggers any active intentions (FTS5) | | dismiss_intention | Deactivate an intention | | list_intentions | List active (or all) intentions for an entity |

Spatial / location memory (Tier 5)

| Tool | Description | |--------------------|-------------------------------------------------------------------| | locate | Store or update where an object was last seen | | find | Return last known location with confidence + age ("where are my keys?") | | seen_at | Confirm object is still at its location; bumps confidence | | location_history | Full trail of past sightings in reverse-chronological order |

Cross-tier

| Tool | Description | |----------------|-------------------------------------------------------| | cross_query | Semantic search across memories AND live readings |

Maintenance

| Tool | Description | |------------------------|----------------------------------------------------------------| | prune | Delete raw readings older than RETENTION_DAYS (default 30d) | | get_fading_memories | Return memories whose confidence has fallen below a threshold, most faded first |

HTTP API endpoints (api.py)

GET  /health                    liveness + row counts
GET  /entities                  list all entities
POST /remember                  store a memory
POST /recall                    semantic search (mode=vector|keyword|hybrid, recency_weight, min_confidence)
POST /get_context               relevance-filtered context snapshot
GET  /profile/{entity_name}     full profile
POST /relate                    create relationship
POST /forget                    delete memory or entity
POST /record                    ingest a reading
POST /record/bulk               ingest multiple readings at once
POST /query_stream              query time-series
POST /get_trends                trend summary
POST /schedule                  add schedule event
POST /cross_query               unified search
POST /prune                     delete readings older than RETENTION_DAYS
GET  /fading                    memories below a confidence threshold (most faded first)

POST /open_session              open a conversation session for an entity
POST /log_turn                  append a turn (user/assistant/system) to a session
POST /close_session             close a session with optional summary
GET  /get_session/{id}          retrieve full session transcript
POST /extract_and_remember      LLM-extract facts from text and store as memories

POST /wm/open                   open a working-memory task scope
POST /wm/set                    set a key/value slot in an open task
POST /wm/get                    get one slot or all slots from a task
GET  /wm/list                   list tasks (?status=open|closed|expired|all&entity_name=X)
GET  /wm/{task_id}              get all slots and metadata for a task
POST /wm/close                  close a task (promote=true bundles slots into long-term memory)

POST /locate                    store/update last-known location of an object
POST /find                      return last known location with confidence + age
POST /seen_at                   confirm object is still at a location; bumps confidence
GET  /location_history/{name}   full location trail for an object

POST /search_sessions           keyword search across session turn content (FTS5/BM25)
POST /get_context_budget        token-budget context snapshot (greedy fill, truncated flag)
POST /intend                    store a prospective intention (trigger_text → action_text)
POST /check_intentions          check if text triggers any active intentions (FTS5)
POST /dismiss_intention         deactivate an intention (soft-delete)
GET  /intentions                list intentions (?entity_name=X&active_only=true)

GET  /voices/unknown            list unenrolled provisional speaker entities
POST /voices/enroll             rename provisional entity to real person
POST /voices/merge              merge provisional entity into enrolled entity
POST /voices/update_print       update voiceprint embedding (running average)

GET  /graph                     vis.js entity relationship graph (SPA)
GET  /api/graph                 entity graph data { nodes, edges }

GET  /export/markdown           export all entities as Obsidian-compatible Markdown
GET  /export/markdown/{name}    export single entity as .md file download
POST /import/markdown           import entities from Markdown files (two-way sync)

GET  /admin/                    dashboard
GET  /admin/entities            entity list
GET  /admin/entity/{name}       entity detail
GET  /admin/readings            readings stream
POST /admin/prune               prune (HTMX-friendly HTML response)

Usage examples

Ability: build context before responding to Brian

# Pull full profile (memories + latest readings + schedule)
profile = await mem.tool_get_profile("Brian")
# → inject as <memory>...</memory> in system prompt

# Or cross-query to pull what's relevant to the current question
context = await mem.tool_cross_query("how is Brian feeling today?")

Home Assistant → record sensor readings via HTTP

# configuration.yaml — rest_command
rest_command:
  push_temperature:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ room }}","metric":"temperature",
       "value":{{ temp }},"unit":"F","source":"ha","entity_type":"room"}

  push_presence:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ person }}","metric":"presence",
       "value":"{{ state }}","source":"ha"}

  push_mood:
    url: http://localhost:8900/record
    method: POST
    content_type: application/json
    payload: >
      {"entity_name":"{{ person }}","metric":"mood",
       "value":{"mood":"{{ mood }}","confidence":{{ conf }}},"source":"avatar_ability"}

Avatar ability: store inferred mood state

# After detecting mood from conversation
await mem.tool_record(
    entity_name="Brian",
    metric="mood",
    value={"mood": "focused", "confidence": 0.87},
    source="avatar_ability",
)
# The pattern engine will promote this to a memory like:
# "Brian's mood is predominantly 'focused' (72% of days)"

Query last week of temperature with daily rollup

result = await mem.tool_query_stream(
    entity_name="living_room",
    metric="temperature",
    granularity="day",
    start_ts=time.time() - 7 * 86400,
)

Cross-entity semantic query

result = await mem.tool_cross_query("who in the house prefers a cooler environment?")
# Returns: matching memories (explicit preferences) + live temperature readings scored by relevance

Swapping embedding models

# 1. Set the new model and dimension via env vars
export MEMORY_EMBED_MODEL=mxbai-embed-large
export MEMORY_EMBED_DIM=1024

# 2. Pull the new model
ollama pull mxbai-embed-large   # 1024-dim — richer but slower

# 3. Re-embed all memories (non-destructive — only rebuilds memory_vectors)
python reembed.py --dry-run     # preview
python reembed.py               # run it

Expanding the schema

• New entity types: pass any string — SQLite won't enforce the enum
• New metric names: pass any string to record() — fully dynamic
• New memory categories: same — add to enum in schema or free-text
• New pattern detectors: write _detect_*(entity_name, metric, data) → list[tuple], call it via _maybe_promote() in _promote_patterns(). See docs/pattern-engine.md.
• New rollup statistics: add columns to reading_rollups and compute in _build_rollups()
• Structured entity attributes: use the meta JSON column on entities

(e.g. {"age": 35, "diet": "vegetarian", "wake_time": "06:30"})

• Retention window: change RETENTION_DAYS in server.py. See docs/retention.md.