Mnemosyne

Mnemosyne is a transparent HTTP proxy designed to manage LLM context memory. It sits between LLM clients and providers like Anthropic or OpenAI, providing semantic segmentation, multi-fidelity compression, and live monitoring to reduce context costs and improve agent performance.

Overview

Mnemosyne acts as a drop-in proxy. It intercepts API requests, segments content into labeled blocks, and applies multi-fidelity compression to manage the context window. As context grows, Mnemosyne automatically degrades content fidelity to keep the most relevant information available while minimizing token usage.

Architecture

Client → Mnemosyne Proxy → Provider API
             |
      +------+------+
      |             |
  BlockStore    PageStore
      |             |
  FidelityManager   |
      |             |
 SSECleanupFilter   |
      |             |
  ObjectStore <-----+

Features

• Transparent Proxy: Operates as a drop-in replacement for LLM API endpoints. No client-side changes are required.
• Semantic Segmentation: Automatically labels conversation content with tensor/block IDs, allowing the model to reference specific segments.
• Multi-Fidelity Compression: Supports five levels of fidelity (L0-L4):
  • L0: Full content (no compression)
  • L1: Detailed summary (~30% size)
  • L2: Compact summary (~5% size)
  • L3: Metadata stub (~50-100 tokens)
  • L4: Evicted (not in context)
• Pressure-Based Degradation: Automatically downgrades object fidelity based on context window pressure zones (Normal, Caution, Warning, Critical, Emergency).
• SSE Cleanup Filter: Intercepts streaming responses to strip internal memory management tags (e.g., <memory_cleanup>, <yuyay-response>) and executes cleanup operations in real-time.
• Live Monitoring: Includes an embedded dashboard at /dashboard providing real-time metrics on context reduction, fidelity distribution, admission control, and latency.
• REST API: Exposes endpoints for session management, benchmarking, and memory state inspection.
• OpenCode Integration: Includes a plugin for OpenCode to enable memory-aware context injection and status reporting.

Installation

Mnemosyne is a Python project managed with uv.

1. Clone the repository:

   git clone https://github.com/jyapayne/mnemosyne
   cd mnemosyne
   

2. Install dependencies:

   uv sync
   

Configuration

Mnemosyne is configured via environment variables:

• ANTHROPIC_API_KEY: API key for the provider.
• MNEMOSYNE_PORT: Port for the proxy server (default: 8080).
• MNEMOSYNE_HOST: Host for the proxy server (default: 127.0.0.1).

Usage

Run the proxy server:

uv run mnemosyne

Point your LLM client (e.g., Claude Code) to the proxy URL (e.g., http://127.0.0.1:8080/v1).

API Reference

• /api/sessions: List active conversation sessions.
• /api/benchmark: Retrieve performance and token usage metrics.
• /api/memory: Inspect the current state of the object store.
• /api/blocks: View tracked conversation blocks.

How It Works

Request Lifecycle

1. Inbound:
   • label_messages: Injects [block:xxxx] markers into message content.
   • apply_to_messages: Drops or collapses blocks based on cleanup operations.
   • compact_messages: Compresses remaining content.
   • apply_fidelity: Adjusts content based on current fidelity levels.
2. Outbound:
   • SSECleanupFilter: Strips cleanup tags from streaming responses, executes operations, and forwards the clean stream.
3. Post-Response:
   • Updates fidelity pressure, schedules summaries, and records benchmarks.

License

MIT