Production LLM coding agents drift over long sessions: they forget user-specified constraints, slip into mistakes the user already flagged, and confabulate prior agreements. White-box approaches such as persona vectors require model weights and so cannot be applied to closed APIs (Claude, GPT-4) that most users actually interact with. We present Nautilus Compass, a black-box persona drift detector and agent memory layer for production coding agents. The method operates entirely at the prompt-text layer: cosine similarity between user prompts and behavioral anchor texts, aggregated by a weighted top-k mean using BGE-m3 embeddings. Compass is, to our knowledge, the only public agent memory layer (among Mem0, Letta, Cognee, Zep, MemOS, smrti verified May 2026) that does not call an LLM at index time to extract facts or build a graph; raw conversation text is embedded directly. The system ships as a Claude Code plugin, an MCP 2024-11-05 A2A server (Cursor, Cline, Hermes), a CLI, and a REST API on one daemon, with a Merkle-chained audit log for tamper-evident anchor updates. On a held-out test set built from real Claude Code session traces and labeled by an independent LLM judge, Compass reaches ROC AUC 0.83 for drift detection. The embedded retrieval pipeline scores 56.6% on LongMemEval-S v0.8 and 44.4% on EverMemBench-Dynamic (n=500), topping the four published EverMemBench Table 4 baselines. LongMemEval-S 56.6% is ~30 points below recent white-box leaders (90+%); we treat that as the architectural ceiling of the no-extraction design. End-to-end reproduction cost is $3.50 (~14x cheaper than GPT-4o-judged stacks). A paired cross-vendor behavior A/B accompanies these numbers as preliminary system-level evidence. Code, anchors, frozen test data, and audit-log tooling are MIT-licensed at github.com/chunxiaoxx/nautilus-compass.
翻译:[translated abstract in Chinese]
生产环境中的LLM编码智能体在长会话过程中会发生漂移:它们会遗忘用户指定的约束条件,重现已被用户标记的错误,并虚构先前达成的协议。诸如人格向量之类的白盒方法需要访问模型权重,因此无法应用于大多数用户实际交互的封闭API(如Claude、GPT-4)。我们提出Nautilus Compass,一种面向生产编码智能体的黑盒人格漂移检测器与智能体记忆层。该方法完全在提示文本层运行:通过BGE-m3嵌入计算用户提示与行为锚点文本之间的余弦相似度,并采用加权top-k均值进行聚合。据我们所知,Compass是唯一公开的智能体记忆层(在Mem0、Letta、Cognee、Zep、MemOS、smrti(2026年5月验证)中),其在索引时不调用任何LLM来提取事实或构建知识图谱,而是直接嵌入原始对话文本。该系统以Claude Code插件、MCP 2024-11-05 A2A服务器(支持Cursor、Cline、Hermes)、命令行界面以及守护进程上的REST API形式交付,并附带用于防篡改锚点更新的默克尔链审计日志。在由真实Claude Code会话轨迹构建并由独立LLM评审者标注的预留测试集上,Compass的漂移检测ROC AUC达到0.83。其嵌入检索流水线在LongMemEval-S v0.8上得分为56.6%,在EverMemBench-Dynamic(n=500)上得分为44.4%,超越了EverMemBench表4中已发布的四个基线方法。LongMemEval-S上56.6%的得分比近期白盒领先方法(90%以上)低约30个百分点——我们认为这是无提取架构设计的天花板。端到端复现成本为3.50美元(比GPT-4o评审的堆栈便宜约14倍)。伴随这些数据提供了一个跨厂商配对行为A/B测试,作为初步系统级证据。代码、锚点、冻结测试数据及审计日志工具基于MIT许可协议开源,地址为github.com/chunxiaoxx/nautilus-compass。