Single HTTP server per machine

The stdio transport worked. Five agents talking to ai-raccoon at the same time did not.

The Model Context Protocol defines two transport layers: stdio (a process per client, communicating over stdin/stdout) and HTTP (a shared server, communicating over the network). ai-raccoon started with stdio because it is the simplest thing that works: the client spawns the process, sends JSON-RPC messages, and reads responses. No ports, no configuration, no service discovery. Every MCP client supports it out of the box.

The problem is what "a process per client" means when the process is heavy. Every ai-raccoon process was a complete server: it opened the SQLite bank, loaded an ONNX InferenceSession for the embedding model, started file watchers, and ran schema migrations. Five concurrent processes meant five copies of everything. I measured the cost: 484 to 620 MB RSS for five processes versus roughly 193 MB for one. Each process loaded its own copy of the ONNX model into memory (the int8 all-MiniLM-L6-v2 is small, but five copies of it add up). Each started its own set of watchers, 35 concurrent watchers across 7 directories, and they raced each other on digestion: two watchers seeing the same file change would both try to ingest it, producing duplicate entries or conflicting updates.

The concurrency bugs got worse from there. Mixed binary versions could not be detected: if you upgraded the ai-raccoon binary but one old process was still running, the two versions could write incompatible data to the same bank. Concurrent migrations could corrupt the schema: both processes see "migration 14 needed," both try to run it, one succeeds and the other fails or, worse, both succeed and produce inconsistent state. Every memory_search was a writer because BumpAccessAsync updated access counts and ratings without a transaction, causing rating and access_count divergence across processes writing to the same rows.

ADR-0020, shipped in v1.6.0, collapsed all of that into one server per machine.

What you get

Zero-config. Your existing .mcp.json with "command": "ai-raccoon" works unchanged. One server, one ONNX model, one set of watchers. The client does not know anything changed. The stdio transport still works exactly as before; the difference is what happens inside the process.

How it works

The stdio entry point became a thin proxy (McpTransport.Proxy). The proxy resolves no encryption key, opens no bank, loads no ONNX model. It is a pass-through: it reads JSON-RPC from stdin, forwards it to the real server, and writes the response to stdout. On its first call it probes http://localhost:7721/mcp. If nothing answers, it spawns ai-raccoon serve as a child process and polls on a bounded budget (the server needs time to initialize its ONNX session and start its watchers). Once the server responds to the probe, the proxy forwards the original request. From the client's perspective, the proxy IS a stdio MCP server. It looks, smells, and behaves exactly like the old monolithic process.

The HTTP backend (ai-raccoon serve) is the real server. It opens the bank, loads the model, starts watchers, and listens on loopback port 7721. It is the only process that touches the database. Every proxy on the machine talks to the same backend.

Complications

The N-way start race was the hardest part. Two agents launching simultaneously both probe, both see nothing, both try to spawn. If both succeed, you have two full servers fighting over the same database. ServeRunner solves this with a probe-attach-catch-reprobe pattern: try to spawn, catch AddressInUseException (the other process won the port), then re-probe and attach to whoever won. The losing proxy becomes a client of the winning server. This works for any N: the first server to bind the port wins, every other proxy attaches to it.

The first call pays a startup tax: 254 ms median versus 43 ms for subsequent calls, because the proxy has to spawn the server and wait for it to become ready. This is a one-time cost per machine boot (or per server restart). Subsequent calls from any client go straight through the proxy to the already-running server.

The loopback-only binding is a deliberate security choice. The server listens on 127.0.0.1:7721, not 0.0.0.0. No remote access, no firewall rules needed. The authenticated loopback check (the proxy sends a shared token that the server validates) prevents other local processes from impersonating a proxy.

Event pump for predictable load

The metrics writer had a hand-rolled bounded queue (ADR-0074). It worked until the initial codebase embedding process blocked producers, and then the server stalled under load.

The server has two families of background work that run continuously. Metrics collection happens every 30 seconds: the server flushes timing data, search hit rates, and resource usage to an internal log. Embed processing happens whenever a watch digest or a memory write produces new entries that need vector embeddings: the embed consumer pulls batches from an outbox, runs them through the ONNX model, and writes the results back. Both families produce events that need to reach their consumers, but they have opposite drop tolerances.

Metrics data is distinct per measurement: if you drop one, that data point is gone forever. The 2:30 PM flush that recorded a latency spike is not recoverable from any durable store. Embed signals, by contrast, are wake-ups over a durable outbox: the actual work items live in SQLite, and the signal just tells the consumer "there is something to process." Dropping an embed signal only delays the next poll, because the outbox still has the work. A single queue with a single policy cannot serve both families correctly.

ADR-0091, shipped in v1.33.0, replaced the hand-rolled queue with a shared EventPump<T>.

What you get

Predictable load. Memory writes, watch digests, and maintenance polls never block on downstream processing. The server stays responsive when the embed engine is busy chewing through a large batch or the metrics flush is running. No more stalls under load.

How it works

EventPump<T> wraps .NET's `Channel<T>` with one rule: TryEnqueue never blocks. It calls ChannelWriter.TryWrite only, never WriteAsync. The channel is configured with BoundedChannelFullMode.Wait as the full mode, but since only TryWrite is called (never the blocking WriteAsync), the Wait mode is irrelevant: TryWrite returns false immediately when the channel is full, and the caller drops the event according to its topic's policy. This reproduces the original "never block the producer" contract without hand-rolling a queue.

ApplyCapacity can shrink or grow the soft cap at runtime without rebuilding the channel. This matters because the embed consumer's throughput varies: a large model like bge-m3 processes entries much slower than the default MiniLM, so the channel capacity needs to adjust to avoid dropping every signal during a slow embed run.

Two topics run through the pump. Metrics (Coalesce: false) treats every measurement as distinct data; the consumer processes each event individually, and dropping one loses that data point permanently. Embed (Coalesce: true) coalesces signals: if three embed events arrive while the consumer is busy, only one needs to survive, because the consumer will drain the entire outbox on its next run regardless of how many wake-up signals it received.

Complications

Every Drop* full mode was rejected. Under DropOldest or DropNewest, TryWrite reports success even when it silently drops the item. The caller thinks the event was enqueued; it was not. That is worse than a clear "channel full" signal, because the caller has no way to know its data was lost. With BoundedChannelFullMode.Wait plus TryWrite-only, a full channel returns false, and the caller can log the drop, coalesce the signal, or take whatever action the topic's policy dictates.

AllowSynchronousContinuations = true was rejected because it lets a producer's thread run the consumer's drain callback inline. When a memory write enqueues an embed signal, the embed consumer's drain could execute on the memory write's thread, blocking the MCP request that triggered the write. A memory write should complete in microseconds; running the embed drain (which calls ONNX inference) on the same thread would turn that into seconds.

A round-robin single consumer was rejected because the metrics flush (every 30 seconds, fast) would block behind the embed drain (which can take multiple seconds per batch, especially with larger models). If both topics share one consumer loop, a metrics flush scheduled at T+30s has to wait for the embed drain that started at T+29s and runs until T+35s. Separate consumers with separate drain rates was the only design that kept both families predictable.

Custom embedding models

The original embedding engine was hardcoded: all-MiniLM-L6-v2, 384 dimensions in the vec0 DDL, a WordPiece tokenizer reading a bundled vocab.txt, mean pooling, 256-token window. Every one of those was a constant in the source code. The vec0 virtual table was created with dimensions=384 baked into the DDL. The tokenizer loaded vocab.txt from a path relative to the assembly. The pooling mode was a switch statement that always returned "mean." "Use a different model" meant "change the build."

ADR-0084, shipped in v1.29.0, made the embedding engine swappable.

What you get

Download any Hugging Face model, activate it, search with it. Two commands:

ai-raccoon model download BAAI/bge-m3 --yes
ai-raccoon model set local <path>

SHA-256 pin verification runs on both download and activation: every file in the model directory is hashed and compared against the manifest. Sentencepiece tokenizers are supported alongside WordPiece, which opens up models like multilingual-e5-large and bge-m3 that use Sentencepiece instead of WordPiece.

How it works

A model directory must carry ai-raccoon.manifest.json. The manifest is the contract between the model and the server. It declares:

• dimensions: the embedding vector size (384 for MiniLM, 1024 for bge-m3, 768 for code-daemon)
• context window: the maximum token count before truncation (256 for MiniLM, 8192 for bge-m3)
• tokenizer family: wordpiece or sentencepiece, plus the file paths for the tokenizer data
• pooling mode: how to collapse token-level embeddings into a single vector (mean, cls, last)
• normalization: whether to L2-normalize the output vector
• ONNX input/output names: the tensor names the model expects and produces
• per-file SHA-256 pins: every file that makes up the model (ONNX, tokenizer, config) is pinned

A directory without a manifest is refused. The server will not guess at a model's configuration. The tokenizer abstraction (IEmbeddingTokenizer) has WordPiece and Sentencepiece implementations, selected by the manifest's tokenizer.family field. The vec0 dimension is reconciled at activation time: if the vec0 table does not exist, create it with the new dimensions; if it exists with matching dimensions, use it; if it exists with different dimensions, error out and tell the user to re-embed.

The embedding options available today:

Complications

Pooling mode must come from the ONNX graph, not just config files. Some models pool inside their graph: the output tensor is already [batch, dimensions], not [batch, tokens, dimensions]. If the server applies mean pooling on top of that, it gets garbage (mean of a single vector is the vector itself, but the dimensions are wrong). The fix is to read the output rank at load time: rank-2 means the model handled pooling internally, and the server skips its own pooling step.

The fairseq-offset tokenizer with no added_tokens_decoder breaks piece-table numbering for the xlm-roberta family. These models use a tokenizer format where the piece IDs are offset by a constant, and the offset is not declared in the standard config files. The server cannot determine the correct offset automatically. The fix is to refuse the model and ask the user to hand-write the manifest with the correct offset. Automatic detection is not reliable enough, and a wrong offset produces silently garbage embeddings.

Byte-identity is architecture-local. A qint8 (quantized int8) model requantizes differently on x64 versus ARM because the quantization kernels use platform-specific instructions. Measured cosine similarity between x64 and ARM embeddings of the same input: 0.9895. That is close enough for search (the ranking is preserved) but not close enough for exact reproducibility. CI enforces token-id equality (the tokenizer must produce the same tokens on both platforms) plus a coarse breakage floor (cosine similarity must stay above 0.98) to catch architecture-specific regressions.

A model swap costs a full re-embed. Every entry in the bank has an embedding vector stored as a blob. When you switch models, every vector is wrong (different dimensions, different meaning). The server re-embeds everything from the stored text.

Re-embed cost: Switching to bge-m3 (1024 dimensions, 2.27 GB) re-embeds 23,520 entries at roughly 1.85 entries per second. That is about 3.4 hours. One-time, background, server stays responsive. The CLI reports progress so you are not staring at a blank terminal.

Provenance files (config.json, tokenizer_config.json) are now pinned alongside the ONNX model and tokenizer. A model directory with tampered provenance is rejected at activation, not just at download. This prevents a supply-chain attack where someone replaces a config file to change the model's behavior without changing its weights.

Semantic code ingestion

Agents ask "how does X work" about their own repository. The only answer was prose memory: notes an agent wrote about the code, stored as text. Nothing indexed the source line by line, embedded for semantic retrieval. An agent that wanted to understand the proxy architecture had to either read the files itself (expensive, every time) or hope a previous agent had written good notes (unreliable).

ADR-0085, shipped in v1.30.0, added a code corpus. The dedicated code embedding model (faxenoff/code-daemon-embed-v1, 768 dimensions, 187 MB) shipped in v1.32.0.

What you get

memory_search kind=code or kind=both searches across 24 source-file extensions (.cs, .ts, .js, .py, .rs, .go, .java, .cpp, .h, .hpp, .c, .rb, .php, .swift, .kt, .scala, .sh, .bash, .zsh, .ps1, .sql, .tf, .yaml, .yml). code_get reads a chunk's full source by its content hash, so the agent can go from a search hit to the actual code in one call. One command activates the code model:

ai-raccoon model set code default

This downloads and activates faxenoff/code-daemon-embed-v1. The code corpus has its own embedding engine, configured independently from the prose memory engine. Activating the code model never touches the memory engine settings. You can run MiniLM for prose and code-daemon for source simultaneously.

How it works

A second, code-only corpus lives in the same memory.db. The code_entries table stores hash, path, value (the chunk text), source_file (the original file path), line_start, line_end, project_id, embed_state, embedding (the vector blob), chunk_index, and total_chunks. Two indexes back it: code_fts (FTS5 for lexical search) and vec_code (vec0 with 768 dimensions for vector search). Watches and file ingest feed the code corpus automatically alongside prose memory: when a watched file changes, both the prose and code ingest pipelines process it.

The chunking strategy splits source files at function and class boundaries where possible, falling back to line-count-based splitting for files without clear structural markers. Each chunk carries its line range so code_get can return the exact source. The chunk index and total chunks fields let the agent reconstruct a full file from its chunks if needed.

Design decision: code is a cache

Code never syncs to the cloud, never sweeps (no degradation reaper), has no TTL, no promotion path, no workspaces. It is an explicit re-derivable cache. Losing it costs a re-ingest from disk, never lost knowledge. The code_entries table deliberately omits columns that prose memory has: scope, workspace_id, rating, ttl_days, access_count. Those concepts do not apply to source code that is always available on disk and never needs curation.

Why keep it in the same bank? A separate bank duplicates connection management, encryption setup, settings storage, and the entire SQLite lifecycle. Folding code into the prose entries table would mean every memory query needs a kind filter to avoid mixing prose and code results, and code rows get meaningless rating and TTL columns that clutter the schema. A separate table in the same bank is the least-wrong option: shared connection pool, shared encryption, separate schema, separate indexes.

The kind=both search mode runs the query against both corpora independently and returns two separate ranked lists: memory results and code results. Each corpus applies its own internal RRF (FTS5 + vector) independently. There is no cross-corpus fusion. The agent gets the full result set from memory and the full result set from code, each self-contained with its own ranking.

Project identity and operational hardening

Project IDs were arbitrary strings. A typo in projectId silently created a new project with its own isolated partition. There was no registry of which projects existed, no way to list them, and no validation at write time. Cloud snapshots had no authenticity check: a tampered snapshot downloaded from S3 or Azure Blob would attach without complaint, potentially injecting corrupted or adversarial entries into the bank.

Three releases addressed this: HMAC snapshot verification (v1.31.0), embedding engine runtime knobs (v1.33.0), and registered project identities (ADR-0089, v1.33.2).

What you get

project_id_token_get mints a guidv7 and registers it in the bank's project registry. A write to an unregistered project ID is refused with a clear error message. Cloud snapshots are HMAC-verified before attach: the server computes an HMAC over the snapshot contents using a key derived from the bank's encryption key, and rejects snapshots whose HMAC does not match. Model activation re-verifies SHA-256 pins at activation time, not just at download time. Two new operational knobs give operators control over resource usage: settings model threads <n> to control the ONNX runtime's thread count (important on shared machines where you do not want the embedding engine consuming all cores), and settings maintenance embed-rows-per-run <n> to throttle background embedding (important when a model swap triggers a multi-hour re-embed and you want the server to stay responsive).

How it works

GuidV7 (UUID version 7) is sortable and encodes creation order: the first 48 bits are a Unix timestamp in milliseconds, so sorting by ID sorts by creation time. This matters for listing projects in creation order without a separate timestamp column. IDs are canonicalized at the tool boundary: the D format (lowercase with hyphens) is the canonical form, so project_id_token_get always returns the same string for the same ID regardless of how the caller formats it.

A project exists when it is registered, not when someone first writes to it. This is the key change: the old behavior was "write creates the project if it does not exist," which meant typos created ghost projects. The new behavior is "write fails if the project is not registered," which forces explicit creation via project_id_token_get.

Legacy text IDs with existing rows keep working. The server detects that a project ID has existing entries but no registry row, logs a warning suggesting migration, and allows the write. This preserves backward compatibility for existing installations while nudging users toward the registered model.

What holds it together

Every feature here started as a problem I hit in production, not as a feature I planned on paper. The proxy solved memory exhaustion (five processes burning 620 MB). The event pump solved load stalls (the embed drain blocking the metrics flush). Custom embeddings solved the "what if someone needs a bigger model" question (the answer: download it, activate it, search with it). Code search solved the "how does this repo work" question (a second corpus, one command). Project identity solved the silent-typo problem (a guidv7 that must be registered).

The test suite has grown to roughly 3,700 tests. The proxy architecture was validated by RSS measurements before the code was written. The event pump was designed around the specific blocking behavior of the two consumer families. The embedding engine was built to fail loudly on unsupported models rather than guess.

ai-raccoon is open source. The proxy architecture, event pump, and code corpus are all in the current release. Install it from NuGet:

dotnet tool install -g ai-raccoon

The repository is at github.com/Arasz/ai-raccoon. For the search pipeline, the isolation model, and the features that did not survive measurement, see the introduction article.