Shared Memory for Several AI Tools: SQLite Holds the Truth, Qdrant Only Searches

My assistant agent, Claude Desktop and an agent in another IDE share one Qdrant collection as memory through MCP. In daily use, the vector database turned out to be the part I worried about least. The trouble was deciding which copy of the data to trust and who could write to it, then keeping the two stores in sync and working out when to retire old entries.

SQLite holds the memory events and audit log as the canonical store. Qdrant handles retrieval through MCP; an older collection stays available as a read-only reference. The sync job used to read files, but now reads incrementally from SQLite using a cursor. Along with that arrangement, I’ll cover two problems I ran into. The examples were run on Python 3.13, SQLite 3.45 and Qdrant 1.19.2.

1. Decide which copy counts

The current setup looks roughly like this:

Write: tool ──► SQLite (canonical)
                 memory_events: event memory
                 audit_log: who wrote what, and when
                 sync_status: last row synced
                     │
                     └── incremental sync, written via MCP ──► Qdrant collection (retrieval, L3)
                                                                    ▲
Read: assistant agent, Claude Desktop, IDE agent ── MCP qdrant-find ──┘

Old collection (384 dims): read-only reference

The current collection uses Gemini Embedding at 3072 dimensions with cosine distance. An older 384-dimension collection is kept as read-only reference.

The official mcp-server-qdrant supports only FastEmbed, so it can’t talk to Gemini. The MCP server I run is my own patched version, published as qdrant_mcp_hybrid in docker-compose-templates; the template collection as a whole is described in Introducing My Docker Compose Template Collection. The patch adds a Gemini Embedding 2 provider (3072 dimensions), switched on with EMBEDDING_PROVIDER=gemini; the default dense vector is still fastembed’s BAAI/bge-small-en-v1.5 (384 dimensions). Retrieval becomes a hybrid search over a dense and a sparse vector, the sparse one from fastembed’s BM25 (Qdrant/bm25), with the two result lists merged by RRF, and qdrant-find gains an optional metadata filter. All three tools share one collection and keep their memories apart with a metadata.agent field.

Speed isn’t the issue. Measured locally against a local Qdrant and this MCP server, a write took about 0.05 seconds and a query about 0.002 seconds. Those were quick measurements at the time, not a benchmark, but they make it clear where the effort goes.

There is one write path: the tool writes the memory and its audit record to SQLite, then the sync job copies it to Qdrant. The catch is qdrant-store in the official mcp-server-qdrant: it writes straight into Qdrant. Going through MCP doesn’t mean going through SQLite. A tool that calls it directly still bypasses the canonical store.

Every tool has to follow those write rules for the arrangement to work reliably. Enforce them in each tool’s configuration; a line in a design document isn’t enough. Of the two mistakes I see most often, one is treating the retrieval store as the source of truth and accepting whatever Qdrant returns as fact. Stale or contradictory memories keep coming back.

The other is writing directly to the vector store, bypassing MCP because it was convenient. Like calling qdrant-store directly, this leaves data in Qdrant that SQLite doesn’t have. Rebuild the retrieval layer from the canonical store and that data disappears, with no record of who wrote it.

You should be able to throw the retrieval layer away and rebuild it from SQLite when switching embedding models, changing how text is chunked, or recovering from a broken collection. Being able to rebuild it is the most practical reason to keep the stores separate. A single entry that exists only in Qdrant breaks that guarantee.

To keep the old collection read-only at the MCP layer, use the official mcp-server-qdrant with its QDRANT_READ_ONLY environment variable enabled. That removes qdrant-store from the available tools. With one server instance per collection, the read-only instance has no write tool to call.

2. The sync job lost its files: cursor-based incremental sync

The original sync script read a Markdown file generated each day and wrote its contents into Qdrant. When memory moved into SQLite, the daily files stopped appearing and the sync job started reporting Read failed. The data was all there; the script was still looking in the old place.

I changed the sync job to read SQLite directly and added a sync_status table to keep the last synced id as a cursor. It fetches rows with ids above the cursor, at most 20 per batch. Only after the write to Qdrant succeeds does the cursor advance.

The simplified version below uses a deterministic hash vector in place of an embedding. The container couldn’t reach the model download sources, and vector quality makes no difference to the sync logic. The named vector follows the mcp-server-qdrant naming rule:

import hashlib
import math
import sqlite3

from qdrant_client import QdrantClient, models

COLLECTION = "memory_demo"
VECTOR_NAME = "fast-all-minilm-l6-v2"  # must match the named vector the collection was created with
DIM = 384
BATCH = 20


def toy_embed(text: str) -> list[float]:
    # deterministic vector for the demo; use a real embedding model in practice
    raw = hashlib.sha512(text.encode()).digest() * 6
    vec = [b - 127.5 for b in raw[:DIM]]
    norm = math.sqrt(sum(x * x for x in vec))
    return [x / norm for x in vec]


def init_db(conn: sqlite3.Connection) -> None:
    conn.executescript("""
    CREATE TABLE IF NOT EXISTS memory_events (
        id         INTEGER PRIMARY KEY,
        scope      TEXT NOT NULL,
        source     TEXT NOT NULL,
        content    TEXT NOT NULL,
        created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
    );
    CREATE TABLE IF NOT EXISTS sync_status (
        target  TEXT PRIMARY KEY,
        last_id INTEGER NOT NULL
    );
    INSERT OR IGNORE INTO sync_status (target, last_id) VALUES ('qdrant', 0);
    """)


def sync_once(conn: sqlite3.Connection, client: QdrantClient) -> int:
    (cursor,) = conn.execute(
        "SELECT last_id FROM sync_status WHERE target = 'qdrant'").fetchone()
    rows = conn.execute(
        "SELECT id, scope, source, content FROM memory_events "
        "WHERE id > ? ORDER BY id LIMIT ?", (cursor, BATCH)).fetchall()
    if not rows:
        return 0
    points = [
        models.PointStruct(
            id=row_id,  # SQLite id as point id, so a resend overwrites
            vector={VECTOR_NAME: toy_embed(content)},
            payload={"document": content,
                     "metadata": {"scope": scope, "source": source,
                                  "sqlite_id": row_id}},
        )
        for row_id, scope, source, content in rows
    ]
    client.upsert(COLLECTION, points=points, wait=True)  # raises on failure, cursor stays put
    with conn:
        conn.execute("UPDATE sync_status SET last_id = ? WHERE target = 'qdrant'",
                     (rows[-1][0],))
    return len(rows)

The test loads 45 rows, with a wrapper around the client that raises a connection error on the second upsert. After rerunning with a normal client, it resets the cursor to 0 by hand and sends everything again. This simulates a crash after a successful upsert but before the cursor was written back:

class FlakyClient:
    """Simulates a network failure on the fail_on-th upsert."""

    def __init__(self, inner: QdrantClient, fail_on: int):
        self.inner, self.fail_on, self.calls = inner, fail_on, 0

    def upsert(self, *args, **kwargs):
        self.calls += 1
        if self.calls == self.fail_on:
            raise ConnectionError("simulated: qdrant unreachable")
        return self.inner.upsert(*args, **kwargs)

synced batch of 20
sync aborted: simulated: qdrant unreachable
after failed run:            cursor=20  points=20
synced batch of 20
synced batch of 5
after retry:                 cursor=45  points=45
after replaying everything:  cursor=45  points=45

When the second batch fails, the cursor stays at 20 and Qdrant holds only the first 20 points, so the rerun picks up from row 21. With the cursor reset to 0 and all 45 rows sent again, the point count is still 45: the point id is the SQLite id, so sending the same row twice overwrites it.

Advancing the cursor only on success means nothing is skipped. If the process dies between a successful upsert and the cursor update, though, the batch will be sent again. That gives us at-least-once delivery; the writer must make those resends idempotent to avoid duplicates.

Using an increasing id as the cursor here assumes that SQLite generates the ids and rows are only ever inserted, never updated or deleted. INTEGER PRIMARY KEY on its own doesn’t promise that. Without AUTOINCREMENT, deleting the row with the highest id hands that id to the next insert:

id INTEGER PRIMARY KEY                   new id after deleting 3: 3
id INTEGER PRIMARY KEY AUTOINCREMENT     new id after deleting 3: 4

If the cursor has already moved past 3, the reused id never gets synced. If memory entries can be edited, you also need an updated_at column or a change sequence number, or edited content will never reach Qdrant.

Writing through MCP can prevent the caller from making resends idempotent, and my own stack is a case in point. With a default collection configured, qdrant-store in the official mcp-server-qdrant takes only information and metadata; without one, it also needs a collection name. The server generates the point id with uuid.uuid4(). The caller has no say in it, so sending the same batch twice with random ids gives this:

attempt 1: points=20
attempt 2: points=40

If the sync job uses the official qdrant-store, rerunning before the cursor has advanced leaves two copies of the same memory in Qdrant. My patched server changes the embedding and the search, but its writes still use uuid.uuid4().hex as the point id, so the same problem exists in my setup. That is why the incremental sync has to rely on the SQLite cursor and advance it only on success. The server won’t deduplicate anything, and the cursor is the only thing limiting resends to the batch it hasn’t moved past yet; without it, every rerun would write the whole memory store again. Putting the SQLite id in metadata lets you deduplicate by id after querying, but only among the results you fetched. The duplicate points remain and can crowd out other results within the query limit.

An MCP server that accepts caller-supplied ids avoids that problem. The sync job could also write directly with qdrant-client and explicit ids. That opens a write path around MCP, so the boundary rules must restrict it to this one job.

3. Not existing vector name: configuration and collection disagree

At one point, queries through the MCP server kept failing with Not existing vector name. The server’s embedding model setting was sentence-transformers/all-MiniLM-L6-v2, while the existing collection’s named vector was fast-all-minilm-l6-v2. Adjusting the model setting so the server queried the collection’s vector name fixed it.

A Qdrant collection can hold several named vectors; a query selects one with using. The FastEmbed provider in mcp-server-qdrant takes the part of the model name after /, converts it to lowercase and adds the prefix fast-. That turns sentence-transformers/all-MiniLM-L6-v2 into fast-all-minilm-l6-v2. A source comment says the rule preserves compatibility with FastEmbed’s naming before 0.6.0.

Following that rule, the official server configured with sentence-transformers/all-MiniLM-L6-v2 already queries fast-all-minilm-l6-v2. A mismatch means either the server or the code that created the collection used a different naming rule, or the configured model differs from the one used to create it. Adjust the model setting until both sides derive the same name. The field still needs a model name: putting a vector name such as fast-... there may break again when you switch server implementations.

I reproduced this against a Qdrant 1.19.2 server, with only fast-all-minilm-l6-v2 in the collection, by querying each name:

from qdrant_client import QdrantClient
from qdrant_client.http.exceptions import UnexpectedResponse

from sync_demo import COLLECTION, toy_embed

client = QdrantClient(url="http://localhost:6333")
info = client.get_collection(COLLECTION)
print("named vectors:", list(info.config.params.vectors))

query = toy_embed("memory event #7")
for name in ["sentence-transformers/all-MiniLM-L6-v2", "fast-all-minilm-l6-v2"]:
    try:
        hits = client.query_points(COLLECTION, query=query, using=name, limit=1).points
        print(f"{name}: top hit = {hits[0].payload['document']}")
    except UnexpectedResponse as err:
        print(f"{name}: HTTP {err.status_code} {err.content.decode()}")

named vectors: ['fast-all-minilm-l6-v2']
sentence-transformers/all-MiniLM-L6-v2: HTTP 400 {"status":{"error":"Wrong input: Not existing vector name error: sentence-transformers/all-MiniLM-L6-v2"},"time":0.002283536}
fast-all-minilm-l6-v2: top hit = memory event #7

The error prints the missing name. Compare it with the names from get_collection and the mismatch is usually obvious. A name is only a label, so check that the dimensions and model match too. Keep the name while changing the model, and Qdrant will reject a different dimension. If the dimensions happen to match, the query may succeed with results that are no longer comparable. That is harder to spot than an error.

The official mcp-server-qdrant currently supports only one embedding provider, FastEmbed, with sentence-transformers/all-MiniLM-L6-v2 (384 dimensions) as the default, which is why I had to patch in a Gemini provider myself. Gemini’s gemini-embedding-001 and gemini-embedding-2 both output 3072 dimensions by default, with 768, 1536 or 3072 recommended, and the collection has to be created with the dimension the model actually produces.

4. curl against an MCP endpoint returns 406

I once tried to check whether an MCP server was alive by calling it with curl instead of a client, and got a 406. The Streamable HTTP transport requires a client’s POST to list both application/json and text/event-stream in its Accept header. Revisions up to 2025-11-25 also supported a GET to open an SSE stream, which required text/event-stream; the current revision has dropped GET. Servers check these requirements and reject requests that don’t meet them.

Sending an initialize request to a Streamable HTTP server running MCP Python SDK 2.2.0, with only application/json in Accept, produces:

HTTP/1.1 406 Not Acceptable
server: uvicorn
content-type: application/json
mcp-session-id: 5b40f4097d714f659abc508a89c6791e
content-length: 142

{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"Not Acceptable: Client must accept both application/json and text/event-stream"}}

With Accept: application/json, text/event-stream, the server returns 200 with text/event-stream. This SDK version treats */* as accepting both. The default header in curl, Accept: */*, therefore gets through; explicitly setting application/json gets it rejected. I haven’t verified the behaviour of older versions or SDKs in other languages, which may differ. When you get a 406, check Accept first.

Debugging by hand also depends on the protocol revision. Under 2025-11-25, you send initialize first. The server may create a session; if it returns an MCP-Session-Id, every later request must carry it.

The current 2026-07-28 revision removes the initialize handshake and protocol-level sessions, including the Mcp-Session-Id header. Every POST needs MCP-Protocol-Version and Mcp-Method headers; tools/call also needs Mcp-Name. The body’s _meta must hold the protocol version and client capabilities. If headers and body disagree, the server returns 400. Following an old tutorial step by step won’t work against a server that only supports the new revision.

The transport caused trouble outside curl too. When I connected the IDE agent over MCP, its transport clashed between SSE and HTTP. I ended up switching the MCP service to JSON-RPC over HTTP (the server started with --transport streamable-http, exposed on port 8800) before the IDE could connect.

5. Memory is a knowledge governance problem first

I want rules for what to keep and what to trust before spending time tuning retrieval. Which parts of a conversation qualify as long-term memory? When entries repeat or contradict each other, there needs to be a rule for merging duplicates and deciding which version to trust. Memories from different tools or projects need to stay separate, and each entry needs a record of where it came from.

Those decisions determine the table columns. In the example, scope and source, along with a separate audit_log table, record the information the rules need. Retrieval quality can wait.

SQLite can hold local memory on its own; add Qdrant when you need semantic search. For operational memory that can be updated and audited, PostgreSQL with pgvector may fit more naturally than a dedicated vector database. It keeps vectors in the same table as the other columns, so updates and audit records commit in one transaction without copying data between SQLite and Qdrant. Embeddings still need recomputing when the content changes. I noted this down while working through a Hacker News discussion.

6. Some things belong in documents

Project conventions, wording rules and prohibitions apply every time an agent works. Put them in documents such as AGENTS.md and CLAUDE.md, where they can be versioned and reviewed. Check which files each tool reads and where it loads them; this varies by tool. Loading a file means the model can see it. Compliance is another matter. In a vector store, even visibility is uncertain: the rule reaches the model only if a query happens to retrieve it. Rephrase the query and it may be missed.

A vector store suits events and experience. Decisions and past fixes accumulate, as do records of who asked for what; they are useful when relevant to the task, which makes them a good fit for semantic retrieval. Keeping fixed rules in documents avoids repeatedly writing them to the memory store, where old and new versions could contradict each other. From Messaging to Community and Short Video has a minimal Qdrant recommendation example. The vector query there is the feature itself; here it serves as a retrieval layer.

Notes

For shared AI memory, SQLite is canonical. Qdrant handles retrieval and must be disposable: tools write to SQLite, then a sync job writes to Qdrant, so the retrieval layer can be rebuilt. Calling the official qdrant-store directly bypasses SQLite too.
The sync job keeps the last synced id in sync_status as a cursor and works in batches of 20. Advancing only on success gives at-least-once delivery; idempotent upserts keyed on the SQLite id prevent duplicates. The id must never be reused.
qdrant-store in the official mcp-server-qdrant uses uuid4 point ids, so resending a batch through it creates a second copy. My patched version with Gemini and hybrid search does the same, so only the SQLite cursor limits what gets resent.
Not existing vector name means the query and collection disagree on the named vector. The official FastEmbed provider takes the part of the model name after the /, lowercases it and adds a fast- prefix. Fix the model setting; the model field still needs a model name.
If curl gets a 406 from a Streamable HTTP endpoint, check first that Accept lists both application/json and text/event-stream. The 2026-07-28 revision has removed sessions and the initialize handshake.
Decide the rules for promotion, deduplication, conflicts, scope and provenance before tuning long-term memory retrieval. Rules that always apply belong in files like AGENTS.md. Events and experience go in the vector store.