rag-memory

# RAG Memory Skill ## Architecture (as of 2026-04-04) All files live under `~/.openclaw/skills/rag-memory/` — the single source of truth. | File | Purpose | |---|---| | `config.env` | Credentials and tuning (gitignored) | | `config.env.example` | Template with no secrets (tracked in git) | | `sync_to_qdrant.py` | Ingest pipeline: Postgres + .md files → Qdrant | | `requirements.txt` | Python deps for sync script | | `venv/` | Python virtualenv (gitignored) | | `plugin/index.js` | OpenClaw plugin — registers `vector_search` tool and `before_prompt_build` hook | | `plugin/package.json` | Plugin manifest | | `plugin/openclaw.plugin.json` | Plugin config schema | | `systemd/` | Source-of-truth copies of systemd units (deploy with `sudo cp` + `daemon-reload`) | | `references/config-paths.example.md` | File and credential location template (copy to `config-paths.md` and fill in) | | `references/operational.md` | When/how to use vector_search | Plugin is installed at `~/.openclaw/extensions/rag-memory/` and configured in `~/.openclaw/openclaw.json` under `plugins.entries.rag-memory`. Qdrant collections: `{QDRANT_COLLECTION_PREFIX}_memory` (daily logs + facts) and `{QDRANT_COLLECTION_PREFIX}_docs` (tools + skill docs). 768-dim, nomic-embed-text via OpenAI-compatible embedding endpoint. --- ## When to use vector_search Always prefer `vector_search` over reading full memory files: - Before reading any daily log or MEMORY.md → `scope: memory` - Before reading TOOLS.md or any SKILL.md → `scope: docs` - Unsure → `scope: all` Only fall back to reading the file directly if vector_search returns no results. --- ## Sync management | Task | Command | |---|---| | Incremental sync (unsynced rows only) | `~/.openclaw/skills/rag-memory/venv/bin/python ~/.openclaw/skills/rag-memory/sync_to_qdrant.py` | | Full resync (rebuild all vectors) | `... sync_to_qdrant.py --full` | | Single file | `... sync_to_qdrant.py --file /path/to/file.md` | Incremental sync also fires automatically when any `.md` file changes in `~/.openclaw/workspace/memory/` (via `sysclaw-rag-watch.path`). Nightly full sync runs at 03:00 (`sysclaw-rag-sync.timer`). --- ## Updating the plugin If `plugin/index.js` is changed: ```bash SKILL=~/.openclaw/skills/rag-memory openclaw plugins install $SKILL/plugin openclaw gateway restart ``` If `config.env` tuning values change, also update the matching values in `plugins.entries.rag-memory.config` in `openclaw.json` (especially `score_threshold` and `top_k`). --- ## Deploying systemd unit changes Edit files in `systemd/` (source of truth), then deploy using the included script — it substitutes the correct username automatically: ```bash sudo ~/.openclaw/skills/rag-memory/systemd/deploy.sh ``` Or manually, replacing `youruser` with the local OpenClaw user: ```bash sudo sed "s/User=openclaw/User=youruser/g" ~/.openclaw/skills/rag-memory/systemd/*.service \ | sudo tee /etc/systemd/system/sysclaw-rag-{sync,watch}.service > /dev/null sudo cp ~/.openclaw/skills/rag-memory/systemd/sysclaw-rag-sync.timer /etc/systemd/system/ sudo cp ~/.openclaw/skills/rag-memory/systemd/sysclaw-rag-watch.path /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl restart sysclaw-rag-sync.timer sysclaw-rag-watch.path ``` --- ## Key config values | Key | Example / Default | |---|---| | `QDRANT_HOST` | `https://qdrant.example.com` | | `EMBED_BASE_URL` | `https://litellm.example.com` | | `EMBED_MODEL` | `nomic-embed-text` | | `QDRANT_COLLECTION_PREFIX` | any UUID or unique prefix string | | `MEMORY_DIR` | `~/.openclaw/workspace/memory` | | `score_threshold` | `0.70` (in both config.env and openclaw.json plugin config) | | `top_k` | `5` | --- ## If Qdrant is unreachable 1. Try once — do not retry in a loop 2. Fall back to reading the relevant file directly 3. Note in your response that vector search was unavailable 4. Check: `curl $QDRANT_HOST/readyz` 5. Do not restart Qdrant automatically — alert the user --- ## Not for Do not activate this skill for: - General file search or code search unrelated to memory/RAG - Editing files (use read/write tools directly) - Tasks that don't involve looking up past context, facts, or tool docs - Syncing or resyncing when the user has not asked to update memory - Any request where the user explicitly says "don't search memory" --- ## External endpoints This skill sends data to three external services configured via `config.env`: | Service | Variable | What is sent | |---|---|---| | Embedding endpoint (OpenAI-compatible) | `EMBED_BASE_URL` | Text chunks from memory files and Postgres records, for vector embedding | | Qdrant | `QDRANT_HOST` | Embedded vectors + metadata for storage and search queries | | Postgres | `POSTGRES_DSN` | Read-only queries against `memory_entries`, `daily_logs`, `tools` tables | No data is sent to any Anthropic or third-party endpoint by this skill. --- ## Privacy statement Memory content (daily logs, facts, tool descriptions) leaves the local machine in two ways: 1. Text is sent to your configured embedding endpoint (`EMBED_BASE_URL`) to generate vectors 2. Vectors + metadata are stored in and queried from your Qdrant instance (`QDRANT_HOST`) Both services are self-hosted and configured by the user. No data is sent to any external party by this skill. --- ## Security manifest Environment variables accessed: - `QDRANT_HOST`, `QDRANT_API_KEY`, `QDRANT_COLLECTION_PREFIX` - `EMBED_BASE_URL`, `EMBED_API_KEY`, `EMBED_MODEL` - `POSTGRES_DSN` - `MEMORY_DIR`, `CHUNK_SIZE`, `CHUNK_OVERLAP`, `TOP_K_DEFAULT`, `SYNC_BATCH_SIZE`, `EMBED_BATCH_DELAY_MS` Files read: - All `.md` files under `MEMORY_DIR` - `config.env` (loaded at sync script startup) Files written: - None (no local file writes) Database writes (Postgres): - `memory_entries.qdrant_synced_at` — updated after successful vector upsert - `daily_logs.qdrant_synced_at` — same - `tools.qdrant_synced_at` — same - `qdrant_sync_log` — one row inserted per sync run (records_synced, duration_ms) Network: - Outbound HTTPS to `EMBED_BASE_URL` (embedding) - Outbound HTTPS to `QDRANT_HOST` (vector storage/search) - Outbound TCP to Postgres host in `POSTGRES_DSN` --- ## Trust statement By installing this skill, text from your memory files and database will be sent to the embedding endpoint configured in `EMBED_BASE_URL`. Vectors are stored in the Qdrant instance at `QDRANT_HOST`. Both are self-hosted services you configure. Only install if you trust those services with your memory contents. When `auto_inject` is enabled (default: true), Qdrant search results are prepended to every prompt context. This data originates from your own self-hosted Qdrant instance — no external party supplies it. The content is sanitized of session boilerplate before storage but is otherwise injected as-is; only enable `auto_inject` if you trust your Qdrant data as prompt input. --- ## License MIT-0 — no attribution required.