diff --git a/.planning/ONBOARDING-SPEC.md b/.planning/ONBOARDING-SPEC.md new file mode 100644 index 0000000..6de696a --- /dev/null +++ b/.planning/ONBOARDING-SPEC.md @@ -0,0 +1,105 @@ +# Spec — Install / Onboarding Rework + +**Status:** DRAFT → building. +**Why:** install/onboarding is the #1 adoption blocker (claim `mm-300f`). 3 overlapping docs, no from-zero happy path, interactive-only setup that isn't detect-first, no verify, no full-stack orchestration. +**Locked decisions:** (1) BOTH — agent install prompt drives a detect-first idempotent setup, then verifies; (2) FULL-STACK default, friction absorbed by the agent, with a graceful no-Docker fallback; (3) Claude Code + Codex from v1; brownfield-reuse; consolidate docs. +**Constraint (STOP rule):** ENHANCE `setup_hooks.py` additively — do NOT rewrite from scratch. Preserve every existing wiring function and its idempotent remove-then-add behavior. + +--- + +## 1. New module — `memorymaster/surfaces/setup_detect.py` + +Pure, side-effect-free environment probing. Subprocess calls are read-only, `shell=False`, timeout-bounded (≤5s each), and NEVER raise (degrade to "unknown/absent"). + +```python +@dataclass(frozen=True) +class Detected: + python_version: str + pip_ok: bool + os: str # "Windows" | "Linux" | "Darwin" + docker: bool # `docker --version` ok + docker_compose: bool # `docker compose version` ok + ollama: bool # http GET OLLAMA_URL/api/tags ok OR `ollama --version` + ollama_models: tuple[str, ...] + qdrant: bool # http GET QDRANT_URL/healthz ok + obsidian_vault: str | None # path if an obsidian-vault/ found under cwd + gitnexus: bool # `.gitnexus/` present OR `npx gitnexus` resolvable + claude_code: bool # ~/.claude/ exists + codex: bool # ~/.codex/ exists + mm_installed: bool # `import memorymaster` works + mm_mcp_registered: bool # memorymaster already in ~/.claude.json mcpServers + existing_hooks: tuple[str, ...] # memorymaster hooks already in settings.json + +def detect_environment(*, cwd: Path | None = None) -> Detected: ... +def format_plan(d: Detected, *, want_full_stack: bool) -> list[str]: + """Human-readable 'will do / will skip (already present) / can't (missing dep)' lines.""" +``` + +Brownfield = the report drives skip-if-present everywhere. + +--- + +## 2. `setup_hooks.py` — additive enhancements + +Keep all existing functions (`install_hooks`, `install_mcp`, `append_instructions`, `setup_steward_cron`, `install_obsidian_skills`). Add: + +1. **argparse, non-interactive mode.** `main()` parses: + `--yes/-y` (no prompts, accept defaults), `--db PATH`, `--provider {google,openai,anthropic,ollama}`, `--api-key`, `--model`, `--project-root PATH`, `--full-stack/--no-full-stack`, `--no-cron`, `--no-obsidian-skills`, `--codex/--no-codex`, `--verify-only`, `--json` (machine-readable result for the agent prompt). When a flag is given, never prompt for that value. `ask`/`ask_yn` must honor a global non-interactive flag (return the default/flag value instead of calling `input()`). +2. **Detect-first.** Call `detect_environment()` early; print `format_plan(...)`; in interactive mode confirm, in `--yes` proceed. +3. **Idempotent + brownfield.** Skip MCP/hook/cron/skill steps whose target is already present (use the Detected report). Re-running must be a no-op-ish (already idempotent for hooks — extend to MCP: don't clobber unless `--force`). +4. **Fix the MCP-path bug.** Register the server via the entry point: `command="memorymaster-mcp"` (preferred) or `[PYTHON_EXE, "-m", "memorymaster.surfaces.mcp_server"]` — NOT the deprecated `memorymaster.mcp_server`. Also register for Codex when `codex` detected (its MCP config location). +5. **Full-stack orchestration (the locked default).** New `setup_full_stack(detected, *, interactive, yes)`: + - If `docker_compose` present: offer/run `docker compose up -d qdrant ollama` (reuse repo `docker-compose.yml`); wait for health; `ollama pull` the configured model. Reuse if `qdrant`/`ollama` already healthy (brownfield). + - **No-Docker fallback:** if Docker absent AND Qdrant/Ollama not already running → print a clear, non-fatal message: "Running in SQLite-only mode. Vector recall + local LLM auto-ingest are OFF. To enable them: install Docker and re-run with `--full-stack`, or point QDRANT_URL/OLLAMA_URL at existing services." Setup continues and succeeds in degraded mode. + - Never block the core install on stack failures. +6. **Verify step.** New `verify_install(db_path)`: init a throwaway check — ingest a sentinel claim then `query_memory` it back via `service`, confirm round-trip; if MCP registered, note "restart session to load MCP". Print PASS/PARTIAL with exactly what works. Reachable via `--verify-only`. +7. **`--json` output** for the agent prompt: emit `{detected, planned, applied, verify, degraded}` so the agent can parse and report. + +--- + +## 3. Agent install prompt — `docs/AGENT-INSTALL.md` + +A copy-paste block the user gives Claude Code **or** Codex. It instructs the agent to: +1. `pip install "memorymaster[mcp,security,qdrant,embeddings]"` (full-stack extras). +2. Run `memorymaster-setup --yes --full-stack --json` and read the JSON. +3. Report the plan (what was wired, what was reused/brownfield, what degraded). +4. Run `memorymaster-setup --verify-only` and show the round-trip result. +5. Tell the user to restart the session to load hooks + MCP. +Include a Claude-Code variant and a Codex variant (Codex MCP config path differs; Codex has no Stop hook → point at the session-end script, mirroring current `append_instructions`). + +--- + +## 4. Docs consolidation + +- **README.md:** replace the install section with a **30-second quickstart** — the single happy path: `pip install` → paste the agent prompt (link `docs/AGENT-INSTALL.md`) → restart → verified. One short "manual / advanced" pointer to `INSTALLATION.md`. +- **INSTALLATION.md:** stays as the reference matrix (extras, Docker, Helm, env vars, troubleshooting). Add the new flags + `--verify-only`. +- **docs/INTEGRATING.md:** remove install overlap; keep only the 3-beat agent contract / integration semantics. Cross-link, don't duplicate. + +--- + +## 5. Tests (MANDATORY, hermetic) + +All tests MUST patch `HOME`/`CLAUDE_DIR`/`CODEX_DIR` to `tmp_path` and mock `subprocess.run`/HTTP — **never touch the real `~/.claude`**. Cover: +- `setup_detect`: each probe parses canned output; all degrade to absent on timeout/error; no exceptions escape. +- `setup_hooks` non-interactive: `--yes --provider ... --db ...` wires hooks/MCP into a tmp HOME; re-run is idempotent (no duplicate hook entries; settings.json stays valid JSON). +- MCP registration uses the correct (non-deprecated) command. +- No-Docker fallback: Docker absent → setup still succeeds, degraded message emitted, exit 0. +- `verify_install`: round-trip ingest+query on a tmp DB returns PASS. +- `--json` emits valid parseable JSON. + +--- + +## 6. Safety constraints +- Never corrupt existing `settings.json` / `.claude.json` — read-merge-write, preserve unknown keys, keep valid JSON on every path. +- All external-tool probes timeout-bounded, `shell=False`, never fatal. +- The build/test run must NOT execute the real installer against the developer machine. + +## 7. Acceptance criteria (verifiable) +- [ ] `memorymaster-setup --yes --db --provider ollama --no-cron --no-obsidian-skills` runs non-interactively to completion in a patched HOME, exit 0. +- [ ] Re-running it produces no duplicate hooks and identical settings.json (idempotent). +- [ ] MCP entry uses `memorymaster-mcp` / `surfaces.mcp_server`, not the deprecated path. +- [ ] Docker-absent path prints the degraded-mode message and still exits 0. +- [ ] `--verify-only` round-trips a sentinel claim (PASS) on a tmp DB. +- [ ] `--json` output parses. +- [ ] New + existing tests green; ruff clean; full suite collects clean. +- [ ] README quickstart ≤ ~30 lines to first verified memory; `docs/AGENT-INSTALL.md` present with Claude + Codex variants. diff --git a/INSTALLATION.md b/INSTALLATION.md index 30b4edf..37f02ef 100644 --- a/INSTALLATION.md +++ b/INSTALLATION.md @@ -47,27 +47,70 @@ pip install -e ".[dev,mcp,security]" memorymaster --db memory.db init-db ``` -### Interactive setup (hooks + MCP + cron) +### Setup (hooks + MCP + cron) -After `pip install memorymaster`, run the interactive installer to wire -MemoryMaster into Claude Code (hooks, MCP server, steward cron) and -optionally into Codex: +After `pip install memorymaster`, run the installer to wire MemoryMaster into +Claude Code (hooks, MCP server, steward cron) and optionally Codex. + +**Recommended — let your agent drive it** (paste [`docs/AGENT-INSTALL.md`](docs/AGENT-INSTALL.md) +into Claude Code or Codex): ```bash -memorymaster-setup +memorymaster-setup --yes --full-stack --json ``` -This will: +**Manual / interactive:** -- Ask for your LLM provider (Gemini / OpenAI / Anthropic / Ollama) and API key -- Ask where `memorymaster.db` should live (default: current directory) -- Copy 7 hook scripts to `~/.claude/hooks/` and wire them into `~/.claude/settings.json`: recall, classify, validate-wiki, session-start, auto-ingest, pre-compact, plus the 6-hour steward cron -- Register the `memorymaster` MCP server globally in `~/.claude.json` -- Optionally append Codex `AGENTS.md` + global `CLAUDE.md` integration snippets -- Optionally install the steward cron (Linux/macOS) or Task Scheduler job (Windows) +```bash +memorymaster-setup +``` Running from a cloned repo? `python scripts/setup-hooks.py` also works — it is -a 3-line shim that calls the same `memorymaster.setup_hooks:main` function. +a 3-line shim that calls the same `memorymaster.surfaces.setup_hooks:main` function. + +#### `memorymaster-setup` flags + +| Flag | Default | Description | +|------|---------|-------------| +| `-y` / `--yes` | off | Non-interactive; accept all defaults (no `input()` prompts) | +| `--db PATH` | `/memorymaster.db` | Path to the SQLite database | +| `--provider {google,openai,anthropic,ollama}` | prompted | LLM provider for the auto-ingest Stop hook | +| `--api-key KEY` | prompted | API key for the chosen provider | +| `--model MODEL` | provider default | LLM model id | +| `--project-root PATH` | cwd | Directory where `memorymaster.db` lives | +| `--full-stack` | on | Bring up Qdrant + Ollama via Docker Compose | +| `--no-full-stack` | off | Skip the vector + local-LLM stack | +| `--no-cron` | off | Skip steward cron setup | +| `--no-obsidian-skills` | off | Skip Obsidian skills install | +| `--codex` | auto-detect | Force Codex MCP + instructions wiring | +| `--no-codex` | — | Skip Codex wiring | +| `--force` | off | Overwrite existing MCP entries (default: skip if present) | +| `--verify-only` | off | Run only the sentinel round-trip verify and exit | +| `--json` | off | Emit machine-readable JSON result on stdout (human chatter goes to stderr) | + +#### What the installer does + +- Probes your environment first (Python, Docker, Qdrant, Ollama, `~/.claude/`, `~/.codex/`) and prints a plan; existing components are reused, not overwritten. +- Copies 7 hook scripts to `~/.claude/hooks/` and wires them into `~/.claude/settings.json`: recall, classify, validate-wiki, session-start, auto-ingest, pre-compact, plus the 6-hour steward cron. +- Registers the `memorymaster` MCP server globally in `~/.claude.json` (using `memorymaster.surfaces.mcp_server` — not the deprecated path). +- Optionally appends Codex `AGENTS.md` + global `CLAUDE.md` integration snippets. +- Optionally installs the steward cron (Linux/macOS) or Task Scheduler job (Windows). +- Runs a sentinel round-trip verify at the end (`--verify-only` to run this step alone). + +#### No-Docker degraded mode + +If Docker is absent and Qdrant/Ollama are not already running, the installer +continues without them and prints: + +``` +Running in SQLite-only mode. Vector recall + local LLM auto-ingest are OFF. +To enable them: install Docker and re-run with --full-stack, or point +QDRANT_URL / OLLAMA_URL at existing services. +``` + +Setup exits 0. Core hooks, MCP, and SQLite-based recall all work normally in +degraded mode. Add vector search later by installing Docker and re-running +`memorymaster-setup --full-stack --yes`. ## Docker Compose diff --git a/README.md b/README.md index 376c87b..eee770d 100644 --- a/README.md +++ b/README.md @@ -104,72 +104,38 @@ Reproduce: `python tests/bench_longmemeval.py --retrieval-only`. Full methodolog - **Docker** for Qdrant — vector retrieval. SQLite FTS5 is the default and works out of the box; add Qdrant when you want semantic recall on top of keyword search. -## 15-minute quickstart +## 30-second quickstart -From zero to a recalled claim and a live dashboard. No Qdrant, no Postgres, no LLM key required for these steps (SQLite + FTS5 is the default). - -**1. Install (2 min)** +**1. Install** ```bash -pip install "memorymaster[mcp]" -memorymaster --db memorymaster.db init-db +pip install "memorymaster[mcp,security,qdrant,embeddings]" ``` -**2. Configure a provider (3 min, optional for this walkthrough)** - -Recall and ingest below work with zero config. An LLM provider is only needed for the steward/wiki cycles — pick one when you're ready (see [Pick your LLM provider](#pick-your-llm-provider)). For a Claude Code subscriber, the cheapest path is: +**2. Let your agent do the rest** -```bash -export MEMORYMASTER_LLM_PROVIDER=claude_cli # reuses your Claude Code OAuth, no API key -``` +Paste the contents of [`docs/AGENT-INSTALL.md`](docs/AGENT-INSTALL.md) into Claude Code or Codex. The agent will: +- run `memorymaster-setup --yes --full-stack --json` (detects your environment, wires hooks + MCP, starts Qdrant + Ollama if Docker is present, or falls back to SQLite-only mode gracefully) +- report what was wired, what was reused (brownfield), and what degraded +- run `memorymaster-setup --verify-only` and show the round-trip result -**3. Ingest a claim via CLI (1 min)** +**3. Restart your session** -```bash -memorymaster --db memorymaster.db ingest \ - --text "Server uses PostgreSQL 16" \ - --source "session://chat|turn-3|user confirmed" -``` +Hooks and MCP load on session start. Restart Claude Code / Codex once. -**4. Recall it (1 min)** +**4. Verify** -A freshly-ingested claim starts life as a `candidate` (unvalidated). The CLI `query`/`context` paths *exclude* candidates by default — that's the governance model: unvalidated facts don't silently leak into recall until the steward promotes them. To see your brand-new claim before a validation cycle, pass `--include-candidates`: +After restart, run in your agent: -```bash -# Hybrid retrieval (lexical + freshness + confidence) -memorymaster --db memorymaster.db query "database version" \ - --retrieval-mode hybrid --include-candidates - -# Token-budgeted context block — the killer feature for agents -memorymaster --db memorymaster.db context "database" \ - --budget 4000 --format xml --include-candidates ``` - -You should see the PostgreSQL 16 claim come back, ranked, with its citation. (Drop `--include-candidates` and you'll get zero results until step 7's `run-cycle` promotes it to `confirmed` — that's working as designed, not a bug.) - -**5. Open the dashboard (2 min)** - -```bash -memorymaster --db memorymaster.db run-dashboard # serves on http://127.0.0.1:8765 -``` - -Open the URL: you'll see your claim in **Claims**, plus governance panels — **Conflicts**, **Review Queue**, **Recall Analysis** (why each claim ranked where it did), **Audit Log**, **Provenance by Agent**, and **Reliability**. - -**6. Wire it into your agent (3 min)** - -```bash -memorymaster-setup # interactive: hooks, MCP, steward cron, CLAUDE.md / AGENTS.md +query_memory("test") ``` -That installs the MCP server and the auto-ingest Stop hook so your agent recalls and stores memory automatically. See [MCP server](#mcp-server) for the config block. - -**7. Run a validation cycle (1 min, needs a provider)** +You should get a recall response from the MCP server. Done. -```bash -memorymaster --db memorymaster.db run-cycle # extract, validate, decay, compact -``` +--- -For the one-prompt agent install (paste into any agent with shell access), see [`docs/handbook.md#one-prompt-agent-install`](docs/handbook.md#one-prompt-agent-install). +For manual setup, advanced flags (`--provider`, `--db`, `--no-cron`, `--no-full-stack`, `--verify-only`, `--json`, and more), Docker, Helm, and Postgres, see [INSTALLATION.md](INSTALLATION.md). ## Pick your LLM provider diff --git a/docs/AGENT-INSTALL.md b/docs/AGENT-INSTALL.md new file mode 100644 index 0000000..97d1c9d --- /dev/null +++ b/docs/AGENT-INSTALL.md @@ -0,0 +1,159 @@ +# MemoryMaster — Agent Install Prompt + +Copy the appropriate block below and paste it as a prompt to your coding agent. +The agent will run the installer, read the machine-readable result, and report +what was wired, what was reused (brownfield), and what degraded. + +--- + +## Claude Code variant + +``` +Install MemoryMaster in this environment. Follow these steps exactly. + +1. Install the package with the full-stack extras: + + pip install "memorymaster[mcp,security,qdrant,embeddings]" + +2. Run the detect-first, non-interactive installer and capture the JSON output: + + memorymaster-setup --yes --full-stack --json + + The installer prints human-readable progress to stderr and emits a single + JSON document to stdout. Parse the JSON and report back to me: + + - detected: which tools were found (Docker, Qdrant, Ollama, Claude Code, + Codex, existing hooks, MCP registration state). + - planned: the list of [will-do / skip-present / cant-missing] lines the + installer printed — paste them verbatim so I can see the plan. + - applied: what was actually wired (hooks, mcp_claude, cron, full_stack, etc.). + - degraded: true/false. If true, explain the reason from the JSON + (typically: Docker not found → SQLite-only mode, vector recall + local LLM + auto-ingest are off). + - verify.status: PASS, PARTIAL, or FAIL and the detail string. + +3. Run the verify round-trip to confirm hooks and DB are functional: + + memorymaster-setup --verify-only + + Show me the output. + +4. Tell me to restart my Claude Code session. MCP and hooks take effect only + after a full session restart. + +Notes for you (the agent): +- The JSON payload shape is: + {"detected": {...}, "planned": [...], "applied": {...}, "verify": {"status": "...", "detail": "...", "mcp_note": "..."}, "degraded": bool} +- If --full-stack brings up Docker services it may take up to 2 minutes; wait + for the process to exit before reading stdout. +- If degraded is true the install still succeeded (exit 0). Report the + degraded message from applied.full_stack.message so I know what to do next. +- The MCP server is registered as memorymaster.surfaces.mcp_server (not the + deprecated memorymaster.mcp_server path). Do not edit the MCP entry manually. +- Hooks installed: UserPromptSubmit (recall + classify), PostToolUse + (validate-wiki on Edit/Write), SessionStart (context injection on + startup/resume), Stop (auto-ingest after each response), PreCompact (save + before context compaction). All are idempotent; re-running the installer is + safe. +``` + +--- + +## Codex variant + +``` +Install MemoryMaster in this environment. Follow these steps exactly. + +1. Install the package with the full-stack extras: + + pip install "memorymaster[mcp,security,qdrant,embeddings]" + +2. Run the detect-first, non-interactive installer and capture the JSON output: + + memorymaster-setup --yes --full-stack --codex --json + + The --codex flag ensures ~/.codex/config.toml is updated with the MCP + server entry (Codex reads MCP config from there, not from ~/.claude.json). + Parse the JSON and report back to me: + + - detected: which tools were found (Docker, Qdrant, Ollama, Codex + presence, MCP registration state). + - planned: paste the [will-do / skip-present / cant-missing] lines verbatim. + - applied: what was actually wired (mcp_codex, cron, full_stack, etc.). + - degraded: true/false and the reason if true. + - verify.status: PASS, PARTIAL, or FAIL and the detail string. + +3. Run the verify round-trip to confirm DB and core service are functional: + + memorymaster-setup --verify-only + + Show me the output. + +4. Session-end memory distillation (no native Stop hook in Codex): + Codex has no Stop hook equivalent, so learnings from each session are NOT + distilled automatically. To enable this, wire the reference script at the + end of each Codex session (or as a notify/exit hook in your Codex config): + + python scripts/agent_session_end_ingest.py \ + --db /memorymaster.db \ + --transcript \ + --source-agent codex-session \ + --cwd + + This distills up to 3 learnings per session, sets source_agent, and routes + through service.ingest (never raw-INSERTs). The script path is relative to + the MemoryMaster repo root; adjust as needed for your environment. + +5. Tell me to restart my Codex session so the updated ~/.codex/config.toml + is picked up. + +Notes for you (the agent): +- The JSON payload shape is: + {"detected": {...}, "planned": [...], "applied": {...}, "verify": {"status": "...", "detail": "...", "mcp_note": "..."}, "degraded": bool} +- Codex MCP config lives in ~/.codex/config.toml as [mcp_servers.memorymaster] + TOML tables, not in ~/.claude.json. The installer writes a marker-bounded + block so re-runs are idempotent and your existing config is preserved. +- The MCP server command registered is the same non-deprecated path: + python -m memorymaster.surfaces.mcp_server +- If degraded is true (Docker absent / services unreachable) the install still + succeeded (exit 0). SQLite-only mode is fully functional; vector recall and + local LLM auto-ingest are simply off until Docker or QDRANT_URL/OLLAMA_URL + are available. +- Claude Code hooks (UserPromptSubmit, Stop, SessionStart, PreCompact) are NOT + registered for Codex — those are Claude Code-specific. The session-end script + above is the Codex equivalent for distilled ingest. +``` + +--- + +## Flag reference (for advanced / manual use) + +| Flag | Effect | +|---|---| +| `-y` / `--yes` | Non-interactive; accept all defaults, no prompts | +| `--db PATH` | Path to `memorymaster.db` (default: `/memorymaster.db`) | +| `--provider {google,openai,anthropic,ollama}` | LLM provider for the auto-ingest Stop hook | +| `--api-key KEY` | API key for the chosen provider | +| `--model MODEL` | LLM model id | +| `--project-root PATH` | Directory where `memorymaster.db` lives | +| `--full-stack` | Bring up Qdrant + Ollama via Docker Compose (default when omitted) | +| `--no-full-stack` | Skip the vector + local-LLM stack | +| `--no-cron` | Skip steward cron setup | +| `--no-obsidian-skills` | Skip Obsidian skills install | +| `--codex` | Force Codex MCP + instructions wiring (auto-detected otherwise) | +| `--no-codex` | Skip Codex wiring | +| `--force` | Overwrite existing MCP entries (default is brownfield-safe skip) | +| `--verify-only` | Run only the sentinel round-trip and exit | +| `--json` | Emit machine-readable JSON result to stdout; human output goes to stderr | + +## Degraded mode + +If Docker is absent and Qdrant/Ollama are not already reachable at +`QDRANT_URL`/`OLLAMA_URL`, the installer continues in SQLite-only mode: + +> Running in SQLite-only mode. Vector recall + local LLM auto-ingest are OFF. +> To enable them: install Docker and re-run with `--full-stack`, or point +> QDRANT_URL / OLLAMA_URL at existing services. + +The exit code is still 0. Core claim storage, recall hooks, and MCP tools +remain fully functional. diff --git a/docs/INTEGRATING.md b/docs/INTEGRATING.md index 3fba18f..ffe3db1 100644 --- a/docs/INTEGRATING.md +++ b/docs/INTEGRATING.md @@ -3,6 +3,9 @@ > Scope: how ANY AI coding agent (Claude Code, Codex, Gemini, droid, opencode, or a > remote VM bridge like Hermes) uses MemoryMaster for cross-session memory. > Companion design map: `.planning/P4-AGENTS-CONTRACT.md` (file:line evidence). +> +> **Install / setup:** see [INSTALLATION.md](../INSTALLATION.md) and [docs/AGENT-INSTALL.md](AGENT-INSTALL.md). +> This document covers the integration contract only — not how to install or configure MemoryMaster. Every agent that uses MemoryMaster runs the **same three beats**. The beats are the contract; the *mechanism* differs per agent class (installed hooks vs. a reference @@ -71,7 +74,9 @@ and is a defect. ### Claude Code — fully turnkey (installed hooks) -`setup_hooks.py` installs three hooks; no manual wiring needed. +`memorymaster-setup` (or `python scripts/setup-hooks.py`) installs the hooks automatically. +See [INSTALLATION.md](../INSTALLATION.md) for setup flags and [docs/AGENT-INSTALL.md](AGENT-INSTALL.md) +for the one-paste agent prompt. Once installed, all three beats fire automatically with no manual wiring. | Beat | Mechanism | Template | |------|-----------|----------| @@ -79,11 +84,9 @@ and is a defect. | 2 RECALL | `UserPromptSubmit` hook runs `recall()` read-only and injects `[MemoryMaster recall]` | `config_templates/hooks/memorymaster-recall.py` | | 3 INGEST | `Stop` hook: block-to-save every 15 msgs + passive ≤3 distill → `service.ingest` with `source_agent="llm-stop-hook"`; the block reason tells Claude to ingest with `source_agent="claude-session"` | `config_templates/hooks/memorymaster-auto-ingest.py` | -Nothing to do beyond running `python scripts/setup-hooks.py`. - ### Codex / generic MCP agents — AGENTS.md (instruction) + reference script (automation) -There are **two layers**, and you need both: +There are **two layers**, and you need both. `memorymaster-setup` (see [INSTALLATION.md](../INSTALLATION.md)) wires both automatically when `~/.codex/` is detected. **Instruction layer** — `setup_hooks.append_instructions()` appends `config_templates/codex-agents-md-append.md` to `~/.codex/AGENTS.md` (CODEX_DIR-aware). diff --git a/memorymaster/surfaces/setup_detect.py b/memorymaster/surfaces/setup_detect.py new file mode 100644 index 0000000..3d361a1 --- /dev/null +++ b/memorymaster/surfaces/setup_detect.py @@ -0,0 +1,345 @@ +"""Pure, side-effect-free environment probing for MemoryMaster onboarding. + +All probes are read-only, shell=False, timeout-bounded (<=5 s each), and +degrade gracefully to absent on ANY error — they never raise. + +Public API +---------- +detect_environment(*, cwd=None) -> Detected +format_plan(d, *, want_full_stack) -> list[str] +""" +from __future__ import annotations + +import json +import os +import platform +import subprocess +import sys +import urllib.error +import urllib.request +from dataclasses import dataclass +from pathlib import Path +from typing import Optional + + +# --------------------------------------------------------------------------- +# DTO +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class Detected: + python_version: str + pip_ok: bool + os: str # "Windows" | "Linux" | "Darwin" + docker: bool # `docker --version` ok + docker_compose: bool # `docker compose version` ok + ollama: bool # HTTP GET OLLAMA_URL/api/tags ok OR `ollama --version` + ollama_models: tuple[str, ...] + qdrant: bool # HTTP GET QDRANT_URL/healthz ok + obsidian_vault: Optional[str] # path if an obsidian-vault/ found under cwd + gitnexus: bool # `.gitnexus/` present OR `npx gitnexus` resolvable + claude_code: bool # ~/.claude/ exists + codex: bool # ~/.codex/ exists + mm_installed: bool # `import memorymaster` works + mm_mcp_registered: bool # memorymaster already in ~/.claude.json mcpServers + existing_hooks: tuple[str, ...] # memorymaster hook names already in settings.json + + +# --------------------------------------------------------------------------- +# Internal probe helpers — every one returns a value, never raises +# --------------------------------------------------------------------------- + +_PROBE_TIMEOUT = 5 # seconds + + +def _run(args: list[str]) -> Optional[str]: + """Run a subprocess (shell=False, timeout<=5s). + + Returns stdout text on success, None on any failure. + """ + try: + result = subprocess.run( + args, + capture_output=True, + text=True, + shell=False, + timeout=_PROBE_TIMEOUT, + ) + if result.returncode == 0: + return result.stdout + return None + except Exception: # noqa: BLE001 — intentional catch-all for probe + return None + + +def _http_get(url: str) -> Optional[bytes]: + """HTTP GET with timeout. Returns body bytes on 2xx, None otherwise.""" + try: + req = urllib.request.urlopen(url, timeout=_PROBE_TIMEOUT) # noqa: S310 + return req.read() + except Exception: # noqa: BLE001 + return None + + +def _probe_python_version() -> str: + return f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + + +def _probe_pip_ok() -> bool: + return _run([sys.executable, "-m", "pip", "--version"]) is not None + + +def _probe_docker() -> bool: + return _run(["docker", "--version"]) is not None + + +def _probe_docker_compose() -> bool: + return _run(["docker", "compose", "version"]) is not None + + +def _probe_ollama() -> tuple[bool, tuple[str, ...]]: + """Return (reachable, model_names). + + Tries HTTP first (OLLAMA_URL env or default localhost:11434), then CLI. + """ + ollama_url = os.environ.get("OLLAMA_URL", "http://localhost:11434") + tags_url = ollama_url.rstrip("/") + "/api/tags" + + body = _http_get(tags_url) + if body is not None: + try: + data = json.loads(body) + models = tuple( + m.get("name", "") for m in data.get("models", []) if m.get("name") + ) + return True, models + except Exception: # noqa: BLE001 + return True, () + + # Fallback: CLI probe + if _run(["ollama", "--version"]) is not None: + return True, () + + return False, () + + +def _probe_qdrant() -> bool: + qdrant_url = os.environ.get("QDRANT_URL", "http://localhost:6333") + healthz_url = qdrant_url.rstrip("/") + "/healthz" + return _http_get(healthz_url) is not None + + +def _probe_obsidian_vault(cwd: Path) -> Optional[str]: + candidate = cwd / "obsidian-vault" + if candidate.is_dir(): + return str(candidate) + return None + + +def _probe_gitnexus(cwd: Path) -> bool: + if (cwd / ".gitnexus").is_dir(): + return True + return _run(["npx", "gitnexus", "--version"]) is not None + + +def _probe_claude_code(home: Path) -> bool: + return (home / ".claude").is_dir() + + +def _probe_codex(home: Path) -> bool: + return (home / ".codex").is_dir() + + +def _probe_mm_installed() -> bool: + try: + import importlib + + spec = importlib.util.find_spec("memorymaster") + return spec is not None + except Exception: # noqa: BLE001 + return False + + +def _probe_mm_mcp_registered(home: Path) -> bool: + """Check ~/.claude.json for a memorymaster entry in mcpServers.""" + claude_json = home / ".claude.json" + if not claude_json.is_file(): + return False + try: + data = json.loads(claude_json.read_text(encoding="utf-8")) + servers = data.get("mcpServers", {}) + if isinstance(servers, dict): + return any("memorymaster" in k for k in servers) + return False + except Exception: # noqa: BLE001 + return False + + +def _probe_existing_hooks(home: Path) -> tuple[str, ...]: + """Return hook event names that already contain a memorymaster entry.""" + settings_path = home / ".claude" / "settings.json" + if not settings_path.is_file(): + return () + try: + data = json.loads(settings_path.read_text(encoding="utf-8")) + hooks_section = data.get("hooks", {}) + if not isinstance(hooks_section, dict): + return () + found: list[str] = [] + for event, entries in hooks_section.items(): + raw = json.dumps(entries) + if "memorymaster" in raw: + found.append(event) + return tuple(found) + except Exception: # noqa: BLE001 + return () + + +# --------------------------------------------------------------------------- +# Public API +# --------------------------------------------------------------------------- + + +def detect_environment(*, cwd: Optional[Path] = None) -> Detected: + """Probe the environment and return a frozen Detected snapshot. + + All probes are read-only, shell=False, timeout-bounded, and never raise. + """ + effective_cwd = cwd if cwd is not None else Path.cwd() + home = Path.home() + + ollama_ok, ollama_models = _probe_ollama() + + return Detected( + python_version=_probe_python_version(), + pip_ok=_probe_pip_ok(), + os=platform.system(), + docker=_probe_docker(), + docker_compose=_probe_docker_compose(), + ollama=ollama_ok, + ollama_models=ollama_models, + qdrant=_probe_qdrant(), + obsidian_vault=_probe_obsidian_vault(effective_cwd), + gitnexus=_probe_gitnexus(effective_cwd), + claude_code=_probe_claude_code(home), + codex=_probe_codex(home), + mm_installed=_probe_mm_installed(), + mm_mcp_registered=_probe_mm_mcp_registered(home), + existing_hooks=_probe_existing_hooks(home), + ) + + +# --------------------------------------------------------------------------- +# Plan formatter +# --------------------------------------------------------------------------- + +_STEP_WILL_DO = "will-do" +_STEP_PRESENT = "skip-present" +_STEP_CANT = "cant-missing" + + +def _line(tag: str, description: str) -> str: + icons = { + _STEP_WILL_DO: "[will-do]", + _STEP_PRESENT: "[skip-present]", + _STEP_CANT: "[cant-missing]", + } + return f" {icons[tag]} {description}" + + +def format_plan(d: Detected, *, want_full_stack: bool) -> list[str]: + """Return a human-readable list of 'will-do / skip-present / cant-missing' lines. + + Each line describes one setup action and whether it will be applied, + skipped (already present), or blocked by a missing dependency. + """ + lines: list[str] = [] + + # --- pip / memorymaster install --- + if d.mm_installed: + lines.append(_line(_STEP_PRESENT, "memorymaster already installed")) + else: + lines.append(_line(_STEP_WILL_DO, "pip install memorymaster")) + + # --- Claude Code hooks --- + if d.claude_code: + if d.existing_hooks: + joined = ", ".join(sorted(d.existing_hooks)) + lines.append( + _line(_STEP_PRESENT, f"Claude Code hooks already registered ({joined})") + ) + else: + lines.append(_line(_STEP_WILL_DO, "install Claude Code hooks (~/.claude/)")) + else: + lines.append( + _line(_STEP_CANT, "Claude Code hooks — ~/.claude/ not found (install Claude Code first)") + ) + + # --- MCP registration --- + if d.mm_mcp_registered: + lines.append(_line(_STEP_PRESENT, "MCP server already registered in ~/.claude.json")) + elif d.claude_code: + lines.append(_line(_STEP_WILL_DO, "register MCP server in ~/.claude.json")) + else: + lines.append( + _line(_STEP_CANT, "MCP registration — ~/.claude/ not found") + ) + + # --- Codex integration --- + if d.codex: + lines.append(_line(_STEP_PRESENT, "Codex (~/.codex/) present — will wire MCP + instructions")) + else: + lines.append(_line(_STEP_CANT, "Codex integration — ~/.codex/ not found (optional)")) + + # --- Full-stack: Docker / Qdrant / Ollama --- + if want_full_stack: + if d.qdrant: + lines.append(_line(_STEP_PRESENT, "Qdrant already reachable")) + elif d.docker_compose: + lines.append(_line(_STEP_WILL_DO, "docker compose up -d qdrant")) + else: + lines.append( + _line( + _STEP_CANT, + "Qdrant — Docker Compose not found; running in SQLite-only mode. " + "Install Docker or point QDRANT_URL at an existing service to enable vector recall.", + ) + ) + + if d.ollama: + lines.append( + _line( + _STEP_PRESENT, + "Ollama already reachable" + + (f" (models: {', '.join(d.ollama_models)})" if d.ollama_models else ""), + ) + ) + elif d.docker_compose: + lines.append(_line(_STEP_WILL_DO, "docker compose up -d ollama")) + else: + lines.append( + _line( + _STEP_CANT, + "Ollama — Docker Compose not found; local LLM auto-ingest is OFF. " + "Install Docker or point OLLAMA_URL at an existing service.", + ) + ) + else: + lines.append(_line(_STEP_CANT, "Full-stack (Qdrant + Ollama) skipped — not requested")) + + # --- Obsidian vault --- + if d.obsidian_vault: + lines.append(_line(_STEP_PRESENT, f"Obsidian vault found at {d.obsidian_vault}")) + else: + lines.append(_line(_STEP_WILL_DO, "create obsidian-vault/ skeleton")) + + # --- GitNexus --- + if d.gitnexus: + lines.append(_line(_STEP_PRESENT, "GitNexus index (.gitnexus/) already present")) + else: + lines.append( + _line(_STEP_CANT, "GitNexus index — .gitnexus/ not found (run `npx gitnexus analyze` to enable)") + ) + + return lines diff --git a/memorymaster/surfaces/setup_hooks.py b/memorymaster/surfaces/setup_hooks.py index 22f3fae..e529dc3 100644 --- a/memorymaster/surfaces/setup_hooks.py +++ b/memorymaster/surfaces/setup_hooks.py @@ -15,19 +15,24 @@ - Steward cron (every 6h) - Obsidian skills installation """ +import argparse import json import os import platform import shutil import subprocess import sys +import time from pathlib import Path +from typing import Any, Optional try: from importlib.resources import files as _resource_files except ImportError: # pragma: no cover - Python < 3.9 from importlib_resources import files as _resource_files # type: ignore +from memorymaster.surfaces.setup_detect import Detected, detect_environment, format_plan + # Templates are shipped inside the package as importable resources. TEMPLATES_DIR = Path(str(_resource_files("memorymaster") / "config_templates")) @@ -43,16 +48,60 @@ CODEX_DIR = HOME / ".codex" PYTHON_EXE = sys.executable +# --------------------------------------------------------------------------- +# Non-interactive mode (set by main() when --yes is passed). When True, +# ask()/ask_yn() never call input() — they return the supplied default. This +# is what makes the installer scriptable + hermetic-testable. +# --------------------------------------------------------------------------- +NON_INTERACTIVE = False + + +def set_non_interactive(value: bool) -> None: + """Toggle module-level non-interactive mode (no input() prompts).""" + global NON_INTERACTIVE + NON_INTERACTIVE = bool(value) + + +def _load_json_preserving(path: Path) -> dict: + """Load a JSON object from *path*, returning ``{}`` when it's absent. + + If the file EXISTS but is malformed JSON, back it up to + ``.corrupt-.bak`` and warn BEFORE returning ``{}`` — so a + later write never silently overwrites (and loses) a user's hand-edited + config. If the backup itself can't be written we raise rather than wipe: + losing the data in place is worse than a clear, actionable abort. + """ + if not path.exists(): + return {} + try: + data = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError: + backup = path.with_name(f"{path.name}.corrupt-{time.strftime('%Y%m%d-%H%M%S')}.bak") + try: + shutil.copy2(path, backup) + except OSError as exc: + raise RuntimeError( + f"{path} is not valid JSON and a backup could not be written ({exc}); " + "refusing to overwrite. Fix or move the file, then re-run." + ) from exc + print(f" WARNING: {path} is not valid JSON — backed up to {backup} before rewriting.") + return {} + return data if isinstance(data, dict) else {} + def ask(prompt, default=""): - """Prompt user for input.""" + """Prompt user for input. In non-interactive mode, returns the default.""" + if NON_INTERACTIVE: + return default suffix = f" [{default}]" if default else "" result = input(f" {prompt}{suffix}: ").strip() return result or default def ask_yn(prompt, default=True): - """Yes/no prompt.""" + """Yes/no prompt. In non-interactive mode, returns the default.""" + if NON_INTERACTIVE: + return default suffix = " [Y/n]" if default else " [y/N]" result = input(f" {prompt}{suffix}: ").strip().lower() if not result: @@ -118,14 +167,10 @@ def install_hooks(llm_config): dest.write_text(content, encoding="utf-8") print(f" Installed: {dest}") - # Update settings.json with hooks config + # Update settings.json with hooks config. A malformed pre-existing file is + # backed up (never silently wiped) by _load_json_preserving. settings_path = CLAUDE_DIR / "settings.json" - settings = {} - if settings_path.exists(): - try: - settings = json.loads(settings_path.read_text(encoding="utf-8")) - except json.JSONDecodeError: - settings = {} + settings = _load_json_preserving(settings_path) # Add env vars env = settings.setdefault("env", {}) @@ -234,46 +279,127 @@ def install_hooks(llm_config): # --------------------------------------------------------------------------- # 3. MCP config # --------------------------------------------------------------------------- -def install_mcp(): - banner("MemoryMaster MCP Server (Global)") +def _mcp_command_args() -> tuple[str, list[str]]: + """Return the (command, args) pair for the MCP server entry. - if not CLAUDE_JSON.exists(): - print(f" {CLAUDE_JSON} not found — creating") - CLAUDE_JSON.write_text("{}", encoding="utf-8") + Uses the non-deprecated module path memorymaster.surfaces.mcp_server. + The installed console-script entry point is ``memorymaster-mcp`` (see + pyproject), but we register the explicit interpreter + module form so the + registration is robust even when the script dir is not on PATH. + """ + return PYTHON_EXE, ["-m", "memorymaster.surfaces.mcp_server"] - data = json.loads(CLAUDE_JSON.read_text(encoding="utf-8")) - servers = data.setdefault("mcpServers", {}) - if "memorymaster" in servers: - if not ask_yn("memorymaster MCP already configured. Overwrite?", False): - return - - db_path = str(PROJECT_ROOT / "memorymaster.db") - servers["memorymaster"] = { +def _mcp_server_entry(db_path: str) -> dict[str, Any]: + command, args = _mcp_command_args() + return { "type": "stdio", - "command": PYTHON_EXE, - "args": ["-m", "memorymaster.mcp_server"], + "command": command, + "args": args, "env": { "MEMORYMASTER_DEFAULT_DB": db_path, "MEMORYMASTER_WORKSPACE": str(PROJECT_ROOT), - } + }, } + +def install_mcp(*, force: bool = False, already_registered: bool = False): + """Register the MemoryMaster MCP server in ~/.claude.json. + + Brownfield-safe: if an entry already exists, skip (no clobber) unless + ``force`` is set. ``already_registered`` lets the caller pass the + Detected report so we avoid touching the file when nothing would change. + """ + banner("MemoryMaster MCP Server (Global)") + + # Read-merge-write. A malformed pre-existing .claude.json is backed up + # (never silently wiped) by _load_json_preserving; absent -> {}. + data = _load_json_preserving(CLAUDE_JSON) + servers = data.setdefault("mcpServers", {}) + + if "memorymaster" in servers and not force: + if not ask_yn("memorymaster MCP already configured. Overwrite?", False): + print(" Keeping existing MCP entry (brownfield) — pass --force to replace.") + return + + db_path = str(PROJECT_ROOT / "memorymaster.db") + servers["memorymaster"] = _mcp_server_entry(db_path) + CLAUDE_JSON.write_text(json.dumps(data, indent=2, ensure_ascii=False), encoding="utf-8") print(f" Added memorymaster to {CLAUDE_JSON}") +# Marker-bounded block for the Codex config.toml MCP table. We do NOT parse +# the whole TOML (no writer in stdlib for 3.10); instead we manage an +# idempotent fenced block so re-runs replace cleanly and unknown content is +# preserved untouched. +_CODEX_MCP_BEGIN = "# >>> memorymaster mcp (managed by memorymaster-setup) >>>" +_CODEX_MCP_END = "# <<< memorymaster mcp (managed by memorymaster-setup) <<<" + + +def install_mcp_codex(*, force: bool = False): + """Register the MCP server for Codex (~/.codex/config.toml). + + Codex reads MCP servers from ``[mcp_servers.]`` TOML tables. We + write a marker-bounded block so re-runs are idempotent and any + pre-existing user TOML outside the block is preserved verbatim. + """ + if not CODEX_DIR.exists(): + return + banner("MemoryMaster MCP Server (Codex)") + config_path = CODEX_DIR / "config.toml" + existing = config_path.read_text(encoding="utf-8") if config_path.exists() else "" + + has_block = _CODEX_MCP_BEGIN in existing + if has_block and not force: + if not ask_yn("memorymaster MCP already in Codex config. Overwrite?", False): + print(" Keeping existing Codex MCP entry (brownfield) — pass --force to replace.") + return + + db_path = str(PROJECT_ROOT / "memorymaster.db") + command, args = _mcp_command_args() + args_toml = ", ".join(json.dumps(a) for a in args) + block = ( + f"{_CODEX_MCP_BEGIN}\n" + "[mcp_servers.memorymaster]\n" + f"command = {json.dumps(command)}\n" + f"args = [{args_toml}]\n" + "[mcp_servers.memorymaster.env]\n" + f"MEMORYMASTER_DEFAULT_DB = {json.dumps(db_path)}\n" + f"MEMORYMASTER_WORKSPACE = {json.dumps(str(PROJECT_ROOT))}\n" + f"{_CODEX_MCP_END}\n" + ) + + if has_block: + before, _, rest = existing.partition(_CODEX_MCP_BEGIN) + _, _, after = rest.partition(_CODEX_MCP_END) + # Drop a single trailing newline left by the old block end marker. + new_content = before.rstrip("\n") + "\n\n" + block + after.lstrip("\n") + else: + sep = "\n\n" if existing.strip() else "" + new_content = existing.rstrip("\n") + sep + ("\n" if existing.strip() else "") + block + if not existing.strip(): + new_content = block + + config_path.write_text(new_content, encoding="utf-8") + print(f" Registered memorymaster MCP in {config_path}") + + # --------------------------------------------------------------------------- # 4. Append to CLAUDE.md and AGENTS.md # --------------------------------------------------------------------------- def append_instructions(): banner("Append MemoryMaster Instructions") - # Claude global CLAUDE.md + # Claude global CLAUDE.md — only when ~/.claude exists (Claude Code + # installed). Mirrors the Codex guard below; without it a from-zero box + # (no Claude Code yet) crashes writing into a non-existent ~/.claude. claude_md = CLAUDE_DIR / "CLAUDE.md" marker = "## MemoryMaster (Cross-Session Memory)" - if claude_md.exists() and marker in claude_md.read_text(encoding="utf-8"): + if not CLAUDE_DIR.exists(): + print(f" {CLAUDE_DIR} not found — skipping CLAUDE.md (install Claude Code, then re-run)") + elif claude_md.exists() and marker in claude_md.read_text(encoding="utf-8"): print(f" {claude_md} already has MemoryMaster section — skipping") elif ask_yn("Append MemoryMaster instructions to ~/.claude/CLAUDE.md?"): append = (TEMPLATES_DIR / "claude-md-append.md").read_text(encoding="utf-8") @@ -377,56 +503,414 @@ def install_obsidian_skills(): print(" You can install manually: git clone https://github.com/kepano/obsidian-skills.git") +# --------------------------------------------------------------------------- +# 7. Full-stack orchestration (Qdrant + Ollama via Docker Compose) +# --------------------------------------------------------------------------- +SQLITE_ONLY_MESSAGE = ( + "Running in SQLite-only mode. Vector recall + local LLM auto-ingest are OFF.\n" + " To enable them: install Docker and re-run with --full-stack, or point\n" + " QDRANT_URL / OLLAMA_URL at existing services." +) + + +def setup_full_stack(detected: Detected, *, interactive: bool, yes: bool, model: str = "") -> dict[str, Any]: + """Bring up the optional vector + local-LLM stack (Qdrant + Ollama). + + Brownfield: reuse already-healthy services. No-Docker fallback: print the + SQLite-only degraded message and CONTINUE (never block core install). + + Returns a dict describing what happened (for --json), including a + ``degraded`` boolean and a human ``message`` when degraded. + """ + banner("Full-Stack (Qdrant + Ollama)") + result: dict[str, Any] = { + "qdrant": "absent", + "ollama": "absent", + "degraded": False, + "message": "", + "compose_run": False, + } + + # Brownfield: already-healthy services are reused as-is. + if detected.qdrant: + result["qdrant"] = "reused" + print(" Qdrant already reachable — reusing.") + if detected.ollama: + result["ollama"] = "reused" + print(" Ollama already reachable — reusing.") + + qdrant_needed = not detected.qdrant + ollama_needed = not detected.ollama + + if not (qdrant_needed or ollama_needed): + print(" Full stack already healthy — nothing to do.") + return result + + # No Docker Compose → degraded fallback (non-fatal). + if not detected.docker_compose: + result["degraded"] = True + result["message"] = SQLITE_ONLY_MESSAGE + print(f" {SQLITE_ONLY_MESSAGE}") + return result + + if interactive and not yes: + if not ask_yn("Start Qdrant + Ollama via docker compose?", True): + result["degraded"] = True + result["message"] = SQLITE_ONLY_MESSAGE + print(" Skipped full-stack at user request.") + print(f" {SQLITE_ONLY_MESSAGE}") + return result + + compose_file = PROJECT_ROOT / "docker-compose.yml" + up_args = ["docker", "compose"] + if compose_file.is_file(): + up_args += ["-f", str(compose_file)] + services = [s for s, needed in (("qdrant", qdrant_needed), ("ollama", ollama_needed)) if needed] + up_args += ["up", "-d", *services] + + try: + proc = subprocess.run(up_args, capture_output=True, text=True, timeout=120) + if proc.returncode == 0: + result["compose_run"] = True + for svc in services: + result[svc] = "started" + print(f" docker compose up -d {' '.join(services)} — OK") + else: + result["degraded"] = True + result["message"] = SQLITE_ONLY_MESSAGE + print(f" docker compose failed (rc={proc.returncode}); continuing degraded.") + print(f" {SQLITE_ONLY_MESSAGE}") + return result + except Exception as exc: # noqa: BLE001 — stack failure must never block core install + result["degraded"] = True + result["message"] = SQLITE_ONLY_MESSAGE + print(f" docker compose error: {exc}; continuing degraded.") + print(f" {SQLITE_ONLY_MESSAGE}") + return result + + # Best-effort model pull (non-fatal). + if ollama_needed and model: + try: + subprocess.run(["ollama", "pull", model], capture_output=True, text=True, timeout=300) + print(f" ollama pull {model} — requested") + except Exception as exc: # noqa: BLE001 + print(f" ollama pull {model} failed (non-fatal): {exc}") + + return result + + +# --------------------------------------------------------------------------- +# 8. Verify install (sentinel round-trip) +# --------------------------------------------------------------------------- +def verify_install(db_path: str | Path) -> dict[str, Any]: + """Ingest a sentinel claim via the core service, then query it back. + + Returns {"status": "PASS"|"PARTIAL"|"FAIL", "detail": str, "mcp_note": str}. + Never raises — a failure degrades to a FAIL/PARTIAL result. + """ + banner("Verify Install") + db_path = str(db_path) + result: dict[str, Any] = {"status": "FAIL", "detail": "", "mcp_note": ""} + + try: + from memorymaster.core.models import CitationInput + from memorymaster.core.service import MemoryService + except Exception as exc: # noqa: BLE001 + result["detail"] = f"could not import core service: {exc}" + print(f" FAIL — {result['detail']}") + return result + + sentinel = "memorymaster setup verification sentinel claim" + try: + svc = MemoryService(db_path) + svc.init_db() + svc.ingest( + sentinel, + [CitationInput(source="setup-verify", locator="verify_install")], + scope="project:memorymaster-verify", + source_agent="memorymaster-setup", + idempotency_key="memorymaster-setup-verify-sentinel", + ) + hits = svc.query( + "verification sentinel", + limit=10, + include_candidates=True, + scope_allowlist=["project:memorymaster-verify"], + ) + round_tripped = any("sentinel" in (getattr(c, "text", "") or "") for c in hits) + except Exception as exc: # noqa: BLE001 + result["detail"] = f"round-trip failed: {exc}" + print(f" FAIL — {result['detail']}") + return result + + if round_tripped: + result["status"] = "PASS" + result["detail"] = "sentinel claim ingested and recalled successfully" + print(" PASS — sentinel claim ingested and recalled.") + else: + result["status"] = "PARTIAL" + result["detail"] = "ingest succeeded but query did not return the sentinel" + print(" PARTIAL — ingest OK but recall did not find the sentinel.") + + if detect_environment(cwd=PROJECT_ROOT).mm_mcp_registered: + result["mcp_note"] = "MCP registered — restart your session to load it." + print(f" {result['mcp_note']}") + + return result + + +# --------------------------------------------------------------------------- +# argparse +# --------------------------------------------------------------------------- +def build_arg_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser( + prog="memorymaster-setup", + description="Detect-first, idempotent MemoryMaster onboarding.", + ) + p.add_argument("-y", "--yes", action="store_true", help="non-interactive; accept all defaults") + p.add_argument("--db", help="path to memorymaster.db (overrides project-root default)") + p.add_argument( + "--provider", + choices=["google", "openai", "anthropic", "ollama"], + help="LLM provider for the auto-ingest Stop hook", + ) + p.add_argument("--api-key", help="API key for the chosen provider") + p.add_argument("--model", help="LLM model id") + p.add_argument("--project-root", help="directory where memorymaster.db lives") + p.add_argument( + "--full-stack", + dest="full_stack", + action="store_true", + default=None, + help="bring up Qdrant + Ollama (default)", + ) + p.add_argument( + "--no-full-stack", + dest="full_stack", + action="store_false", + help="skip the vector + local-LLM stack", + ) + p.add_argument("--no-cron", action="store_true", help="skip steward cron setup") + p.add_argument("--no-obsidian-skills", action="store_true", help="skip Obsidian skills install") + p.add_argument( + "--codex", + dest="codex", + action="store_true", + default=None, + help="force Codex MCP + instructions wiring", + ) + p.add_argument("--no-codex", dest="codex", action="store_false", help="skip Codex wiring") + p.add_argument("--force", action="store_true", help="overwrite existing MCP entries") + p.add_argument("--verify-only", action="store_true", help="run only the verify round-trip and exit") + p.add_argument("--json", action="store_true", help="emit machine-readable JSON result") + return p + + +def _resolve_provider_config(args: argparse.Namespace) -> dict[str, str]: + """Build the llm_config from flags, falling back to interactive prompts.""" + defaults = { + "google": ("GEMINI_API_KEY", "gemini-3.1-flash-lite-preview"), + "openai": ("OPENAI_API_KEY", "gpt-4o-mini"), + "anthropic": ("ANTHROPIC_API_KEY", "claude-haiku-4-5-20251001"), + "ollama": ("", "llama3.2:3b"), + } + if args.provider: + _, default_model = defaults[args.provider] + return { + "provider": args.provider, + "api_key": args.api_key or "", + "model": args.model or default_model, + } + # No provider flag: prompt (honors NON_INTERACTIVE → returns defaults). + cfg = setup_llm_provider() + if args.api_key: + cfg["api_key"] = args.api_key + if args.model: + cfg["model"] = args.model + return cfg + + # --------------------------------------------------------------------------- # Main # --------------------------------------------------------------------------- -def main(): +def _emit_json(payload: dict[str, Any]) -> None: + print(json.dumps(payload, indent=2, ensure_ascii=False)) + + +def main(argv: Optional[list[str]] = None) -> int: + """Entry point. In --json mode, all human chatter is routed to stderr so + that stdout contains ONLY the parseable JSON document.""" + args = build_arg_parser().parse_args(argv) + if not args.json: + rc, _payload = _run_main(args) + return rc + + import contextlib + + with contextlib.redirect_stdout(sys.stderr): + rc, payload = _run_main(args) + if payload is not None: + _emit_json(payload) + return rc + + +def _run_main(args: argparse.Namespace) -> tuple[int, Optional[dict[str, Any]]]: global PROJECT_ROOT - banner('MemoryMaster Setup') - default_root = str(Path.cwd()) - root_input = ask('Project root (where memorymaster.db lives)', default_root) + + # Never prompt when stdout may be parsed (--json), when explicitly + # non-interactive (--yes), or for a quick --verify-only check — stdin may + # not be a tty (CI, `docker exec`, an agent), where input() raises EOFError. + set_non_interactive(bool(args.yes) or bool(args.json) or bool(args.verify_only)) + want_full_stack = True if args.full_stack is None else bool(args.full_stack) + + # --verify-only short-circuits BEFORE any project-root prompt: it only needs + # --db (defaulting to cwd/memorymaster.db) and must never block on input. + if args.verify_only: + vdb = Path(args.db).expanduser().resolve() if args.db else Path.cwd() / "memorymaster.db" + verify = verify_install(vdb) + rc = 0 if verify.get("status") in ("PASS", "PARTIAL") else 1 + return rc, ({"verify": verify} if args.json else None) + + # --- Resolve project root (flag > prompt > cwd) --- + if args.project_root: + root_input = args.project_root + else: + banner("MemoryMaster Setup") + root_input = ask("Project root (where memorymaster.db lives)", str(Path.cwd())) PROJECT_ROOT = Path(root_input).expanduser().resolve() PROJECT_ROOT.mkdir(parents=True, exist_ok=True) - print(f' Using project root: {PROJECT_ROOT}') - banner("MemoryMaster Setup") - print(" This will configure hooks, MCP, steward cron, and optional integrations.") + db_path = Path(args.db).expanduser().resolve() if args.db else PROJECT_ROOT / "memorymaster.db" + + # --- Detect-first --- + detected = detect_environment(cwd=PROJECT_ROOT) + plan = format_plan(detected, want_full_stack=want_full_stack) + banner("Detection + Plan") print(f" Project root: {PROJECT_ROOT}") print(f" Python: {PYTHON_EXE}") print() + for line in plan: + print(line) + print() - if not ask_yn("Continue?"): - return + if not args.json and not ask_yn("Continue?"): + return 0, None - # Init DB if needed - db_path = PROJECT_ROOT / "memorymaster.db" + applied: dict[str, Any] = {} + + # --- Init DB if needed --- if not db_path.exists(): - print("\n Initializing database...") - subprocess.run([PYTHON_EXE, "-m", "memorymaster", "init-db", "--db", str(db_path)], check=True) + if not args.json: + print("\n Initializing database...") + try: + subprocess.run( + # `--db` is a GLOBAL arg — it MUST precede the subcommand, + # else argparse exits 2 (unrecognized arguments). + [PYTHON_EXE, "-m", "memorymaster", "--db", str(db_path), "init-db"], + check=True, + # capture_output: the subprocess writes to the REAL stdout fd, + # which contextlib.redirect_stdout (a sys.stdout swap) does NOT + # cover — without this its "initialized db" line corrupts the + # --json document on stdout. + capture_output=True, + text=True, + ) + applied["db_init"] = True + except Exception as exc: # noqa: BLE001 + applied["db_init"] = f"error: {exc}" + + # --- LLM provider config --- + llm_config = _resolve_provider_config(args) + applied["provider"] = llm_config["provider"] + + # --- Claude Code hooks (idempotent remove-then-add; brownfield-safe) --- + if detected.claude_code: + install_hooks(llm_config) + applied["hooks"] = "installed" + else: + applied["hooks"] = "skipped (no ~/.claude/)" + + # --- MCP registration (skip-if-present unless --force) --- + if detected.claude_code: + install_mcp(force=args.force, already_registered=detected.mm_mcp_registered) + applied["mcp_claude"] = "present" if (detected.mm_mcp_registered and not args.force) else "registered" + else: + applied["mcp_claude"] = "skipped (no ~/.claude/)" - llm_config = setup_llm_provider() - install_hooks(llm_config) - install_mcp() + # --- Instructions (CLAUDE.md / AGENTS.md / Codex session-end) --- append_instructions() - setup_steward_cron() - install_obsidian_skills() + + # --- Codex MCP wiring --- + want_codex = detected.codex if args.codex is None else bool(args.codex) + if want_codex and detected.codex: + install_mcp_codex(force=args.force) + applied["mcp_codex"] = "registered" + else: + applied["mcp_codex"] = "skipped" + + # --- Steward cron (references the Claude hook script; needs ~/.claude) --- + if not args.no_cron and detected.claude_code: + setup_steward_cron() + applied["cron"] = "configured" + else: + applied["cron"] = "skipped" if args.no_cron else "skipped (no ~/.claude/)" + + # --- Obsidian skills (installed into ~/.claude/skills) --- + if not args.no_obsidian_skills and detected.claude_code: + install_obsidian_skills() + applied["obsidian_skills"] = "attempted" + else: + applied["obsidian_skills"] = "skipped" + + # --- Full-stack orchestration --- + degraded = False + if want_full_stack: + stack = setup_full_stack( + detected, interactive=not args.yes, yes=bool(args.yes), model=llm_config.get("model", "") + ) + applied["full_stack"] = stack + degraded = bool(stack.get("degraded")) + else: + applied["full_stack"] = {"skipped": True} + + # --- Verify --- + verify = verify_install(db_path) banner("Setup Complete!") - print(" Restart all Claude Code / Codex sessions to apply changes.") - print() - print(" What's configured:") - print(" - Recall hook (UserPromptSubmit) — injects relevant claims into each prompt") - print(" - Auto-ingest hook (Stop) — extracts learnings via LLM after each response") - print(" - MemoryMaster MCP — 21 tools available in all sessions") - print(" - Steward cron — validates claims every 6 hours") + print(" What actually happened (skips reflect what's installed on this box):") + if detected.claude_code: + print(" - Claude hooks (recall + auto-ingest + session-start) — installed") + print(" - MemoryMaster MCP — registered (memorymaster.surfaces.mcp_server)") + else: + print(" - Claude Code not detected — hooks + MCP SKIPPED. Install Claude") + print(" Code, then re-run `memorymaster-setup` to wire them.") + print(f" - Codex MCP — {applied.get('mcp_codex')}") + print(f" - DB: {db_path} | verify: {verify.get('status')}") print(f" - LLM provider: {llm_config['provider']}") + if degraded: + print(" - Stack: SQLite-only (degraded) — vector recall + local LLM OFF") print() print(" Next steps:") - print(" 1. Restart Claude Code sessions") - print(" 2. Run: memorymaster --db memorymaster.db run-cycle") - print(" 3. Open Obsidian vault at: obsidian-vault/") + if detected.claude_code or detected.codex: + print(" 1. Restart Claude Code / Codex sessions to load hooks + MCP") + else: + print(" 1. Install Claude Code or Codex, then re-run memorymaster-setup") + print(" 2. Re-check anytime with: memorymaster-setup --verify-only") print() + if args.json: + from dataclasses import asdict + + payload = { + "detected": asdict(detected), + "planned": plan, + "applied": applied, + "verify": verify, + "degraded": degraded, + } + return 0, payload + return 0, None + if __name__ == "__main__": - main() + raise SystemExit(main()) diff --git a/tests/test_setup_detect.py b/tests/test_setup_detect.py new file mode 100644 index 0000000..a002c28 --- /dev/null +++ b/tests/test_setup_detect.py @@ -0,0 +1,643 @@ +"""Hermetic tests for memorymaster.surfaces.setup_detect. + +Rules enforced here: +- subprocess.run is ALWAYS mocked — no real processes spawned. +- HTTP (_http_get) is patched — no real network calls. +- HOME / CLAUDE_DIR / ~/.claude.json are redirected to tmp_path. +- The real ~/.claude, ~/.codex, ~/.claude.json are NEVER touched. +- All probes must degrade to absent on timeout / error without raising. +- No exceptions must escape detect_environment(). +""" +from __future__ import annotations + +import json +import subprocess +from pathlib import Path +from typing import Optional +from unittest.mock import MagicMock + +import pytest + +import memorymaster.surfaces.setup_detect as det +from memorymaster.surfaces.setup_detect import ( + Detected, + detect_environment, + format_plan, +) + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +_MODULE = "memorymaster.surfaces.setup_detect" + + +def _make_completed(stdout: str = "", returncode: int = 0) -> MagicMock: + """Return a mock CompletedProcess.""" + m = MagicMock(spec=subprocess.CompletedProcess) + m.stdout = stdout + m.returncode = returncode + return m + + +def _patch_home(tmp_path: Path) -> dict[str, Path]: + """Return a dict of tmp_path-based home dirs for patching.""" + home = tmp_path / "home" + home.mkdir(parents=True, exist_ok=True) + return {"home": home} + + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + + +@pytest.fixture() +def fake_home(tmp_path: Path) -> Path: + """A tmp home dir with no .claude / .codex pre-existing.""" + h = tmp_path / "home" + h.mkdir(parents=True, exist_ok=True) + return h + + +@pytest.fixture() +def fake_cwd(tmp_path: Path) -> Path: + cwd = tmp_path / "project" + cwd.mkdir(parents=True, exist_ok=True) + return cwd + + +# --------------------------------------------------------------------------- +# Individual probe unit tests (mock subprocess.run + _http_get) +# --------------------------------------------------------------------------- + + +class TestProbeDocker: + def test_present_when_version_returns_zero(self, monkeypatch): + monkeypatch.setattr( + det, + "_run", + lambda args: "Docker version 24.0" if args == ["docker", "--version"] else None, + ) + assert det._probe_docker() is True + + def test_absent_when_run_returns_none(self, monkeypatch): + monkeypatch.setattr(det, "_run", lambda args: None) + assert det._probe_docker() is False + + +class TestProbeDockerCompose: + def test_present(self, monkeypatch): + monkeypatch.setattr( + det, + "_run", + lambda args: "Docker Compose version v2" if args == ["docker", "compose", "version"] else None, + ) + assert det._probe_docker_compose() is True + + def test_absent(self, monkeypatch): + monkeypatch.setattr(det, "_run", lambda args: None) + assert det._probe_docker_compose() is False + + +class TestProbeOllama: + def test_http_path_parses_models(self, monkeypatch): + body = json.dumps( + {"models": [{"name": "llama3.2:3b"}, {"name": "mistral:7b"}]} + ).encode() + monkeypatch.setattr(det, "_http_get", lambda url: body) + ok, models = det._probe_ollama() + assert ok is True + assert "llama3.2:3b" in models + assert "mistral:7b" in models + + def test_http_path_empty_models(self, monkeypatch): + body = json.dumps({"models": []}).encode() + monkeypatch.setattr(det, "_http_get", lambda url: body) + ok, models = det._probe_ollama() + assert ok is True + assert models == () + + def test_cli_fallback_when_http_fails(self, monkeypatch): + monkeypatch.setattr(det, "_http_get", lambda url: None) + monkeypatch.setattr( + det, + "_run", + lambda args: "ollama 0.1.0" if args == ["ollama", "--version"] else None, + ) + ok, models = det._probe_ollama() + assert ok is True + assert models == () + + def test_absent_when_both_fail(self, monkeypatch): + monkeypatch.setattr(det, "_http_get", lambda url: None) + monkeypatch.setattr(det, "_run", lambda args: None) + ok, models = det._probe_ollama() + assert ok is False + assert models == () + + def test_uses_ollama_url_env(self, monkeypatch, tmp_path): + monkeypatch.setenv("OLLAMA_URL", "http://myhost:11434") + captured: list[str] = [] + + def fake_get(url: str) -> Optional[bytes]: + captured.append(url) + return None + + monkeypatch.setattr(det, "_http_get", fake_get) + monkeypatch.setattr(det, "_run", lambda args: None) + det._probe_ollama() + assert captured and "myhost:11434" in captured[0] + + def test_http_malformed_json_degrades_gracefully(self, monkeypatch): + monkeypatch.setattr(det, "_http_get", lambda url: b"not json") + ok, models = det._probe_ollama() + # HTTP responded (truthy body) but JSON parse fails — should still be ok=True + assert ok is True + assert models == () + + +class TestProbeQdrant: + def test_present_on_http_200(self, monkeypatch): + monkeypatch.setattr(det, "_http_get", lambda url: b"ok") + assert det._probe_qdrant() is True + + def test_absent_on_failure(self, monkeypatch): + monkeypatch.setattr(det, "_http_get", lambda url: None) + assert det._probe_qdrant() is False + + def test_uses_qdrant_url_env(self, monkeypatch): + monkeypatch.setenv("QDRANT_URL", "http://qdrant.internal:6333") + captured: list[str] = [] + + def fake_get(url: str) -> Optional[bytes]: + captured.append(url) + return b"ok" + + monkeypatch.setattr(det, "_http_get", fake_get) + det._probe_qdrant() + assert captured and "qdrant.internal" in captured[0] + + +class TestProbeObsidianVault: + def test_found_when_dir_exists(self, fake_cwd: Path): + vault = fake_cwd / "obsidian-vault" + vault.mkdir() + result = det._probe_obsidian_vault(fake_cwd) + assert result == str(vault) + + def test_none_when_absent(self, fake_cwd: Path): + assert det._probe_obsidian_vault(fake_cwd) is None + + +class TestProbeGitnexus: + def test_found_via_dir(self, fake_cwd: Path, monkeypatch): + monkeypatch.setattr(det, "_run", lambda args: None) + (fake_cwd / ".gitnexus").mkdir() + assert det._probe_gitnexus(fake_cwd) is True + + def test_found_via_npx(self, fake_cwd: Path, monkeypatch): + monkeypatch.setattr( + det, + "_run", + lambda args: "gitnexus/0.1" if "gitnexus" in args else None, + ) + assert det._probe_gitnexus(fake_cwd) is True + + def test_absent(self, fake_cwd: Path, monkeypatch): + monkeypatch.setattr(det, "_run", lambda args: None) + assert det._probe_gitnexus(fake_cwd) is False + + +class TestProbeClaudeCode: + def test_present_when_dir_exists(self, fake_home: Path): + (fake_home / ".claude").mkdir() + assert det._probe_claude_code(fake_home) is True + + def test_absent_when_missing(self, fake_home: Path): + assert det._probe_claude_code(fake_home) is False + + +class TestProbeCodex: + def test_present(self, fake_home: Path): + (fake_home / ".codex").mkdir() + assert det._probe_codex(fake_home) is True + + def test_absent(self, fake_home: Path): + assert det._probe_codex(fake_home) is False + + +class TestProbeMmInstalled: + def test_true_when_package_importable(self, monkeypatch): + # memorymaster IS importable in this test environment + result = det._probe_mm_installed() + assert isinstance(result, bool) + + def test_false_when_find_spec_returns_none(self, monkeypatch): + import importlib.util + + monkeypatch.setattr(importlib.util, "find_spec", lambda name: None) + assert det._probe_mm_installed() is False + + def test_false_on_exception(self, monkeypatch): + import importlib.util + + def boom(name): + raise RuntimeError("nope") + + monkeypatch.setattr(importlib.util, "find_spec", boom) + assert det._probe_mm_installed() is False + + +class TestProbeMmMcpRegistered: + def test_true_when_entry_present(self, fake_home: Path): + claude_json = fake_home / ".claude.json" + claude_json.write_text( + json.dumps({"mcpServers": {"memorymaster": {"command": "memorymaster-mcp"}}}), + encoding="utf-8", + ) + assert det._probe_mm_mcp_registered(fake_home) is True + + def test_false_when_no_entry(self, fake_home: Path): + claude_json = fake_home / ".claude.json" + claude_json.write_text( + json.dumps({"mcpServers": {"other-tool": {}}}), + encoding="utf-8", + ) + assert det._probe_mm_mcp_registered(fake_home) is False + + def test_false_when_file_absent(self, fake_home: Path): + assert det._probe_mm_mcp_registered(fake_home) is False + + def test_false_on_malformed_json(self, fake_home: Path): + (fake_home / ".claude.json").write_text("not json", encoding="utf-8") + assert det._probe_mm_mcp_registered(fake_home) is False + + def test_false_when_mcp_servers_not_dict(self, fake_home: Path): + (fake_home / ".claude.json").write_text( + json.dumps({"mcpServers": ["list-not-dict"]}), encoding="utf-8" + ) + assert det._probe_mm_mcp_registered(fake_home) is False + + +class TestProbeExistingHooks: + def test_returns_event_names_containing_memorymaster(self, fake_home: Path): + claude_dir = fake_home / ".claude" + claude_dir.mkdir() + settings = { + "hooks": { + "UserPromptSubmit": [ + {"hooks": [{"command": 'python "memorymaster-recall.py"'}]} + ], + "Stop": [{"hooks": [{"command": "other-tool"}]}], + } + } + (claude_dir / "settings.json").write_text( + json.dumps(settings), encoding="utf-8" + ) + result = det._probe_existing_hooks(fake_home) + assert "UserPromptSubmit" in result + assert "Stop" not in result + + def test_empty_tuple_when_no_settings(self, fake_home: Path): + assert det._probe_existing_hooks(fake_home) == () + + def test_empty_tuple_on_malformed_json(self, fake_home: Path): + claude_dir = fake_home / ".claude" + claude_dir.mkdir() + (claude_dir / "settings.json").write_text("bad json", encoding="utf-8") + assert det._probe_existing_hooks(fake_home) == () + + def test_empty_tuple_when_hooks_section_not_dict(self, fake_home: Path): + claude_dir = fake_home / ".claude" + claude_dir.mkdir() + (claude_dir / "settings.json").write_text( + json.dumps({"hooks": "string-not-dict"}), encoding="utf-8" + ) + assert det._probe_existing_hooks(fake_home) == () + + +# --------------------------------------------------------------------------- +# Degradation tests — every probe degrades to absent on error +# --------------------------------------------------------------------------- + + +class TestRunDegradation: + """_run() must return None (not raise) on every failure mode.""" + + def test_returns_none_on_timeout(self, monkeypatch): + def boom(*args, **kwargs): + raise subprocess.TimeoutExpired(cmd=args[0], timeout=5) + + monkeypatch.setattr(subprocess, "run", boom) + assert det._run(["docker", "--version"]) is None + + def test_returns_none_on_file_not_found(self, monkeypatch): + def boom(*args, **kwargs): + raise FileNotFoundError("no such binary") + + monkeypatch.setattr(subprocess, "run", boom) + assert det._run(["nonexistent-binary"]) is None + + def test_returns_none_on_nonzero_exit(self, monkeypatch): + monkeypatch.setattr( + subprocess, "run", lambda *a, **kw: _make_completed("", returncode=1) + ) + assert det._run(["docker", "--version"]) is None + + def test_returns_none_on_permission_error(self, monkeypatch): + def boom(*args, **kwargs): + raise PermissionError("denied") + + monkeypatch.setattr(subprocess, "run", boom) + assert det._run(["something"]) is None + + +class TestHttpGetDegradation: + """_http_get() must return None (not raise) on every failure mode.""" + + def test_returns_none_on_connection_error(self, monkeypatch): + import urllib.request + + def boom(url, timeout): + raise OSError("connection refused") + + monkeypatch.setattr(urllib.request, "urlopen", boom) + assert det._http_get("http://localhost:9999/healthz") is None + + def test_returns_none_on_timeout(self, monkeypatch): + import urllib.request + import urllib.error + + def boom(url, timeout): + raise urllib.error.URLError("timed out") + + monkeypatch.setattr(urllib.request, "urlopen", boom) + assert det._http_get("http://localhost:9999/healthz") is None + + +# --------------------------------------------------------------------------- +# detect_environment — no exception escapes, returns correct types +# --------------------------------------------------------------------------- + + +class TestDetectEnvironment: + """detect_environment() must never raise and must return a Detected.""" + + def _all_absent_patches(self, monkeypatch, fake_home: Path, fake_cwd: Path) -> None: + """Patch everything to absent/False so tests are fast and hermetic.""" + monkeypatch.setattr(det, "_run", lambda args: None) + monkeypatch.setattr(det, "_http_get", lambda url: None) + monkeypatch.setattr(Path, "home", staticmethod(lambda: fake_home)) + + def test_returns_detected_instance(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + assert isinstance(result, Detected) + + def test_frozen_dataclass(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + with pytest.raises((AttributeError, TypeError)): + result.docker = True # type: ignore[misc] + + def test_no_exception_when_all_probes_fail(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + # Should not raise + result = detect_environment(cwd=fake_cwd) + assert result.docker is False + assert result.docker_compose is False + assert result.ollama is False + assert result.qdrant is False + assert result.claude_code is False + assert result.codex is False + + def test_ollama_models_is_tuple(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + assert isinstance(result.ollama_models, tuple) + + def test_existing_hooks_is_tuple(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + assert isinstance(result.existing_hooks, tuple) + + def test_python_version_populated(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + parts = result.python_version.split(".") + assert len(parts) == 3 + assert all(p.isdigit() for p in parts) + + def test_os_field_is_string(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + result = detect_environment(cwd=fake_cwd) + assert isinstance(result.os, str) + assert len(result.os) > 0 + + def test_uses_cwd_for_vault_probe(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + vault = fake_cwd / "obsidian-vault" + vault.mkdir() + result = detect_environment(cwd=fake_cwd) + assert result.obsidian_vault == str(vault) + + def test_uses_cwd_for_gitnexus_probe(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + (fake_cwd / ".gitnexus").mkdir() + result = detect_environment(cwd=fake_cwd) + assert result.gitnexus is True + + def test_claude_code_detected_from_home(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + (fake_home / ".claude").mkdir() + result = detect_environment(cwd=fake_cwd) + assert result.claude_code is True + + def test_codex_detected_from_home(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + (fake_home / ".codex").mkdir() + result = detect_environment(cwd=fake_cwd) + assert result.codex is True + + def test_mcp_registered_detected(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + (fake_home / ".claude.json").write_text( + json.dumps({"mcpServers": {"memorymaster": {}}}), encoding="utf-8" + ) + result = detect_environment(cwd=fake_cwd) + assert result.mm_mcp_registered is True + + def test_existing_hooks_detected(self, monkeypatch, fake_home, fake_cwd): + self._all_absent_patches(monkeypatch, fake_home, fake_cwd) + claude_dir = fake_home / ".claude" + claude_dir.mkdir() + settings = { + "hooks": { + "Stop": [ + {"hooks": [{"command": "python memorymaster-stop.py"}]} + ] + } + } + (claude_dir / "settings.json").write_text(json.dumps(settings), encoding="utf-8") + result = detect_environment(cwd=fake_cwd) + assert "Stop" in result.existing_hooks + + def test_no_exception_even_on_subprocess_timeout(self, monkeypatch, fake_home, fake_cwd): + """If subprocess.run raises TimeoutExpired, detect_environment still returns.""" + monkeypatch.setattr(Path, "home", staticmethod(lambda: fake_home)) + monkeypatch.setattr(det, "_http_get", lambda url: None) + + def timeout_run(*args, **kwargs): + raise subprocess.TimeoutExpired(cmd=args[0], timeout=5) + + monkeypatch.setattr(subprocess, "run", timeout_run) + # Should not raise + result = detect_environment(cwd=fake_cwd) + assert isinstance(result, Detected) + + +# --------------------------------------------------------------------------- +# format_plan — structural + content tests +# --------------------------------------------------------------------------- + + +class TestFormatPlan: + def _base_detected(self, **overrides) -> Detected: + defaults = dict( + python_version="3.12.0", + pip_ok=True, + os="Linux", + docker=False, + docker_compose=False, + ollama=False, + ollama_models=(), + qdrant=False, + obsidian_vault=None, + gitnexus=False, + claude_code=False, + codex=False, + mm_installed=False, + mm_mcp_registered=False, + existing_hooks=(), + ) + defaults.update(overrides) + return Detected(**defaults) + + def test_returns_list_of_strings(self): + d = self._base_detected() + result = format_plan(d, want_full_stack=False) + assert isinstance(result, list) + assert all(isinstance(line, str) for line in result) + + def test_nonempty(self): + d = self._base_detected() + result = format_plan(d, want_full_stack=False) + assert len(result) > 0 + + def test_will_do_mm_install_when_not_installed(self): + d = self._base_detected(mm_installed=False) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "[will-do]" in lines + assert "pip install" in lines + + def test_skip_present_when_mm_installed(self): + d = self._base_detected(mm_installed=True) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "[skip-present]" in lines + assert "already installed" in lines + + def test_cant_hooks_when_no_claude_dir(self): + d = self._base_detected(claude_code=False) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "[cant-missing]" in lines + assert "claude" in lines.lower() + + def test_will_do_hooks_when_claude_present_no_hooks(self): + d = self._base_detected(claude_code=True, existing_hooks=()) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "[will-do]" in lines + assert "hooks" in lines + + def test_skip_present_hooks_when_already_registered(self): + d = self._base_detected(claude_code=True, existing_hooks=("UserPromptSubmit", "Stop")) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "[skip-present]" in lines + assert "already registered" in lines + + def test_skip_present_mcp_when_registered(self): + d = self._base_detected(claude_code=True, mm_mcp_registered=True) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "already registered" in lines + + def test_will_do_mcp_when_claude_present_not_registered(self): + d = self._base_detected(claude_code=True, mm_mcp_registered=False) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "register MCP" in lines + + def test_full_stack_skipped_message_when_not_requested(self): + d = self._base_detected() + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "Full-stack" in lines + assert "skipped" in lines.lower() or "not requested" in lines.lower() + + def test_full_stack_docker_will_do_when_compose_present(self): + d = self._base_detected(docker_compose=True, qdrant=False) + lines = "\n".join(format_plan(d, want_full_stack=True)) + assert "docker compose up" in lines + assert "qdrant" in lines.lower() + + def test_full_stack_qdrant_skip_when_already_reachable(self): + d = self._base_detected(qdrant=True) + lines = "\n".join(format_plan(d, want_full_stack=True)) + assert "Qdrant already reachable" in lines + + def test_full_stack_cant_qdrant_when_no_docker(self): + d = self._base_detected(docker_compose=False, qdrant=False) + lines = "\n".join(format_plan(d, want_full_stack=True)) + assert "[cant-missing]" in lines + assert "SQLite-only" in lines + + def test_full_stack_ollama_skip_when_reachable(self): + d = self._base_detected(ollama=True, ollama_models=("llama3.2:3b",)) + lines = "\n".join(format_plan(d, want_full_stack=True)) + assert "Ollama already reachable" in lines + assert "llama3.2:3b" in lines + + def test_full_stack_cant_ollama_when_no_docker(self): + d = self._base_detected(docker_compose=False, ollama=False) + joined = "\n".join(format_plan(d, want_full_stack=True)) + assert "auto-ingest is OFF" in joined or "Ollama" in joined + + def test_obsidian_vault_skip_when_found(self): + d = self._base_detected(obsidian_vault="/tmp/project/obsidian-vault") + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "obsidian-vault" in lines.lower() + assert "[skip-present]" in lines + + def test_obsidian_vault_will_do_when_absent(self): + d = self._base_detected(obsidian_vault=None) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "obsidian-vault" in lines.lower() + + def test_gitnexus_skip_when_present(self): + d = self._base_detected(gitnexus=True) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "GitNexus" in lines + assert "[skip-present]" in lines + + def test_gitnexus_cant_when_absent(self): + d = self._base_detected(gitnexus=False) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "GitNexus" in lines + assert "[cant-missing]" in lines + + def test_codex_present_line_included(self): + d = self._base_detected(codex=True) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "codex" in lines.lower() or "Codex" in lines + + def test_codex_absent_line_included(self): + d = self._base_detected(codex=False) + lines = "\n".join(format_plan(d, want_full_stack=False)) + assert "codex" in lines.lower() or "Codex" in lines diff --git a/tests/test_setup_hooks.py b/tests/test_setup_hooks.py new file mode 100644 index 0000000..cd10a36 --- /dev/null +++ b/tests/test_setup_hooks.py @@ -0,0 +1,505 @@ +"""Hermetic tests for memorymaster.surfaces.setup_hooks. + +SAFETY (mandatory): every test patches the module-level HOME / CLAUDE_DIR / +CLAUDE_JSON / CODEX_DIR to ``tmp_path`` and mocks ``subprocess.run`` so the +REAL ``~/.claude`` / ``~/.codex`` is NEVER touched and the REAL installer is +NEVER executed against this machine. detect_environment is also stubbed so no +real Docker/Ollama/Qdrant probes run. + +Coverage (spec §5): +- non-interactive ``--yes`` wires hooks/MCP into a tmp HOME +- re-run is idempotent (no duplicate hook entries; settings.json valid JSON) +- MCP registration uses the non-deprecated command +- no-Docker fallback: setup succeeds, degraded message emitted, exit 0 +- verify_install round-trips a sentinel claim on a tmp DB (PASS) +- ``--json`` emits valid parseable JSON +""" +from __future__ import annotations + +import json +import subprocess +from pathlib import Path + +import pytest + +import memorymaster.surfaces.setup_hooks as sh +from memorymaster.surfaces.setup_detect import Detected + + +# --------------------------------------------------------------------------- +# Helpers / fixtures +# --------------------------------------------------------------------------- + + +def _detected(**overrides) -> Detected: + defaults = dict( + python_version="3.12.0", + pip_ok=True, + os="Linux", + docker=False, + docker_compose=False, + ollama=False, + ollama_models=(), + qdrant=False, + obsidian_vault=None, + gitnexus=False, + claude_code=True, + codex=False, + mm_installed=True, + mm_mcp_registered=False, + existing_hooks=(), + ) + defaults.update(overrides) + return Detected(**defaults) + + +@pytest.fixture() +def hermetic_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> dict[str, Path]: + """Redirect every filesystem target into tmp_path. The real HOME is safe.""" + home = tmp_path / "home" + claude_dir = home / ".claude" + codex_dir = home / ".codex" + claude_json = home / ".claude.json" + claude_dir.mkdir(parents=True, exist_ok=True) + + monkeypatch.setattr(sh, "HOME", home) + monkeypatch.setattr(sh, "CLAUDE_DIR", claude_dir) + monkeypatch.setattr(sh, "CLAUDE_JSON", claude_json) + monkeypatch.setattr(sh, "CODEX_DIR", codex_dir) + + project = tmp_path / "project" + project.mkdir(parents=True, exist_ok=True) + monkeypatch.setattr(sh, "PROJECT_ROOT", project) + + return { + "home": home, + "claude_dir": claude_dir, + "codex_dir": codex_dir, + "claude_json": claude_json, + "project": project, + } + + +@pytest.fixture() +def no_real_subprocess(monkeypatch: pytest.MonkeyPatch) -> list[list[str]]: + """Record subprocess.run calls and return a benign success — never spawn.""" + calls: list[list[str]] = [] + + def fake_run(args, *a, **kw): + calls.append(list(args) if isinstance(args, (list, tuple)) else [args]) + m = subprocess.CompletedProcess(args=args, returncode=0, stdout="", stderr="") + return m + + monkeypatch.setattr(sh.subprocess, "run", fake_run) + return calls + + +def _stub_detect(monkeypatch: pytest.MonkeyPatch, detected: Detected) -> None: + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: detected) + + +# --------------------------------------------------------------------------- +# ask / ask_yn non-interactive honoring +# --------------------------------------------------------------------------- + + +class TestNonInteractive: + def test_ask_returns_default_when_non_interactive(self, monkeypatch): + sh.set_non_interactive(True) + try: + # input() must NOT be called — if it is, this raises. + monkeypatch.setattr("builtins.input", lambda *_: (_ for _ in ()).throw(AssertionError("input called"))) + assert sh.ask("anything", "the-default") == "the-default" + assert sh.ask_yn("ok?", True) is True + assert sh.ask_yn("ok?", False) is False + finally: + sh.set_non_interactive(False) + + def test_ask_uses_input_when_interactive(self, monkeypatch): + sh.set_non_interactive(False) + monkeypatch.setattr("builtins.input", lambda *_: "typed-value") + assert sh.ask("q", "def") == "typed-value" + + +# --------------------------------------------------------------------------- +# MCP registration — non-deprecated command + brownfield idempotency +# --------------------------------------------------------------------------- + + +class TestInstallMcp: + def test_uses_non_deprecated_command(self, hermetic_home): + sh.install_mcp(force=True) + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + entry = data["mcpServers"]["memorymaster"] + assert entry["args"] == ["-m", "memorymaster.surfaces.mcp_server"] + # Must NOT be the deprecated path. + assert "memorymaster.mcp_server" not in json.dumps(entry["args"]) + + def test_skips_existing_without_force(self, hermetic_home): + sh.set_non_interactive(True) # ask_yn returns its default (False here) + try: + hermetic_home["claude_json"].write_text( + json.dumps({"mcpServers": {"memorymaster": {"sentinel": "keep-me"}}}), + encoding="utf-8", + ) + sh.install_mcp(force=False) + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + # Brownfield: untouched. + assert data["mcpServers"]["memorymaster"] == {"sentinel": "keep-me"} + finally: + sh.set_non_interactive(False) + + def test_force_overwrites(self, hermetic_home): + hermetic_home["claude_json"].write_text( + json.dumps({"mcpServers": {"memorymaster": {"old": True}}, "keepKey": 1}), + encoding="utf-8", + ) + sh.install_mcp(force=True) + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + assert data["mcpServers"]["memorymaster"]["args"] == ["-m", "memorymaster.surfaces.mcp_server"] + # Unknown keys preserved. + assert data["keepKey"] == 1 + + def test_preserves_other_servers(self, hermetic_home): + hermetic_home["claude_json"].write_text( + json.dumps({"mcpServers": {"other": {"x": 1}}}), encoding="utf-8" + ) + sh.install_mcp(force=True) + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + assert data["mcpServers"]["other"] == {"x": 1} + assert "memorymaster" in data["mcpServers"] + + +class TestInstallMcpCodex: + def test_writes_managed_block_with_correct_command(self, hermetic_home): + hermetic_home["codex_dir"].mkdir(parents=True, exist_ok=True) + sh.install_mcp_codex(force=True) + content = (hermetic_home["codex_dir"] / "config.toml").read_text(encoding="utf-8") + assert "[mcp_servers.memorymaster]" in content + assert "memorymaster.surfaces.mcp_server" in content + assert sh._CODEX_MCP_BEGIN in content and sh._CODEX_MCP_END in content + + def test_idempotent_block_no_duplication(self, hermetic_home): + hermetic_home["codex_dir"].mkdir(parents=True, exist_ok=True) + sh.install_mcp_codex(force=True) + sh.install_mcp_codex(force=True) + content = (hermetic_home["codex_dir"] / "config.toml").read_text(encoding="utf-8") + assert content.count("[mcp_servers.memorymaster]") == 1 + + def test_preserves_unmanaged_toml(self, hermetic_home): + hermetic_home["codex_dir"].mkdir(parents=True, exist_ok=True) + cfg = hermetic_home["codex_dir"] / "config.toml" + cfg.write_text('model = "gpt-5"\n', encoding="utf-8") + sh.install_mcp_codex(force=True) + content = cfg.read_text(encoding="utf-8") + assert 'model = "gpt-5"' in content + assert "[mcp_servers.memorymaster]" in content + + def test_noop_when_no_codex_dir(self, hermetic_home): + # codex_dir does not exist → no file created + sh.install_mcp_codex(force=True) + assert not (hermetic_home["codex_dir"] / "config.toml").exists() + + +# --------------------------------------------------------------------------- +# install_hooks idempotency +# --------------------------------------------------------------------------- + + +class TestInstallHooksIdempotent: + def test_no_duplicate_hooks_on_rerun(self, hermetic_home, monkeypatch): + # config_templates/hooks must exist as package resource; stub the copy + # loop is unnecessary — install_hooks reads real templates and writes + # into the tmp CLAUDE_DIR. We only need settings.json behavior. + llm = {"provider": "ollama", "api_key": "", "model": "llama3.2:3b"} + sh.install_hooks(llm) + sh.install_hooks(llm) + settings = json.loads( + (hermetic_home["claude_dir"] / "settings.json").read_text(encoding="utf-8") + ) + ups = settings["hooks"]["UserPromptSubmit"] + mm_entries = [h for h in ups if "memorymaster" in json.dumps(h)] + # recall + classify = exactly 2, not 4. + assert len(mm_entries) == 2 + + def test_settings_json_stays_valid(self, hermetic_home): + llm = {"provider": "ollama", "api_key": "", "model": "llama3.2:3b"} + sh.install_hooks(llm) + # Must parse without error. + json.loads((hermetic_home["claude_dir"] / "settings.json").read_text(encoding="utf-8")) + + +# --------------------------------------------------------------------------- +# Full-stack: no-Docker fallback +# --------------------------------------------------------------------------- + + +class TestSetupFullStack: + def test_no_docker_fallback_degraded(self, hermetic_home, no_real_subprocess): + det = _detected(docker_compose=False, qdrant=False, ollama=False) + result = sh.setup_full_stack(det, interactive=False, yes=True) + assert result["degraded"] is True + assert "SQLite-only" in result["message"] + # No docker compose invoked. + assert not any("compose" in c for call in no_real_subprocess for c in call) + + def test_reuses_already_healthy(self, hermetic_home, no_real_subprocess): + det = _detected(qdrant=True, ollama=True) + result = sh.setup_full_stack(det, interactive=False, yes=True) + assert result["degraded"] is False + assert result["qdrant"] == "reused" + assert result["ollama"] == "reused" + + def test_compose_up_when_available(self, hermetic_home, no_real_subprocess): + det = _detected(docker_compose=True, qdrant=False, ollama=False) + result = sh.setup_full_stack(det, interactive=False, yes=True, model="llama3.2:3b") + assert result["compose_run"] is True + assert result["degraded"] is False + assert any("compose" in c for call in no_real_subprocess for c in call) + + def test_compose_failure_degrades_not_raises(self, hermetic_home, monkeypatch): + def boom(args, *a, **kw): + return subprocess.CompletedProcess(args=args, returncode=1, stdout="", stderr="nope") + + monkeypatch.setattr(sh.subprocess, "run", boom) + det = _detected(docker_compose=True, qdrant=False, ollama=False) + result = sh.setup_full_stack(det, interactive=False, yes=True) + assert result["degraded"] is True + + +# --------------------------------------------------------------------------- +# verify_install — round-trip on a tmp DB +# --------------------------------------------------------------------------- + + +class TestVerifyInstall: + def test_round_trip_pass(self, tmp_path, hermetic_home, monkeypatch): + # MCP not registered → no mcp_note required; keep detect stub absent. + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected(mm_mcp_registered=False)) + db = tmp_path / "verify.db" + result = sh.verify_install(db) + assert result["status"] == "PASS", result + assert "sentinel" in result["detail"] + + def test_mcp_note_when_registered(self, tmp_path, hermetic_home, monkeypatch): + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected(mm_mcp_registered=True)) + db = tmp_path / "verify2.db" + result = sh.verify_install(db) + assert result["status"] == "PASS" + assert "restart" in result["mcp_note"].lower() + + +# --------------------------------------------------------------------------- +# main() end-to-end (non-interactive) — wiring + idempotency + JSON + exit 0 +# --------------------------------------------------------------------------- + + +def _run_main(monkeypatch, hermetic_home, det, argv): + """Run main() fully hermetic: detect stubbed, subprocess mocked.""" + _stub_detect(monkeypatch, det) + + def fake_run(args, *a, **kw): + return subprocess.CompletedProcess(args=args, returncode=0, stdout="", stderr="") + + monkeypatch.setattr(sh.subprocess, "run", fake_run) + # main resets PROJECT_ROOT from --project-root; point it at the tmp project. + return sh.main(argv) + + +class TestMainNonInteractive: + def _argv(self, hermetic_home, extra=None): + argv = [ + "--yes", + "--provider", + "ollama", + "--project-root", + str(hermetic_home["project"]), + "--db", + str(hermetic_home["project"] / "memorymaster.db"), + "--no-cron", + "--no-obsidian-skills", + "--no-full-stack", + ] + if extra: + argv += extra + return argv + + def test_exit_zero_and_wires_hooks_mcp(self, monkeypatch, hermetic_home, capsys): + det = _detected(claude_code=True, mm_mcp_registered=False) + rc = _run_main(monkeypatch, hermetic_home, det, self._argv(hermetic_home)) + assert rc == 0 + # settings.json written into tmp HOME, valid JSON. + settings = json.loads( + (hermetic_home["claude_dir"] / "settings.json").read_text(encoding="utf-8") + ) + assert "UserPromptSubmit" in settings["hooks"] + # MCP registered with correct command. + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + assert data["mcpServers"]["memorymaster"]["args"] == ["-m", "memorymaster.surfaces.mcp_server"] + + def test_rerun_idempotent(self, monkeypatch, hermetic_home): + det1 = _detected(claude_code=True, mm_mcp_registered=False) + _run_main(monkeypatch, hermetic_home, det1, self._argv(hermetic_home)) + first = (hermetic_home["claude_dir"] / "settings.json").read_text(encoding="utf-8") + + # Second run: detection now reports hooks + MCP already present. + det2 = _detected( + claude_code=True, + mm_mcp_registered=True, + existing_hooks=("UserPromptSubmit", "Stop"), + ) + _run_main(monkeypatch, hermetic_home, det2, self._argv(hermetic_home)) + second = (hermetic_home["claude_dir"] / "settings.json").read_text(encoding="utf-8") + + s2 = json.loads(second) + # No duplicate memorymaster hooks accumulated. + ups = s2["hooks"]["UserPromptSubmit"] + assert len([h for h in ups if "memorymaster" in json.dumps(h)]) == 2 + # MCP entry not clobbered (idempotent) — still the correct command. + data = json.loads(hermetic_home["claude_json"].read_text(encoding="utf-8")) + assert data["mcpServers"]["memorymaster"]["args"] == ["-m", "memorymaster.surfaces.mcp_server"] + assert json.loads(first) and json.loads(second) # both valid JSON + + def test_no_docker_fallback_exits_zero(self, monkeypatch, hermetic_home, capsys): + det = _detected(docker_compose=False, qdrant=False, ollama=False) + argv = [ + "--yes", + "--provider", + "ollama", + "--project-root", + str(hermetic_home["project"]), + "--db", + str(hermetic_home["project"] / "memorymaster.db"), + "--no-cron", + "--no-obsidian-skills", + "--full-stack", + "--json", + ] + rc = _run_main(monkeypatch, hermetic_home, det, argv) + assert rc == 0 + out = capsys.readouterr().out + # JSON must be parseable and report degraded. + payload = json.loads(out) + assert payload["degraded"] is True + + def test_json_output_parses(self, monkeypatch, hermetic_home, capsys): + det = _detected(claude_code=True) + rc = _run_main( + monkeypatch, hermetic_home, det, self._argv(hermetic_home, extra=["--json"]) + ) + assert rc == 0 + payload = json.loads(capsys.readouterr().out) + assert set(["detected", "planned", "applied", "verify", "degraded"]).issubset(payload) + assert payload["verify"]["status"] in ("PASS", "PARTIAL") + + +class TestVerifyOnly: + def test_verify_only_short_circuits(self, monkeypatch, hermetic_home, capsys): + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected(mm_mcp_registered=False)) + db = hermetic_home["project"] / "vo.db" + rc = sh.main( + [ + "--verify-only", + "--project-root", + str(hermetic_home["project"]), + "--db", + str(db), + "--json", + ] + ) + assert rc == 0 + payload = json.loads(capsys.readouterr().out) + assert payload["verify"]["status"] == "PASS" + + +class TestMalformedConfigBackup: + """A malformed pre-existing config must be BACKED UP, never silently wiped. + + WHY: ~/.claude/settings.json and ~/.claude.json are hand-edited daily-driver + configs. The pre-fix code reset malformed files to {} and overwrote them, + losing user data. These tests fail if that data-loss regression returns. + """ + + def test_malformed_settings_json_is_backed_up(self, hermetic_home, no_real_subprocess): + settings = hermetic_home["claude_dir"] / "settings.json" + settings.write_text('{ this is : not valid json,,, ', encoding="utf-8") + original = settings.read_text(encoding="utf-8") + + sh.install_hooks({"provider": "ollama", "api_key": "", "model": "llama3.2:3b"}) + + # settings.json is now valid JSON with our hooks wired in... + rewritten = json.loads(settings.read_text(encoding="utf-8")) + assert "memorymaster" in json.dumps(rewritten["hooks"]) + # ...and the original malformed content was preserved in a .corrupt-*.bak + backups = list(hermetic_home["claude_dir"].glob("settings.json.corrupt-*.bak")) + assert len(backups) == 1 + assert backups[0].read_text(encoding="utf-8") == original + + def test_malformed_claude_json_is_backed_up(self, hermetic_home, no_real_subprocess): + sh.set_non_interactive(True) + try: + cj = hermetic_home["claude_json"] + cj.write_text('{"mcpServers": BROKEN', encoding="utf-8") + original = cj.read_text(encoding="utf-8") + + sh.install_mcp(force=False) + + data = json.loads(cj.read_text(encoding="utf-8")) + assert "memorymaster" in data["mcpServers"] + backups = list(hermetic_home["home"].glob(".claude.json.corrupt-*.bak")) + assert len(backups) == 1 + assert backups[0].read_text(encoding="utf-8") == original + finally: + sh.set_non_interactive(False) + + +class TestFromZeroRegressions: + """Regressions found by a real from-zero container install (not catchable by + the stub-detect/populated-HOME unit harness). Each fails if its bug returns. + """ + + def test_no_claude_code_completes_without_crash(self, hermetic_home, no_real_subprocess, monkeypatch, capsys): + """~/.claude absent + claude_code=False: setup must finish (exit 0), not + crash in append_instructions writing into a non-existent ~/.claude.""" + import shutil + shutil.rmtree(hermetic_home["claude_dir"], ignore_errors=True) + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected(claude_code=False)) + db = hermetic_home["project"] / "fz.db" + rc = sh.main( + ["--yes", "--no-full-stack", "--no-cron", "--no-obsidian-skills", + "--project-root", str(hermetic_home["project"]), "--db", str(db), "--json"] + ) + assert rc == 0 + payload = json.loads(capsys.readouterr().out) + assert payload["applied"]["hooks"].startswith("skipped") + assert payload["applied"]["mcp_claude"].startswith("skipped") + assert not (hermetic_home["claude_dir"] / "CLAUDE.md").exists() # never created a fake ~/.claude + + def test_db_init_passes_db_before_subcommand(self, hermetic_home, monkeypatch): + """init-db is invoked as `--db init-db` (global arg first), not + `init-db --db ` which argparse rejects with exit 2.""" + calls: list[list[str]] = [] + + def fake_run(args, *a, **kw): + calls.append(list(args) if isinstance(args, (list, tuple)) else [args]) + return subprocess.CompletedProcess(args=args, returncode=0, stdout="", stderr="") + + monkeypatch.setattr(sh.subprocess, "run", fake_run) + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected(claude_code=False)) + db = hermetic_home["project"] / "order.db" # must NOT pre-exist + sh.main(["--yes", "--no-full-stack", "--no-cron", "--no-obsidian-skills", + "--project-root", str(hermetic_home["project"]), "--db", str(db), "--json"]) + init_calls = [c for c in calls if "init-db" in c] + assert init_calls, "init-db subprocess was never invoked" + c = init_calls[0] + assert c.index("--db") < c.index("init-db"), f"--db must precede init-db: {c}" + + def test_verify_only_is_non_interactive_without_yes(self, hermetic_home, monkeypatch): + """--verify-only (no --yes) must never call input() — stdin may be a + non-tty (CI/docker/agent), where input() raises EOFError.""" + sh.set_non_interactive(False) + monkeypatch.setattr("builtins.input", lambda *a, **k: (_ for _ in ()).throw(AssertionError("input() called"))) + monkeypatch.setattr(sh, "detect_environment", lambda *a, **kw: _detected()) + db = hermetic_home["project"] / "vo2.db" + rc = sh.main(["--verify-only", "--db", str(db), "--json"]) + assert rc == 0