second-brain
Health Uyari
- License — License: MIT
- Description — Repository has a description
- Active repo — Last push 0 days ago
- Low visibility — Only 7 GitHub stars
Code Gecti
- Code scan — Scanned 12 files during light audit, no dangerous patterns found
Permissions Gecti
- Permissions — No dangerous permissions requested
Bu listing icin henuz AI raporu yok.
Local-first RAG assistant: answers only from your own notes/docs/code, every claim cited, graded by an eval harness. OpenAI-compatible gateway (free local by default), exposed over MCP.
second-brain
A personal RAG assistant: point it at your notes, docs, and code, then ask questions and
get answers cited to your own sources. Runs on the home AI lab — free local models by
default, one env-flip to Claude quality.
uv sync
cp .env.example .env
artjeck # start talking to Artjeck directly
sb ingest ~/Notes # or any file/folder (md, txt, pdf, code…)
sb ask "what did I decide about the gateway routing?"
sb learn "My preferred invoice format is PDF with line items."
sb agent # ask questions, teach it with /learn <fact>
sb chat # interactive
sb overnight # safe nightly scan: ingest changed files + write a morning report
sb morning # daily briefing from overnight runs, tasks, and cited RAG
sb status
sb gc --dry-run # show chunks whose source file is gone; drop the flag to remove them
sb keyword-reindex --corpus all # one-off backfill for collections ingested before the keyword index
sb eval # retrieval benchmark (add --answers for chat-model checks)
SB_COLLECTION=second_brain_regression sb eval evals/regression.json --ingest-corpus
sb eval --answers --trace-output data/eval-traces.json
sb eval-intake # build a private 100-question benchmark in the terminal
Why it's built this way (the interesting decisions)
- OpenAI-compatible everywhere.
llm.pytalks to one OpenAI-style endpoint. Default is
a local Ollama directly (free, no key, both models warm). Switch to.env.gatewayand the
same code routes through the LiteLLM gateway — free local embeddings + Claude for the
answer, with a budget cap. Develop for free, ship with quality, zero code change. - Citations are mandatory. The system prompt forces the model to answer only from
retrieved context and cite[n]; the CLI prints the source files + distances. No source,
no claim — that's the trust signal that separates a real RAG product from a demo. - Hybrid retrieval rescues exact lookups. Vector search finds semantic matches; a
persisted SQLite FTS5 index rescues the exact section and list lookups embeddings miss.
The two ranked lists are fused by reciprocal rank rather than by score, because a cosine
distance and a BM25-derived one share no scale and pretending otherwise is how a fusion
quietly becomes a coin flip. Keyword hits expand to adjacent chunks, the result is capped
atk, and every answer still has to be cited.SB_HYBRIDtoggles it for A/B checks. - Learning is explicit and inspectable.
sb learnand/learnin agent mode write
Markdown memory files underSB_MEMORY_DIR, then ingest them through the same cited RAG
pipeline. The model does not silently save its own guesses as truth. - Agent workflow is explicit. Artjeck routes each console turn through a LangGraph
workflow that can answer, learn, ingest files, and manage tasks. - Tasks are structured.
SB_STATE_DBis a SQLite database for durable tasks and
assistant state. It is separate from semantic memory. - Lean, not framework-soup. Small codebase, the OpenAI SDK + Chroma/Qdrant, no LangChain. You can
read every step of the pipeline (chunk → embed → store → retrieve → ground → cite), which
is the point — it shows you understand RAG, not just how to import it. - Local-first, lab-ready vector store. Chroma on disk is the default zero-infra path.
SetSB_STORE=qdrantto use the AI Lab's hosted Qdrant service without changing CLI code.
Architecture
files ──▶ ingest.py ──chunk──▶ llm.embed ──▶ Store (Chroma or Qdrant)
│
learn ──▶ memory.py ──markdown──▶ ingest.py ──────┤
│
question ──▶ LangGraph ──ask──▶ Store.query ──top-k──▶ llm.answer ──▶ cited answer
│
├─learn──▶ memory.py
├─ingest─▶ ingest.py
└─task───▶ SQLite (`SB_STATE_DB`)
Roadmap (each item maps to a JD checkbox)
- Swap Chroma → lab Qdrant (hosted vector DB)
- Explicit learned memory (
sb learn,/learnin agent mode) - LangGraph turn workflow + SQLite task store
- Hybrid search (vector + BM25 keyword) — keyword rescue for exact section/list lookups, off via
SB_HYBRID=0 - Wrap as an MCP server so Claude Desktop / Claude Code can query your brain directly (docs)
- Eval harness: benchmarks, rubric checks, local traces/spans, and trajectories (docs)
- [~] Web UI (Next.js) — anonymous cited-Q&A demo live; owner mode + sandbox in progress
MCP server
Expose the brain to any MCP client (Claude Code, Claude Desktop, a native mobile or desktop
client) so an agent can query and teach it directly. It reuses this engine, so it inherits
the free-local-model gateway defaults — no extra cost. Full guide: docs/MCP.md.
uv run sb-mcp # stdio (local clients launch it as a subprocess)
SB_MCP_TRANSPORT=http \
SB_MCP_HOST=0.0.0.0 SB_MCP_PORT=8848 \
SB_MCP_TOKEN=$(openssl rand -hex 24) uv run sb-mcp # Streamable HTTP, token-protected
Tools: ask (cited answer), recall (raw chunks, no model call), ingest, learn,list_tasks, add_task, complete_task, status. Claude Code auto-loads the stdio
server from the repo's .mcp.json — just run claude here (works over SSH too).
Inspect interactively with npx @modelcontextprotocol/inspector.
Config
.env (see .env.example / .env.gateway.example): OPENAI_BASE_URL, OPENAI_API_KEY,EMBED_MODEL, CHAT_MODEL, SB_STORE, SB_COLLECTION, SB_MEMORY_DIR, SB_STATE_DB,
and retrieval knobs SB_CHUNK_SIZE / SB_CHUNK_OVERLAP / SB_TOP_K / SB_HYBRID.
Chroma default
SB_STORE=chroma
SB_DATA=./data/chroma
SB_MEMORY_DIR=./data/memory
SB_STATE_DB=./data/artjeck.sqlite3
Qdrant
SB_STORE=qdrant
QDRANT_URL=http://127.0.0.1:6333
QDRANT_API_KEY=<your Qdrant API key>
After switching stores, ingest again because Chroma and Qdrant keep separate collections.
Storage Policy
Split by cost, so the machine you sit at stays light:
- The host you run the CLI on keeps the code and the small SQLite state DB.
- A larger volume keeps learned Markdown memories, big source files, exports, and backups.
- Qdrant holds the heavy vector index once
QDRANT_API_KEYis configured — on a GPU host
if you have one, since that is usually where the embedding model already lives.
An .env shape that follows it (adjust the paths to your own volume):
SB_MEMORY_DIR=/path/to/volume/memory
SB_STATE_DB=./data/artjeck.sqlite3
QDRANT_URL=http://127.0.0.1:6333
Put large documents under that volume's inbox, then ingest from there:
artjeck
/ingest /path/to/volume/inbox
Learning
Teach a fact directly:
sb learn "ArtJack prefers short implementation summaries with exact file paths."
sb ask "How does ArtJack prefer implementation summaries?"
Use agent mode:
artjeck
/learn The AI Lab gateway is the preferred front door for shared model routing.
What is the preferred front door for shared model routing?
/ingest ~/Notes
/task Review new lease documents
/tasks
/done 1
/exit
Learned memories are normal Markdown files in SB_MEMORY_DIR, so you can inspect, edit,
delete, or re-ingest them. This keeps learning controlled: the system learns from what you
teach it, not from unverified model guesses.
Overnight worker
The overnight worker is the safe "make him smarter while I sleep" loop. It scans configured
folders, ingests new or changed supported files, extracts likely tasks into a report, and
writes an audit trail under data/overnight/. It does not edit, move, delete, send, or
buy anything.
First run it manually:
uv run sb overnight --dry-run
uv run sb overnight
The first run creates data/overnight/config.json. Edit targets there to point at your
real inboxes or synced network folders. Reports land in data/overnight/reports/.
Routing a target to a collection
A target is normally a string, and everything found under it is ingested into the default
collection:
"targets": ["~/Projects/second-brain", "~/Projects/engineering-docs"]
A target may instead be an object naming the collection it belongs to:
"targets": [
"~/Projects/second-brain",
{ "path": "/Volumes/DISK/AI/artjeck/reference", "collection": "second_brain_reference" }
]
Both forms can appear in the same list, so an existing config needs no changes. The report
names the collection for a file only when it is not the default.
This exists because the reference corpus — syllabi, standards, study material you keep but did
not write — was split out of the personal collection so a question about your own work could
not be answered out of someone else's textbook. The scan did not know about the split and
ingested everything into the default collection, so the migration had to be protected by
excluding istqb outright. That bought safety at the cost of the reference corpus having no
automatic refresh at all. Naming the collection on the target is the version of that which
keeps working.
To schedule it daily (macOS, launchd):
chmod +x deploy/install-overnight-service.sh
SB_OVERNIGHT_HOUR=3 SB_OVERNIGHT_MINUTE=15 deploy/install-overnight-service.sh
Launchd runs deploy/run-nightly.sh, which refreshes project-context notes, performssb overnight, syncs tasks, runs health checks, and then writes sb morning. Logs land
in data/overnight/logs/. Keep the first few runs read-only and review the reports before
giving Artjeck any action permissions beyond ingesting and reporting.
Morning briefing
After the overnight worker runs, generate a daily briefing:
uv run sb morning
It writes a Markdown briefing under data/morning/ with the latest overnight run, open
tasks, possible follow-ups found in recent reports, and a project inventory from configured
scan targets. To also ask the indexed brain with model calls:
uv run sb morning --rag
Refresh source-backed project notes any time:
uv run sb project-context --ingest
Sync explicit follow-ups into the durable task store:
uv run sb task-sync
uv run sb task-sync --dry-run
Run read-only health checks for lab services and projects:
uv run sb health
Fetch a website into the brain as a cited source:
uv run sb web-check https://example.com
uv run sb web-check https://example.com --no-ingest
For JavaScript-rendered pages, use a real browser capture:
uv run sb browser-check https://example.com
uv run sb browser-check https://example.com --no-screenshot --no-ingest
sb agent starts the same conversation loop. The artjeck command is just the named
shortcut intended for daily use.
Yorumlar (0)
Yorum birakmak icin giris yap.
Yorum birakSonuc bulunamadi