stateprobe

# StateProbe

The attention layer for LLM agents.

Your agent already drifted — wrong focus, stale context, confidently editing files you never asked about. StateProbe catches it before the agent ships the answer. Works with Claude Code, Cursor, Cline, Continue, and any MCP host.

For closed-source agents, this is fast task-level attention inferred from text. Open-weight models unlock the optional Lab / future Runtime Probe path for activations and vectors.

English | 简体中文

Why

LLM agents drift. They miss the user's actual point, get steered by stale context, or burn cycles on the wrong subtopic. Today's fix is "rewrite the prompt and pray." StateProbe gives you a sharper tool:

• See what the agent is about to focus on, before it answers
• Decide to continue, rewrite the focus, ask a boundary question, or cut stale context
• Audit the actual output afterwards and surface drift

Runs locally. Costs zero LLM tokens by default. Plugs into any MCP host.

Install

pip install stateprobe

For MCP integration (Claude Code, Cursor, Cline, Continue):

pip install "stateprobe[mcp]"

For activation projection on open-source DeepSeek (optional):

pip install "stateprobe[lab]"

30-second demo

Copy-paste this after pip install stateprobe to catch a bad plan before it ships:

stateprobe skill preview \
  --context-text "Focus on safety; do not include deprecated APIs." \
  --plan-text "I will list deprecated APIs and explain why they are unsafe."

After the agent answers, audit alignment with user requirements:

stateprobe skill overlay \
  --context-text "Focus on safety; do not include deprecated APIs." \
  --output-text "The answer recommends a deprecated API first."

Or run the legacy prompt diagnostic, with the included
smart_but_not_answering demo:

stateprobe demo

What it gives back

stateprobe skill preview returns a JSON activation_decision — your agent host branches on it:

Every decision now ships with confidence (low / medium / high) and evidence — the concrete user requirements and gaps the decision was built on. Destructive hard stops (rewrite_planned_focus, cut_context_contamination) only fire on high confidence. ask_boundary_question can fire at medium because the cost is one user yes/no, not a rewrite. Everything weaker downgrades to continue_with_warning so StateProbe never destroys a workflow on a hunch. This is the contract that keeps the layer usable instead of a noisy referee.

stateprobe skill overlay returns an interrupt_level (ok / watch / interrupt) plus attention_gaps and control_levers for the next turn.

Full schemas: Skill spec, MCP server.

What StateProbe is / is not

StateProbe is a preflight attention HUD for agent workflows. It exposes — as structured control signals — what the agent is about to focus on, what it is about to ignore, and whether it is still attached to stale context, before the agent takes expensive actions (calls a tool, writes code, sends an email, renders an image).

StateProbe is not:

• Not an oracle. Rule-based judges have false positives and false negatives. That is why every verdict ships with confidence and evidence, and only high confidence ever stops an agent.
• Not a replacement for human or LLM review. Review agents look at finished output. StateProbe looks at planned focus. They are complementary, not substitutes.
• Not a semantic correctness checker. It does not know whether your code is right, whether your essay is true, or whether your design is good. It checks attention alignment, not domain truth.
• Not a "spin up another agent to judge this one" wrapper. The whole point is that the default path is local, deterministic, zero-API-cost, and emits a structured activation_decision your host can branch on — not a paragraph of LLM critique.

The honest one-line pitch: low-cost, explainable, host-integratable attention preflight; not a referee, not a benchmark, not a guarantee.

Known failure modes

StateProbe will misjudge in these cases. The full list is in tests/fixtures/skill_cases.jsonl (currently 19 documented known_issue cases out of 51 total). The recurring families:

• Paraphrase / antonym blindness. must_not 提缺点 violated by 列举需要改进的地方 — no lexical overlap, no concept lexicon, false negative.
• Avoidance-phrase false positive. Plan 避免使用营销话术 literally contains the must_not keyword, so the matcher fires even though the intent is the opposite.
• Implicit-realisation false positive. A drawing plan that captures 放松 via 闭眼微笑、背景柔和 is hard-stopped because the abstract concept word is not literally present.
• Modifier loss. Plan covers the head must (解释 RAG) but loses a softer modifier (面向新手). Algorithm hard-stops; a human would warn.
• Plan-substance gaps. A plan that is just 好的 or just a clarifying question soft-warns instead of hard-stopping.

The fix path for each one is named in the fixture's notes field, not handwaved. None of them are blockers for the preflight contract; they are exactly the kinds of cases users will surface and grow the calibration suite around.

Two product lines

Boundary: the Skill HUD never claims neural interpretability; it makes task-level attention visible and steerable from text. Closed-source APIs (OpenAI, Claude) cannot expose hidden states — OpenAI/Claude 物理上读不到 hidden states — so they run the Skill layer only. Open-source models (DeepSeek, Qwen, Llama) unlock the Lab path today and the future Runtime Probe line later.

How it differs

Complementary, not competitive. promptfoo / Guardrails check what came out; StateProbe shapes what's about to come out.

Architecture

Hybrid evidence pipeline (ADR_009): independent contributors emit confidence-weighted evidence, aggregated into 8 behavior axes. Static rules are always on (zero cost); the LLM and Lab layers are opt-in and stack on top.

Theoretical foundation:

• Anthropic — Persona Vectors: Monitoring and Controlling Character Traits in Language Models
• DeepSeek-AI — DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

DeepSeek-first, not DeepSeek-only — see DeepSeek roadmap.

Roadmap

• v0.3 — Skill HUD, MCP server, Lab activation projection on 4 axes
• v0.3.1 — Windows CLI encoding + launch demo polish
• v0.4 (current) — Evidence-driven activation_decision: every verdict ships confidence + evidence, only high confidence hard-stops, plus a 51-case hand-labelled calibration suite (32 agree / 19 transparently documented known issues) and the contamination / must_not / modality-gate fixes that go with it
• v0.4.x — Close the documented known issues: antonym / paraphrase lexicon (ISSUE-005/006/020), modifier-aware coverage (ISSUE-001/009), plan-substance detection (ISSUE-012/018), more pivot markers (ISSUE-007); host feedback channel once user volume justifies it
• v0.5 — MoE expert routing contributor on DeepSeek-MoE; named steering vectors; output-time intervention API

See CHANGELOG for the full version history.

Documentation

• Skill spec — attention HUD reference
• MCP server — Claude Code / Cursor / Cline / Continue setup
• Architecture — hybrid evidence pipeline
• FAQ — common objections answered

• Evidence model — three-layer evidence boundaries
• DeepSeek roadmap — DeepSeek-first, not DeepSeek-only
• Architecture decisions — ADRs for hybrid pipeline and lab contributor
• Publishing — release process
• CHANGELOG / CITATION / CODE_OF_CONDUCT / CONTRIBUTING

中文文档（含 China 镜像、PowerShell 编码 fix、完整命令样例）：README.zh-CN.md

Contributing

Rule library quality = the project's core value. If you find a prompt pattern that isn't detected, a misfire, or want a new target preset — open an issue or PR.

Each rule contribution must include: pattern / affected axis / direction / weight / mechanism / paper citation. See CONTRIBUTING.md.

python scripts/acceptance_check.py

License

MIT — see LICENSE.

Built on Anthropic interpretability and DeepSeek-AI open research. This tool turns those findings into something agent hosts and prompt engineers can use every day, without having to actually answer the question of how the model "thinks" — just what it's about to focus on next.