Context Runtime

An efficiency optimizer for a fleet of apps — a database query planner for LLM
context. The application says "I need an answer"; the runtime decides what the model
sees — what to retrieve, compress, route, and verify — emits an inspectable,
replayable plan, and learns from the outcome. It does for AI context what query
planners did for SQL. See POSITIONING.md for the thesis.

It optimizes any app with (a) a decision point about what context/config to use and
(b) a measurable outcome. Eleven tenants are built and green (each number is the
learned-vs-baseline reward its offline examples/<tenant>.py prints):

PYTHONPATH=. python examples/sidekick_learning.py   # discrete-strategy bandit
PYTHONPATH=. python examples/rag_tuning.py          # numeric-knob tuning
PYTHONPATH=. python examples/soc_triage.py          # tool-using cybersecurity tenant

Plus the ToolPlugin seam (context_runtime/tools/ — how plans reach external systems,
with an approval-gated audit trail) and trace exporters (context_runtime/observability/ exporters.py — JSONL offline, or Langfuse / OpenLLMetry-OTel when the extras are
installed).

Status: v0.1 vertical slice. Runs fully offline with stub plugins; the real
redevops-rag retrieval and LiteLLM
model bindings are wired and lazy-imported. See SPEC.md §10 for the
conformance checklist these tests assert against.

Install

pip install -e .                 # core (offline stub path, zero heavy deps)
pip install -e ".[litellm]"      # real models across 100+ providers
pip install -e ".[rag]"          # redevops-rag — single-hop hybrid retrieval
pip install -e ".[hipporag]"     # HippoRAG — multi-hop graph retrieval (the planner picks per query)

Single-hop vs multi-hop is a per-query decision. The planner classifies intent and
routes: BM25/hybrid (redevops-rag) when the answer is in one chunk, graph (HippoRAG)
when it lives in the connections between documents — and the cost model only pays the
graph premium when it's warranted. python examples/hop_routing.py shows single-hop
missing the bridge document that multi-hop surfaces.

30-second tour

from context_runtime import ContextRuntime, SourceRef

rt = ContextRuntime.default(docs)          # offline: stub model + in-memory store

# RUN — the core abstraction (plan → build_context → execute → verify)
res = rt.run("Explain why deployment X failed",
             sources=[SourceRef("docs", "docs")],
             constraints={"max_cost_usd": 2.0, "require_citations": True})
print(res.answer, res.cost_usd, res.trace)

# EXPLAIN — debug the plan like SQL (add analyze=True for EXPLAIN ANALYZE)
ex = rt.explain("Explain why deployment X failed")
print(ex.intent.bucket, len(ex.candidates), ex.chosen.score.total)

# SIMULATE — forecast cost/latency/tokens with confidence intervals, no execution
sim = rt.simulate("Explain why deployment X failed")
print(sim.expected_cost_usd, sim.expected_models, sim.based_on_samples)

Or from the CLI / config:

PYTHONPATH=. python examples/incident_review.py
context-runtime --corpus ./docs run "what's our incident process?"
context-runtime --config context_runtime.yaml explain --analyze "why did deploy X fail?"

What's implemented (v0.1)

Architecture

The decision layer is thin; the substrate is reused. See:

• ARCHITECTURE.md — the layered design and the cost-based optimizer loop
• SPEC.md — the normative interface contracts (six plugin seams, IR, trace, plan-cache key)
• ROADMAP.md — v0.1 → v2 phasing with per-phase exit benchmarks

Test

pip install -e ".[dev]" && pytest      # 18 tests; test_conformance.py == SPEC §10