JevLoop

Every fork in your agent loop is a full LLM call. Not one of them is generation.

Should I act? Which tool? Is this safe? Did it work? Am I done? Can I ship this? A conventional agent answers each of those by writing a sentence and parsing it back. But each is a pick, a score or a yes/no answer: one forward pass over a fixed candidate set, ~10–40 ms, no tokens generated.

JevLoop routes them to a decision model (Jev / Laya) and keeps the LLM for the one thing only it can do: writing.

English · 中文

$ npm run demo          # fresh clone: no key, no network, no npm install

JevLoop · demo
  decision  : laya→rule-judge
  generator : scripted — set DEEPSEEK_API_KEY for a real LLM

  ── loop trace ──────────────────────────────────────────
  ▲ laya unavailable (fetch failed), falling back to rule-judge
  cleared: list_dir (auto)
  cleared: read_file (auto)

  ── every decision ──────────────────────────────────────
  step 1
     decide  loop.needsTool         use_tool           4.3ms  needs_tool=0.95
     decide  loop.pickTool          call               4.3ms  tool=list_dir
     decide  loop.gradeRisk         auto               4.2ms  risk=0.0 needs_auth=0.05
     decide  loop.stepOk            continue           4.1ms  ok=0.92
     decide  loop.isDone            keep_going         4.0ms  done=0.10
  ...
  step 3
     decide  loop.canDeliver        deliver            4.3ms  deliverable=0.90 unsupported=0.08
     model   generate (scripted)                       601ms

  ── accounting ──────────────────────────────────────────
  decisions  12     50ms (4.2ms each)
  model       1     601.2ms

  decisions : model = 12.0:1   decisions are 7.7% of wall clock

Zero dependencies. Zero build step. Runs offline with no API key.

That run's judge is a rule table, not a model — it shows the shape of the loop, not the quality of a decision. Real backends and what they actually cost: Honest numbers.

The problem

Take a task that needs two tool calls. A conventional agent burns a model call on each of these:

You were paying generation prices for decisions.

Quick start

Needs Node ≥ 22.6 (it runs TypeScript directly, no build).

git clone https://github.com/zjunlp/JevLoop
cd JevLoop
npm run demo

That's it — no npm install, no API key, no network. The demo falls back to a deterministic rule judge so the whole loop runs offline.

Use a real decision model:

npm run demo -- --laya     # local Laya sidecar on :7789 (open weights, free)
npm run demo -- --jev      # official Jev API (needs TYPESAFE_API_KEY)

DECISION.md — the decisions, compiled

Every generation of agent framework leaves behind a .md. AGENTS.md holds conventions, SKILL.md holds capabilities — and both are prose for a model to read. The model pays tokens for them every turn, it can ignore them, and nothing tells you whether it did.

DECISION.md is the first one that gets compiled.

Not a decision record. A record is written afterwards, to explain what an agent did. DECISION.md declares what the loop is going to decide, and a program turns it into the questions the decision model is asked.

One file, two consumers:

structure blocks  →  questions + policy  →  the decision model   (tens of ms, no tokens)
prose             →  system prompt       →  the LLM              (the one expensive step)

So it is subtraction: every block you move into the file is one question the LLM no longer has to be asked. headline() counts them from the file itself — change a kind and the sentence changes with it.

And the file cannot quietly rot. tests/decisiondoc.test.ts compiles it and asserts, in both directions, that it matches what src/decisions.ts actually asks: no decision missing, none invented, every question's primitive type the same.

## grade_risk
kind: mixed
when: before every tool call that actually runs

### risk
ask: How risky is this tool call?
- read-only
- reversible write
- irreversible
- destructive

### needs_auth
ask: This call must be explicitly authorised by a human before it runs
- true — it can destroy data, spend money, or leave the machine
- false — it only reads or writes inside the working directory

policy:
  - score:risk >= 2 → ask_human
  - prob:needs_auth >= 0.5 → ask_human
  - score:risk >= 1 → auto_audit
  - else → auto

The question type is inferred from how the options are written, never declared. Two options named true and false is a noul; every option named is a choice; none named is a score; a mix is an error rather than a guess. kind then has to agree with what the writing implies.

Predicates are a closed vocabulary. else, top >= n / top < n (single-question blocks only), prob:<id> (on a noul), score:<id> >= n (on a score), picked:<id> = <option> (on a choice). There is deliberately no > and no <=: a condition you cannot write here is a condition that belongs in code.

A predicate aimed at the wrong kind of question is rejected, not compiled. Left alone it would become a rule that never fires — the author believes they wrote a gate, there is no gate, and it fails open. Actions are a closed list too, and an unknown one is reported with its line number. Nothing is ever silently dropped: everything unrecognised lands in problems, with the line it came from.

How it works

 step ─┬─ loop.needsTool   ↗ do I need to act?  ──no──▶ generate
       │
       ├─ loop.pickTool    ↗ which tool?  (options rebuilt every step)
       │
       ├─ loop.gradeRisk   ↗ how dangerous is this?  ──▶ ask a human
       │
       ├─ [ tool runs ]     ← the only place with real side effects
       │
       ├─ loop.stepOk      ↗ did it work?
       │
       └─ loop.isDone      ↗ am I done?  ──no──▶ next step
                            │
                            ▼
                       [ LLM generates ]   ← the only expensive call
                            │
                       loop.canDeliver  ↗ is this shippable?

All six decision points live in one file: src/decisions.ts. If you read one file in this repo, read that one — it's the whole idea.

A decision is three things

export const pickTool = defineDecision({
  id: "loop.pickTool",

  // ① Project the agent state into a BOUNDED decision frame.
  //    This caps what the model can judge: what isn't in the
  //    frame cannot be decided.
  state: (ctx: AgentCtx) => ({
    task: clip(ctx.task, 400),
    files_known: (ctx.files ?? []).slice(0, 20),
    recent: (ctx.history ?? []).slice(-3).map(h => `${h.tool}(${h.input}) → ${clip(h.result, 120)}`),
  }),

  // ② Typed questions. Answered in ONE forward pass.
  questions: (ctx: AgentCtx) => ({
    tool: choice("Which tool should the agent call next?", toolsFor(ctx)),
  }),

  // ③ Policy: answers → action. Pure code. No model involved.
  policy: [
    { when: gte("tool", 0.6), action: "call" },
    { action: "escalate", reason: "not confident enough — hand back, don't guess" },
  ],
});

Three primitives, taken straight from the Jev wire protocol:

Two things worth stealing

Rebuild the options every step. A fixed action list makes the model pick something that no longer applies — write_file should not still be a candidate after you've written the file. That's why questions can be a function of the context.

Never let a probability bypass authorisation. Risk gating is a hard rule, not a threshold:

policy: [
  // irreversible ⇒ explicit authorisation. No confidence score overrides this.
  { when: scoreGte("risk", 2), action: "ask_human" },
  // the model's own read is a SECOND, independent gate
  { when: probGte("needs_auth", 0.5), action: "ask_human" },
  { action: "auto" },
]

A decision model may decide whether to ask a human. It must never decide whether to skip authorisation.

Honest numbers

The same loop against three decision backends. The ratio that matters is decisions : model calls, and the one that surprised us is how much of the wall clock the decisions take.

Two conclusions we are not going to soften:

• The whole claim holds on a locally-served decision model. 30 ms decisions make the loop's thinking essentially free next to one generation call.
• Over the hosted API it does not. ~390 ms per decision is network round-trips, and with 13 decisions for 1 generation the decisions dominate the clock. Still ~5–8× faster than a frontier LLM call and orders of magnitude cheaper, but "decisions are free" would be a lie at that latency.

The obvious sweet spot is a strong decision model served locally. Neither of the two we could test is that: one is fast but not accurate enough, the other is accurate but round-trips.

Two gotchas we hit so you don't have to

Both were found by running this loop against a real Laya checkpoint on an A100, not by reading docs.

confidence is not the top probability. Laya's confidence for a choice is normalised Shannon entropy (1 - H(p)/log(k)) — p = [0.80, 0.20] gives confidence = 0.269. So a fixed threshold means a completely different thing at 2 options than at 20. Gate a choice on the winning option's probability instead; that's what topGte() is for. (The official docs call confidence a solid default and hand you the full probabilities for exactly this reason.)

A base checkpoint will not do a novel decision task zero-shot. Asked "which tool next?", laya-typed-decisions chose done at 0.660 on step 2 while the right answer on step 1 scored 0.646 — the wrong answer scored higher, and everything landed in a 0.55–0.66 band with no separation. No threshold fixes that; it's a capability gap. The open-weight checkpoint is a fast base to specialise, not a drop-in judge.

Both are the same lesson from Jev Engineering: the call is the easy part — the work is in the state you send and the threshold you act on.

Bring your own backends

Decision backend — anything that answers {state, questions} → {answers}:

import { Decider, HttpProvider, FallbackProvider, MockProvider } from 'jevloop';

const decider = new Decider({
  provider: new FallbackProvider([
    new HttpProvider({ baseUrl: "https://api.typesafe.ai", apiKey: process.env.TYPESAFE_API_KEY, name: "jev" }),
    new HttpProvider({ baseUrl: "http://127.0.0.1:7789", name: "laya" }),
    new MockProvider(),   // never fails
  ]),
});

Generation backend — anything that turns a prompt into text:

import { HttpGenerator } from 'jevloop';
// any OpenAI-compatible /chat/completions endpoint
new HttpGenerator({ baseUrl: "https://api.openai.com/v1", apiKey, model: "gpt-5" });
new HttpGenerator({ baseUrl: "http://localhost:11434/v1", model: "qwen3" });  // ollama

Swapping either one touches exactly one file. The loop and the decision specs don't move.

Not on npm yet. Install from git — the prepare script builds dist/ for you:
npm install github:zjunlp/JevLoop

What this is not

• Not a replacement for an LLM. Drafting, coding and summarising still need one.
• Not "zero hallucination". A decision model can't return an answer outside the type you asked for, but the answer can still be wrong. That's what the threshold is for.
• Not benchmarked against a conventional agent on the same task yet. The 12:1 above is from the bundled demo. That comparison is the obvious next step and it isn't done.
• Not production-hardened. Tool sandboxing covers path escape only. Read src/tools.ts before pointing it at anything you care about.

Layout

DECISION.md      ★ the decisions as a file — compiled, and checked against the code
src/
  vocab.ts       Question / Answer / Decision — the whole vocabulary
  decisions.ts   ★ all six of the agent's judgements, one file
  decisiondoc.ts the DECISION.md parser (nothing is silently dropped)
  decision-compile.ts  blocks → questions + policy
  decide.ts      the six steps of one decision
  policy.ts      answers → action (pure code, unit-testable)
  seam-provider.ts     the decision-backend interface
  provider-http.ts     Jev / Laya — swap by baseUrl
  provider-mock.ts     a stand-in that never guesses
  provider-fallback.ts try them in order, report every downgrade
  meter.ts       ★ decisions vs model calls
  agent.ts       ★ the loop
  tools.ts       list / read / write, path-locked to cwd
  llm.ts         the one place that generates
examples/
  demo.ts        runs offline
  rule-judge.ts  deterministic stand-in for a decision model

License

Apache-2.0