Recommend Agentic Trust Layer

A fact-checker for the things you tell your AI agents. It checks a statement against real sources (the web, or your own documents) and tells you whether it holds up, and how sure it is.

Here's the problem we kept running into. AI answers everything with the same confident
tone, whether it's right or not. Ask a model "how sure are you?" and it says 95% almost
every time. That's fine for a chat. It's not fine when an agent is about to act on that
information: write it into a report, put it in a knowledge base, make a decision from it.

So we built the thing that checks first. You give it a statement. It goes and looks, using
real sources fetched right now (never the model's memory), and comes back with three things:

• Is it true? A score from 0 to 100.
• How much should you trust that answer? A separate confidence number. It drops when
  the sources are thin, out of date, or disagree with each other.
• Why? The sources it used and a few sentences of plain-language reasoning.

You can use it yourself in a web page, plug it into an agent as a tool, or run it over a
whole knowledge base and stamp the files that hold up.

Try it in two minutes

git clone https://github.com/recommend-dev/recommend-agentic-trust-layer.git
cd recommend-agentic-trust-layer
pip install -r requirements.txt
cp .env.example .env          # add GEMINI_API_KEY and EXA_API_KEY
python3 server.py             # open http://localhost:8899

You need two keys: a free Gemini key and an
Exa key. More keys unlock more sources (see the engineering notes below),
but these two are enough to get going.

Type something like "Humans only use 10% of their brains" and watch it work.

What you get back

Here's what it says about "Sugar makes children hyperactive":

Verdict:     REFUTED
Truth score: 9 / 100
Confidence:  0.86

3 of 4 evidence sources agree. Lane spread 0.07, consistent.

A 1995 meta-analysis of 23 studies (JAMA) found no effect of sugar on
children's behaviour; the perceived effect is explained by parents'
expectations. Sources: jamanetwork.com, nih.gov, ...

The two numbers are independent on purpose. A claim can be clearly false and the system
can be very sure about that. Or the evidence can be genuinely split, in which case the
confidence tells you so instead of quietly picking a side.

How it works, in plain terms

The easiest way to picture it is a small fact-checking desk.

1. Intake. The claim gets rewritten into one precise, checkable sentence, split into
   the smaller facts it depends on, and labelled by type (factual, statistical, causal,
   about the future, opinion). One rule here: the claim is never "corrected". If you type
   in a myth, the myth is what gets checked, not its debunking.

2. Independent researchers. Several evidence sources go and look at the same time,
   each using only its own tool. We call these lanes. There are two different web
   search engines, a live Google index, and, when the claim calls for it, scientific
   literature or a prediction market. Which lanes run depends on the kind of claim. A
   study can settle "does X cause Y"; a news article can't.

3. A judge for each lane. Each lane's findings get their own verdict, made only from
   the text that lane actually fetched. The judge is told to say "not enough evidence"
   rather than guess.

4. Opposing counsel. Every verdict is then attacked. Does the evidence really cover
   this time period, this population, this exact wording? A verdict that breaks keeps
   only about half its weight.

5. Adding it up. The verdicts are combined, weighted by how strong their evidence was.
   If the lanes disagree, confidence goes down. If a lane came back empty, confidence
   also goes down, because silence isn't coverage.

6. One last check that can only go down. A final pass looks for overconfidence. It's
   allowed to lower the confidence and never to raise it.

7. Readout. Two or three sentences on what the evidence showed, plus the sources.

A few things it refuses to do, on purpose. It won't give a factual verdict on a claim about
the future (you get a lean with capped confidence instead). It won't let a prediction
market decide anything on its own, because that's belief, not fact. And it won't stamp
something it couldn't check.

Three ways to use it

1. Yourself, in the browser

python3 server.py and open http://localhost:8899. Type a claim, watch the lanes land,
read the sources. You can also pin a claim, and it gets re-checked daily so you can see
the score move as the evidence changes.

2. From an agent, as an MCP tool

Give your agent a verify_claim tool and it stops answering factual questions from memory:

claude mcp add --transport http recommend-trust \
  http://localhost:8899/mcp \
  --header "Authorization: Bearer <key printed at startup>"

The tool returns the verdict, both scores, a per-source breakdown and the sources.
A demo key is generated into keys.json the first time you run the server.

3. On a whole knowledge base

If your agents write into a shared knowledge base (for example in Google's
Open Knowledge Format),
each file can carry a verified: field. Today that field is just a name someone typed in.
These tools put an actual check behind it. There are two modes:

Against the world. Is what this document says actually true?

python3 okf_verify.py examples/okf-demo      # --dry to report without writing

Against its own citations. Does the source this document cites really say this?
No web search here; the evidence is the files and links the document itself points to.
This is the mode for internal material the web has no opinion on: policies, metric
definitions, schema docs.

python3 okf_adapter.py path/to/bundle        # --dry · reports in <bundle>-reports/

In both modes a document only gets the verified stamp if every checked claim holds up.
Anything refuted, inconclusive or uncheckable stays unstamped, and the reasons are written
into the file and into the bundle's log.md. The demo bundle in examples/okf-demo shows
all three outcomes:

There's a drag-and-drop UI for this too: python3 okf_server.py, then http://localhost:8898.

Things to know before you rely on it

• Every check costs real API credit. There are daily caps built in (PER_IP_DAY=25,
  GLOBAL_DAY=400). Change them in .env before you put this on a public host.
• A missing key isn't an error. That source just sits the check out, and confidence
  is a bit lower because of it. The verdict tells you which sources actually ran.
• It's not a search engine. One check takes 10 to 20 seconds, because it fetches,
  judges and cross-examines. Use it for the things your agent is about to act on, not for
  every sentence it produces.
• No framework, no build step. It's one Python file using the standard library, and
  one HTML file. Read it in an afternoon.

For engineers

Weights multiply each lane's own evidence strength (0–1, set by the judge). A lane that the
opposing-counsel pass breaks has its strength multiplied by 0.55. The prediction market is
capped at strength 0.35 regardless.

• Truth score = weighted average of each lane's p_true, weight = lane weight × evidence strength.
• Confidence = evidence mass × (1 − 0.55 × spread) × (0.45 + 0.55 × decisiveness), clamped to [0.05, 0.96], where
  evidence mass is total weight divided by the number of lanes started (so a silent lane counts against you),
  spread is the gap between the most and least supportive lane, and
  decisiveness is how far the score sits from 50/50.
• A refuted sub-claim caps the score at 45 and confidence at 0.6. Predictive or opinion claims never get
  SUPPORTED/REFUTED; they get LEANS TRUE/FALSE with confidence capped at 0.45.
• The calibration pass (calibration_review) may only lower confidence.

Everything lives in server.py, sections numbered 1–5 in the source.

• Unfalsifiable claims never get a factual verdict. "X is going to attack Y" used to
  return REFUTED at 96%. Absence of reporting about the future is not proof.
• Intake never corrects the claim. "Humans use 10% of their brain" once got normalized
  to "no areas of the brain are dormant", every lane supported that, and the myth scored
  100/100. There is now a second model call that checks the rewritten claim still points
  the same direction, and falls back to the user's wording if it doesn't.
• Every prompt knows today's date. Without it, models guess, and penalise good
  evidence for being "in the future".
• (p_true or 0.5) when every lane said 0. 0.0 is falsy in Python.

Stamps are spec-conformant and additive: a verified: {by: recommend-trust-layer/0.1, at: …}
event (the machine-confirmed trust tier per SPEC §5.3) plus an x_verification
extension block with per-claim results and sources. Existing frontmatter is never deleted.

Citation-integrity verdicts (okf_adapter.py): supports · contradicts · related-only ·
no-evidence · unverifiable (unreachable source). A claim can be true and still fail
here: true-but-miscited is an attribution failure. A supports verdict whose own reasoning
denies support is downgraded mechanically. Needs only a Gemini key; a 26-concept bundle
cost about $0.02 end to end in our test.

Deprecated concepts and Attested Computations are noted and skipped.

Reverse proxy. The front-end derives every path from location.pathname, so it works
at / locally and under any prefix in production. If your proxy buffers responses (Caddy,
nginx), disable buffering for this route (Caddy: flush_interval -1) or SSE will never
stream. Set HOST=0.0.0.0 to listen beyond localhost. A Dockerfile is included for the
OKF UI (Cloud Run ready).

Tests. Real-Chrome end-to-end tests; the macOS Chrome path is hardcoded, adjust
executablePath for your OS.

cd tests && npm install
node uitest.js      # UI invariants
node tracktest.js   # tracking flow
node cmptest.js     # compare endpoint
node scaletest.js   # layout at widths

Two servers. server.py (port 8899) is the claim checker and MCP endpoint.
okf_server.py (port 8898) is the knowledge-base upload UI. Both read the same .env.

Tracking. Pinned claims are re-checked every TRACK_EVERY_H hours (default 24),
scoped per browser session.

Roadmap

• Pluggable lanes. Add your own evidence source (Brave Search, an internal corpus,
  BigQuery schema checks) without touching the pipeline.
• Continuous verification. Re-run on a schedule and on every knowledge-base change,
  so a stamp ages honestly.

License

MIT · built by Recommend