Base Layer

base-layer.ai · Paper · Research · Dataset

If something represents you, it has to know what it means to represent you. You can write that down yourself, or it can be derived from what you have written. Base Layer does the second.

Base Layer is an open-source pipeline: it takes a person's own writing and produces a specification, plain-text claims about how that person reasons, communicates and decides, which an agent reads before acting for them. The claims describe how the person interprets facts, not only the facts: what they count as evidence, what they treat as settled, where they refuse a tradeoff.

In the main method, the cited build, every claim cites the facts it rests on, and every fact quotes the person's own words. You can see what any claim rests on and correct it where it is wrong.

If you are an agent, start with Using it with an agent and AGENTS.md.

How it works

Each step leaves an output you can inspect and a check that guards it. These are the steps of the
cited build, in order. A turn is one message. The leaf stage is distill: one model call per chunk
of facts.

Extraction runs on Claude Haiku 4.5, the leaf stage on Claude Sonnet 5, and authoring on Claude
Opus 5 by default. Each line is in this repository unless it says otherwise.

Today the cited build is eight commands, not one, and it is not yet served to an agent
automatically: consolidate writes files you put in a system prompt by hand
(Using it with an agent).

baselayer run is the older, quicker pipeline. It is one command, and its result is served to
Claude Code over MCP (Model Context Protocol) today. It makes far fewer checks. Extraction does not require the person's
own words unless BASELAYER_TURN_CONTRACT=1 is set. Claude Sonnet 4.6 writes the three layers in
three separate calls, each from a selection of facts made by code and capped in size, so on a
large corpus each call reads a small share of the facts. Claims may cite facts but need not, and
there is no verification or consolidation step. Afterwards each claim is linked to the facts
nearest it in meaning, which shows that a fact is related, not that the writer used it.

What a specification contains

A specification has three layers:

• ANCHORS: where the person reasons from, the certainties that settle a question before the
  details arrive.
• CORE: how they communicate, and the context to read them in.
• PREDICTIONS: given a kind of situation, how they are likely to respond.

Every claim has an id and can say when it applies. One claim, from the specification built from
Zitkala-Sa's American Indian Stories (1921, public domain):

P15 TOLERANT TO PEOPLE HARSH TO SYSTEMS. Their severity is aimed at institutions and at
settler power in the aggregate, not at individuals in front of them: in face-to-face
disagreement, including religious disagreement, they extend respect, pity for suffering, and
forgiveness over punishment. Predict the polemic in writing and the courtesy in the room.

Active when: They disagree with, or witness the hardship of, a particular person present

Evidence: [F-86fc472d] [F-cabf4c6c] [F-124a3cd5] ... (17 facts)

The Evidence line lists the facts the claim was written from. This example was made with an
earlier version of the cited build, before its facts quoted the text they came from. The full
specification is in examples/zitkala_sa/.

What it is not:

• Not a memory system. Memory systems retrieve facts. A specification is the lens those facts
  are read through, and it sits above whatever memory you use.
• Not a copy of the person. It is closer to what someone who knows you notices about how you
  approach a situation, written down for an agent.
• Not a biography or a summary. It describes how someone reasons, not what happened to them.
• Not confirmed. It is a reading made by models, and it is not confirmed until the person it
  describes says it is them.
• Not a portrait for the person. It is written for the agent working for them; the person reads
  it to correct it.

Quick start

To see a finished specification before spending anything, read
examples/zitkala_sa/.

Requires Python 3.10 or later and git. Base Layer needs a model API key, which it reads from the
ANTHROPIC_API_KEY environment variable (get a key);
set it in your shell profile or a secrets manager rather than typing it into a command.

pip install git+https://github.com/agulaya24/BaseLayer.git

Run it on your own writing, or on writing by someone who has agreed to it. Use the cited build
when you want every claim traceable to the person's quoted words and checked; use baselayer run
to try Base Layer in one command and have Claude Code load the result.

The cited build is eight commands into a fresh data root (the directory that holds the
database; see below): init, import,
extract --turn-contract, distill and assemble for each layer, author-from-package,
verify-spec and consolidate, plus a file listing the names the person goes by. distill and
author-from-package print an estimate and refuse to start without a spend ceiling at or above
it, and until you confirm the date of the rate table they price from. The commands in order, and the batch setup the measured builds used:
docs/PIPELINE.md.

baselayer run, to try it and use the result in Claude Code today (the second line needs
Claude Code):

baselayer run chatgpt-export.zip     # or claude-export.json, a folder of journals, notes.md
claude mcp add --scope user --transport stdio base-layer -- baselayer-mcp

The first time, it asks for consent and for the name and pronouns of the person the writing is by
(--accept-data-processing answers the consent prompt). It prints an estimate of the extraction
cost and asks before spending; the estimate covers extraction only and has run low, so treat it
as a floor.

The result is three markdown files in data/identity_layers/ inside the data root, which is
~/.baselayer unless you set MEMORY_SYSTEM_ROOT to an existing directory (a source checkout
differs; see docs/PIPELINE.md). The second line registers the MCP server
(Model Context Protocol, how Claude Code loads outside tools and context), which serves those
files to new Claude Code sessions in any folder. For anyone other than you, build into a fresh
data root (export MEMORY_SYSTEM_ROOT=... and mkdir -p it first), so their facts do not mix
with yours, and register that server inside one project only:
claude mcp add --scope local -e MEMORY_SYSTEM_ROOT=<that root> --transport stdio base-layer -- baselayer-mcp.

Base Layer is pre-1.0 (version 0.5.0) and not on PyPI; the baselayer name there belongs to
another project.

What it costs

API spend for one specification, by size of corpus. Measured figures are recorded token usage
from cited builds made in September and October 2026. Projected figures scale those builds'
per-fact costs, at 17 to 23 facts per 1,000 words of the person's writing. The cited build column
is the batch setup every measured build used: extraction and the leaf stage (one model call per 50 facts
per layer) sent through the Message Batches API at half price, and the layers written by Claude
Opus 5.5.

• The cited build spends about two thirds to four fifths on its leaf stage. Run as the plain
  command sequence instead, at full rate with Claude Opus 5 writing, it costs roughly 1.6 to 1.9
  times as much.
• baselayer run is nearly all extraction: writing the three layers measured about $0.12 on
  three corpora of about 600 to several thousand facts in July 2026; it does not grow with the
  corpus, because each layer reads a capped selection. Its extractor has not been priced on a recorded
  run, so the column is projected from the cited build's extractor; read it as an order of
  magnitude.
• Rates, per million input / output tokens, from Anthropic's
  pricing page as read on 19 October
  2026: Claude Haiku 4.5 $1 / $5, Claude Sonnet 4.6 $3 / $15, Claude Sonnet 5 $2 / $10, Claude
  Opus 5 $5 / $25, Claude Opus 5.5 $4 / $20; the Message Batches API at half these rates.

Stage by stage, with the basis of every figure: docs/PIPELINE.md.

Using it with an agent

• A cited build, by hand: put served.txt in the system prompt, and when a situation it lists
  applies, add the claims it names from index.json. Serving a cited build automatically is not
  in this repository yet.
• A baselayer run build, over MCP: in Claude Code, the claude mcp add line above; in
  Claude Desktop, through its config file, as
  recipes/serve_specification_via_mcp.md shows. The
  server sends the three layers with every session and offers tools, among them
  recall_memories and search_facts to find facts, trace_claim to show what a claim rests on,
  and verify_claims to check it against the database (it writes its results there). Some clients truncate long MCP
  instructions; the full text is always the memory://specification resource.
• Any model, by hand: put the three layer files in the system prompt. You lose fact search.

How an agent should read it:

1. A claim with an Active when line (or a prediction's When ... trigger) applies when that
   situation holds, not everywhere. The always-on claims at the top of served.txt apply in every
   exchange. A claim with no condition is context to read the person in, not a rule for every
   situation.
2. A claim marked CONTESTED keeps both sides of evidence that disagrees. Do not resolve it for the
   person.
3. Facts call the subject "user" or "this person". That is the person the specification describes.

How to check it

To check a cited build, follow a claim's fact ids to the words they quote, and run verify-spec,
which re-checks every citation and quote against the database at no cost.

What is checked by code, and what is a model's reading

The measured builds used the quote gate. The quote check shows that a fact's evidence is text from a turn the source attributes to the
person, not that the fact reads it correctly.

baselayer run lets the writer cite facts but does not require it, and links every claim to the
facts nearest it in meaning. baselayer provenance --claim P3 (or trace_claim over MCP) prints
each link with how it was made.

No check here shows that the specification is faithful. A citation that resolves proves
the reference is real, not that the fact caused the claim. A specification that another model can
tell apart from others is distinguishable, which is not the same as faithful. Only the person can
confirm it. Every term used in a fixed sense is defined in docs/DEFINITIONS.md.

How to correct or update it

Facts do not carry their own significance. An extractor can find that someone rewrote a plan three
times; it cannot tell whether that was diligence or avoidance. That judgement comes from the person,
by editing the text.

• Edit a cited build. Change a claim in spec/<layer>.md and spec/<layer>.json together
  (consolidate fails when they disagree), then run verify-spec and consolidate again.
• Edit a baselayer run build. Open the files in data/identity_layers/. Delete what is
  wrong, rewrite what is close, add what the writing never said, then restart the MCP server.
  Running author or run again rewrites these files from the facts; earlier generated versions
  are kept in data/identity_layers/history/, but hand edits are not, so copy yours first.
• Remove a wrong fact. baselayer review walks through facts one at a time to hide or flag
  them; baselayer forget hides facts by id, by conversation, or all of them. Neither rewrites
  the layers already written: for a baselayer run build, run baselayer author again, then
  baselayer verify --generate; for a cited build, start again from distill.
• Add more text. For a baselayer run build, run baselayer run on the new file; extraction
  only processes what is new. For a cited build, baselayer import the file, run
  baselayer extract --turn-contract, then rebuild from distill.
• Start over or remove everything. Use a new, empty data root (baselayer forget --all is not
  a reset). To remove everything, delete the data root, the MCP session logs in
  ~/.baselayer/sessions/, and any trees/, packages/, spec/, verify/ and served/
  directories you created for a cited build.

For an agent making these changes on someone's behalf, docs/CORRECTIONS.md
sets out how a correction moves through facts, layers and what is served, in order. Base Layer
Live, which keeps a specification updated and served as corrections arrive, is in development.

Limits and what is not included yet

• Self-report. The corpus is what the person wrote or said: text only, nothing observed by
  others.
• No time axis. Claims are not dated; the specification does not record how the person
  changed.
• Little gain on the well known. It helps most where the model knows the person least.
• Anthropic models. Base Layer has been tested almost entirely on Anthropic's models, and
  every language-model call goes to Anthropic except extraction, which can run on a local model
  through Ollama (baselayer extract --backend ollama). The local models we tried fell well short
  of what the pipeline needs: Qwen 2.5 14B failed at writing across 12 attempts and was kept for
  extraction only (D-030).
• The cited build is experimental. Its tests run against fake model clients, most of its design
  was measured on one book-length corpus of about 400 facts, and the builds in the cost table ran on a
  development version of this code with the same stages.
• Not in this repository yet: one command for the cited build; serving a cited build to an
  agent; the support filter that judges each fact against its quote before distillation, and a sort
  of the facts that go beyond their quote by how they do so; a separate check by model judges of
  each claim's scope; and the
  fire-rate measurement for choosing always-on claims. The measured builds in the cost table used
  the support filter, from scripts outside the package. ROADMAP.md lists each.

Research

The Beyond Recall study tested specifications on 14 historical subjects with public-domain
autobiographies, scored by panels of model judges. Prediction is the test of whether a
specification represents the person accurately, not the goal.

• Gains were largest where the model knew the person least.
• The direction reproduced across response models; absolute sizes depend on the judge panel.

A later test, separate from the paper, asked where the difference shows. Given recorded responses
to the same decisions under nine specifications (some of them variants of one another), a
deliberately simple classifier, not a model judge, named the right specification 51.6% of the time
from the reasoning and 13.4% from the decision alone, against 11.1% chance. Differences of tone
and length were not ruled out.

Specifications change how decisions are argued in every situation tested, and change the decision
itself in some. An analysis plan was locked partway through the study, after most data had been
collected; it is not a preregistration of the full study, and it names a seven-judge panel while
the primary results use five
(ANALYSIS_PLAN_LOCK.md).
Method and results: the paper, the
research page, and the
study code and data. The version the paper used:

pip install git+https://github.com/agulaya24/[email protected]

Privacy

The database, facts and specification are stored only on your machine. There are no accounts
and no sync. Extraction, distillation, writing, the tension check and any check a model judges send
text to the Anthropic API, whose retention policy applies; extraction can instead
run on a local model through Ollama (baselayer extract --backend ollama). The embedding model is
downloaded from Hugging Face on first use. Base Layer sends no usage data and turns off the
telemetry of its vector store, ChromaDB. A specification is meant to be legible to the
person it describes and under their control: they can read it, correct it, or delete it.

Documentation

Contributions on evaluation, source adapters, other interpretive-layer implementations and local
models are welcome: CONTRIBUTING.md.

@software{baselayer2026,
  title     = {Base Layer: An Open-Source Reference Pipeline for the Interpretive Layer Above Memory},
  author    = {Gulaya, Aarik},
  year      = {2026},
  url       = {https://github.com/agulaya24/BaseLayer},
  license   = {Apache-2.0}
}

Apache 2.0. See LICENSE. The Beyond Recall paper is CC BY 4.0.