CiteProof

Audit tools for AI literature reviews. CiteProof checks that a review's citations, quotes, counts and risk-of-bias judgements hold up. It comes in two parts: an MCP server for Claude Science and other research agents, and a full research assistant app that writes evidence-grounded papers.

🔗 Live demo of the research app → deep-research-ai-xi.vercel.app

Contents

• What is CiteProof?
• CiteProof MCP
  • What it checks
  • Install for Claude Science
  • Install for other MCP clients
  • The 24 tools
  • What it does not do
• CiteProof Research (the app)
  • Two modes
  • Architecture
  • Research pipeline
  • 3 human-in-the-loop quality gates
  • Academic integrity and research invariants
  • Tech stack
  • Academic API coverage
  • Getting started
• Repository layout
• Contributing
• Licence
• Star history

What is CiteProof?

AI research agents are now good at finding and reading papers. They are less good at proving their work: which databases were searched, how many records were screened, whether a quoted sentence is really in the paper, whether two reviewers agreed. CiteProof is built around that gap.

A small first benchmark (two reviews against Claude Science) showed that agents with strong search find far more of the known included studies than CiteProof Research does. That is why the MCP server does not try to search: it audits the work of the agent that does.

CiteProof MCP

CiteProof MCP keeps the records a systematic review needs and checks them: a protocol, a PRISMA record, a screening log, an evidence table, claim checks, risk-of-bias records and an exportable audit trail. It works with Claude Science and with other tools that can run a stdio MCP server, such as research agents and desktop AI clients.

The server has no language model inside. The agent searches, reads, decides and writes. The server stores each decision, checks the evidence it points to, computes the numbers and refuses bad input.

Full documentation: MCP/README.md · tool reference: MCP/docs/SPEC.md.

What it checks

Install for Claude Science

Claude Science runs a Local command connector inside a sandbox that cannot read your project folder. The install therefore copies the server into the connector's own workspace folder and starts it with Claude Science's own python3:

python MCP/scripts/build_claude_science_bundle.py "<connector workspace folder>/bundle"

Then add a connector with the command python3 <connector workspace folder>/bundle/run.py. The exact steps, the folder path and troubleshooting are in MCP/docs/CLAUDE_SCIENCE.md. The server was checked on Claude Science 0.1.67 on Windows: it starts, and all 24 tools are listed. A full review run with every tool in Claude Science has not been benchmarked yet.

Install for other MCP clients

pip install ./MCP

Point the client at the citeproof-mcp command (stdio):

{
  "mcpServers": {
    "citeproof": {
      "command": "citeproof-mcp",
      "env": { "CITEPROOF_MCP_HOME": "/path/to/your/review-data" }
    }
  }
}

Review files go to CITEPROOF_MCP_HOME, or to ~/.citeproof-mcp when it is not set. Other clients were not tested for this release.

The 24 tools

What it does not do

• It does not search for papers. Use your agent's search, or the CiteProof Research app below.
• It does not judge anything. A label such as "supported" is the agent's judgement. The server only checks that the quote is in the text.
• It does not read PDFs. The agent supplies the text.
• The PRISMA checklist and the report number check are keyword checks. They show what is mentioned, not whether it is reported well.

CiteProof Research (the app)

An AI-powered multi-agent research workspace with two modes: fast web research that returns a cited report, and a full evidence-grounded academic pipeline that writes complete research papers whose cited claims link back to source passages, with human-in-the-loop quality gates.

Two modes

Architecture

Both modes run as separate LangGraph state machines behind one FastAPI backend, sharing the LLM layer, checkpointer, and SSE streaming transport.

flowchart TB
    UI["🖥️ Frontend<br/>Vanilla JS · Swiss Modernism 2.0"]
    API["⚡ FastAPI<br/>SSE streaming & Heartbeats"]

    UI <-->|"Server-Sent<br/>Events"| API

    API --> DS["🔎 DeepSearch<br/>Graph"]
    API --> RM["📚 Research Mode<br/>24-Agent Graph"]

    DS --> TAV["🌐 Tavily<br/>Web Search"]
    RM --> ACAD["🎓 Multi-Source Academic APIs<br/>8 discovery providers · citation expansion<br/>author enrichment · full-text resolution"]

    DS --> LLM["🧠 LLM Layer<br/>Structured JSON Outputs"]
    RM --> LLM
    LLM --> CACHE[("💾 SQLite<br/>response cache")]

    DS --> CP[("🔁 Checkpointer<br/>resumable state")]
    RM --> CP

    style UI fill:#4f46e5,stroke:#3730a3,color:#fff
    style API fill:#4f46e5,stroke:#3730a3,color:#fff
    style DS fill:#0891b2,stroke:#0e7490,color:#fff
    style RM fill:#0891b2,stroke:#0e7490,color:#fff
    style TAV fill:#ea580c,stroke:#c2410c,color:#fff
    style ACAD fill:#ea580c,stroke:#c2410c,color:#fff
    style LLM fill:#db2777,stroke:#be185d,color:#fff
    style CACHE fill:#475569,stroke:#334155,color:#fff
    style CP fill:#475569,stroke:#334155,color:#fff

Research pipeline

Research question → Scope & protocol → Literature retrieval → Evidence extraction → Synthesis → Validation & export

24 specialized agents · 5 phases · 3 human approval gates

3 human-in-the-loop quality gates

Each gate genuinely pauses graph execution via LangGraph interrupt(), allowing the author to approve or request targeted revisions in natural language:

Academic integrity and research invariants

1. Deterministic Flow-Tracking State Machine (PRISMA 2020-aligned):
   PRISMATracker.validate_invariants() asserts these internal conservation relations:

   records_identified - duplicates_removed     == records_after_dedup
   records_screened   - excluded_title_abstract == full_text_requested
   full_text_assessed - excluded_full_text      == studies_included
   

   full_text_requested is the tracker's internal shortcut for PRISMA's reports sought for retrieval. The distinct PRISMA stages reports not retrieved (full_text_unavailable) and reports assessed for eligibility (full_text_assessed) are tracked as separate fields, but the hand-off between them is not asserted—so the equations above are internal invariants rather than the complete PRISMA 2020 flow. Every count is tracked deterministically in code—never generated or estimated by an LLM.

2. Immutable Evidence Store:

   • PaperRecord.paper_id: first 16 hex characters of a SHA-256 digest (a truncated representation, not the full digest) computed over a source-qualified key: doi:<canonicalized DOI> when a DOI is available, otherwise title:<normalized title>:<year>. Normalized titles are not assumed unique, so the fallback key also folds in the publication year (nd when unknown).
   • EvidenceRecord.evidence_id: deterministic composite identifier {paper_id}_ev001 (the composite is not itself hashed) anchoring exact quote, metric, baseline value, and effect direction.
   • ReviewClaim.claim_id: deterministic composite identifier {section}_cl001 (section slug truncated to 8 characters) linking paper claims to supporting evidence IDs.

3. Deterministic Validation Pipeline:

   • citation_validator: Resolves every in-text citation against known PaperRecord metadata and every ReviewClaim through its claim → evidence → paper → DOI/URL chain; flags citations and claims that do not resolve. This is a resolution check only: it does not test that the cited passage supports the sentence, so a resolving claim is marked resolved, never verified.
   • claim_validator: Matches numbers in quantitative sentences against extracted EvidenceRecord benchmark metrics (numeric match only).
   • integrity_auditor: Validates PRISMA invariants and issues comprehensive ValidationReport; passed_all_gates means the resolution and numeric-match gates passed, not that sources support the prose.

Tech stack

Academic API coverage

Research Mode uses each academic API for a specific stage:

Getting started

1. Clone and configure

git clone https://github.com/Aryan-Pardeshi/CiteProof-AI-MCP.git
cd CiteProof-AI-MCP
cp .env.example .env

Minimum .env configuration:

LLM_API_KEY="your_api_key_here"
LLM_BASE_URL="https://api.deepseek.com"
TAVILY_API_KEY="your_tavily_key_here"
OPENALEX_EMAIL="[email protected]"
NCBI_EMAIL="[email protected]"
NCBI_TOOL_NAME="your_tool_name"

Optional academic API configuration:

SEMANTIC_SCHOLAR_API_KEY="your_api_key_here"
NCBI_API_KEY="your_api_key_here"
CORE_API_KEY="your_api_key_here"
ORCID_AUTHOR_ENRICHMENT=0

2. Run with Docker

docker compose up -d --build

Access the UI at http://localhost:8000.

3. Local development

python -m venv .venv
.venv\Scripts\activate      # Windows (or source .venv/bin/activate on Unix)
pip install -r requirements.txt
uvicorn backend.app.main:app --reload --port 8000

Repository layout

Contributing

Contributions are welcome, from typo fixes to new academic source adapters and new MCP checks. Start with CONTRIBUTING.md for setup and a project map, then pick an issue labelled good first issue or help wanted. The test suites run offline with pytest: run pytest --ignore=MCP from the repository root for the app, and python -m pytest inside MCP/ for the MCP server.

Licence

MIT, see LICENSE. Two files inside the MCP server are adapted from Anthropic's Claude Science under the Apache License 2.0; see MCP/NOTICE.md.

Star history