Zero Error is the Baseline. Absolute Balance is the Goal.
Make it exist first, then make it good later.

🏵 Overview

Klaudia is an end-to-end agentic finance accountant AI platform designed to automate financial document processing, receipt extraction, data entry operations, and spreadsheet workflows.

Unlike traditional OCR systems, Klaudia combines document intelligence, multi-agent orchestration, human-in-the-loop validation, and production-grade LLMOps into a single architecture.

Core Capabilities

• Receipt & invoice extraction
• Multi-page PDF processing
• Human-in-the-loop correction workflow
• Google Sheets automation
• SQLite financial document registry
• LangGraph multi-agent orchestration
• MCP-based tool execution boundary
• Fine-tuned Qwen 3.5 financial document understanding
• Async document processing pipeline
• Langfuse observability & tracing
• Mobile-first conversational interface

🐸 System Architecture

Agentic Flow

User
 │
 ▼
Guardrails
 │
 ▼
Extraction Agent
 │
 ▼
Supervisor Agent (Klaudia)
 │
 ├── SQL Agent
 │      ▼
 │   MCP SQLite
 │
 └── Data Entry Team
        ▼
    MCP Google Sheets

🐝 Training Pipeline

Fine-Tuning Strategy

• Base Model: Qwen 3.5 4B
• PEFT LoRA Fine-Tuning
• Hyperparameter Search
• Experiment Tracking with Weights & Biases
• KIEVal Evaluation Suite
• ANLS*, Digit Accuracy, and JSON Validity Assessment

🍀ྀི Benchmark Results

The Klaudia model (Qwen 3.5 4B fine-tuned on our financial receipt dataset) was evaluated using KIEVal, ANLS*, Digit Accuracy, and JSON Validity benchmarks.

Across all evaluation metrics, Klaudia consistently outperformed the base Qwen 3.5 and Gemma 4 models, demonstrating superior entity extraction, document structure understanding, numerical accuracy, and reduced human correction effort.

Evaluation Summary

Key Improvements Over Base Qwen 3.5 4B

• +17.09% KIEVal Entity F1
• +43.71% KIEVal Group F1
• +21.85% KIEVal Aligned
• +16.77% ANLS*
• +16.71% Digit Accuracy
• Maintained 100% JSON Validity

🎞️ Dataset & Models

[!WARNING]
The dataset and fine-tuned model will be released after the research paper is published. They are currently archived in a private Oxen.ai repository.

Dataset Sources

• Hugging Face
• Kaggle
• Roboflow
• Pinterest
• X (Twitter)
• Custom Collected Receipts
• Human Verified Labels

🏞️ Mobile Application

Mobile Stack

• React Native
• Expo
• TypeScript
• Session Management
• Conversational Finance Assistant
• Receipt Upload Workflow
• Spreadsheet Operations

🍏 Tech Stack

🪲 Key Features

Financial Document Intelligence

• Receipt Extraction
• Invoice Extraction
• OCR + KIE Pipeline
• Multi-page PDF Support
• Structured JSON Generation

Agentic AI System

• Supervisor Architecture
• Multi-Agent Collaboration
• MCP Tool Execution
• Human-in-the-Loop Workflow
• Session-Aware Context Management

Production Infrastructure

• Async Processing
• Queue-Based Execution
• Observability & Tracing
• Object Storage
• Deduplication Pipeline
• Fault-Tolerant Processing

🌱 Harness Engineering (Planned)

[!NOTE]
This architecture is currently under active development on the development branch and is not yet part of the stable release.

The next evolution of Klaudia focuses on a production-grade agent harness that brings long-term memory, procedural skills, evaluation loops, and continuous LLMOps into a unified runtime.

Planned Capabilities

• 🧠 Persistent agent memory (episodic & semantic)
• 📚 Skill-based procedural knowledge
• 🔍 Hybrid RAG for retrieval and context
• 🛠️ Tool execution & scheduled agent jobs
• 📈 Continuous LLM evaluation with Langfuse
• 🤖 LLM-as-a-Judge evaluation pipeline
• 🚦 Safe release gates before deployment
• 🔄 Self-improving feedback loop

🗂️ Repository Structure

app/
├── routes/
├── services/
├── extraction/
├── guardrails/
└── core/

klaudia/
├── supervisor/
├── sql_agent/
├── data_entry_team/
└── tools/

mcp-sqlite/
mcp-gsheets/

docs/
tests/
sample-data/

🍀 Mission

Where financial records lose balance, Klaudia restores order.

Klaudia is designed around a simple philosophy:

Inputting financial data is not merely typing numbers. It is preserving the financial truth of an organization.

📗 Research

This project is part of an undergraduate research thesis focused on:

• Agentic AI Systems
• Financial Document Intelligence
• LLMOps Pipelines
• Human-in-the-Loop Data Entry Automation
• Domain Adaptation for Financial OCR

Dataset and model release will follow publication.

🔰 Author

Yudhy Prayitno

Building agentic systems for real-world financial automation.