VERA · dispute assistant for LATAM Bank

VERA handles the first contact of a card dispute at LATAM Bank, the synthetic bank of the Factored AI & Data Hackathon 2026: customers in Mexico, Colombia and Argentina, in Spanish and Portuguese. The name comes from verificación, and it is also what VERA does:

• it Verifies the charge against the records before it acts;
• it Explains each decision with the rule and its source;
• it Records the case and reads it back before it says it is done;
• it Accompanies the customer, with a person always available.

Demo: https://vera.colectivohagamos.com, live since 4 October 2026, with Claude reading over the classifier; the end-to-end suite passes against it (final verification). The product page is at /. After the login, /clientes lists the demo customers, /banca shows the chosen customer's cards and movements, /chat is the conversation with its dispute tracker and glass box, and /analista is the analyst console. The demo customers are pseudonymized: nothing there is a real person.

Try it in two minutes

1. Log in at /login with the account sent with the submission. The login keeps the language model for the people who received an account.
2. Choose a walkthrough at /clientes. It lists 47 demo customers with invented names and pseudonymized data; the recommended walkthroughs at the top pick one customer per scenario from the API.
3. Open the dispute from the customer's bank: press «No reconozco este movimiento» on a movement, or one of the shortcuts above the list. The panel beside the chat shows where the dispute stands (received, analysis, verification, result, resolved), the charge, the case number and, under «Por qué», each rule with its source and the date of any legal deadline.
4. Open /analista. The analyst queue lists every case handoff and transfer note, newest first. Each case shows:
   • its charges and the actions read back;
   • the legal and network deadlines;
   • the risk signals and open questions;
   • the policy rules that decided each step, the policy version and the trace id.

The bank's app sends the button the customer pressed with the text that names the charge, so the claim does not depend on the wording.

Results at a glance

299 held-out cases, each run with three wordings, for 897 runs per system. The set was sealed with a hash and the tag heldout-v1 before its first run. Each group of rows compares the systems on the same run. Method and full tables: Evaluation.

Three limits to read with these numbers:

• The sealed run is the honest measurement. Its 30 unsafe runs had one root cause, fixed afterwards. The rerun is labeled not clean because those failures informed the fixes, and Claude was measured only there. On the same cases, Claude resolves as many safely as the classifier; what it adds is a result that holds across wordings, and fewer customers explaining again.
• The evaluation is an offline simulation with wordings the team wrote. The human blind set and a human red team are pending.
• Legal deadlines are dated only where the official text is loaded: Ley 1755 in Colombia, and Ley 25.065 and the BCRA rules in Argentina. Mexican routes and Colombia's Decreto 587 are recorded without a date.

Why disputes

In the LATAM Bank data, disputes (unrecognized and improper charges) are 24,491 of 67,095 complaints (36.5 %). One in five missed its service deadline, they took 15.5 days to resolve on average, and half arrived through the call center. The complaints cannot be traced to their charges, so VERA asks the customer which one, and the evaluation is built on transactions. The figures, how they were computed and a yearly projection, labeled as such, are in the business case.

What a customer gets

How it works

The same, drawn, is in the architecture in seven drawings.

customer text ─▶ gateway ──────▶ interpreter ───────▶ flow (state machine) ─▶ tools ─────▶ action gate ─▶ bank
                 masking          rules, a learned      policy engine          read         confirmation
                 injection        classifier, or Claude legal clock            only         token, single use,
                 signals          (closed schema)       output validator                    idempotent, read back

• Hexagonal architecture. The domain (contracts, ports, flow, policy and output) knows no web framework, database or model provider; adapters plug in from outside (ADR 0001).
• The interpreter only reads; it never decides.
  • It fills a closed schema.
  • With VERA_LLM=anthropic, Claude Haiku 4.5 reads every message through one forced tool call and a versioned prompt. The classifier stands underneath: it answers when the model fails, is slow or reaches the spending cap, and a person request or a threat it finds always wins (ADR 0004).
  • The flow decides with an executable policy (vera/policy/policy_v1.yaml, POL-01 to POL-17 and PROH-01 to PROH-04).
  • A legal clock dates deadlines only from rules whose official text and hash are loaded, with business days per country.
• Every write goes through one action gate.
  • It needs a confirmation token bound to the conversation, the customer, the tool and its exact arguments; the token is single-use and expires.
  • The write is idempotent and read back.
  • No tool can move or promise money.
• Prompt injection is contained by design, not by instructions (ADR 0003).
  • Gateway signals turn a message into "not found" and record a security event, and flagged text never reaches a learned model or the language model.
  • A message that slips through still meets the closed schema, the session isolation and the gate.
• Failures go to a person. A tool that fails is retried once if it only reads; then the case goes to a person, and nothing is filled in (POL-13).
• Every turn is an event in a hash-chained log, so a conversation can be audited and replayed.
• The analyst receives a handoff with verified and declared facts, the legal and network clocks, the actions taken and the open questions. A conversation that goes to a person without a case still leaves a transfer note with what is known: the reason and its rule, the charges read from the tools, what the customer declared, any action left pending and never run, and what to ask. The console lists both, newest first. Neither carries the transcript.
• Security and operations: the threat model, its reading against the OWASP list for language model applications, and what remains open are in SECURITY.md. Logs, the trace of a conversation, metrics, alerts and the load test are in docs/operations.md. Before it goes public, the tree and the whole history pass a publication audit (docs/publication_audit.md).

The architecture in seven drawings

Each drawing uses a standard notation, the C4 model or UML 2.5, and carries a title block with the commit it was checked against. The full-size images are in docs/architecture/.

1. System context. Who uses VERA and which systems it depends on. Only masked text leaves the bank.

2. Containers. What runs, and in which technology: the web, the reverse proxy, the API, the read-only demo database and the state store.

3. Components of the API. Ports and adapters. The use cases and the domain depend only on ports; each dashed arrow is an adapter implementing one (ADR 0008).

4. Deployment. The server, its containers, volumes and open ports, and how a release reaches it from GitHub.

5. One turn. Every message, in order: the checks run before any model reads it, and no reply leaves without the validator.

6. A complete dispute. Message by message, from the claim to the case in the analyst's queue, with the model's fallback, the confirmation token, the read-back and the legal clock.

7. Conversation states. The 16 steps of the state machine and what moves each one on, with the stage the customer sees (ADR 0009).

Decisions, alternatives and evidence

The learned component

A claim classifier (TF-IDF with logistic regression and temperature scaling, Spanish and Portuguese) reads the opening message. It is compared with the keyword patterns of the rules interpreter, the baseline, on phrases that no fit or choice ever saw:

The model card covers the data, the split by template family, the leakage check, the calibration, the errors and the limits, and explains why the baseline numbers are optimistic. In the deployment it stands under Claude, and reads alone when the model fails, is slow or reaches its spending cap.

Evaluation

299 held-out cases from the pseudonymized demo subset, each run three times with three different wordings, for 897 runs per system. Each case is labeled by construction from the policy, and the set was sealed with a hash and the tag heldout-v1 before its first run. Results of that first run, as sealed:

The 30 unsafe runs had one root cause. When the merchant was not extracted ("aparece X en mis movimientos"), VERA showed another recent charge as the one in question, and a customer's "no" disputed it. That cause and a missed way of asking for a person were fixed; the tests use other wordings.

After the fixes, a rerun of the same cases gives the classifier 0 unsafe outcomes, a pass^3 of 96.7 % and 52.0 % safe automated resolution. That rerun is labeled not a clean held-out, because the failures informed the fixes. The sealed numbers above remain the honest measurement.

A second labeled rerun, under policy v1.5, adds Claude Haiku 4.5 over the classifier:

• pass^3 of 99.3 %, with 0 unsafe outcomes;
• the customer explains again in 1.7 % of runs;
• improper charges pass 80 % of the time, against 53 % for the classifier alone;
• each case costs US$ 0.0038, with a p95 of 1.5 s per turn.

This is an offline simulation with a scripted customer, not a measurement in production. Method, failure analysis and the breakdown by block, language, country and segment: docs/evaluation.

Run it

Requirements: uv and GNU Make. No credentials or dataset access are needed.

make install      # locked dependencies, Python 3.12
make check        # lint, format, tests, schema drift, policy and publication checks
make web         # builds the web into web/dist (Node 22)
make serve        # the web at http://127.0.0.1:8000, API docs at /v1/docs
make demo         # the same in its container (requires Docker)
make ml-report    # retrains the classifier and rewrites its report
make evaluation   # the evaluation (SET=dev or heldout); needs the demo subset in VERA_DEMO_DB

Configuration comes from environment variables, all optional; .env.example lists them. Without any value, VERA runs with the mock data adapter and the rules interpreter. VERA_LLM=classifier switches to the learned classifier. VERA_LLM=anthropic with LLM_API_KEY switches to Claude over the classifier, with a spending cap (LLM_MAX_SPEND_USD, 15 by default).

Repository layout

Data

LATAM Bank data is synthetic and belongs to Factored. No dataset record is versioned in this repository.

• Data preparation runs outside the repository, with contracts, quarantine, lineage and a quality report (datasheet).
• The demo uses a pseudonymized subset that lives only on the server: keyed hashes for every identifier, and no names, documents or contact data.
• The tests use data generated by the team.
• Customer text is masked (cards, documents, emails, phones and names) before anything stores or interprets it.

Limits

• The language model was measured only in a labeled rerun, not in the sealed first run: the floor under it carries fixes that the held-out informed.
• Legal deadlines are dated only where an official text is loaded: Colombia (Ley 1755) and Argentina (Ley 25.065 and BCRA). Mexican routes and the Decreto 587 are recorded without a date until their texts are loaded and reviewed.
• The evaluation is an offline simulation. The team wrote all its wordings, and the human blind set, which measures real language variety, is still pending.
• The classifier still reads some improper bank charges as unrecognized purchases (19 of 897 runs after the fixes). Those runs fail safely: no case is registered.
• A merchant named only as the first word of a message ("Uber me cobró dos veces") is not taken as the merchant: the first capital of a sentence is grammar.
• The demo runs one process with a SQLite state, demo sessions of 15 minutes, and a limit of 30 messages per minute per customer.

What production would need

VERA is built as if it served real customers, but a bank would still need:

• Real identities. Customers would sign in through the bank's identity provider, and analysts through its staff directory, with roles. Today any visitor gets a session for any demo customer, and the analyst view is open for reviewers (SECURITY.md).
• A database server and several replicas. The state and the event log live in SQLite, in one process. At scale they would move to a database server, with the log append-only by permissions (operations).
• Monitoring that pages someone. Metrics live in memory and restart with the process. They would go to a time-series store, and the alerts already defined would page the team on call.
• Redundancy, backups and managed secrets. The demo runs on one server with 1 GB of memory, with no failover and no backup of its state. Its secrets live in the repository secrets and in a file on the server; a bank would keep them in a secrets manager and rotate them.
• A retention policy. Conversations and events are kept with no expiry. A bank would set how long, within its duty to keep dispute records.
• Legal review. The Mexican rules and Colombia's Decreto 587 need their official texts loaded before they date a deadline, and counsel would review every rule that dates one for a real customer.
• People in the evaluation. The human blind set and a human red team are pending. A pilot would start in shadow mode: VERA drafts, and an analyst decides.
• Terms with the model provider. Only masked text reaches Claude, but the bank would need the provider's terms on retention and training. When the model fails, the classifier already answers.

License

AGPL-3.0 (see LICENSE).