Tax automation. Human sign-off.
Built for a CPA firm: a pipeline that reads source documents, computes return values, and preserves the evidence a preparer needs to review the result.
Automation with a review boundary.
The problem
Move information from varied tax documents into a return without losing the connection between a value and its source.
What was built
Document capture, vision-model extraction, a deterministic tax engine, form fill, and an audit chain.
What remains human
The CPA checks values, resolves exceptions, and signs every return. Extraction can still be wrong even when an output is well-formed.
This case study describes the shipped architecture and supported scope. Client performance figures and confidential records are not published here.
Supported scope
- Client
- A CPA firm (name withheld)
- Scope
- Build: capture, extraction, tax engine, form fill
- Returns
- IRS Form 1040 and schedules; California 540 and 540NR
- Automated inputs
- W-2, 1098, 1099-INT, 1099-DIV, 1099-R, SSA-1099 (core boxes)
- Model slot
- Vision-language model evaluated against the extraction schema
- Sign-off
- The CPA reviews every value and signs
Every number has to be right — and provably so.
Tax preparation runs on paper: wage statements, interest and dividend forms, mortgage statements — scanned, photographed, or downloaded. Every number on a return has to be exact, and a preparer has to be able to show where each one came from.
Four boundaries. One model call per page.
Capture
Every page becomes a normalized image, whether it arrived as a scan, a phone photo, or a PDF. This pipeline uses page images rather than relying on an embedded PDF text layer.
Extraction
One vision-model call per page. The model reads the page image against that page’s extraction spec — exactly the fields the page carries — and emits typed claims. No transcription step, no second model.
Engine
A first-party, deterministic engine derives each line of the return from those claims. Every rule cites the IRS or California FTB instruction it implements.
Fill
Engine outputs are written into the return. No language model takes part in the fill step.
Claims ledger — every claim stored with its provenance
Audit chain — replayed by a verification gate that fails on any unsourced value
CPA review — every value checked, then signed
Missing data is recoverable. A wrong number isn’t.
Each claim passes through a deterministic validator. Impossible values are rejected; absent fields, unresolved labels, and cross-field mismatches go to CPA review with a named reason. A plausible but incorrect extraction can still pass a structural check, which is why professional review remains required.
Before the model runs, the system decides which extraction spec applies from the upload’s checklist slot, and checks the printed tax year against the engagement year. A mismatch goes to review, not to the engine.
Every number has exactly one source.
A value on the return is a captured claim with page-image evidence, a provided claim from a structured source, or an output the engine computed from claims. That provenance is the spine of the system, not telemetry: a verification gate replays the chain and fails if any value is unsourced or any link is broken.
The model is a replaceable part.
The vision-model slot is designed to be replaceable. A candidate must produce the required schema and be evaluated on representative documents. The evaluation harness measures what changes when a model is swapped; schema compatibility alone does not establish equivalent accuracy.
A person signs every return.
The CPA reviews every value, resolves anything the automation could not, and signs.