BriefPilot
in developmentsince 07.2026Photograph a German official letter and it reads the text back to you. That is the first step toward telling you, in plain language, what it means and what to do.
- 01 Upload
- 02 Preprocessing
- 03 OCR
- 04 Quality gate
Interactive walkthrough of the shipped pipeline. A real screen recording replaces this at the next milestone.
field: problem
The problem
German official letters are stressful and easy to miss deadlines on.
field: constraints
Constraints
- Solo
- Zero-budget
- Privacy of personal mail
- No GPU
field: architecture
Architecture
field: decisions
Three decisions
Provider-Agnostic LLM Architecture Under a Zero-Budget Constraint
- Chose
- Designed a provider-agnostic AIService interface (dependency-inversion pattern) with three interchangeable adapters (Gemini, OpenAI, Azure OpenAI), and set the free-tier provider (Gemini Flash) as the hard default. No paid API is ever called unless a developer explicitly opts in.
- Rejected
- Defaulting to the paid OpenAI adapter with the free tier as opt-in (the original scaffolding); renaming the interface to llm_client to match the spec literally.
- Cost
- Every future AI provider (Anthropic, a self-hosted model) must now implement three abstract methods (extract_document, classify_document, summarize). That is a stricter integration bar per provider, accepted in exchange for one single contract that defines what an AI provider means across the whole app.
- Why
- “A missing API key should never break the product. It degrades one feature (classification returns null, never a guess), not the whole pipeline. Cloning the repo and running it locally should carry zero risk of an unexpected bill. This is cost-aware architecture: the safe default has to be the free one, engineered so the failure mode of "no key configured" is invisible to the user rather than a crash.”
Terminal-State Modeling Over Boolean Flags
- Chose
- Modeled OCR quality as a third distinct terminal job state (low_quality), sitting alongside done and failed, each with its own frontend render path (TypeScript discriminated union). When quality fails, the app withholds the raw OCR text entirely instead of displaying it behind a confidence-score disclaimer.
- Rejected
- A quality_ok: boolean flag bolted onto the existing done result; a "show anyway" toggle that surfaces low-confidence text with a warning label.
- Cost
- Three terminal states to model instead of two, and more frontend branches to maintain. Each one is exhaustive by construction, since a discriminated union cannot be silently half-handled the way a boolean flag can.
- Why
- “failed means the pipeline broke (an engineering problem); low_quality means the pipeline worked exactly as designed and still produced output too unreliable to act on (a user problem, retake the photo). Conflating the two blurs two different remediations into one ambiguous signal. Showing "confidence: 20%, here's what we think it says" trains users to trust unverified output. That directly undermines the product's core trust claim that everything shown is provably grounded in the source document.”
Confidence Capping Over Zeroing or Silent Pass
- Chose
- Built a source-span linking layer that matches every LLM-extracted field back to its literal word-level bounding box in the original OCR output. When a value cannot be matched to any source span, its confidence score is capped at a fixed ceiling. It is never raised, never zeroed, and never left untouched.
- Rejected
- Zeroing confidence on an unlinkable value (overclaims it's wrong, when it may simply be paraphrased); leaving the model's original confidence untouched (lets an unverified 0.9-confidence claim look exactly as trustworthy as a verified, bounding-box-linked one).
- Cost
- Introduces a third confidence state to design and test against: "wrong," "right but unproven," and "verified." It also needs a placeholder threshold value that requires real-data tuning once production fixtures exist, rather than a clean binary pass or fail.
- Why
- “This extends the project's existing deterministic-validator rule, that failures downgrade confidence and get flagged instead of silently passing, to a new failure mode: "we could not verify this." In a product whose single differentiator is click-to-highlight provenance, letting an ungrounded claim wear the same confidence badge as a verified one would quietly break the entire trust story the UI is built to tell.”
field: hard_part
The hard part
The hard part so far has been coordinate integrity. Tesseract returns word positions in the coordinate space of the preprocessed image. The browser shows the original photo, scaled to whatever viewport the user has. Every bounding box has to survive that round trip: preprocessing transforms, OCR, JSON serialization, and CSS scaling, and still land on the right word. Getting this wrong by a few pixels quietly breaks the product's core promise, because a highlight pointing at the wrong word is worse than no highlight.
The second hard part was the quality gate threshold. Too strict, and usable phone photos get rejected. Too loose, and garbled text reaches the user with a straight face. Tuning it meant deliberately taking bad photos and deciding, case by case, where "readable" ends.
field: testing
Testing and guarantees
- ✓ CI runs lint, type-check, test, and build on every push.
- ✓ mypy --strict across the backend.
- ✓ Typed contracts between frontend and backend (TypeScript discriminated unions on job states).
- ✓ The quality gate is the first trust guarantee: unreadable input is rejected with retake guidance rather than rendered as unreliable text.
No accuracy numbers exist yet. None are claimed here.
field: roadmap
Roadmap
LLM structured extraction behind the existing adapter.
why: The adapter contract already defines it.
Deterministic validators on extracted fields.
why: Hallucinated deadlines are the worst failure mode in this product.
Click-to-verify source highlighting.
why: The word-level bounding boxes already shipped exist to power it.
field: role
My role, and where AI assisted
I designed the architecture and made every decision in the ADR log. Claude Code works as a pair programmer under a rule I enforce: it posts a plan and waits for my approval before writing code. No plan, no code.
I review every diff before it merges. Each sprint, I implement at least one component fully by hand, because reading generated code is not the same as being able to write it. mypy --strict and CI catch what my review misses. Where the model was uncertain about a value, I made the system return nothing rather than a confident guess. That decision was mine, not the model's.