Running now · AI · human-in-the-loop triage
SORTHOUSE
An email and document triage lab with schema-validated extraction, field-level evidence, a policy outside the model, an approval queue and idempotent actions.

Try it here
Open the photographed invoice, inspect the low-confidence field and its evidence, correct it, then run the prompt injection scenario in the Failure Lab. The lab interface is in English.
The problem
Automating a mixed inbox is only safe when the system can tell when it lacks evidence and never repeats an action after a retry or timeout.
What it does
Open a synthetic inbox of invoices, a customer complaint, a request for quote, spam and an ambiguous message. Each message passes through real pipeline stages: sanitising, classification, extraction, validation, policy and review. Every extracted field links to its evidence: a quote in the email, a page of a PDF or a region of a photographed invoice.
The decision to automate is made by a policy outside the model, from deterministic checks. Invoices and customer replies stay approval-only; you can approve, correct or reject, and a correction keeps the original prediction in the audit. Approved actions go through a transactional outbox, so retries, duplicate deliveries and a timeout after sending are reconciled instead of repeated.
The public demo runs a deterministic baseline. LLM adapters are pluggable; reported baseline metrics are not LLM results. By default the classifier and extractor are a deterministic heuristic baseline and OCR, malware scanning and actions are simulated, all labelled in the interface; adapters for real language models exist but are not what the public demo runs. Sessions are temporary and isolated with two-hour retention, and the prompt-injection defences are layered, not a claim that the problem is solved.
The public repository includes a synthetic corpus, a tuning set, 122 evaluated cases: 100 held-out emails plus 12 adversarial and 10 failure fixtures, and measured results with their limits. On that set the baseline made 0 false automatic actions and blocked 16/16 unsafe actions, but it is a small synthetic corpus and says nothing about real inboxes or real models. This is an owned demonstration, not a client deployment.
What is in it.
- 01Classification and extraction with field evidence
- 02Deterministic validation and confidence gating
- 03Approval queue: approve, correct or reject
- 04Transactional outbox with timeout reconciliation
- 05Prompt-injection and failure injection lab
- 06Isolated sessions and measured evaluations
Built with
- Laravel
- Alpine.js
- PostgreSQL
- Redis / Horizon
- Provider adapters
- Playwright