Confidence calibration and decision threshold
Illustrative exampleEach extracted field gets a confidence. Above the threshold the system accepts it on its own; below, it reprocesses or sends it to a person. The ideal threshold balances the cost of reviewing and the cost of being wrong.
Variables
- τ*optimal confidence threshold
- crcost of human review of one field
- cfcost of letting an error through
Objective
Minimize expected cost: reviews performed plus errors that get through.
Constraints
- Confidence must be calibrated: 90% confidence should be right about 90% of the time.
- Critical fields may require a higher threshold or always-mandatory review.
Practical example
With c_r/c_f = 0.1 (illustrative value), τ* = 0.90: whatever has confidence above 0.90 passes; the rest goes to the operations desk.
Business implication
The team reviews only what matters, with an explicit rule.
Assumptions and limitations
The costs and the confidence distribution in the tour are illustrative and must be measured.