Agent Evaluation

Purpose

Show how confidence, accuracy and cost interact: the reliability diagram, the expected calibration error and the confidence threshold that minimizes cost.

Variables

• pconfidence reported by the agent
• koverconfidence exponent: true accuracy is p^k
• c_r / c_fcost of a review relative to the cost of a mistake
• τthreshold: accept when p ≥ τ, review otherwise

Assumptions

• 1,000 synthetic predictions generated from the seed.
• With k = 1 the agent is calibrated; with k > 1 it is overconfident.
• A mistake costs 1 and a review costs c_r/c_f; correct accepted outputs cost nothing.

Model

Data this would need in practice

• Real predictions with confidence and ground truth.
• Real costs of a review and of a mistake.
• Enough samples per confidence bin.

Interpretation

With k = 1 the formula and the empirical best threshold agree. As overconfidence grows, ECE rises and the empirically best threshold moves above the formula, because accepted outputs are wrong more often than their confidence says.

Limitations

• The agent is simulated; real systems fail in more ways.
• ECE depends on the number of bins.
• Costs are assumed constant across fields.