Risk & safety
The Verification Dial
How hard you check AI's work should track its stakes and its track record - not sit at 100% forever.
How bad is an undetected error, and would anyone catch it in time? For high-stakes work the dial is welded at 100% - the thousandth draft gets the same check as the first.
For routine work, start at 100% and earn your way down: every line, then every second one, then one in five. You are not trusting blindly - you are sampling, like an auditor.
A new kind of task resets you to 100%, however long you have trusted the same tool on something else. New task shape, new dial, back to the start.
One mistake in a spot check and you are back to checking everything, earning the reduction all over again.
Spot checks only work if they are honest: random, and on a schedule. If an output looks dodgy, checking it does not count as your sample - that is diligence, not sampling.
When to use When deciding how much of an AI's output to check - and when it is safe to check less.
Read the full post →