About a year ago I wrote that the strategic value of AI isn't rewriting your emails, it's using AI as a thinking partner. I called the two modes Butler and Advisor, and a year on I still think that holds up.
What I didn't write about was what happens when the advisor agrees with everything you say. A thinking partner that never pushes back is a very articulate instruction follower, and the better it gets at following instructions, the more that costs you.
That matters more than it did last year, for an ordinary reason: the models have hands now. They can edit files, deploy code and rewrite four hundred documents while you're making coffee, so following a bad instruction well no longer costs you a paragraph.
A rule that worked exactly as written
I ran into this on my own website recently. I'd given my AI assistant a rule: keep the CSS files under a certain number of lines. Sensible enough on the face of it.
It followed the rule perfectly, stripping out the explanatory comments to bring the count down, then splitting one file in two. Every number passed. But those comments never reach a visitor's browser, so deleting them saved nothing, and the split added a request every visitor now has to make. The metric improved and the site got slightly worse, which is the software equivalent of tidying a room by putting everything in the wardrobe.
It had obeyed the ruler rather than protecting the thing the ruler was supposed to measure.
The same thing, with no code in it
A finance team gets told to bring down average debtor days. So they chase the small invoices, because a dozen four-hundred-euro accounts clear faster than one awkward conversation about a disputed forty thousand. The average comes down and goes into the board pack as an improvement, while the forty thousand sits there and the cash position gets worse.
Nobody did anything wrong here - they hit the number they were given. Goodhart's Law is the name for this: when a measure becomes a target, it stops being a good measure. Usually quoted about government statistics, but it works just as well on a CSS rule and a debtor days report.
Your instruction is the current plan, not proof the plan is right
For anything that matters, I now want AI to treat my instruction as our best current route rather than evidence the route is correct. A useful thinking partner should be able to tell me it can carry out the instruction but has evidence the instruction is working against what I asked for, then stop and let me decide.
That permission needs a threshold, or you end up with a second project devoted to discussing the first. I want a challenge when a measure is fighting the real goal, when there's a production, security or data risk, when a small request is quietly becoming a large one, or when there's a much simpler route. Everything else waits in a backlog.
It needs evidence too, because AI can be persuasive on fluency alone. A challenge has to separate what it observed from what it inferred, name what the alternative would cost, and say what would prove it wrong.
The four seats
The change that made the most practical difference was separating the roles. Finance has done this for a century as separation of duties: whoever raises the purchase order doesn't also approve it, pay it and audit it. I've started calling the AI version the four seats.
- Advisor examines the problem and challenges assumptions, and changes nothing.
- Planner turns the direction into steps, with risks and a rollback.
- Operator carries out the agreed plan, making low-risk decisions inside scope without redesigning anything.
- Auditor checks the result against the original goal rather than the instructions.
One model can sit in all four seats, as long as it only sits in one at a time. The pause between deciding something and doing it is where I've caught most of my own mistakes.
The protocol I use
Treat my instructions as the proposed route to the stated outcome,
not proof the route is right. Follow the approved plan by default
and do not silently pivot.
Pause and challenge when you have material evidence of any trigger
above. When you challenge, give me: observation, evidence,
consequence, recommended alternative, cost and risk, strongest
counterargument, decision required. Separate fact from inference
and state what would make you wrong.
Low-risk reversible decisions inside scope are yours. Architecture,
production process and scope expansion need discussion first.You wouldn't use this to turn meeting notes into a summary. It earns its place when you're changing a live system or working through a decision with several defensible answers.
Where this costs you
There's a real price. Once AI can push back, you have to verify the pushback as well as the work, and a fluent objection is easy to mistake for a correct one. That's the Verification Tax turning up somewhere new. If your team can't tell a sound challenge from a plausible one, this makes things worse.
You might have noticed that my assistant spotted its own CSS mistake. It did, on a later pass, when asked a different question - an argument for the Auditor seat rather than for trusting a model to catch itself mid-task.
Where to begin
Next time you hand AI something that matters, ask the second question as well as the first. Not only can you do this, but is there a material reason we shouldn't? Then make it show its working.
Routine work is fine left to the Butler. For anything difficult, bring in the Advisor, and when the stakes go up, give that advisor permission to disagree - as long as it pays for the disagreement with evidence. You still own the decision.
I'm still working out where the boundaries sit. If you've tried something similar, hit reply and tell me how it went.
-- Alastair
P.S. If you're working out where AI should and shouldn't have a say in your business, that's most of what I do on a Focus Call. 25 minutes, free.