This is an archive episode, first broadcast in 2024 under the show's previous name. It carries a short synthetic announcer introduction; the conversation underneath it is untouched human audio.
You can train an AI on data containing no bias at all and still end up with a biased system. Understanding why is the difference between a working deployment and a reputational problem.
Tom Murphy works in AI ethics and opens by saying that every talk he gives needs a content warning. He spends most of the episode on failures that have actually happened rather than on ones that might.
Recorded in December 2024, when the show carried a different name. It is the last episode of the archive backfill.
What this episode covers
- Alignment, without the science fiction. What the word means in practice, and why Murphy points out you cannot verify a human's alignment either.
- Explainable AI and black boxes. Where the line sits between a system you can interrogate and one you can only observe, and why testing guardrails turns into whack-a-mole once people start trying to escape them.
- Proxies. The heart of the episode. Remove gender from the data and the system finds it again through titles, car colours, postcodes. Bias does not need the protected field to reappear.
- The perfect dataset that still became racist. His worked example, and the reason it is more instructive than any of the famous public failures.
- How to test for subgroup bias. Concretely: hold a profile constant, change one attribute, compare outcomes across a range of tests. This is the part to write down.
- An exchange rate for people. The phrase he uses for what these systems end up encoding when they weigh one group's outcomes against another's, and why it is worth sitting with rather than resolving quickly.
- Baseline performance and your reputation. What to measure before deployment, so that you can demonstrate afterwards that you looked.
- The trolley problem and four hundred data scientists. What happened when he put it to a room of them, and what it revealed.
- The cancer drug distribution dilemma. The hardest question in the episode, and one nobody in it pretends to answer.
- "I just want to use ChatGPT." Whether any of this applies if you are not building anything, and Murphy's answer about what your AI says in your voice.
- What to tell your staff. His practical guidance on keeping a human in the loop, and why it has to be a real check rather than a signature at the end.
- Cleaning your data before you build. What that involves, and the protected traits nobody thought to protect until a system started using them.
About the guest
Tom Murphy is an AI ethics specialist who works with organisations on bias, alignment and the governance of deployed systems. Alastair has known him a long time, which shows in how quickly the conversation gets to the difficult material and how little of it is hedged. Murphy opens by saying every talk he gives needs a content warning, and by the halfway mark it is clear why.
Find him at TomMurphyAI.com and on LinkedIn.
Resources mentioned
- GDPR and protected traits - the legal floor, and the traits nobody thought to protect
- The tank detector - the classic blue-skies-and-grey-skies training failure
- The jewellery field that predicted car crashes - a proxy nobody would have looked for
- Human in the loop - what Murphy tells staff, and why it is not a formality
Quotes
"You have to be ensured that you have run at least a modicum of testing for subgroups."
Tom Murphy
"You can't verify a human's alignment either."
Tom Murphy
Books - the Human-First AI Series
- Hello! I'm New Here
- An Absolute Beginner's Guide to AI
- Use AI, Stay Human
- The AI Pilot Handbook
- When AI Becomes Your Responsibility
Connect with Alastair on LinkedIn.