AI Ethics and Hidden Bias, with Tom Murphy

How this transcript was made

Transcribed by machine from the published audio, then corrected by hand. It is not a verbatim record: the changes below were made, and are counted.

This episode

60 speaker turns, 172 changes.

• opening voiceover replaced with the authored script (match 98%) and attributed to Max; 11 words of episode audio that shared the turn were kept
• closing CTA identified as CTA 3 - Resources (match 98%), replaced with the authored script and attributed to Max

The timing in the .vtt files

Each speaker turn's start time is measured by the transcription engine, at 1-second resolution. Cue times within a turn are interpolated by character count, because a third of turns run past 30 seconds and the longest is 644 seconds - one cue per turn would leave an 11-minute block highlighted. Every turn boundary re-anchors to a real measurement, so interpolation error cannot accumulate down the episode.

What was changed in these transcripts, and what was not

Generated: 2026-09-20 from 2026-09-20-named/, itself transcribed from the audio that is live on the feed. Every change below is mechanical and counted. No sentence was reworded, no claim altered, nothing removed for being awkward.

The editorial decision

Owner's choice, 2026-09-20: light clean. "Um", "uh" and stammered word repeats ("and, and") are removed, because nobody intends to say them. "You know", "I mean", "kind of" and "sort of" are KEPT - they run to roughly 2,300 instances across the catalogue and stripping them would make the page read as somebody else.

Counts across the whole set

Change	Count
stammered repeat removed	667
um / uh removed	654
space before punctuation	241
split contraction	132
guest turn named (Joe Casabona)	101
organization -> organisation	71
diarisation fragment returned to the host	62
ai -> AI	57
empty turn dropped (was filler only)	36
synthetic outro restored from script	21
organizations -> organisations	18
hyphen followed by space	14
realize -> realise	14
A.I. -> AI	10
recognized -> recognised	9
gray -> grey	9
summarize -> summarise	9
color -> colour	9
behavior -> behaviour	8
synthetic intro restored from script	8
analyze -> analyse	7
behavioral -> behavioural	7
recognize -> recognise	6
percent -> %	5
personalized -> personalised	5
skeptical -> sceptical	5
analyzing -> analysing	5
center -> centre	5
optimization -> optimisation	4
realized -> realised	4
favorite -> favourite	4
realizing -> realising	4
canceled -> cancelled	4
recognizing -> recognising	4
announcer turn relabelled from Guest	4
summarizing -> summarising	4
labor -> labour	4
behaviors -> behaviours	4
colors -> colours	4
organizational -> organisational	4
customized -> customised	3
analog -> analogue	3
optimize -> optimise	3
honor -> honour	2
optimizing -> optimising	2
summarizes -> summarises	2
personalize -> personalise	1
prioritize -> prioritise	1
prioritized -> prioritised	1
organize -> organise	1
rumor -> rumour	1
organized -> organised	1
capitalize -> capitalise	1
centered -> centred	1
organizes -> organises	1
emphasize -> emphasise	1
split name repaired (Schaefer)	1
humor -> humour	1
traveling -> travelling	1
defense -> defence	1
maximize -> maximise	1
Organizations -> Organisations	1
apologize -> apologise	1
digitized -> digitised	1
monetize -> monetise	1
analyzed -> analysed	1
narrator opener re-attributed to Max, words left as transcribed	1
theater -> theatre	1
specialized -> specialised	1
split name repaired (Wadsworth)	1
aluminum -> aluminium	1
specializes -> specialises	1
skeptic -> sceptic	1
customize -> customise	1
personalization -> personalisation	1
optimized -> optimised	1
visualize -> visualise	1
spoken domain -> humanspark.ai	0

What was deliberately left alone

• "Program". A computer program keeps that spelling in UK English; only a broadcast or a scheme is a "programme", and only a reader can tell which was meant.
• "License" and "practice". Both have a noun/verb spelling split in UK English that a rule cannot decide.
• Capitalised words mid-sentence that would otherwise have been respelled: 4 of them, most often part of a name. Left as spoken.
• Retired show and brand names. Spoken by the host and guests. A transcript that removes what was said is not a transcript.
• "Co-pilot". Sometimes Microsoft's product, sometimes the metaphor. A rule would corrupt the honest ones.

The synthetic segments

The opening voiceover on the archive episodes and the closing call to action are read by Max, the show's AI narrator, from a script written in advance. Transcribing them means guessing at words already on file, and the guesses were wrong in places - one opener came back as "an AI that boned him up" where the script says "phoned him up". Those passages carry the authored text instead, and are attributed to Max rather than to the guest: the diariser groups Max with the non-host speaker, which would otherwise put a real person's name on words they never said.

MaxYou're listening to Something About AI.

The example from this one that tends to stay with people is a facial recognition system, trained on a perfectly balanced and carefully curated set of faces,

which still learned to ignore an entire group of people, because discarding them made the headline accuracy number go up.

Tom Murphy also explains why a postcode is rarely just a postcode, and what a military model was really looking at when everyone believed it could spot a tank. Here's the conversation.

Alastair McDermottTom Murphy is a powerful tool. It can solve many complex

Business problems, but there's also a dark side. We can accidentally reveal biases or create biases. We can destroy our own reputations if we misuse it.

There's so many things that we need to be aware of that are not really talked about because we're all very excited about the amazing possibilities of AI.

But in today's episode, I'm going to speak with an AI ethics expert, somebody I've known a very long time, a very smart guy.

And we're going to talk about some things that you might even find mildly terrifying. So that's coming up next. And today, my guest is Tom Murphy. Tom, you're very welcome.

Tom MurphyOh, thank you. Thank you for having me.

Alastair McDermottSo just for context, I was trying to do the maths. I think I've known you for about 28, 29 years now. It would be around that time.

So when I was talking to you about what we could talk about, some of the things you mentioned were kind of mildly terrifying, I think is a fair enough way to describe that.

So I just want to mention that up front in this podcast, just so people know, you know, this will be like an adults-only conversation possibly.

So with that out of the way, let's just talk about AI and ethics and biases and that kind of dark side that we

Tom MurphyMentioned.

Alastair McDermottCan you just give me kind of like a general introduction to that topic so people know what we're talking about?

Tom MurphySure. I mean, I always put a content warning on all of my talks and I say, look, because this is about bias and ethics, and we by necessity have to talk about topics that are problematic,

that are difficult. And, you know, people want to talk about safety, AI safety, and they think it's about like Terminator-style robots.

But really, it's more about how we can do damage to ourselves, to our society, by misusing AI in various different places,

and not being aware that there are these concerns, that there are these biases that can sneak in, that can creep up on businesses particularly.

And when you get into it, then it doesn't sound as scary, and then actually it gets scarier and scarier until people start saying, please stop, Tom.

Alastair McDermottYeah, and like, I think that, you know, when we're very excited and we see, you know, these tools able to do some amazing things, we get caught up in that excitement.

And when you're not aware of just how biased they can be or the negative implications, I think we do need that wake-up call as well, and people do need to be aware of these

Tom MurphyThings.

Alastair McDermottSo that's why I'm happy to chat with you today.

One of the things I want to ask you about is the word alignment in the context of AI and AI safety, because quite often we see the AI safety teams called the AI safety and alignment team.

And I think that it's a word that people aren't really used to hearing in that context. So that might be a good place to start to kind of introduce the concept.

Tom MurphySure, yeah, and alignment is really just, is the AI int ending to do the thing that we think it's intending to do?

So when we train for, let's say, a game playing AI that's going to play Mario Brothers, and we want it to get as many coins as possible,

and it heads to the right, and it picks up coins as it goes, and it appears to be doing what we're intending it to do.

Then if you take the coins out and you put them to the left, it will continue to go to the right. What it's learned and what its alignment is to go to the right. That's all it learned.

It didn't think twice about the coins. We projected that onto it in the sense that we thought it wanted the coins, but actually it just wanted to go right.

And when you put the coins somewhere else, it doesn't bother with the coins. It continues to go right, and then we discover, oh, look, this is a misalignment.

What we thought we wanted it to do and what it actually wanted it to do were two entirely different things.

And that's okay when you're talking about a game where you're going to get a low score. It's not okay when you're talking about the distribution of medicines, the driving of cars, et cetera.

Then we have to be really clear that its goal alignment is the same as ours. Its goal is aligned with our goals. And that is non-trivial. In fact, it's probably impossible.

So what's worrying the AI community at the moment is that we can never be certain of goal alignment.

Alastair McDermottWell, that's, again, mildly terrifying. So what do we do in this scenario?

Tom MurphyHow do we deal with that? Well, there are various different things. There are two kinds of AI. There's explainable and unexplainable, inexplicable.

So we can often be required to use explicable AI that we can explain why it made every decision along the way.

And therefore, we can have some sort of insight into it and say, look, it chose this route because A was bigger than B. And it chose this route because C was bigger than D.

So in this instance, it went that way for these reasons. And that gives us some internal working viewpoint on the AI.

But then the more powerful models like the neural networks, large language models, et cetera, that are built on top of neural networks, they're very black boxy as it's referred to.

So they're inexplicable. You can't say, look, because of these things, the AI made these decisions. So how do we align?

How do we check the alignment of a neural network, a large language, say, neural network? And one thing is testing.

So you give it many different scenarios and see how it responds in each of these scenarios. And that's a blanket kind of generic whack-a-mole test.

And if you can think of a lot of scenarios, then you can test for a lot of instances. But, of course, you can never think of them all.

And there will be something that immediately will come up in the real world that you don't have in the test model.

We've seen this again and again with escaping AI where we think we've constrained them or we think we've got them on guardrails. And then something happens and they jump those rails.

And so in terms of explicability within neural networks, you can say, look, there's this piece of data.

And if I keep everything else the same and I tweak this piece of data, how much does it change the outcome?

And so you can do that for all pieces of data, keeping all data the same and tweaking that one piece of data. And that gives you an idea of how sensitive is it to these

Alastair McDermottPieces of data.

Tom MurphySo how much is it paying attention to those? But that's quite a crude metric. So the fact of the matter is that we don't really know and we probably won't really know.

One way I often calm people's fears about that is that you also don't know what my alignment is.

So when you ask me to do a job, I appear to do the job you asked me to do, but you don't know that I'm not planning or plotting or scheming or a coup at the same time.

So you have no more insight into my alignment than you do into an AI's alignment. The difficulty is that AI works at scale and at speed.

So if I'm a bad person, you can constrain me in some way, shape or form. If there's an AI that's a bad AI, it's going to go wrong very quickly and it's less easy to control. Yeah.

Alastair McDermottOkay. So that's alignment and some of the problems that go with it. What other things should we be thinking about?

I'm thinking particularly from the perspective of business leadership, trying to look at AI and how they can implement it.

Not wanting to wander into this quagmire or, well, I guess we're kind of in this quagmire already.

Tom MurphyYeah. The way to not wander into this quagmire is don't use AI because once you engage with AI, you have to engage with safety.

We have to engage with bias and ethics and balance and issues around that. Because if you don't, then you're trading irresponsibly really at that point.

There are requirements in GDPR to be able to explain decisions that your systems are making. There are requirements to be fair, to not bias subgroups.

So gender, race, age, sexual orientation, religion.

All of these are protected pieces of information about you, sort of private information and somewhere banned from being used in AI in general.

Some industries have exceptions to that, like the insurance industry is allowed to be ageist.

It's a very weird case, but they're allowed to say, no, you're too old for insurance or you're too young and you get expensive insurance.

But depending on your industry, you're going to have to engage with this to ensure that any AI you're using is not biased, is not picked up traits from your history.

Because what people fail to remember is that AI is just the crunching of all previous history you have, right? So if you feed it biased data, then guess what?

The AI is going to learn biased behaviours. And it thinks that that's what you want because that's what you fed it.

So it'll learn these biased behaviours and repeat them at scale and repeat them uniformly.

So if you are training an AI on your past data, for example, insurance data, which is an area I've worked in before,

then you need to be sure that your past data isn't biased itself or else you're creating a biased AI, a bigoted AI. You need to not feed it things like gender, but AI is very perceptive.

And when it's training, it learns very quickly to latch onto things that are kind of like gender, but not really.

So when you feed in things like a colour of car, it very quickly latches onto the fact that pink cars and mauve cars are driven by women.

And it can predict based on the colour of the car, the behaviours of the driver. And maybe we don't want that to be a proxy for gender.

So that's a really tricky question then is like, well, it turns out the AI thinks that cars which are pink will crash slightly more often,

but have slightly less expensive crashes when they do, which is the behaviour that we see if we analyse by male, female. Very slightly more common, very slightly less expensive crashes.

Is that something we wanted to pick up on? Is that biased in terms of male, female and gender? Or is it okay to say, well, look, it's just looking at the colour.

We know behind the scenes the colour really means gender, but nobody says that that's wrong. So there's a real gap in the legislation for covering things like proxies.

It could also pick up on things like Mr. and Mrs. You know, when we feed in the prenum, it picks up on that.

So we have to isolate those out and not put in Mr. and Mrs. We just put in married. Married makes a difference as well. So we want to keep the married bit, but not the gender bit.

We're allowed to use the married data, but not the gender data. So yeah, so it's really easy for them to become quite biased over time if they're fed biased data.

So if what you're feeding into it is biased data.

Alastair McDermottSo I'm just thinking of other proxies. So postcode, for example, could be a proxy either for income or for race. All sorts of other kind of proxies that we could have in our data.

Let's bring this back to, again, to that business perspective. So what does this mean? Why is this important and what does this mean in terms of how we approach using these systems?

Tom MurphySo what it means is that you have to test to see that your installation of an AI is not doing anything like this.

You have to be ensured that you have run at least a modicum of testing for subgroups.

So you test everybody who, you know, a profile which is one gender and then keep exactly the same change of gender.

Ensure that these people are not being treated differently or significantly differently across a range of tests.

So that's for you to ensure for yourselves that your AI is not doing something crazy when it gets down to specific sub groups. And that might be a combination of things.

One of the examples I use is because people say to us, oh, well, why don't we just feed it an unbiased data set? Like, let's just give the AI, let's train it on proper pure data.

We'll data engineer a training set that's perfect. And that's fine. Okay, so we do that. And let's say it's on something like facial recognition.

You've got a business that's working on AI in the home or some sort of home automation. It's a big area at the moment.

You've got a business leader who wants to put AI into this and recognise people's faces, etc. A very common use case now for AI.

So you feed it a data set which is perfectly balanced in terms of diversity.

So it has the right number of Caucasian, black, Mexican, Asian, men and women of varying types and body sizes, etc. Diligently manage that. And you train on that.

And you tell the AI, get better at recognising people's faces. That's your instruction for your alignment. So it does. And it trains.

But you train multiple AI because that's how you train AI is that you repeatedly train until it improves. And often varies different kinds of AI.

And suppose one of those AI decides that, hey, or discovers that if I throw away all the data relating to Asian women of colour, it's a very small subset of the data set.

But it's difficult for the AI to train on, to get a proper read on them.

And it finds that, oh, if I just completely ignore these people and not attempt to identify them at all, I lose half a percent of the data set.

But I'll get 3% better at the main chunk of the data set, say Caucasian men and women. So the AI has done its job. It's better at recognising people now. And that's what you asked her to do.

You said get better at recognising people. And it did. On a perfectly unbiased data set. So here's a case where we handed an AI a perfectly good data set.

We gave it what seemed like a perfectly reasonable alignment and validation system. And it still became horribly racist and sexist. So what the hell? How do we stop that, right?

So the way you stop that is that you employ a data safety and ethics expert to point this out to you. But then you start to train and you test for subsets.

So you say, look, you can't just get good at people. You have to get good at all of these people simultaneously.

So all these different groups of people, I want to see at least a base level of achievement on all of these. And that raises its own problems.

Because now you say, well, look, I'll trade you half a percent off of black Asian women if I can have 3% more in Caucasian men. All right. Or you might say, well, no, I don't do that trade.

What if I get half a percent better at black Asian women, but it costs me 5% of my performance in another area or it costs me a quarter percent?

You know, you're starting to get an exchange rate for people. And that's just nightmare fuel for ethicists because now you're saying these people are twice as much as those people.

And you're starting to get back to like three-fifths of a vote. And it's all getting a little dark now and we don't really want to go there.

But what you need to do is as a business leader be saying, look, these are my customers.

I want to be able to achieve at least a baseline level of performance for all possible types of customers that I'm selling it to.

And within that, try and make the best decisions that we can in terms of where we plant our flag. Because you don't want to be Dell, right? You don't want to be, I think it was Dell.

I don't want to, I don't want to be defamatory towards Dell. But I think it was their AI recognition laptop that didn't see black people. And that's just an absolute disaster.

Like they don't want that. That wasn't their intention. And no marketing person wants to hear that your product has been found to not see Mexicans or something. You know, it's a nightmare.

So as a business leader, this is business reputation stuff that you don't want to let out the door. Yeah, I mean, it's fascinating.

Alastair McDermottAnd it just reminds me of, you know, the original trolley problem, which is, you know, there's this escape train or something.

Tom MurphyAnd you have a switch.

Alastair McDermottYou can't stop the train, but you can switch it going from

Tom MurphyOne track to the other.

Alastair McDermottOn one track, there's five people.

Tom MurphyOn the other track, there's one person.

Alastair McDermottAnd, you know, you have the power to choose. You know, do you choose the power of five over one? What if the one is a child? What if it's an old person?

You know, all of these different factors. What if the five people are five people you don't like?

Tom MurphyWell, I did this. I did this case with them. I gave a talk in Galway to about 400 data scientists. And I asked them at the start.

I said, how many of you consider yourself to make logic-based decisions? Like that you make decisions based on your own logic. And almost all of them put their hands on and said yes.

And then we went through the trolley problem, et cetera. And some really fascinating stuff kind of bubbled out of it.

But at the end of it, I said, you know, we're back to the one person versus five. But the one person is your child or your spouse or your parents.

It's somebody who's really close to you in the family. So now what do you do? And almost all of them are like, kill all five of them. Like toast. Just drive on. Like no problem.

So I pointed out and say, well, look, this is, this is, you all said at the start that you were, you know, logic-based people.

And you'd like to think that you were logical about your decisions, et cetera. But the truth of the matter is that logic comes into play when emotion has no say in it.

Like if emotion doesn't care, emotion looks at it and goes, oh, yeah, no, nothing in here for me. I don't care about this. Yeah, logic, you do your thing now.

So, but a second that emotion goes, hang on, I really like this person. I'm in love with this person or this is my child. It's like, oh, yeah, logic can get lost.

We don't care about logic now. So emotion trumps logic, you know, and logic will let, you know, emotion will let logic do it like so long as emotion has no interest in the question, right?

So, yeah, that happens repeatedly when I give talks to people.

Alastair McDermottLet's, let's talk.

Tom MurphyOkay.

Alastair McDermottSo let's just talk at a very nuts and bolts level.

If the listener is a business owner who is saying, I just want to use ChatGPT and get faster at what I'm doing, does what we're talking about apply to them? Yeah, it does.

Tom MurphyBut it depends on their, on their use case.

So if it's like, hey, I want you to get ChatGPT to talk to my customers in a support role and I'll feed it a bunch of documents that I have here and I'll ask it to learn those documents and

get better at responding to questions around those documents, then fine. A, you're inheriting a lot of the guide rules of ChatGPT in the first place.

So they've got an ethics and safety team in there that are looking at this to see, is it doing what it should be doing? If it does something it shouldn't do, it'll jump.

There's a kind of a hypervisor will jump in and stop it.

And they said, supposed to, as we've seen recently with Gem ini, that doesn't always happen where the AI told the human that they should go kill themselves and that they were horrible

people. so you would expect though, that with something as matured as ChatGPT, that it is reasonably, safe from that point of view.

However, you are feeding it a bunch of new documentation as well. And then it's, you're going to let it talk directly to your customers.

So anything that talks directly to your customers, you probably want to be sure that it's saying the sorts of things that you want it to say. So again, we're back to the alignment problem.

Like, is it, is it doing and saying the things I wanted to do and say with my voice, because you're putting your logo over it, you're telling people that it represents your company.

And now it's speaking directly to your customers. That's not something you want to have, happen without review. So you need to go in and again, run some tests.

Like suppose you say that you're from Alabama, you don't want it to suddenly presume that you're black, right? which is a problem I run in with copilot and Microsoft copilot.

If you start giving a context, it very quickly projects from those contexts onto the people.

So if you say, Hey, I'm from Alabama and, and, you know, in the 1950s, I did this back, this, that, the other, and, you know, you give it a background to yourself.

It will make some, race decisions and race presum ptions about you.

so you want to be sure that if this is important to you in your, in your business case, in your use case, that you're testing for these sorts of responses.

and that it's not going off the rails or doing anything with something like ChatGPT, because you're in a constraint,

already trained, pre-trained AI with a safety team or checking it, you're probably fine.

You should still check, but you can rely on a certain amount of their work, to already have been done.

And, you know, if it does go completely mad or something, you've got a news article you can write about, but that's not really what you want.

So you want to test this system, and make sure that it's, it's speaking with your voice, what you want it to say.

Alastair McDermottAnd then if we, if we go to next, so one use case there is using, you know, ChatGPT and you're using it as a, as a business owner. You've, maybe you've deployed it, to your team.

They're all using it as well. what do we tell our staff about this? Like, how do we explain this to our staff that, you know, this is, this is something to watch out for?

Tom MurphyWell, first of all, like I said, have an ethicist or a safety person either on board or at least come and talk to your staff about it.

Because it is a, it is a dedicated area where you need to talk about, well, all the little possible things that you, that could go wrong that you need to be, to be aware of.

And it'll depend on the, on the business case in terms of staff get very concerned about AI because they think it's going to take their jobs.

And what you want to tell them is, look, this is a sidekick. And I very much like this kind of approach that we're, we seem to be trending towards of co-pilot and sidekicks, et cetera,

because what we need is in the ethics community, we call it person in the loop, right?

So you're not having an AI making a decision based solely on its own internal decision making processes. It's coming to the human going, you've given me this task. I've done it.

Here's my output. Is this okay, human? Like, have I, have I been a good AI? Am I a good being like, you are a bad human? I am a good being. Right.

so, you know, that's a much better way of installing AI is having human in the loop and having it check it. The human needs to be aware that, Hey, you know what?

You need to think about these things. They should probably be doing that anyway.

If they're, if their coworker is bringing them a piece of work and always seems to refuse female pilots or drivers, or, you know, that should be ringing some bells as well.

Like this isn't wholly an AI problem. it is something where you are required by law to make sure that you're not being, biased against these subgroups.

so I would love to say you should use the same practices that you use for your colleagues and your human, workers that you do for your AI, but they may also be quite poor.

So maybe improve your, reviews of those sorts of things and apply them to AI as well.

Alastair McDermottAre there any other practical steps, a business, leader or business, small business should be taking when they're, when they're looking at this

Tom Murphy? I mean, it depends if you're, if you're building your own AI from scratch, then you're really into the deep waters, right? So my previous company, we built AI for fraud.

We built it for, insurance pricing, insurance decision-making, whether we want to insure people or not, crash propensity projections, et cetera.

so in that sense, you're, you're working with historical data. You're taking the data on, you're cleaning it up. There is no ChatGPT involved in this.

It's a very specific, sorry to me, AI in those cases, if you're going to be a good business leader, then you, then you really need to take this seriously and start looking at subgroup

testing and bias testing and constructing tests specifically to try and find this and be able to repeat those tests time and again, because when you retrain, you need to retest.

so that's kind of the hard end of the, of the scale is if you're building your own, or if you intend to collect data, you need to be sure that that data is not biased.

If it is biased, then you need to take actions before you train with it. so when we looked at our data, it was very clear that there was some bias in there.

and the insurance industry in the UK is, I would say has a per history, from the bias side of things.

so the problem being that if you take that data, train on it, scale it, you're now in instant institutionalizing that bias and that scale.

so you need to take the actions of cleaning the data, having a look at the data first, ensuring that you've got somebody who's looking at distributions that don't have weird bumps in them.

So if you look at, well, at age, you know, you should have a nice curve for age. You should have a nice curve for male and female should match these sorts of things.

You can test in the data beforehand, then when you train your AI, then you test it for subgroup bias as well.

So, it is much more difficult if you're not using something off the shelf, but even if you are using something off the shelf,

you have to be sure it's your responsibility to ensure that your systems are not biasing against, subgroups and protected,

Alastair McDermottData.

Tom MurphyAnd just on that.

Alastair McDermottSo is that the major, is that the major area?

So, the subgroups you talked about, is that the major thing we need to be concerned about or are there other areas that we need to be concerned about?

Tom MurphyWell, the alignment issue is always one that's, that's, like, cause bias is our idea.

Like it's a human idea, like we don't like being biased against, gender or race or, you know, we have protected for sexuality, for, religion. We don't have a protection for eye colour.

And so you can, you can, you can make a system that said blue people never getting insurance. I hate them for whatever reason. I, it's kind of silly.

It's equally silly to race and gender, but it's not one that we have constructed before. AI doesn't really care about any of these subgroups, right?

So it doesn't care about gender, about race. It isn't ethically programmed to care about them. So it will quite happily bias against blue eyed people.

If it finds a reason in the data to do that, right? So when we're looking at alignment again, if it sees a

Alastair McDermottCorrelation, right?

Tom MurphyIf it sees a correlation, if it sees a pattern, it'll go, Oh, I'm going to do this. Do we care about that? Is that something we as humans care about? Probably not.

we might investigate as to why, like, why is this coming up? Right. And, but it's, it's, it's not a, it's not a concern to us. What would be a concern is if it's alignment is off.

So if what it turns out it's doing is something that we don't want it to do, or it has learned something that we don't want it to learn.

A classic example of that was when the American military, they're targeting AI for tanks. They showed them tanks. These are Russian tanks. These are bad ones.

These are American tanks in silhouettes. These are, these are good ones. Right.

So, and they had to learn these and they tested against it and it was fine when they went into, when they went into,

production, like when they went to actually physically testing it, it went haywire and it did not identify tanks at all.

what it transpired was that they had shown them tanks that they had photographed in Texas of American tanks, which all blue sky backgrounds and the tanks that they had pictures of from

Russia were all grey skies, dark, star skies, backgrounds. And what the AI had learned is blue sky, no shooty, grey sky shooty. So, you know, which is perfectly reasonable.

It was getting them right because it was identifying something that we thought it was, but actually it wasn't, it had learned something else entirely.

And our test was still showing us, oh, look, it's still getting it right.

It's still, it's still identifying that the American tanks, and it's still identifying that the Russian tanks correctly. but actually it's, it's not.

And when it got into the real world, the draw went completely AOR and it would just shoot or not shoot at things that, you know, whether depending on the weather.

So, this is alignment problem. This is AI safety.

And this is the sorts of things that you need to kind of think about if you're going to train your own AIs and release them into the wild is that the wild is called the wild for a reason.

Alastair McDermottIs there anything that has really caught you by surprise? you know, any kind of dilemma, sir?

Tom MurphyEvery day, like every day we'd be like, what's it doing now? Stop it, turn it off. yeah, I mean, there are various different things.

I mean, sometimes you would find something in the data that you just wouldn't think would be predictive and it is, a good example of that was we found a piece of data in our data set.

And cause remember you've got these huge data sets, the hundreds of columns, and you're looking to cut out the columns that you don't need because training on that many columns and that many

combinations is very expensive. So you want to find columns that have nothing to do with insurance and just throw them away.

But you need to be careful because you're biasing your data already yourself. And one of these columns was, has a jewelry insured in house? And we were like, right, that's great.

We can test for that. That's bound to be useless. And when we tested for it, we actually found it was predictive of car crash.

And we sat there and went, why would this be predictive of car crash? And the jewelry in the safe isn't driving the car, but it makes sense.

People who have to declare insurance reasons their jewelry inside their house have jewelry of, I think it's north of 10,000 pounds is the limit or 2, 000 pounds worth of jewelry.

So people who have lots of jewelry are slightly higher risk of having car crashes.

People who have lots of jewelry are wealthy individuals who perhaps pay less attention to the law or perhaps think it applies slightly less to them.

So they maybe run that yellow light just that little bit more often. And it's a tiny fraction, but multiply about 100 million records of data and you get a small predictive bump.

And of course, we're looking for those small predictive bumps. So we sat there and went, oh, this is just weird.

I mean, when it found the Mr. and Mrs. thing, we were like, why is it listening to prenume? Like, I can understand doctor and lawyer and stuff.

They would make sense, but we've got very few of those.

And then, of course, it was like, oh, well, you know, it's getting gender from Mr. and Mrs. So, you know, every day we would sit there and test for stuff and see what it's doing.

You know, just is it going to ever train? Like, is it ever going to stop training is another good question. So from a safety point of view, I mean, yeah, postcode was a big problem.

Like, there are postcodes in the UK that it does not like and does not want. And then you have to say, well, is that because there's a lot of crime there?

Is that because there's a lot of theft? Is that because it's a bad area of roads? Is that because it's learned gender or so I learned to race issues?

Is it the postcode at all or has it just associated the postcode with people being turned down? So people were being turned down previously.

And in the UK, back in the 80s and 90s, there was a horrible saying of never ensure somebody whose name began and end with a vowel, which is code for, you know, racist, bigotry.

So if this is built into the data, then and you learn off of this, then it isn't the postcode. It's just learned a postcode as a proxy for this sort of racism.

So these are things that they're difficult questions to answer because maybe it's a mixture of all of these things. Maybe it's racist, but maybe also this area has bad roads.

Maybe it's racist, but maybe this area also has a lot of cul-de-sacs or has a higher propensity for car theft. You know, like we found that cul-de-sacs and car thefts are related.

Thieves don't like to steal cars where there's a lot of cul -de-sacs because they don't know how to get out of the estate.

So yeah, there's all of these sorts of issues that we would often have to look at each other and go, well, are we? There is no law here. Like we're making this up as we go along.

Do we feel comfortable doing this? And that was generally what we came down to was like, are we as a company comfortable doing this?

Like if this was your mom being refused because of this, would you be okay with that? Because there isn't an ethical law to cover these sorts of situations, sorts of proxies and all of that.

So difficult.

Alastair McDermottYeah, I have a question from a listener reviewer, Jay Tars

Tom MurphyWell. Hey, Jay.

Alastair McDermottSo he asks, how do you work against pushback from the anti-

Tom MurphyDEI crowd?

Alastair McDermottSo DEI is diversity, equity, and inclusion. And DEI and bias go in AI, go hand in hand.

Tom MurphyThat's what he's saying.

Alastair McDermottSo how do you work against pushback from that?

Tom MurphyDo you get that? Yeah, this is a really good question and it's difficult to answer. And I'm going to be as honest as I can about this.

AI can be, like people say, oh, why don't you just give it a data set without any race information in it or gender information.

And I've explained how those can become quite racist and quite gender biased all on their own, you know.

So, but the kind of the anti-DEI crowd are saying, oh, just all, you know, AI is blind and we'll just train it on the data. But that in itself can cause issues.

A good example of that is there's a drug which works twice as effectively in Caucasian people for skin cancer as it does in people of colour.

So it's not that it doesn't work for people in colour. It's just that it works. It has twice the efficacy in Caucasian people. Now, it's an expensive drug.

So it's not possible for like the NHS or the Medicare or Medicaid to buy this for everybody and just blanket cover everyone.

So there's decisions to be made as to who distributes, who gets the drug. How do we distribute this drug?

And if you leave it to AI and you say, as we normally would, optimise for quality of years of life, so therefore a child will get the drug ahead of a person who's 80 because you've got a

predicted 70 years of quality of life. You know, somebody who's 70 is going to get it ahead of somebody who's 80 for the same reasons.

So the typical way we do it with doctors is they try and optimise for quality years of life.

We have this problem with organ transplants and donations, etc. You know, who gets the lung, who gets the kidneys?

So in this case, if you just let AI handle that, it will say, give all the drugs to the whiteys, right? Just like, there you go, because this is optimising on quality of life years, right?

That's going to save more people. And that's obviously not something we feel comfortable saying. It's like, no, black people don't get this drug now. It's all for white people.

However, that is the most efficient way to save the highest number of lives. And by giving some of the drugs to the black people, we are actually, like, kind of punishing the white people.

We are saving fewer lives, and overall the system will perform worse. So this is a difficult question, right?

Both of these sound like bad ways to go, and AI is being asked to make this decision.

Now, what normally happens is the doctors at the end points of all of these consultations, they have a mixture of clients,

and they tend to spread it roughly evenly between their mixture of clients, right?

Because maybe they have five vials of drugs, and they have, you know, three white people and two black people in this particular case, right? Then great.

And some of this drug is getting to people of colour. But when you centralize this into one AI, that's one AI looking over the landscape and distributing drugs as optimal as possible,

and it will improve number of years of life saved, then it becomes an issue.

Because now it will flood the market towards white people only for the drug, and none of this will get to the black people, right?

So again, we've got the same kind of problem where you need to go in and say, at least some of this has to get to each of these subgroups, because humans want it to be that way.

I know we told you different AI, and you trained properly, you were a good AI, but that's not what we want. We want it to be evenly distributed between people.

Now, we don't actually have kind of strong understanding and agreement between ourselves, even as humans.

If you could put a human doctor in charge of it, we don't know what to tell that human doctor to do, because some people will say, optimise for years, blind.

Some people will say, proportional shares based on, we don't have those answers already.

So it's a little unfair to say, oh, AI is bad because it doesn't make these decisions well, because we don't know what to train it to.

We don't know what to tell it to train to become better at. So it's not really its fault. And these are ethical questions we're going to have to face, like, ourselves.

So how do you work against the pushback from the anti-DEI crowd? I think that's not really an AI question. That's a cultural question.

It's a social question where we say, you know, you fight for diversity and inclusion because it makes you a better company. It makes you, like, non-mono-syllabic.

There's a word I'm trying to think of. You know, monoculture is always a bad culture. We have to be aware of issues around, like, subgroup bias in AI.

But there's a much bigger issue around how do we actually want things to work and program for that, you know, build for that. I hope that answers the question. Yeah, I think it does.

Alastair McDermottAnd, I mean, some of these moral dilemmas, and we don't have time to get into some of the more terrifying ones, thankfully.

We discussed some of those privately, and maybe I'll get you back on and talk about some of those more terrifying ones later. But those ethical dilemmas, like, they're real for humans,

Tom MurphyNot just for AI.

Alastair McDermottIt would be, like, the trolley problem is the classic. It's tough to figure out what to do in that scenario.

And then, like you're talking about here with medication, again, difficult questions. And I'm personally glad I don't have to make many of those because it really is tough, you know.

And they have real-world impacts on people, on people's lives.

Tom MurphySure. And, I mean, Tesla are already facing the trolley problem. Like, the self-driving cars are already facing the trolley problem.

And their answer, if you notice, is that at the point of car crash, they'll hand back control to the human, right?

So, the AI won't make that decision because they don't want to program it to make that decision. So, that's why they tell you to always have your hands at 10 and 2 on the steering wheel.

Because, at any point that the AI goes, I don't know what to choose here, it's going to go, hey, human, off you go. Good luck with that. I'm outy, right? I'm gone.

So, they're still not solving the trolley problem. They hand back to the human because the human can be defended in court.

And people can say, well, you know, they drove into the one person rather than the five or whatever.

But if they can show that there's code in there to say, kill this person rather than those three, then this person's family are going to have a court case against you.

So, yeah, that's an issue. And that's a live one, and they still haven't solved it.

Alastair McDermottTom, this has been fascinating. I just want to quickly mention, it's kind of a sponsor. It's actually my own book. It's called An Absolute Beginner's Guide to Using AI.

And you can get that online now at AIpower.biz.

So, if you're looking for help with getting started using AI, I won't take you through some of the terrifying problems that Tom has discussed today.

We'll leave that for the more advanced books. But, yeah, Tom, I'm fascinated by this. It's just such an interesting and complex area.

Where can people find out more about you and maybe see about bringing you in to talk to them and terrify their

Tom MurphyAudience? Yeah. I'm available with children's parties. Yeah, you can find me at TomMurphyAI.com or on LinkedIn. I'm sure you'll have – there'll be a link.

I'm going to point at various places. There'll be a link somewhere to my bio on LinkedIn. But TomMurphyAI.com is where I am.

Alastair McDermottAnd I'd encourage you to reach out to Tom because he's a very smart guy and he knows this stuff. Tom, thank you.

MaxThanks for listening to Something About AI. Everything discussed here, and a good deal more, is backed up by free resources over at humanspark.ai. No cost, no catch.

If this episode was useful, pass it on to someone who would use it, and subscribe so the next one finds you.