Rise of AI Conference 2023 · Berlin

No one should trust AI

Joanna Bryson · Professor of Ethics & Technology at Hertie School

At Rise of AI 2023 in Berlin, Joanna Bryson argues that no one should trust AI and that AI should instead be built to be auditable. She defines trust as a relationship between peers who choose not to micromanage each other because they expect fair treatment, illustrating this with the economic trust game, where a society's trust rises and then collapses once partners start defecting. Robots, she says, are not peers: they are designed and owned artefacts, best understood as parts of corporations embedded in people's homes, so societies build governance structures, such as the EU, to keep those corporations in line rather than trusting individual machines. She walks through intelligence as a subset of computation constrained by time, space and energy, and stresses that AI depends on physical infrastructure, not only data and algorithms, pointing to Google's fibre networks and chips. She then reviews EU rules including the GDPR, DMA, DSA and the AI Act, arguing that most of what they require is documentation practice already familiar from 1980s software engineering, so responsibility can be traced when something goes wrong.

Key takeaways

  • Bryson defines trust as a peer relationship that only develops when people have both real choice among partners and imperfect information about them.
  • She argues robots cannot be peers because they are designed, owned artefacts, best understood as parts of corporations placed in people's homes.
  • Intelligence is a subset of computation that generates action from an agent's necessarily partial view of the world; total knowledge is impossible.
  • AI depends heavily on physical infrastructure, such as Google's own fibre networks and chips, not only on data and algorithms.
  • She contends the EU AI Act mainly requires documentation comparable to 1980s software devops, so accountability can be traced rather than demanding perfection.

Detailed notes

Trust as a relationship, not a property of artefacts

Bryson opens with the economists' and psychologists' trust game: one person can give money to a stranger, who receives it doubled and can return any share. She reports that people trust unknown partners who behave fairly but reject offers from partners who keep a disproportionate share, even when accepting would leave them better off. A simulation she and colleagues built shows trust in a population rising once individuals copy the most successful, fair-returning strategy, then collapsing when others start defecting for short-term gain, before rebuilding. From this model she draws a further point: trust only develops when people have both real choice among partners and imperfect information about them; perfect information (as with Uber, where riders are tracked and paid) removes the need for trust and replaces it with certain knowledge.

Why robots cannot be our peers

Turning to why societies tolerate hierarchy, Bryson describes how subordinate monkeys stay near dominant ones because the cost of driving them off exceeds the benefit, producing near-equal peer relationships that let social species build shared structures. Robots, by contrast, are designed and owned, which she says makes a genuinely peer or consensual relationship with them impossible, drawing a parallel to how modern ethics now treats relationships between owners and the people they owned as inherently non-consensual. She argues the more accurate way to think of robots is as parts of corporations placed in people's homes, so individuals cannot treat them as peers directly; instead governments and structures like the EU are built to let whole corporations interact with other powers, such as the US and China, more or less as peers.

Intelligence as constrained computation, and infrastructure as power

Bryson stresses that computation is a physical process requiring time, space and energy, so omniscience is impossible; she recalls US generals fearing that deep learning would deliver complete knowledge, and their alarm on realising it would not. Intelligence, she says, is the subset of computation that generates action from an agent's necessarily partial perception of the world. She then argues AI is inseparable from infrastructure, not just data and algorithms: Google operates its own fibre-optic networks and chips, and discovered its own government was hacking it despite already cooperating with that government, following the Snowden leaks. She calls the large technology companies significant transnational assets in this infrastructural sense.

Accountability, responsibility and transparency

Bryson argues societies disagree about who counts as a moral agent or moral patient, and that this is a social choice rather than a scientific fact. She holds that responsibility is the defining property of moral agents, that accountability is the ability to trace responsibility when something goes wrong, and that transparency is simply how accountability is implemented, not a goal in itself; open-sourcing code is neither necessary nor sufficient for it. She rejects designing AI to have interests of its own, arguing robots cannot be jailed and that people should worry about animals such as cows and rats, whose experience is far closer to human feeling, rather than about robot welfare. She also dismisses the Kantian argument that people must feel obliged toward AI because they identify with it, since many people do not identify with their phones or search engines.

EU regulation: documentation over hype

Bryson situates the AI Act alongside the already-enacted GDPR, DMA and DSA, describing it as a comparatively small addition covering matters such as valid consent, error correction and the right to know how a decision was made, most of which GDPR already covers. She argues the Act mainly asks companies to document who wrote a system and when, comparable to devops practice from the 1980s, and that most obligations amount to disclosing that a system is an artefact, not a person. High-risk systems, she says, are those affecting outcomes like the UK Post Office and Dutch benefits scandals, where appeal routes are needed. She distinguishes documentation-based audits from consultancies' preferred "risk assessment," arguing audits should instead build on cybersecurity, since a system's authorship cannot be trusted if it is not cyber-secure. On bias, she argues that data reflecting real-world inequities, such as gendered depictions of housework, should not trigger a moral panic; the goal is auditability, not erasing lived experience from the data.

Q&A highlights

Asked whether documentation requirements risk making Europe fall behind, Bryson replies that biased outputs, such as a biased translation, do not require banning a technology outright; instead sellers of large language models should vet and document what biases they contain and how much they corrected, and application developers should communicate this to users so responsibility is distributed and understood. Asked about Amazon's hiring tool, she argues Amazon's disclosure that it caught and discarded a biased model was an ethics success, not a failure, and that Amazon simply handled the public relations badly. Asked how bias gets flagged, she says this depends on the product: some allow user reporting, while developers can choose to inherit a model's current state of bias correction, add their own filtering layer, or pass responsibility on to users, similar to how machine teaching complements machine learning.

Quotes

AI isn't something somebody tells you to trust. AI is something you audit.

Joanna Bryson

Trust is relationship. It's building up a community of people that you have expectations about that are probably not necessarily going to do the right thing.

Joanna Bryson

Robots are not our peers. They're designed and they're owned.

Joanna Bryson

Questions answered in this talk

Should people trust AI systems?

According to Joanna Bryson at Rise of AI 2023, no one should trust AI the way we trust human peers; instead AI should be designed to be auditable, so that responsibility can be traced if something goes wrong.

Why does Joanna Bryson say robots cannot be our peers?

Because, in her Rise of AI 2023 talk, robots are designed and owned artefacts, not near-equals capable of a consensual relationship, and are best understood as parts of corporations rather than independent agents.

What does the EU AI Act actually require, according to Joanna Bryson?

At Rise of AI 2023 she explains that it mainly requires documenting who built a system and when, similar to basic 1980s software devops practice, plus disclosing to users that they are dealing with an artefact, not a person.

Mentioned

AI Act, GDPR, DMA, DSA, trust game, moral agents, moral patients, accountability, transparency, devops

Keep exploring

More from Joanna Bryson

More from Rise of AI Conference 2023

More on Ethics & Trust

Transcript

Auto-generated transcript, not edited. · 5,941 words

Show the full transcript

Okay. Oh, wow. Wow. My my job is done here. My job is done. Okay, I can go. Maybe like two two opinions or or three like uh why do you don't trust an AI here in this role? All over there. I don't even trust In what? I said I don't even trust people Oh, you don't trust people. Okay. That's one point. Another one. Yeah? Uh-huh. Okay. One more. I think it's time I think it's time I think it's time that we find that out why we don't trust, right? So, I have Joanna Bryson here next to me and she couldn't be the better here on stage for this topic because she is professor of ethics and technology.

This is your stage. Okay, thank you. All right, yeah. I'm Joanna Bryson and I have way too many slides. So, I'll just go fast. If you don't understand me, you can raise your hand and tell me to slow down. All right. So, what even is trust? I think that's one of the questions we have to answer before you can answer whether you trust AI. And um there's a something called the trust game that economists and psychologists have worked out. And so, they have two people. One person's given money and then they're say there's another person, you're never going to meet them. Would you like to give them money? And if you do, then I'm going to give them twice as much money as I gave you and then they're going to be allowed to give any amount back to you.

Okay, so think. Would you trust that other person? Okay. So, it turns out that many people will trust an unknown partner. They'll believe that they're going to get more money back. All right, maybe three times more than so two times more. Sorry, I said that wrong. Right? But they'll believe that someone over whom they have no control they've never met will give them more money. On the other hand, if you tell them oh, they'll give you uh they're going to keep like 1.8 times more money that I gave you and they're only going to give you 1.2 times back, even though they would make more money, people reject that offer. They say, "No, I don't want to trade with someone who isn't fair.

Even if it would benefit me, I reject that. They don't get that money. That's my money." Somehow, right? We have this feeling. So, this is what trust really is. Trust is relationship. It's it's building up a community of people that you have expectations about that are probably not necessarily going to do the right thing. That's what trust is, okay? And so, my entire point of this talk is basically why that's not what we should be trying to do with AI. Okay. And just really quickly, if you think, "Well, that's crazy. Why would you trust it?" This is something where we simulated version of that and the blue line there is trust. So, the trust it and what the what's going on here is the different people in the society people dots in the society are copying the most successful ones.

And so, once people start giving a high return value, more than fair return value, then it makes sense to trust everyone, right? And then if those other people start defecting and not giving the return value, which they can do and they can make a lot of money in that brief period of defecting, then trust plummets, the game breaks down and then it builds up again. So, that's the kind of dynamic that we see in humans, all right? And we we just simulated a version of that. That's what the paper's about. But but now that we have a model of that, we can say what what does it take to get trust? And so, what's going on on the bottom of this graph here?

This is how many people you could choose between. And what's going on on the the Y axis there is how much do you how likely is it that you know how much they'll give back to you. Okay? And what you see is that if you have too much information, you don't need trust, okay? You don't need to trust. You can just pick whoever's going to give you the highest return value. So, trust doesn't grow in that society because you have perfect information. Similarly, if you don't have any choice, then people don't become trustworthy. All right? There's no reason to become the kind of person that gives a good return, right? This is the other side.

This is people that are being trusted. And notice that if they are if you have lots of choice and you have lots of information, they basically have to give you all the money back, right? So, that's actually a no a no trust situation. It drives me crazy when people say, "Why is it that now we trust everyone? We just jump in the cars of strangers." No, if you have Uber, you have you know that you're getting mapped where you're going, right? You have way more personal security than jumping into a taxi. You you know that you're they know they're going to be paid. Something like 1/10 of taxi rides do not get paid, right? But you've got two credit cards signed up.

That's not trust, that's knowledge. That's certain knowledge that almost certainly this this transaction is going to work, right? So, that's the difference. This is a different question. Sorry, I didn't get to this. I am British. You can tell by my accent, right? Okay. So, um why do we tolerate hierarchy? I don't usually talk about people. I like to talk about monkeys, it's easier. Okay. So, um there's a big question. Why do all these monkeys hanging around have to act subservient to the to the big monkeys, right? Well, the reason is because even if they get treated a little worse, maybe they don't get as have as much sex as they'd like to, at least they're with the others.

They know where the food is. There's some access, right? And in fact, this whole turn this this question that anthropologists they have eventually got turned on its head. They're like, "Well, wait a minute. Why are why are the big monkeys allowing these these little monkeys to stay around and get some of the food and the sex and stuff?" Well, they don't have time to chase all the other guys away, all right? It just isn't worth it. So, this is a relationship. Even if you say, "Well, these guys aren't equals," they're near equals. This is the kind of context where you can build up a society. And this is what we do. All social species, you know, chickens and stuff.

They all figure out how are we going to get together and we work on it because we're sort of peers. Even Elon, okay, kind of kind of a peer, all right? Robots are not our peers, all right? They're designed and they're owned. You know, I I was horrified the first time I heard. This is not how we used to think about consent in the '70s when I was in school. But now they say you cannot consent to have sex with your owner if you're a slave. So, all the children born out of whatever relationship that people may have felt like there was love or whatever, there's that is now considered uh statutory rape to have had sex with somebody you owned, right?

So, in the current thinking ethical thinking, this means that it's impossible to have a consensual relationship. You are not building a society with something that you've designed or something you've owned. That doesn't make sense. And um the other thing is robots aren't our peers. They just aren't. They're nothing like us, right? So, the right way to think about robots actually, and this is my main concern. I can talk about the ethics if you want, is that they're parts of corporations that we've stuck into our houses, right? They're not they're not people. So, um there's no way that we as individuals can treat an entire corporation as a peer. But we can build other structures such that they can more or less interact as peers.

That's what we've done with the EU, right? We made it so that all of us can sort of be peers with the US and China and India, right? Um so, similarly, we build governments to try to keep the comp the companies around us more or less in line with each other, right? So, I've just talked to you about trust. Uh I hope that wasn't too long. Now, I'm going to do intelligence, all right? First, I'm going to start with computation. A lot of people, I don't think people in this room, so I can go fast on this, forget that computation is a physical process. It takes time, space, and energy. It's not magic. You don't just suddenly know something, right?

That's why you have to plug in your computer. It's not just for the lights, okay? Um that omniscience is impossible. No one is ever going to perfectly know things. I have been in the rooms with American generals who thought that first of all, they thought deep learning was the algorithm and we were now going to have all the knowledge and then they realized we didn't. And they thought, "Oh, no. What if the Chinese find the the the the the algorithm before we do?" And it was just like literally it was one of the most terrifying rooms I've ever been in. Okay. So, so no, you you won't get a perfect understanding of the universe. We don't want that.

We actually want what what we as intelligent organisms usually want is a subset of that. So, intelligence is a subset of computation first of all. So, of all the kinds of computation there are, it's the kind that generates action. Now, this is one of those things that drives some people crazy because that's why I think the AI Act has very broad applicability cuz I'm saying everything. Yes, I don't care if it's an Excel spreadsheet. If it's taking an action that determines where your kid goes to school or what health care you get, then let's call it AI and let's make sure that we can check who wrote it, okay? Which is a basically all the AI Act does.

It's like a little bit of of like meta information about who wrote the code when. All right? So, intelligence agents only perceive a subset of the universe and then they represent a little bit of it to make enough of a prediction that they can do the next thing. We're all limited, right? And that includes with AI. Yeah, this thing about infrastructure, again, this has already been said enough. I can go super fast through this. Just don't forget, it's not just the data, it's not just the algorithms, it's also the infrastructure. Google, and I'm not sure they're the only ones, I just know more about them, they have optic fiber optic networks around the world, their own, right?

They have chip fab, they use their own chips, right? They found out that their own national government was hacking them when Snowden uh did his leaks, and they were really angry cuz they were already cooperating with that same government more than they wanted to, and they still got hacked, right? So, yeah, AI is much more than data or algorithms, and the tech giants are, I would argue, significant transnational assets, and I don't mean in intelligence, the other kind of intelligence, they have, although a bit, right? I mean that this is infrastructure, okay? So, going back to the definitions again, artificial intelligence is really easy to define if you accepted that previous definition. It's just all the intelligent stuff that somebody built, okay?

And then agents, a lot of people think, "Oh, I don't mean that, I mean some kind of agency." There's a lot of definitions of agency. These are not the only definitions, that's why there's this caveat at the top. But, if you think about, like for example, a chemical reaction, the things that change the world are typically called agents. The things we actually care about are the moral agents. Who is it that's responsible for doing something? And the other thing we care about is the moral patients, which don't have to be intelligent. For example, the the ecosystem is something we now think we have to take care of, right? And basically, you need to realize that we don't have agreement about who are moral agents.

We don't all agree about how old do you have to be to be an adult, how old do you have to be to consent to sex, how old do you have to be to fight in a war. These are things that that societies come up with. And so, this stuff is all stuff you could look up in a dictionary. This stuff was really controversial for a while, well, but some people are taking it for granted now. The the idea is that ethics itself is basically composed of who do you think is the moral agents and who do you think the moral patients are, and it is also society specific. Now, why people hate that is because they want to say, "But we're more ethical than we used to be."

That's fine, you just have to say on what metrics. So, like, we're more ethical because more people can work now, or we have lower infant mortality, or we, you know, we we have better health care or something. You can still say it, you just can't just assume there's a single set of ethics, you have to choose one. So, I would argue that responsibility is the fundamental, it's almost the definitional property of the moral agents, right? It's what we assign to each other to keep our societies together, right? That's what we're doing when we build a society, and again, this goes back to why I showed you the monkeys, besides that they're cute, right? This is what we have to do to build a society.

So, trust is, as I mentioned, a relationship between peers, where we're basically realizing we can't micromanage you, but we're going to let you go and do things, right? Accountability is your ability to to trace responsibility. So, if something goes wrong, who did it? And transparency is just how we implement accountability. It's not an end to itself, okay? Open sourcing your code, neither necessary nor sufficient. What we need to know, and this is all we want, again, from like the AI Act, is can we tell who it is that was responsible if something went horribly wrong, okay? So, uh yeah, I already talked about that, so I can skip that. Um again, you can't make the robots responsible.

I should check my own watch. I just realized I can't see time. There's no time Oh, yeah, I have 13 minutes, okay, great. Easy. I want to leave you some time for questions, though. Anyway, you aren't going to build something that cares about being stuck in jail, all right? You just aren't. And if you do, you can unbuild it, all right? So, just we're not going to throw the robots in jail, all right? That's smart law people that helped write that paper. Right. Uh do you care about how the robots feel? Who cares? Anyone care about how the robots feel? Anyway, don't worry that if worry about the cows first and the rats, because the rats and the cows experience much more like our feelings, you know, separation when you pull the calf off the cow so that she can have milk and stuff.

That's much more like what a woman being separated from her child is like than anything that's ever going to happen to a robot, okay? So, we just aren't going to build those things. We aren't going to architect the experience of AI. Even if we could make it suffer, that would probably be immoral given that we're going to own it, right? So, why are people even talking about that? As some people say, "Well, we have to feel obliged to AI because we because we identify with it." And they go back to Kant, okay? So, Kant was saying, "Oh, like, you know, I don't know how God feels about the moral status of dogs, but I notice that people who treat dogs badly treat people badly, so we can't we we shouldn't we shouldn't treat dogs badly."

Okay, I'm down with that. I'm down with treating dogs well, but that doesn't make sense in AI. If you think that's true of AI, you've forgotten that it's a that it's a artifact. The A is for artifact, right? That I think it's wrong to believe that we have to do this because, first of all, a lot of super intelligent things, as was just mentioned on the previous panel, people don't even identify with, right? They don't identify with their phones, they don't identify with Google search. But secondly, yeah, we could just fix it, right? If we know that, then we can And that's one of the things the AI Act is doing, as was correctly identified in the first keynote this morning, you have to say if something is an artifact, right?

You you have to make it clear so people know they're not talking to a person, they're talking to a thing, which is actually a huge challenge to most natural language processing stuff that's going on right now, because actually, there's like usually one person running like 20 sock sock, you know, I don't know, sock puppets, whatever. AI bots, they're sort of talking to you, right? But anyway, I had no idea that that was going to be a challenge, but then I did this Euro European chatbots, super interesting. Anyway, so, AI and law are both authored cultural artifacts. So, it's not a scientific matter about, like, is AI a moral patient or not, right? This is something we pick.

We pick both when we define what our moral patients are and by how we design the AI system. And I would argue, like, let's build it so that we don't have to complicate our law any more than necessary, right? So, all we can do as science is predict what the outcomes of policy is, and philosophy can sometimes discover logical fallacies, but basically, normative stuff, that's what we hire politicians for, and that's what we work as communities to discern. Okay, EU regulations. Remember I said this thing about that keeping corporations to a pure like scale was like what I antitrust law, competition law is about? This is all the rules This is one of my uh colleagues' collaborators, Helena Melchert, who uh Director-General Competition, put this slide together.

And I didn't even know about all these things. These are all the ones that she knows about being in the in the Director-General Competition that are AI-intensive legislation. And look, the DMA, the DSA, and the GDPR are already done. I almost feel like the AI Act is like this trial balloon for like letting Google and Microsoft shoot at, because again, there's so little in it. I can't believe how much fuss there's been, all right? Like, uh the things I think about, like, valid consent, um whether or not you can correct if there's an error, the right to know what how a decision was made about you. That's actually in the GDPR already, all right? What's the Sorry about that if you You can get the slides.

Just email me or Twitter me or something. I'll send you the slides. Okay. Aim to benefit the EU Oh, yeah, the the Digital Services Act is trying to make Again, all this legislation is about growing our digital economy. That's the whole point, So, if we want a safe, predictable, trusted online environment by making it easier to defend users' rights online. And so, the three things that really focuses on is profiling, recommendation systems, and targeted advertisements. So, I just mentioned also the GDPR, it's sort of the profiling, you're making sure that the the information about you is accurate. Recommendation systems um are, you know, I would argue that Facebook and Twitter were both better when when we actually worked to follow people that didn't talk too much, but talked about stuff that was interesting and weren't rude, than getting recommendations.

I think it's actually worse It's it's made the experience worse, and it's also given people In fact, Jack wrote a blog about this, and then Elon bought the blog site and took it down so you couldn't read it anymore. But Jack said, "The big mistake we made at Twitter was 2 years ago, we had an activist person on the board, and we started building more tools for other people to control our user experience, when what we should have been doing was letting the users control their own experience and helping them understand." But literally, they apparently hired some guys from Facebook who said, "If you're asking the users questions, you're losing. You should just learn this stuff."

And that's just wrong, okay? Anyway, recommender systems. Um yeah, and you know, on the other hand, there's nothing you can do but recommend with web search, that's what web search is. And and I kind of do get that it's kind of convenient to have some of the, you know, the trending topics stuff, that is recommender. Targeted advertising, again, some people claim it's not giving sales benefit, yet look at this. All the money that used to go to newspapers is now going to Google and and Facebook, right? So, I don't know what's going on there. I if it's the companies believe they need the information, or if other people are encouraging this information to be gathered, I don't know.

Um yeah, the AI regulation itself, it does this categorization about what we don't want to be, which is basically China, right? Um so, we're supposedly not going to do any biometric tracking or social credit scoring, although in the US, we already do, but of course the US is not part of the EU. Um, but but we don't have only one social credit score, we have multiple um credit scores. But anyway, the high risk is the stuff that affects human outcomes. So we don't want to have the kind of thing that happened in the UK with their post office scandal. We don't want to have that kind of thing that happened in the Netherlands with their benefits scandals.

There ought to be ways to appeal if there's a really bad decisions being routinely taken. If you can give evidence to that. So, you basically just need to document what you did. This is basic devops. This is, you know, if trust me, it was a big deal when we started innovating devops in the 1980s. Development and operations, keeping revision control, knowing who had changed the code when, and like maybe if we were lucky they put a note about why. This is basically all that AI Act is asking you to do. And I recommend even if you don't have a high risk system, that's really a good idea. We figured that out in software engineering in the '80s.

Why AI companies don't do that, I don't know. All right. And it actually isn't that expensive. Um, you do have to go and catalog where where you do this or whatever. Um, and most people, as I mentioned, only have to identify as as was mentioned by the bot in the first keynote. So, um I think that yeah, I would I would encourage people who aren't yet doing high risk AI to voluntarily comply with some of the documentation. It will just help you prove, limit your liability. So this is the big deal. The AI Act and the DSA both say we ought to be able to audit. And it's funny cuz some guy from EY said to me that I was working on a some space project with anyway, said, "You you say that they have all this information, but you know they don't."

I want to be clear about this. AI is easily auditable if you choose to do the right thing. If you do keep those records. It is also easily impossible to track what what was done if somebody throws that information away, right? And that if negligent. Okay? In any other sector, you would be liable like you people keep that information to prove that they followed best practice and so they don't have to be liable, right? And if and for those of you who work with people that have, you know, you know, real products, you know, the already got governed sectors, you know that, right? They're they're like, "Oh yeah, we spent a bunch of time doing those kinds of things."

See, a ordinary product law assumes that you can prove you've done due diligence. What is due diligence? It's what your sector says it is. You don't have to keep writing the law that keeps up with You just have to have trade magazines that say most of us know that, you know, for example, you you have a a duty of care for the the information you scrape from your users, right? And that you shouldn't just sell that, right? Maybe you use it for advertising, but you don't just sell it, right? And that's the kind of thing that was used in these prosecutions. Again, I loved the previous panel. Um, that that were done uh uh in the EU and also in Canada, but not in the US.

Yeah, so financial audits are the consultancy's main revenue sources. And so they're actually kind of nervous about getting involved in this and they can't do it for people that they're already financially auditing. And so they're trying to say, "Oh, we don't want to call it audits, we're going to call it assurance." Um But and they're talking about doing it on top of risk assessment. I think that's wrong. I think we should be starting from our cybersecurity audits because it's much, you know, first of all, if the system isn't cyber secure, then how can you possibly say that you're responsible for it? You need to know that you're the one who actually wrote the code or that your owner operator actually used the code.

That is the normal problem you have like with a car, right? You either you negligently built it or the driver negligently drove it if it smashes into a tree, right? Or there was an act of God or something. I I don't know. Okay. So I'm going to skip this. I'm running out of time and I want to be able to ask questions, but I I just want to say there were so many people worried about uh like, you know, like bad words in their in their data models. Look, what we known I I was one of the people in 2017, what you've known for at least 6 years now, is that if you learn about the world, you find out what it's like.

And and a lot of those things are bad biases, right? Like the stuff about women being more likely to be uh doing the laundry or something, right? It it's going to replicate lived experience and that's okay. I don't think we need to throw that out. We need to make something that's easily auditable. The reason that that corporations don't like having four steps like this is because uh it's a little slower, right? But but they're they're I was just at some thing with World Economic Forum, people are starting to say maybe it's worth paying a little bit of time penalty, maybe taking one or two hops to have something that's clearly auditable and so that we can tell, for example, um you know, what what what was done in the in the language model and what's being done at the app level.

Just making that a little clear, not selling gluing those pieces together. All right. So anyway, the whole thing is the translator don't have a moral panic about what's in the language model. Do other things to do whatever you decide fair was. All right. Okay. So yeah, I already said this. Everything in a digital system is actually easily transparent. You just have to keep track in transparent to the level that you need for accountability. You just need to document all the stuff about where you got your data, where you got your software libraries. You may say, "Well, that's a pain." It's a pain we haven't done before, but SolarWinds happened two chains back in the software library.

If you want to be responsible, if you're doing if you're playing games, fine. I don't care except as long as your user data is bleeding out to the internet or to China or something. But if you're if you're writing a medical application, spend a little extra money buying buying your software libraries, okay? Being sure or doing an open source one that you're really really, you know, it's been very carefully validated, right? So all of this stuff is not only helpful for if you ever get audited, there's a question there's capacity, but also it will benefit you when you go back and try to check what you were doing, right? I can't believe C-suites aren't demanding to be able to know what their systems were architected to do.

So that's all I had to say. I just and this is my point. AI isn't something somebody tells you to trust. AI is something you audit. Okay. Thank you. I'm so I'm really sorry for tired I'm really sorry for tired people. I thought I had a half hour and then they told me I had 20 minutes. Are there any questions? Yes, over there. So, um I have a big fear as many other people have, which is that if we regulate AI in the way that it's currently designed to be regulated, that Europe will fall back a lot in not only AI, but also will fall back in industry and in any sector. So, um when we take the example of the translation, I I agree that this is of obviously a biased translation, but it just take common practice from the past from old books, old texts where these these roles of the role models were were like that, which are not now.

Um, but does it mean that you would say rather than having the the the the risk that this translation is biased like it is, um forbidding all translation systems that could possibly bias or not forbidding, but having the document and and fix this first and audited it first before using anything that could work like that. Okay. Translation is just an example Okay. which you gave. Right. So so the the answer to that question, first of all, we programmers tend to think like things have to be these weird absolutes. So I remember the trolley Remember the trolley problem? Lawyers lawyers are like, "Oh, that happens every time we put up a stop sign that, you know, there's some people that are going to be killed.

They're different than the ones who were going to be killed before. We just have to deal with that, right? It's not it's not the end of the world." So similarly about about um again, I was just at this this thing about uh governing uh generative AI. The um first of all, everybody is trying to vet and at least document what the biases in their system are and where it comes from. And I think people are getting a better understanding of the fact that there will be biases. And then you then what you certify as somebody who's selling a large language model or selling access language large language model is how much vetting you did, how much you you fixed it, whatever.

And then the application developers make sure that people understand that, too. So it is about not necessarily about having this perfect piece of software, but but more about, as I said, building up a standard of good practice and communication. And on this particular thing on bias, we're getting there. I think people are really starting to get that. I got to tell you guys a story. Everyone says, "Oh, what about Amazon because they, you know, they were hiring only male programmers." No, Amazon released that news story to show that how good it was that they checked and they noticed that their program said this and they didn't use it. Okay? That was actually an AI ethics win.

It wasn't an AI ethics lose and Amazon is just bad at PR. It's just really weird. But Um, but these things happen in social media. Um, I hope that I covered most of that question. And how can we how can we fight bias in this field? Oh, uh how do you flag bias? What people do is they like I said, they establish practice about what they look for it. Oh, how do we flag it if we see it? That depends on the product. So some products are allowing you to report, but mostly people are just saying uh yeah, that that it's it's it's something that be checked either at the application level or as I said the model level and then when you're an app builder, you choose whether you want that responsibility.

You can pass it on to users and say you guys know we didn't do bias on this, right? Or you can pass it on be back to you and say we only want an LLM that has the current state of the art bias, you know, erasing and we don't care it's a little slower or whatever. Or you take care of it yourself. You just add a little layer in there that looks for like, you know, gender stuff or whatever and you and you have a little bit of extra work and you slow down your app a little tiny bit. So it's it's like this what we've heard before, not the machine learning but then the machine teaching kind of thing.

It It's the machine being used to measure. I still don't like to ever talk about when you say the machine is doing something, you're giving it agency. So it's what we do with the machines and what we can do is is actually measure and improve our society. Yes. And I mean in the machine learning process we also teach the system, right? With the false back thing. Okay, but we need to continue. Thank you so much. Sure, you're welcome.

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.