At Rise of AI 2023 in Berlin, Joanna Bryson argues that no one should trust AI and that AI should instead be built to be auditable. She defines trust as a relationship between peers who choose not to micromanage each other because they expect fair treatment, illustrating this with the economic trust game, where a society's trust rises and then collapses once partners start defecting. Robots, she says, are not peers: they are designed and owned artefacts, best understood as parts of corporations embedded in people's homes, so societies build governance structures, such as the EU, to keep those corporations in line rather than trusting individual machines. She walks through intelligence as a subset of computation constrained by time, space and energy, and stresses that AI depends on physical infrastructure, not only data and algorithms, pointing to Google's fibre networks and chips. She then reviews EU rules including the GDPR, DMA, DSA and the AI Act, arguing that most of what they require is documentation practice already familiar from 1980s software engineering, so responsibility can be traced when something goes wrong.
Key takeaways
- Bryson defines trust as a peer relationship that only develops when people have both real choice among partners and imperfect information about them.
- She argues robots cannot be peers because they are designed, owned artefacts, best understood as parts of corporations placed in people's homes.
- Intelligence is a subset of computation that generates action from an agent's necessarily partial view of the world; total knowledge is impossible.
- AI depends heavily on physical infrastructure, such as Google's own fibre networks and chips, not only on data and algorithms.
- She contends the EU AI Act mainly requires documentation comparable to 1980s software devops, so accountability can be traced rather than demanding perfection.
Detailed notes
Trust as a relationship, not a property of artefacts
Bryson opens with the economists' and psychologists' trust game: one person can give money to a stranger, who receives it doubled and can return any share. She reports that people trust unknown partners who behave fairly but reject offers from partners who keep a disproportionate share, even when accepting would leave them better off. A simulation she and colleagues built shows trust in a population rising once individuals copy the most successful, fair-returning strategy, then collapsing when others start defecting for short-term gain, before rebuilding. From this model she draws a further point: trust only develops when people have both real choice among partners and imperfect information about them; perfect information (as with Uber, where riders are tracked and paid) removes the need for trust and replaces it with certain knowledge.
Why robots cannot be our peers
Turning to why societies tolerate hierarchy, Bryson describes how subordinate monkeys stay near dominant ones because the cost of driving them off exceeds the benefit, producing near-equal peer relationships that let social species build shared structures. Robots, by contrast, are designed and owned, which she says makes a genuinely peer or consensual relationship with them impossible, drawing a parallel to how modern ethics now treats relationships between owners and the people they owned as inherently non-consensual. She argues the more accurate way to think of robots is as parts of corporations placed in people's homes, so individuals cannot treat them as peers directly; instead governments and structures like the EU are built to let whole corporations interact with other powers, such as the US and China, more or less as peers.
Intelligence as constrained computation, and infrastructure as power
Bryson stresses that computation is a physical process requiring time, space and energy, so omniscience is impossible; she recalls US generals fearing that deep learning would deliver complete knowledge, and their alarm on realising it would not. Intelligence, she says, is the subset of computation that generates action from an agent's necessarily partial perception of the world. She then argues AI is inseparable from infrastructure, not just data and algorithms: Google operates its own fibre-optic networks and chips, and discovered its own government was hacking it despite already cooperating with that government, following the Snowden leaks. She calls the large technology companies significant transnational assets in this infrastructural sense.
Accountability, responsibility and transparency
Bryson argues societies disagree about who counts as a moral agent or moral patient, and that this is a social choice rather than a scientific fact. She holds that responsibility is the defining property of moral agents, that accountability is the ability to trace responsibility when something goes wrong, and that transparency is simply how accountability is implemented, not a goal in itself; open-sourcing code is neither necessary nor sufficient for it. She rejects designing AI to have interests of its own, arguing robots cannot be jailed and that people should worry about animals such as cows and rats, whose experience is far closer to human feeling, rather than about robot welfare. She also dismisses the Kantian argument that people must feel obliged toward AI because they identify with it, since many people do not identify with their phones or search engines.
EU regulation: documentation over hype
Bryson situates the AI Act alongside the already-enacted GDPR, DMA and DSA, describing it as a comparatively small addition covering matters such as valid consent, error correction and the right to know how a decision was made, most of which GDPR already covers. She argues the Act mainly asks companies to document who wrote a system and when, comparable to devops practice from the 1980s, and that most obligations amount to disclosing that a system is an artefact, not a person. High-risk systems, she says, are those affecting outcomes like the UK Post Office and Dutch benefits scandals, where appeal routes are needed. She distinguishes documentation-based audits from consultancies' preferred "risk assessment," arguing audits should instead build on cybersecurity, since a system's authorship cannot be trusted if it is not cyber-secure. On bias, she argues that data reflecting real-world inequities, such as gendered depictions of housework, should not trigger a moral panic; the goal is auditability, not erasing lived experience from the data.
Q&A highlights
Asked whether documentation requirements risk making Europe fall behind, Bryson replies that biased outputs, such as a biased translation, do not require banning a technology outright; instead sellers of large language models should vet and document what biases they contain and how much they corrected, and application developers should communicate this to users so responsibility is distributed and understood. Asked about Amazon's hiring tool, she argues Amazon's disclosure that it caught and discarded a biased model was an ethics success, not a failure, and that Amazon simply handled the public relations badly. Asked how bias gets flagged, she says this depends on the product: some allow user reporting, while developers can choose to inherit a model's current state of bias correction, add their own filtering layer, or pass responsibility on to users, similar to how machine teaching complements machine learning.
Quotes
AI isn't something somebody tells you to trust. AI is something you audit.
Joanna Bryson
Trust is relationship. It's building up a community of people that you have expectations about that are probably not necessarily going to do the right thing.
Joanna Bryson
Robots are not our peers. They're designed and they're owned.
Joanna Bryson
Questions answered in this talk
Should people trust AI systems?
According to Joanna Bryson at Rise of AI 2023, no one should trust AI the way we trust human peers; instead AI should be designed to be auditable, so that responsibility can be traced if something goes wrong.
Why does Joanna Bryson say robots cannot be our peers?
Because, in her Rise of AI 2023 talk, robots are designed and owned artefacts, not near-equals capable of a consensual relationship, and are best understood as parts of corporations rather than independent agents.
What does the EU AI Act actually require, according to Joanna Bryson?
At Rise of AI 2023 she explains that it mainly requires documenting who built a system and when, similar to basic 1980s software devops practice, plus disclosing to users that they are dealing with an artefact, not a person.