At Rise of AI 2018 in Berlin, legal futurist Helen Tung explores who is liable when autonomous machines cause harm, using three case studies. In a fictitious 2050 scenario, a person is told by an AI doctor that new medication will extend their life by fifty years, but that sensors predict a 99.95 percent chance of future cancer, heart disease and depression, triggering a higher insurance premium; the same person's autonomous car is then hacked, and after a crash an AI judge finds them fully liable for retaking manual control, a ruling later reduced to 25 percent liability on appeal to a human judge. A second scenario asks whether a robotic dog left 80 percent of an owner's will could hold legal entity status. A real case she describes involved two horse owners disputing which owner's horse a jockey could ride at England's Derby, resolved by judges the morning of the race. Tung then shifts to a more personal argument, invoking Clayton Christensen's theory of disruptive innovation and life-measurement questions, before leading a short guided breathing exercise and closing with a Wizard of Oz metaphor about people already possessing the intelligence, heart and courage that AI debates worry about outsourcing.
Key takeaways
- In Helen Tung's fictitious 2050 case, an AI judge initially rules a hacked-car driver 100 percent liable, but a human judge on appeal reduces liability to 25 percent.
- Tung argues predictive health sensors reaching 99.95 percent accuracy could be used under future EU rules to justify higher insurance premiums before a condition even develops.
- She raises open legal questions about whether an inherited robotic pet could hold legal entity status or be considered to have consciousness.
- In a real Derby horse-racing dispute Tung witnessed, judges ruled on which owner's horse a jockey could ride just hours before the race began.
- Tung argues that regardless of how capable robots become, people retain something machines lack: a human heart.
Detailed notes
A fictitious 2050 case: AI doctor, hacked car, AI judge
Tung sets up a 2050 scenario in which an AI doctor tells the protagonist that new medication will extend their life by fifty years, but that nanobot sensors predicting a 99.95 percent chance of future cancer, heart disease and depression will trigger a higher insurance premium under incoming EU regulations. On the drive home the protagonist's autonomous car is hacked; a caller threatens to drive it off a cliff unless bitcoin is transferred. The protagonist refuses, tries to retake manual control, and crashes. Facial recognition and satellite footage bring the case to an AI judge within six weeks, who rules the protagonist 100 percent liable, reasoning that nothing would have happened had they not taken control and that their own intervention caused the collision. On appeal, a human judge shows sympathy and reduces liability to 25 percent. Tung poses this as a question for engineers, economists, politicians and researchers in the audience: how would they decide such a case.
A robo-dog inheritance
Tung describes a second scenario: a robotic dog bought in 2018 that by 2050 fetches, delivers mail and acts as a bodyguard, to which its owner leaves 80 percent of their will, prompting family objections. She notes that people today already leave portions of their wills to pets, citing a real case in Australia, where an executor would typically set up a trust fund for the pet's lifetime. For a robo-dog, she raises open questions: whether it could hold legal entity status, what happens if it is leased for a hundred years and repeatedly upgraded by different companies, who then controls it, and whether it could be considered to have consciousness.
A real dispute over a jockey and a racehorse
Tung recounts a real court case she observed involving England's Derby horse race: a jockey who trained with one horse owner fell out with them and began training with a second owner's horse instead. The first owner sued to prevent the jockey from riding any horse but their own; the second owner argued the jockey was now training with him. The judges concluded the dispute was no more complex than any other case before them and ruled, delivering their decision the morning of the race, just hours before it began, that the jockey could ride the second owner's horse.
What robots cannot replace
Tung shifts to what she calls a more personal argument, describing a talk by Clayton Christensen, whom she calls the godfather of disruptive innovation, about being asked to work an extra Saturday, realizing the request conflicted with his family values, and later developing his theory of disruptive innovation after leaving that job. Tung uses this to argue that no matter how intelligent robots become, people retain something machines do not: a human heart. She leads the audience through a short guided breathing exercise focused on releasing a difficult emotion, then closes by comparing the search for answers about AI to Dorothy's journey in The Wizard of Oz, where the brains, heart and courage the characters seek turn out to already be within them, and mentions her involvement with the Moon Village Association.
Questions answered in this talk
Who is liable when an autonomous car is hacked and crashes after the driver retakes manual control?
In Helen Tung's fictitious 2050 case study at Rise of AI 2018, an AI judge initially ruled the driver 100 percent liable for retaking control of a hacked car, but a human judge reduced that liability to 25 percent on appeal.
Could a robotic pet inherit part of someone's will?
According to Helen Tung at Rise of AI 2018, people already leave portions of their wills to pets today, with executors typically setting up a trust fund for the animal's lifetime; for a future robotic pet she raises open questions about whether it could hold legal entity status or be considered to have consciousness.
What does Helen Tung say humans have that robots do not?
Helen Tung argues at Rise of AI 2018 that no matter how intelligent robots become, people retain a human heart, and she closes her talk by comparing the search for answers about AI to Dorothy's journey in The Wizard of Oz, where the qualities sought were already within the characters.