At the Rise of AI Summit 2020 in Berlin, a panel hosted by KI Park Germany discusses how to move ethical principles for AI from theory into practice. The panel brings together Carla Hustedt, head of the Ethics of Algorithms project at Bertelsmann Stiftung, a member of Volkswagen's group board of management responsible for integrity and legal affairs, the co-founder of a privacy-focused data science company, and philosopher Marcus Gabriel of the University of Bonn, moderated by an AI consultant named Peter. Gabriel defines ethics as the discipline studying moral facts, and argues that 'strong AI' in John Searle's sense will never exist, so the real ethical stakes lie in 'weak AI' systems like news feed ranking or predictive policing. Hustedt argues that bias in algorithmic systems usually originates in human decisions about goals, data and implementation, not the technology alone, and describes the algo.rules framework her organization developed as a set of process standards rather than new fundamental rights. The panel also debates predictive policing, responsibility for trustworthy AI, and whether European ethics framing is a genuine competitive advantage over the US and China.
Key takeaways
- Carla Hustedt argues algorithmic bias usually originates in human decisions about goals, data and implementation, not in the technology itself.
- Marcus Gabriel argues 'strong AI' in John Searle's sense will never exist, so the real ethical stakes lie in 'weak AI' systems and their automation limits.
- The panel's supervised-learning hiring example shows how a model trained on historical decisions reproduces embedded discrimination when sensitive attributes are inferable.
- Gabriel argues predictive-policing tools like Clearview AI reinforce statistical correlations mistaken for causation, and calls for shifting toward ethics-by-design rather than after-the-fact risk assessment.
- Hustedt's algo.rules framework, developed for Bertelsmann Stiftung, sets process standards such as informing people of automated decisions and providing appeal mechanisms, rather than proposing new fundamental rights.
Detailed notes
Defining ethics and where algorithmic bias comes from
Marcus Gabriel opens by defining ethics as the philosophical discipline studying moral facts and principles, what humans ought and ought not to do, arguing it must now proceed in a transdisciplinary way that includes politics, society and business. Carla Hustedt says she avoids the term AI in favor of 'socio-technical systems,' because bias and mistakes usually originate not in the technology itself but in the human decisions behind it: the system's goals, the training data, the code, and how it is implemented inside an organization. The co-founder of a privacy-focused data science company illustrates this with a supervised-learning hiring example, where a model trained on historical hiring decisions will reproduce any discrimination against women or ethnic minorities present in that data, especially when sensitive attributes can be inferred from other fields.
Where automation should stop
Asked by the moderator whether certain applications, such as autonomous drones, elections or candidate pre-selection, should be excluded from algorithmic decision-making from the outset, the Volkswagen panelist agrees this needs to be assessed case by case, citing unavoidable dilemma situations in road traffic where codified rules cannot simply be applied. Gabriel argues that 'strong AI' as described by John Searle's Chinese Room argument will never exist, because intelligence is arguably a feature of living systems, so he sees the real ethical problems concentrated in 'weak AI' applications such as pre-selection algorithms and news feed ranking, and in ethical limits on automation, for instance against level-5 fully automated driving in certain dilemma conditions.
Who is responsible for trustworthy AI
Hustedt argues responsibility for trustworthy AI is shared: technical builders must communicate clearly what a system can and cannot do, and organizations implementing it need competence-building among the people who use it. She says her project's case analyses repeatedly find that failures trace back to miscommunication between stakeholders, often driven by a Silicon Valley narrative that technology is a simple fix for complex social problems. The Volkswagen panelist adds that trustworthy AI also requires supranational institutions setting ethical standards, ethical training for developers and programmers, background checks for critical situations, and quality checks that assess ethical as well as technical quality.
Predictive policing and the ethics of correlation
Asked about predictive policing tools such as Clearview AI, Gabriel explains that racism specifically confuses a statistical correlation between a phenotype and a behavior with a genetic explanation for it, when the real explanation for such correlations is sociological. He argues predictive policing as currently practiced reinforces these correlations and is ethically harmful, but that the same underlying methods could instead be used for 'positive AI,' for example directing resources into a neighborhood based on the same data to take genuinely helpful action. He calls for shifting the field's focus from after-the-fact risk assessment to ethics built into systems by design.
Do we need new rules
Hustedt says she does not think fundamentally new rights are needed for the digital age, but that existing oversight mechanisms, such as bodies' rights to access code and their staffing, often fail to keep up with automated processes, and that a strong civil society acting as watchdog is needed. She describes the algo.rules her team developed for Bertelsmann Stiftung as process standards rather than definitions of what is fair, covering things like informing people when a decision about them was made by a machine, explaining that decision, providing mechanisms for appeal, and addressing IT security and privacy.
Closing statements: Europe versus the US and China
In closing, the Volkswagen panelist is cautiously optimistic that Europe, with many ethical stakeholders and supranational institutions, can build AI driven by people's needs rather than by government or corporate priorities alone. The privacy-focused co-founder argues Europe's strong data-protection foundation is a competitive advantage, not a constraint, because people everywhere want control over their own data. Gabriel rejects framing ethical values as European versus American or Chinese, insisting ethical value is by its nature universal, and points to Germany's own recently revised racist language in its basic law as a reminder to criticize one's own mistakes first. Hustedt closes by arguing European digital policy is currently too reactive, chasing technological mistakes after the fact, and calls instead for a positive societal vision of what digitization in Europe should achieve.
Quotes
Intelligence is arguably a feature of a certain living system.
Marcus Gabriel
It's not just the technology that is the cause of bias or mistakes, but mostly the human decisions that lie behind it.
Carla Hustedt
The racist confuses a correlation between the phenotype of a human animal and a certain kind of behavior with a genetic explanation.
Marcus Gabriel
Questions answered in this talk
Why does Marcus Gabriel say 'strong AI' will never exist?
According to Marcus Gabriel at the Rise of AI Summit 2020, intelligence is arguably a feature of living biological systems, so systems not literally built on neuronal biological material, in his view, will never reach the human-level or superintelligent 'strong AI' described by John Searle.
What is the source of bias in algorithmic decision-making according to Carla Hustedt?
According to Carla Hustedt at the Rise of AI Summit 2020, bias mostly stems from the human decisions behind a system, including its goals, its training data, its code and how it is implemented inside an organization, not from the technology alone.
What are the algo.rules discussed on this panel?
According to Carla Hustedt at the Rise of AI Summit 2020, algo.rules are a set of process standards, developed for Bertelsmann Stiftung, covering things like informing people when a machine made a decision about them, explaining that decision, providing appeal mechanisms, and addressing IT security and privacy.