At Rise of AI 2019 in Berlin, Dagmar Monett, professor of computer science at the Berlin School of Economics and Law, presents research into why artificial intelligence still has no agreed definition and what her group is doing about it. Her team ran a survey, opened in July 2017 and still running at the time of the talk, that collected more than 550 responses from people in more than 57 countries and over 180 institutions, about 80% of them from academia; 82% of respondents said they work in AI. The survey asked whether existing definitions of AI are adequate, whether human and machine intelligence need separate definitions (80% said yes), and how respondents would define intelligence themselves, producing more than 4,000 free-text comments. Monett argues the field lacks consensus because it is more than 60 years old, interdisciplinary, contextual and a moving target, since capabilities once called intelligent later stop being seen that way. She presents quality criteria her team derived for good definitions, cognitive biases that distort discussion, and a catalog of properties meant to give researchers, regulators and companies a shared vocabulary for defining intelligent systems, a precondition, she argues, for workable AI regulation and ethics frameworks.
Key takeaways
- Monett's team surveyed 556 people across more than 57 countries and 180 institutions on how they define AI and intelligence.
- 80% of respondents said human and machine intelligence need separate definitions rather than one shared definition.
- The team found cognitive biases, including an anchoring effect and a tendency to comment mainly when disagreeing, that distort apparent consensus.
- The literature definition rated highest by participants emphasizes adapting to the environment, limited resources, real-time processing and learning from experience.
- The team built a quality-criteria catalog and vocabulary, drawing on Aristotle and software-requirements engineering, to help others define intelligent systems well.
Detailed notes
A global survey on how people define intelligence
Monett opens by asking the audience how they would define AI or machine intelligence to a child, framing the talk's central question. Her research group ran a survey, opened in July 2017 and still open at the time of the talk though the main data collection closed a year later, that asked people whether they agreed with existing literature definitions of AI, why or why not, and how they would define intelligence themselves. Of about 10,000 people invited, 556 took part, coming from more than 57 countries and over 180 institutions, roughly 80% from academia; 82% said they work in or do research or teaching in AI. The survey deliberately restricted itself to human and machine intelligence, leaving out animal or other kinds of intelligence, and asked whether the two need separate definitions: 80% of respondents said yes, arguing that what counts as intelligent behavior in machines has changed over time as capabilities that were once called intelligent stopped being seen that way.
Why AI still lacks a consensus definition
Monett lists reasons her team identified for the absence of a shared definition: AI is more than 60 years old but has been through repeated cycles of hype and reinvention; researchers from different disciplines, such as neuroscience, psychology and deep learning, bring incompatible starting assumptions; and the field is a moving target, since growing computing power and data constantly expand what systems can do. She argues that consensus still matters because regulation, governance, transparency and "responsible AI" all require agreement on what is being regulated, and because media hype creates expectations about AI that affect funding, favoring fast-moving subfields like machine learning over other AI research. Many survey respondents, however, told her team it does not pay off to seek a definition at all, comparing it to "pornography: I know it when I see it."
Quality criteria for well-formed definitions
Before proposing a definition of intelligence, the team asked what makes any definition well-formed. Monett drew on the literature on definitions going back to Aristotle and on her own work in software engineering on defining good software requirements, combining both to derive quality criteria for definitions of intelligence. Applying these criteria to more than 4,000 comments from survey respondents, the team found that the literature definition participants rated highest exemplifies intelligence through concrete cognitive capabilities: adapting to the environment, operating with insufficient knowledge and resources, processing information in real time, and learning from experience. Monett notes it surprised her that the definition participants preferred most overall came not from AI but from psychology, even though more than 80% of respondents work in AI themselves.
Cognitive biases that undermine consensus
Analyzing the free-text comments, the team found cognitive biases that shape how people argue about definitions. An anchoring or focalism effect meant definitions presented earlier in a list were agreed with more often than those presented later. Respondents were also far more likely to leave a comment when they disagreed with a definition, trying to refute it, than when they agreed, so silence in the data skews toward agreement rather than reflecting genuine consensus. Monett argues that anyone arguing about how to define intelligence, in research, standards or policy, should be aware of these biases before treating apparent agreement or disagreement as meaningful.
A catalog and vocabulary for defining intelligent systems
The team used natural language processing to analyze the semantics of the comments and compare how AI experts and non-experts talk about intelligence, then built a catalog of properties and a vocabulary intended to help people define intelligent systems well, plus an iterative process for building a definition from scratch. Monett frames this as a resource for practitioners outside AI research: policymakers, people writing standards, companies that want to add AI to their processes, and ethics work, all of which she says need a common vocabulary before they can regulate, standardize or evaluate AI meaningfully. She closes by saying the AI community is responsible for supplying that vocabulary to people making decisions about AI from outside the field.
Q&A highlights
Asked whether it is even worth debating a single definition of intelligence rather than naming specific problems like reasoning or metacognition, Monett compares the situation to definitions of fairness or transparency, which have also proliferated without consensus, and says the field's name is unlikely to change; she points to proposed subfields such as "machine behavior" as an example of terminology shifting over time, much as "configuring algorithms" in metaheuristics became "hyperparameter optimization." Asked what share of AI researchers versus government respondents wanted a definition, she says most AI researchers who answered did want one, but too few government respondents took part to draw conclusions, which is one reason the survey stays open. On how the research could inform ethical frameworks, she says the same quality criteria and bias awareness could help define what makes a system "ethical," and that the vocabulary is meant partly for software engineers with no ethics background. Asked whether definitions differ by AI subfield, she says the survey did not ask about subfield specifically, only institution, role and country, but notes that philosophers in the sample tended to want a single definition of intelligence, and mentions that the European Commission's expert group guidelines on AI, released shortly before the talk, proposed at least 200 new PhD positions in philosophy to support understanding of AI.
Quotes
How would you define AI or machine intelligence or intelligence in general if you have to tell a child what AI is?
Dagmar Monett
We think it is very important to know what the I in AI is.
Dagmar Monett
I think we have to be very careful with the terminology we use.
Dagmar Monett
Questions answered in this talk
Why is there still no consensus definition of artificial intelligence?
According to Dagmar Monett at Rise of AI 2019, AI lacks a consensus definition because the field is more than 60 years old, draws on many incompatible disciplines, and is a moving target whose boundaries shift as computing power and data grow, so capabilities once called intelligent stop being seen that way.
Should human intelligence and machine intelligence be defined the same way?
In the survey presented by Dagmar Monett at Rise of AI 2019, 80% of respondents said human and machine intelligence need separate definitions, because what counts as intelligent in a machine depends on context and changes over time.
What cognitive biases affect how people argue about defining AI?
Dagmar Monett's research, presented at Rise of AI 2019, found an anchoring or focalism effect where definitions listed earlier got more agreement, and a bias where people commented mainly to disagree, so silence does not indicate true consensus.