At Rise of AI 2019 in Berlin, Peter Morgan, CEO of Deep Learning Partnership, argues that deep learning cannot lead to artificial general intelligence and proposes a physics-based alternative. He defines intelligence as an agent's adaptation to its environment and argues there are many distinct types of intelligence — physical, spatial, musical, linguistic, social, emotional — not just the logical-mathematical kind that deep learning approximates. He traces how biology builds intelligence hierarchically, from chemical-gradient navigation in bacteria without neurons, through the roughly 130-neuron nervous system of C. elegans and the million-neuron bumblebee, up to the human brain's roughly 100 billion neurons organized into repeating cortical columns. Comparing digital hardware such as Google's TPU and the Summit supercomputer to neuromorphic systems like SpiNNaker and BrainScaleS, he argues that brain-like hardware that combines memory and processing is a necessary step toward AGI, since digitally simulating the brain is far less efficient than biology itself. He proposes grounding a general theory of intelligence in physics, specifically the principle of least action, and highlights active inference, tied to Karl Friston's free energy principle, as the theoretical approach he finds most promising among several decades-old research programs.
Key takeaways
- Morgan defines intelligence as an agent's adaptation to and prediction of its environment, spanning at least nine distinct types beyond the logical-mathematical.
- He argues deep learning is statistics and linear algebra that can never produce general intelligence, only incremental accuracy gains.
- Biology builds intelligence hierarchically, from neuron-free bacteria through 130-neuron C. elegans and million-neuron bumblebees to the roughly 100-billion-neuron human brain.
- He argues neuromorphic hardware such as SpiNNaker and BrainScaleS, which combines memory and processing like the brain, is a necessary step toward AGI.
- He proposes grounding a general theory of intelligence in the physics principle of least action, favoring active inference as the most promising specific framework.
Detailed notes
Redefining intelligence as environmental adaptation
Morgan opens by arguing that to build artificial general intelligence (AGI) it is first necessary to define what intelligence is. His definition: intelligence is an agent's adaptation to, and successful prediction of, its environment. He argues this is not one faculty but many — logical-mathematical (Einstein), physical (a high jumper who could out-perform Einstein at the high jump), musical (Mozart, Beethoven), linguistic (Shakespeare), interpersonal, emotional, and social intelligence up to the level of nations keeping peace. He contrasts this with narrow systems such as AlphaGo, AlphaStar, Deep Blue and IBM Watson, which he calls statistically brilliant but "completely dumb" and not general.
Why deep learning cannot reach AGI
Morgan states that deep learning is fundamentally statistics and linear algebra — summing weighted vectors through large matrices — and that this can never produce introspection or genuine understanding, only ever-smaller accuracy gains. He cites BERT (from Google) and GPT-2 (from OpenAI) as impressive statistical achievements, noting that Facebook, Microsoft and other big tech companies are likewise pushing ahead with such statistical methods, but says progress on general intelligence via these methods is effectively zero. He notes that Geoff Hinton and Yann LeCun never intended deep learning to imitate the brain, and reports that Hinton has said his own deep learning work now needs to be set aside to reach the next level by studying how biology actually does it.
How biology builds intelligence, from bacteria to the brain
Morgan describes intelligence as hierarchical in biology: bacteria have no neurons at all yet navigate using chemical gradients; C. elegans has about 130 neurons, the simplest nervous system that can still survive, eat and reproduce; bumblebees have about a million neurons and can navigate and perform their waggle dance; the human brain has on the order of 100 billion neurons with roughly 1,000 to 10,000 synapses each, organized into repeating cortical columns of about two million neurons that show generality (visual cortex can be repurposed for sound processing in blind people). He argues any general theory of intelligence must explain this whole hierarchy, from atoms and neurons up to societies and nations.
Neuromorphic hardware as the missing piece
Morgan contrasts von Neumann digital hardware, which separates processing from memory, with neuromorphic hardware that combines both, as biology does. He cites Google's TPU and the Summit supercomputer (three exaflops, GPU- and AMD-processor-based) as extremely powerful but "dumb as a brick" at general intelligence. He points to SpiNNaker, part of the Human Brain Project at the University of Manchester under Steve Furber, which he says has built about a billion artificial neurons across 48 chips per board, 24 boards per rack and 10 racks, and to BrainScaleS at the University of Heidelberg. He states digital simulation of one second of brain activity took the Human Brain Project about a week on a full data center, whereas neuromorphic hardware runs about a thousand times less efficiently than biology, versus about a million times for conventional digital chips.
A physics-based theory: least action and active inference
Morgan proposes that intelligence, like all physical systems, should be describable through the principle of least action, the single underlying formula (in Lagrangian or Hamiltonian form) he says unifies Newtonian mechanics, relativity, thermodynamics and quantum mechanics. He surveys several decades-old theoretical programs and singles out active inference, associated with a theoretical neuroscience group and its free energy minimization approach he links to Karl Friston, as the framework closest to a genuine physics of intelligence. He concludes that the data, theory, algorithms and (nascent) hardware needed to build AGI already exist, but estimates it could take five to ten years as the hardware matures.
Q&A highlights
Asked how society can bring together the computer science, physics, neuroscience and psychology needed for AGI when academic incentives push toward narrow specialization, Morgan points to Karl Friston's theoretical neuroscience group, which mixes strong mathematicians and computer scientists with neuroscientists, as an early case study, and adds that psychologists are still needed. He argues that while many disciplines must contribute ideas, the actual mathematical theory will ultimately have to be written down by one or a few very strong scientists, comparing this to Einstein deriving general relativity.
Quotes
It takes a village to create an AGI, just as it takes a village to raise a child.
Peter Morgan
We do all that in three pounds. We don't need three football fields or ten megawatts, we need a couple of sandwiches, an apple and a drink to discover the laws of the universe.
Peter Morgan
It does not give us a neuron, absolutely not, even close, but never will do.
Peter Morgan
Questions answered in this talk
Can deep learning lead to artificial general intelligence?
According to Peter Morgan at Rise of AI 2019, no: deep learning is a statistical, linear-algebra method good at pattern recognition, but progress toward general, introspective intelligence through these methods is effectively zero and always will be.
What is neuromorphic computing and how does it relate to AGI?
Peter Morgan explains at Rise of AI 2019 that neuromorphic computing, exemplified by the SpiNNaker project at the University of Manchester and BrainScaleS at Heidelberg, replicates the brain in hardware by combining memory and processing, running far more efficiently than simulating the brain on conventional digital chips.
What theoretical framework does Peter Morgan propose for building AGI?
At Rise of AI 2019, Peter Morgan proposes grounding a general theory of intelligence in physics via the principle of least action, and highlights active inference, linked to Karl Friston's theoretical neuroscience group, as the approach closest to that goal.