At Rise of AI 2019 in Berlin, Moritz Helmstaedter, Managing Director at the Max Planck Institute for Brain Research, argues that today's deep learning still corresponds closely to neuroscience ideas from the 1960s, using feed-forward connectivity inspired by early visual system anatomy and neuron nonlinearities, without fundamentally new architecture. He argues artificial and biological intelligence differ most sharply in two respects: energy efficiency, and especially label efficiency, since a child learns to recognize a tree after hearing the word only 50 to 100 times, while machine learning systems need billions of labeled examples. He presents his field, connectomics, which uses electron microscopy to map brain circuits at the scale of individual synapses, describing findings including a fully reconstructed mammalian retinal circuit, the discovery that each cortical nerve cell connects to roughly 1,000 others, and a newly found "triadic" inhibitory circuit motif in the entorhinal cortex that may function as a context-gated hypothesis circuit. He argues brains, unlike computers, do not separate memory and processing, making circuit structure itself informative about function, and closes by proposing that mapping brain circuits in detail could again inspire new, more energy- and label-efficient computing architectures, as neuroscience did for early AI.
Key takeaways
- Moritz Helmstaedter argues today's deep learning still corresponds mostly to neuroscience ideas from the 1960s, based on neuron nonlinearity and feed-forward connectivity.
- He argues label efficiency, not energy use, may be AI's biggest gap versus the brain: children need to hear a word only 50 to 100 times to learn it, versus billions of labeled examples for machine learning.
- Each of the brain's roughly 86 billion neurons connects synaptically to about 1,000 others, often over long range, a connectivity degree no current artificial system matches.
- Unlike computers, the brain does not separate memory from processing, so mapping its circuit structure can reveal both algorithm and storage at once.
- His group found a preserved "triadic" inhibitory circuit motif in the entorhinal cortex that may act as a context-gated hypothesis circuit, blocking signals until unlocked by contextual information.
Detailed notes
Deep learning is still old neuroscience
Helmstaedter argues that today's deep learning, despite being called "neural," draws on only two ideas from biology: neuron nonlinearities, understood to help approximate almost any function, and feed-forward connectivity, inspired by early anatomical studies of the visual system showing signals passing from retina through the thalamus into cortex. He argues this means most of today's AI architecturally resembles neuroscience from the 1960s, and that learning rules such as backpropagation and LSTM lack clear biological correspondence in his view. He stresses he is not claiming brains must be copied to build intelligent machines, comparing this to how aircraft do not need to flap like birds to fly, but argues it may still be worth looking at the brain again for new inspiration.
Energy efficiency and label efficiency
Helmstaedter identifies two key differences between biological and artificial intelligence. The first, energy efficiency, is widely discussed already. The second, which he argues is even more important and less discussed, is label efficiency: today's AI learns to recognize a tree by being shown labeled tree images repeatedly, often billions of times, whereas a child, by his estimate, hears the word "tree" only about 50 to 100 times before reliably distinguishing a tree from a car at around 18 months old, despite having seen millions of trees. He argues this label inefficiency, not just architecture, is the biggest problem in today's AI, and that the brain is far better at it.
Mapping the brain's circuits: connectomics
Helmstaedter describes his field, connectomics, which measures the brain's actual wiring using electron microscopy. Each of the brain's roughly 86 billion nerve cells connects synaptically to about 1,000 others, often over long range rather than only to near neighbors, a degree of connectivity he says is unmatched by any current artificial system. Because the cables involved, axons and dendrites, are as thin as 50 to 100 nanometers, mapping them requires scanning electron microscopes that image and shave away thin tissue layers repeatedly, technology he says has been developed over about 15 years in Germany and the Max Planck Society, using microscopes built by Zeiss, with only three to five such instruments in existence worldwide. He describes his group's first major result, a full reconstruction of roughly 1,000 densely packed nerve cells in a piece of mouse retina, the first stage of visual processing, which sends roughly 30 to 40 parallel filtered representations of a visual scene into the brain, and the resulting adjacency matrices mapping which cells communicate with which.
Brain architecture and inhibitory circuits
Helmstaedter argues that, unlike computers, the nervous system does not separate memory from processing, which means that mapping brain structure can reveal both algorithm and storage at once, unlike slicing open a computer chip, which reveals only its microprocessor architecture and nothing about the software running on it. He highlights inhibitory neurons, roughly 20 to 25 percent of all nerve cells, which dampen rather than excite neighboring cells, as a feature almost absent from standard artificial neural networks. In a recent study of the entorhinal cortex, a region involved in spatial navigation, his group found a highly preserved "triadic" circuit motif in which an inhibitory neuron is always recruited alongside an excitatory connection between two other neurons, creating what he calls cellular feedforward inhibition that can act as a closed gate blocking signal transmission until unlocked by contextual information, which he proposes, as an early hypothesis, could function as a way of wiring in many possible interpretations while activating only the most plausible one given context.
Toward brain-inspired computing
Helmstaedter says his group is now trying to determine how much recurrency actually exists in mammalian cortical circuits, and argues that detailed circuit mapping, still in its early stages, could eventually inspire new, more energy- and label-efficient computing architectures, much as neuroscience inspired early AI roughly 70 years earlier before the two fields drifted apart and machine learning later became a tool for analyzing neuroscience data itself. He frames this connectomics research as happening in Europe and Germany specifically, and hopes it can contribute to genuinely transformative developments in AI.
Q&A highlights
Asked about incorporating chemical as well as electrical signals, Helmstaedter notes neural transmission is inherently electrochemical, and that state-dependent, slower chemical modulation, such as dopamine signaling relevant to reinforcement learning, is a separate, heavily researched layer. Asked whether speeding up biological-style computation to nanosecond timescales would help, he says physical limits on ion channel conduction make that unlikely, and argues the brain's relevant operating speed is set by real-world constraints such as the time needed to react to a falling object under gravity, for which current millisecond-scale reaction times are already sufficient. Asked about a key difference from standard negative-weight artificial networks, he notes mammals have neurons dedicated solely to inhibition, a property known as Dale's law that some species such as flies and worms lack, and whose function is not yet fully understood. Asked about hardware, he argues that current neuromorphic chip initiatives are built on brain knowledge that is now about 20 years out of date and do not match circuit architectures his group is now measuring, and that implementing brain-like wiring density may require three-dimensional rather than planar hardware.
Quotes
We don't have to build birds to fly.
Moritz Helmstaedter
The nervous system combines storage and processing; we have absolutely no evidence for separation of the two.
Moritz Helmstaedter
Label efficiency is really the biggest problem we have, or inefficiency, in today's AI.
Moritz Helmstaedter
Questions answered in this talk
What is the biggest difference between how brains and today's AI learn?
According to Moritz Helmstaedter at Rise of AI 2019, the biggest difference is label efficiency: a child learns to recognize a tree after hearing the word only about 50 to 100 times, while today's AI systems typically need billions of labeled examples.
What is connectomics?
According to Moritz Helmstaedter at Rise of AI 2019, connectomics is the use of electron microscopy to map brain circuits at the scale of individual synapses, revealing that each cortical neuron connects to roughly 1,000 others, often over long range within the brain.
Why is deep learning's architecture still based on old neuroscience?
According to Moritz Helmstaedter at Rise of AI 2019, deep learning draws mainly on neuron nonlinearity and feed-forward connectivity ideas from early visual-system anatomy studied in the mid-20th century, so most current AI architecturally resembles 1960s neuroscience despite newer learning rules.