Rise of AI Conference 2017 · Berlin

AI’s bumpy road and glorious future

Christoph von der Malsburg

At Rise of AI 2017 in Berlin, Christoph von der Malsburg argues that neither of the two historical routes to AI, the general-purpose computer and the brain-inspired neural network, can on its own solve general intelligence. The computer, he says, is a universal but empty machine that depends entirely on human programming, while today's neural networks and deep learning depend entirely on annotated data; in both cases, he argues, AI remains, in his words, human thought loaded into machines in a detailed way. He then discusses how AI systems must widen their contextual scope by decomposing a situation into abstract, relatively independent aspects, and draws an analogy to children, who apply innate, abstract behavioral patterns to concrete situations within their first years of life and learn concepts such as a tricycle from a single exposure. He argues general intelligence will come from a conceptual breakthrough rather than decades of engineering, supporting this with an information-theoretic gap between the cortex's wiring complexity and the genome's size. He closes by describing his own model of recognition through switchable, collectively decided "network fragments," illustrated with a face-recognition system and a Raven's matrices reasoning test.

Key takeaways

  • Von der Malsburg argues neither classical symbolic AI nor today's neural networks can alone solve general intelligence: one frame is too large, the other too narrow.
  • He argues today's AI remains entirely dependent on human input, either through programming or through annotated training data.
  • He argues general intelligence will arrive through a conceptual breakthrough rather than decades of incremental engineering.
  • He calculates that describing the cortex's wiring would take about a petabyte of information while the genome carries roughly a gigabyte, implying the brain relies on built-in regularity rather than fully learned detail.
  • He proposes that recognition arises from switchable network fragments whose decisions are made collectively by the active network rather than by single neurons.

Detailed notes

Two routes to AI, and why both are dependent on humans

Von der Malsburg frames AI's history around two competing routes: the computer, a universal machine that can in principle run any algorithm but starts completely empty, needing everything loaded in by people, tracing back to the 1956 Dartmouth conference and John McCarthy; and the brain, whose mere existence proves general intelligence is possible, which inspired neural networks, machine learning and, today, deep learning. His conclusion is sobering: classical AI got stuck in a combinatorial search space that is too large, while neural networks work within a frame that is too narrow to capture what happens in the brain and depend entirely on annotated data. He summarizes today's AI, in a phrase from one of his blog posts, as nothing but human thoughts in tin cans.

Widening the scope: decomposing situations into abstract aspects

He recalls that a good computer-vision PhD thesis once meant analyzing a single image, with no guarantee the method worked on the next one; systems now handle whole benchmarks of images, but still need much greater contextual scope. He argues scope can be widened by decomposing a situation into aspects, such as motion, texture or depth, that must be abstract and only relatively independent of each other, giving object recognition's need for translation invariance as an example.

Learning from children: instinct, one-shot learning and the case for a breakthrough

Von der Malsburg notes that animals and infants are born with a repertoire of instincts abstract enough to be applied immediately in a natural environment, and asks the audience whether they expect general intelligence to require decades of hard work or a sudden breakthrough, declaring himself in the breakthrough camp. His first argument is that children grasp the reality of their environment within two to three years in a simple, stable nursery setting, and can recognize a new object like a tricycle as something to sit on and steer after a single exposure, which he calls one-shot learning.

The information gap between the cortex and the genome

His second argument is quantitative: describing the wiring pattern of the cerebral cortex would take on the order of a petabyte of information, while the genetic information that builds it is closer to a gigabyte, a gap of about six orders of magnitude. He concludes that the vast majority of the cortex's structure cannot be explained by individually specified connections and must instead come from built-in regularity, a strong structural prejudice tuned to the world we live in, which he believes could in principle be replicated by training in a virtual-reality environment built from a comparatively small amount of information.

Network fragments, the neural code, and a face-recognition system

Von der Malsburg proposes that recognition works through what he calls network fragments: a two-dimensional array of local feature detectors in early visual cortex is linked, through switchable projection connections, to a matching model elsewhere in the brain, with connectivity able to change as fast as the eyes move. He built a face-recognition system on this idea to convince colleagues; he says it failed to convince them but became the basis of two still-active companies, one in Los Angeles and one in Germany. He stresses that recognition decisions are made collectively by an active network, not by a single neuron, and links the idea to the older AI concept of frames associated with Marvin Minsky and Roger Schank. He illustrates abstract, schema-based reasoning with a Raven's matrices intelligence-test item solved by recognizing numerical relationships between figures, and closes by describing his company's goal of integrating separate vision algorithms, for motion, texture and other aspects, into a shared network-fragment language so that ambiguous individual readings resolve into one consistent scene.

Quotes

It is nothing but human thoughts in tin cans.

Christoph von der Malsburg

The brain is obviously totally dominated by regularity.

Christoph von der Malsburg

A decision is taken by the establishment of this whole network. The decision is a collective decision.

Christoph von der Malsburg

Questions answered in this talk

Why does Christoph von der Malsburg think today's AI is limited?

According to Christoph von der Malsburg at Rise of AI 2017, classical computer-based AI works within a search space that is too large, while neural networks and deep learning work within a frame too narrow to capture what happens in the brain and depend entirely on annotated data, so both routes remain dependent on detailed human input.

Does von der Malsburg think general intelligence will come from years of engineering or a sudden breakthrough?

According to Christoph von der Malsburg at Rise of AI 2017, he expects a conceptual breakthrough rather than decades of incremental work, pointing to how quickly children learn to understand their environment and to the large information gap between the cortex's wiring complexity and the genome's size.

What is a "network fragment" in von der Malsburg's model of the brain?

According to Christoph von der Malsburg at Rise of AI 2017, a network fragment is a connectivity pattern, such as a two-dimensional array of feature detectors linked by projection fibers, that can switch on and off as fast as the eyes move, with recognition decided collectively by the active network rather than by any single neuron.

Mentioned

a"": ""According to Christoph von der Malsburg at Rise of AI 2017

Keep exploring

More from Christoph von der Malsburg

More from Rise of AI Conference 2017

More on AGI & Future of Intelligence

Transcript

Auto-generated transcript, not edited. · 3,477 words

Show the full transcript

as you all know there are two routes to AI and my first part is about my assessment of what where we are now with AI these two routes are the computer and the brain the computer has a very powerful argument in its favor it's a universal machine whatever the principle of the brain if you can couch it in terms of an algorithm you can run it on a machine there is no limit to that that was the thought that riveted people in the field in the 50s when they founded this field artificial intelligence as you know it arose with the Dartmouth conference in 56 the town goes back to John McCarthy a problem with that is that the computer as such is completely empty has no tendency to go anywhere everything has to be loaded into it by humans it totally depended on human collaboration I want to kick this slide the brain also has a very convincing winning argument it is a proof of existence it evidently it shows what can be done this

thought spawned the field of neural networks and its ilk machine learning and Bayesian estimation and so on and in its present incarnation it's called deep learning and it has in this version at least it has a drawback which is it is totally dependent on annotated data as you all know so in a way the computer has a frame that is much too large so that classical AI got bogged down by the combinatorial search problem as you know it is too much space from which to pick structures and the main point I'm going to make is that the neural networks approach as a frame that is much too narrow that does it cannot possibly contain the answer they cannot possibly contain the solution to the artificial intelligence problem and in its present version you are in their present version neural networks don't capture what is going on in our brain ok so my conclusion it's a pessimistic view or maybe a sobering view but I think we should face it AI today is totally dependent on

human thought in a detailed way either by needing programming or by me needing annotated data so I once wrote in one of my blogs it is nothing but human thoughts in tin cans a little overstated maybe but we're thinking about I think an important point here is in a detailed way and it is a matter of great of course you can run a computer run a system today in a much more powerful way than was possible 20 years ago so there is a gradient going on and that is the gradient of my second my second topic raining of coke when you have a system it has to deal with a wide range of scopes and and you know I'm old enough to have lived a period when a decent PhD thesis in computer vision was about analyzing one single image one single picture and the usual truth of the matter was that as the next picture wouldn't work so that was very limited we are much further now we now have systems that that referring

to the second now is not what I wanted okay I switch only oh yeah I'm now referring to the second cone here much further along we have systems that can deal with a whole benchmark with a whole series of pictures in that particular case but we will have to go much further in order to increase the contextual scope of oil systems how can you increase the scope of systems I think there are two rather obvious point you have to decompose the situation with which the system is to be a good deal in two aspects envision that would be for instance a text a foreign motion death elimination just you know the type of thing you are handling with and of course the agenda under which you want to analyze the images decomposing two aspects and these aspects have to be relatively abstract so they have to be abstract relative to each other one has to abstract from all the others as far as that is I don't know okay that is abstract and what relative

also refers to the fact that they cannot be totally independent of each other relative aspect so for instance when you and you our about object recognition you want to be translation invariance the recognition or the representation of objects needs the totally oblivious of where the object is in an image in a principled way I think these two points are pretty obvious and I think what is necessary is that the level of abstractness is to be lifted much higher in order to reach much larger contextual scope now I would like to remind you of the fact of the fact that children or animals when they come into the into the world are born with the repertoire of behavioral patterns of instincts and the miracle if you want to see it that way is that they can apply them immediately in a natural environment again or that is being born can behave in the savanna minutes after after birth so the level of abstraction of the drives that are built into animal is very high and there

is a mechanism to link them to link these drives with concrete situations that must be our goal we want of course to be able as humans to formulate on this level here I want to formulate goals agenda purposes and once existent to be able to adapt those purposes in no matter what context that's the goal now there is a very fundamental point of Creed here and maybe I should take a vote with you is generic other little intelligence a matter of sweat will it take decades of hard work as it has in the past or will there be the possibility of a sudden breakthrough I'm who is for sweat interesting who is for breakthrough oh great okay I didn't expect it I am an adherent of the other rudder of hypothesis that there is indeed a place to be found where the solution of this grand problem can be found here in a matter of a breakthrough and I have two arguments one is remember that we bring up our kids in very simple and

very stable nurseries they are expressly made simple and stable and in a matter of two or three years infants are able to grasp the reality of the immediate environment perfectly we forget that we speak about the reality with the baby where was a kid and not thinking about particular features that need to be paid attention to we take it as a given that they see the reality that they think in terms of the reality of their environment at about 3 years of age they begin to speak and they can exert their will if you have young kids you know what I'm talking about they have the world and they can interact with the world in terms of their will even before the second year of the life when you show them at the age of 3 they take they need to learn about the world for about 3 years if you show them a new implement like a tricycle they inspect it and know them all all of them is a great variety of possible

tricycles in the world changing in chase and exact way and a color and so on and so on but they get it this is a vehicle you can sit on it you can steer it you can drive it they know what the tricycle is one shot learning that's what they have my other argument is a very powerful tool I think if you want to describe the wiring pattern of the cerebral cortex it takes you a petabyte the very easy calculation a petabytes of information but what goes into the making of the recording you have one gigabyte of genetic information or probably much less this is an over estimate and I claim it will be possible in principle to train a to create a training environment by virtual reality on the basis of one gigabyte that's about the size of computer graphic systems today and speaking of the size of the program that you download so that's a tremendous gap between one gigabyte and in one terabyte six orders of magnitude so 99.9999% or the structure

of the brain is not explainable on the basis the rectangular basis of Obie's there is no mystery of course you all know of Canoga of complexity the information content of something in this case the cortex is given by the shortest algorithm by which you can create that structure but the good news here the eldorado news if you want to see that way is that the brain is obviously totally dominated by regularity it's totally dominated by regularity you don't need to train each synapse in the brain by statistical input the brain has a very powerful prejudice about in what states to fall and this prejudice is evidently tightly tuned to the world in which we live that's a very important point okay there is an Eldorado we will be able to create electronic organ of the human level intelligence we only have to find the other order where is it where do we have to look here is my essential point I think I know where to look and it is about the neural code with

you when you look at anything like this here or the scene you are grounded by when you open your eyes it takes a tenth of a second for your brain to create a model of that scene to construct that see what you are seeing is of course a construction in your brain in this moment you get visual input you get tactile input and so on but a very large part of the information contained in your scene representation comes from inside in this hindrance because in the pictures nothing about that there is nothing about materials about the significance of patterns important in the opportunities to act and so on all of that comes from inside that is a contraction and the important question that is raised by that is how do you not felt in your brain manage to create this mental theater it's not only vision as you know it's additional its emotions its pain and drives and all of this is all expressed by neurons that's what we all believe I I think and

how do they do that that is the neural code issue how do they generate the phenomenon of your mind and I will give you my answer to that and I hope I will be able to convince you this picture here refers to a face recognition system which I want built with the express intention to convince my colleagues that my way of thinking of the neural code issue is the correct way it was a complete failure my colleagues don't believe it still don't believe it but it was a good basis for two companies one in Los Angeles one here in Germany they are still alive and our way so the idea about facts recognition is this we know that a a picture of an object or a face enters our brain as a two-dimensional in the primary visual cortex as a two-dimensional array of local teacher detectors that is on very solid experimental basis we know that primary visual cortex which is called alpha 3 alpha 3 alpha for a very good reason because it is

dominated by lateral connections we know that these feature detectors are naturally connected as a two-dimensional sheet will rather local connection so that is the representation of the taste when it comes in I claim in order to recognize it you need a model of that phase in another part of the brain that would be in the NGO fusiform complex in your info temporal cortex you need a model which is homeomorphic meaning that it has some approximation the same feature detectors as in primary visual cortex arranged in the same two-dimensional array and the mechanism is to link the 2 is another network pattern which are these protection fibers here at the point in which I am diverging from everybody else in my field in neural x is that I believe that those connectivity patterns can switch as quickly as you move the eyes so when you move your eyes then the image in primary visual cortex move because and you will have to build up in your head of those projection pockets now the point I want

to make this these are free what I call Network fragment this is one this is one and those connection fibers are another networks fragment and the point I'm making is that our brain is totally dominated by such Network fragments which are built in a slow process I'm going to comment on in the next slide a building a slow process which can be switched on and off as quickly as you think they are the Nadeau block of your mind that is the language spoken by and by the by the brain now we have three such Network fragments here they cannot be combined in arbitrary fashion that is a very scary as I said earlier a very potent structural principle dominating the structure of these and that console is that each individual connection like rank with this one here from here to here is embedded in alternate pathway which helped it to to create correlations of signals between here and here and individual connections is is only good if it has partners co-operative thousands that link up

the same places and these connectivity pattern are all constructed of lots of such meshes well since this the link from here to here is gets help from the link from here to here the alternate pathway and this projection patterns are a dense mesh of such cooperative loops that's the idea and let me make another very important point I think this is that a point that is necessary to open the door to two intelligent machine the decision that is that is happening here that this is the correct model for this space here is not taken not necessary taking is not taken by an individual neuron that says I know this guy there was the marvelous period when he was our president currently in this case and a decision is taken by the establishment of this whole network the decision is a collective decision and a particular network may never have been created in the past but it falls together out of pieces or other structures net fragment that has been built in the past now I

worried about this some organization if you are sorry if you have a set of active neurons then symbolized here then it creates spontaneous signal if you give it enough time and these signals act backed by synaptic plasticity by this correlation statistic on the network this usually would be a runaway situation the network exchange and changing change that there are certain networks that are stabilized by this loop and those are the important networks I'm talking about the network pregnant I have the the word net for them and now in order to do vision which is my application domain it suffices to have two types of such network pregnant very simple types one is a two-dimensional neural filled with local connectivity as I had employed for the face of Obama and the other type is a neighborhood preserving projection pattern between them and with that you can do vision that's what I claim let's analyze the particular process when the child for instance looked at a implement a tricycle I would like to refer Jan idea which

was very important in good old fashions a I go for you here red they the name which was trampled over by a neural network from the early eighties you know a people like Marvin Minsky or Roger Strang they talked of frames of cut the idea is that you understand a scene or in this case an object by relating it to a frame to an abstract frame remember we are talking about deep abstraction which is necessary and you link it to an abstract frame which has the same part in the same order in the same connectivity pattern these connectivity is here similar stand symbolically reality is not more complicated for the spatial relationships in the intrinsic coordinates of e of the object which is a front and back and a right part and the left hand although this is left to that in the picture and as the this frame is represented by a net which is composed of a subnet which are either recognitions structures for parts of the object which you have learnt and

had to have learned before before you have spent three years on that you don't have these hard descriptors and you have these these coordinates fragments inside your metal and the to do your model and the two are linked by a set of connections which for instance do the transformation that you need to do if you walk around the tricycle then they change their structure you need to implement a proper control of the network pattern that's what I find is the essence of of intelligence and in order to convince you of that I'm going to show you into a test this is the Raven matrix which is part of the intelligence test and in order to perform on this you first recognize of course we are in and examine you are in the exam here just now you have to solve the problem what the problem well here is a question mark here and there is something like a menu this you recognize as a matrix and as you probably know the problem to be solved

is you have to find some kind of schema that unites the figures in rows and columns and establishes a relationship and what you have to do is to fill this this empty this empty box filled with one of the many items such that does relation is also filled a and to in the last law in the last column now is there any answer what is the letter when I sent you this to here I can see that you are all very intelligent people well what you have to discover you there is something to be counted you girls point here and if you can if you have numbers you can apply our less Mattox so 2 plus 1 3 3 minus 4 is minus 1 and 5 minus 3 is 2 okay that's I just wanted to give another argument that in our view intelligence beings interpreting the environment in terms of abstract schemas very abstractly like number here and in matrix and menu and some now this is my last slide I'm just being sent

a message please finish your thought what I'm trying to do what I am planning to do in my company is solve the visual problem which is hanging the computer vision problem is hanging due to the fact that there are lots of very intelligent algorithm to solve individual problems individual aspects like motion detection just texture and so on but they don't work together the problem of vision is that when you start with a picture here in the middle you have to project it into those different aspects but these projections at first are totally ambiguous you cannot possibly know what the depth is unless you also know about the other aspect so the way that itself is you need an architecture in which those different aspects talk the same language and guess what the language is the cornet fragment and they are connected on the basis of experience by by context net patterns that have storage as a matter of experience combinations of values that you gather in the image make sense so that now of ambiguous

cloud of responsible the image collapses into something that forms a well-structured network that's what I see as a vision problem and if I have at least intrigued you please contact me write me an email I send you information increase this regard those two letters which I do not know they ended up here thank you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.