Rise of AI Conference 2016 · Berlin

AI Kindergarten

Danko Nikolic

At Rise of AI 2016 in Berlin, brain scientist Danko Nikolic argues that current machine learning cannot produce human-level or superhuman AI because it stores knowledge in a single large, uninterruptible box trained by comparatively simple learning rules, whereas human intelligence is distributed across millions of specialized learning rules compressed into our genes by billions of years of evolution. He illustrates the gap with a demonstration of how humans instantly reinterpret an ambiguous image as a car, a train, or a table and chairs, something today's AI cannot do without prior training on each category. Citing the no-free-lunch theorem, he argues no single general algorithm can be both broad and efficient, which is why the brain relies on many specialized learning rules instead of one. Since evolving such a genome from scratch would take billions of years, Nikolic proposes "AI Kindergarten": starting from a simple genome, deploying it to users such as data scientists, and feeding real-world experience back to a central server to gradually evolve better genomes, accumulating knowledge instead of erasing it between tasks. He concludes that AI built this way could approach but never exceed human intelligence, ruling out an intelligence-explosion singularity.

Key takeaways

  • Nikolic argues today's AI stores knowledge in one large box trained by comparatively simple learning rules, unlike the brain's millions of specialized learning rules.
  • He invokes the no-free-lunch theorem to argue no single algorithm can be both general and efficient across all problems.
  • He proposes AI Kindergarten: evolving a machine genome via a central server that cycles feedback from users back into improved genomes.
  • He argues this architecture can only ever approach, never exceed, human intelligence, which rules out an intelligence-explosion singularity.
  • He expects future AI to require constant human supervision, meaning most people would effectively become supervisors of machines rather than be replaced.

Detailed notes

How a child understands a car differently from an AI

Nikolic opens by contrasting how a child and a machine learn to recognize a car: a child needs to play with one single car to recognize cars far better than any AI of the time, and can even build a car from Lego or create something new based on understanding what a car is, while an AI needs thousands of training images. He demonstrates with an ambiguous image that a person can instantly see as a car, then, once told, instantly reinterpret as a train, then as a table with two chairs, and then as a classroom, without ever having been trained on that specific image before. He argues this rapid reinterpretation of the same input shows a fundamental difference from doing statistics on images.

Why today's AI architecture cannot scale to human intelligence

Nikolic describes today's AI as storing knowledge in one large, uninterruptible box, such as a set of network weights or a probability distribution, produced by learning rules whose code is far smaller than the resulting knowledge store. He argues the real intelligence lies in creating the learning rule itself, comparing a compressed rule like backpropagation to the compactness of Einstein's E=mc². By his calculations, reaching human-level intelligence with this single-box architecture would require pre-programming every possible situation a human could encounter, needing a computer roughly the size of a planet.

Millions of specialized learning rules, and the no-free-lunch theorem

Nikolic argues the brain instead stores knowledge across millions of specialized learning rules rather than one big box, which lets it compress and store far more knowledge. He grounds this in the no-free-lunch theorem, which he summarizes as meaning no single general algorithm can be efficient across all possible problems: an algorithm is either general and slow, or specific and efficient. Humans are born not with knowledge of, say, cars, but with genes containing rules for how to quickly learn new things, which he describes as the true site of intelligence, compressing the experience of billions of years of ancestors, from fish and algae onward, in a way he says cannot be replicated by engineers.

AI Kindergarten: evolving a genome instead of training a model

Since evolving such a genome from scratch would take a billion years, Nikolic proposes transferring and adapting knowledge from human genes to a machine genome instead, through a method he calls AI Kindergarten. A central server evolves an AI's genome, delivers it to users such as data-mining practitioners, who apply and learn from it and feed experience back to the server, which uses that feedback to develop an improved genome in a repeating cycle. He contrasts this with deep learning today, where training a network on a new data type typically erases prior knowledge, saying his architecture would instead accumulate knowledge, giving the example of a system that starts by recognizing simple drawings and gradually gains the ability to draw and recognize cars. He suggests a first, simple genome for time-series analysis could be built within a year or two, already outperforming current tools by learning from smaller datasets.

Implications: no singularity, and everyone becomes a supervisor

Nikolic argues this architecture implies an AI can never exceed human intelligence, since it only receives a partial transfer of humanity's evolved genome, meaning the exploding-intelligence singularity cannot occur, which he calls good news because there is no reason to fear it running out of control. He also argues it should reduce fear of job loss: such AI would take over many tasks but, since it can never fully match human intelligence, would remain more like a very capable but limited system requiring constant human supervision, so that in a future built on such machines, he expects everyone to effectively become a supervisor, or "boss," of a machine.

Q&A highlights

Asked about machines already outperforming humans at specific tasks, such as a calculator doing arithmetic faster than any person, Nikolic says that is not the real question; the question is whether a machine can reach human-level general intelligence, an AGI as smart as a human overall, which he maintains cannot be done in a lab because what makes human intelligence possible is billions of years of accumulated work stored in our genes.

Quotes

We have to build AI in a different way than what we have been doing so far if we want to create machines that somehow approach the level of intelligence that we humans have.

Danko Nikolic

There could never ever exist an AI that's more intelligent, that has a better, smarter genome than us.

Danko Nikolic

Questions answered in this talk

Why does Danko Nikolic think today's AI cannot reach human-level intelligence?

According to Danko Nikolic at Rise of AI 2016, today's AI stores knowledge in one large box built from relatively simple learning rules, whereas human intelligence comes from millions of specialized learning rules compressed into our genes over billions of years of evolution, which cannot be replicated in a lab.

What is AI Kindergarten?

According to Danko Nikolic at Rise of AI 2016, AI Kindergarten is his proposed method for evolving a machine's genome through a central server: the genome is delivered to users, their experience is fed back to the server, and the genome is improved in repeating cycles, accumulating knowledge instead of erasing it between tasks.

Does Danko Nikolic believe the AI singularity, an intelligence explosion, could happen?

According to Danko Nikolic at Rise of AI 2016, no: because his proposed architecture can only approach but never exceed human intelligence, an exploding, self-improving superintelligence cannot occur, which he says is good news.

Mentioned

AI Kindergarten, no-free-lunch theorem, backpropagation, deep learning, singularity, AGI, evolutionary genome

Keep exploring

More from Danko Nikolic

More from Rise of AI Conference 2016

More on AGI & Future of Intelligence

Transcript

Auto-generated transcript, not edited. · 3,093 words

Show the full transcript

hello good evening uh my name is Danko Nikolic as you can see i am a brain scientist originally and um i have only recently started working on ai explicitly so what i will present to you is not much of brain science directly you will just see some nice images of brain which has no purpose but i will i will talk about what i think how we should be thinking about ai building a i from the perspective what we have learned in how the brain works right so and what i will tell you is that actually we have to build ai in a different way than what we have been doing so far if we want to create machines that somehow approach the type of intelligence the level of intelligence that we humans have now you know probably all that if you want to teach a machine to recognize a car or a face of a child you have to present thousands of images and so on right but in actually a real child it's enough

to play with one single car and after having played with the car the car the child is able to recognize cars much better than any ai of today not only is able to recognize cars it can actually build a car from say legos or it can actually build something new creative that's based on the understanding of the car so there's a fundamental difference from doing statistics of images and from understanding what an car is about so i'll give you an example of something that you can easily recognize as a car if i tell you this is a car and also a child can do it and the interesting thing is that we can do it if we see it for the first time ever we don't need to be trained on that we just need to understand the concept of where we can do it and now if i do if i present you if you understand this as a car i can show you this image and you can immediately see a train today

ai cannot do it if it hasn't been already trained on trains of this kind of images right and now it's even even better things can happen in our minds i can tell you now that this image is not a car anymore but it's a table with two chairs and you can immediately see a table with two chairs and you see it for the first time no problem at all and then i can show you this image again and then you see a classroom which was the train before we can change our understanding of the world really quickly really fast and that's a big big big difference from what say machine learning today is doing right so i want to tell you how this happens what what are the differences here now uh important part about the ai how it works today is that it stores knowledge in a big box this could be some synthetic weights of a network this could be some probability distribution this could be all kinds of information but it stores

knowledge in a box and it's a big box with lots of knowledge and the way how it computes its answer produces its answer it says that you get an input in and it computes computes and it spits out the answer the important part is that during this computation the box cannot be interrupted it shouldn't be interrupted it's much more flexible and productive if it's not interrupted if it just goes does its own job now how did you get this knowledge and you have a certain algorithm certain learning rules the interesting part about ai today is that algorithms take much less resources much less memory on a computer the your code of your algorithm is much smaller than the amount of memory you need to store the knowledge the knowledge that you have created is really really much larger right now the important part about the um amount of intelligence in each of this these two is just the opposite the amount of intelligence in the learning rules is much much much higher to create the

learning rules like say the back propagation learning rule to create the algorithms for deep learning this is where the real intelligence was this is where actually humans work really really hard to create now you once you have created it you apply it to some data you create this big box this is a relatively modest amount of intelligence compared to what we have in the learning rules in a way this little dot over there has compressed a lot of intelligence in something very short it's like the equation of einstein is equal mc squared a lot of intelligence compressed only in a little little little small equation right so this is the very important part to understand for further later for the later things it turns out that if you want to make human level intelligence with this architecture that i have just shown you i have made some calculations i'm not going to show you uh calculations you have to just kind of trust me right now but if you want to create human level intelligence

with this architecture you would have to have a computer of a size probably of a planet because you basically have to have in this box pre-programmed everything every possible situation every possible thing that could a human that you could be presented with that you could understand right i could come here and on this bean and say all kinds of different things and you as the humans can understand all kinds of different things if you estimate the amount of number of things that you could possibly understand this number this amount requires huge resources if you have everything already stored in the box so there's a problem with that now uh i want to tell you what what is the difference how actually our brain works we have this box of a knowledge actually very small box and we have stored information in learning rules and if you have information stored in your own learning rules you can actually compress like we have compressed uh in in this previous uh image uh one learning very large amount

of knowledge in one learning rule here we have compressed milli large amount of knowledge and millions of learning rules so we can store a lot a lot more knowledge now if we divide it into lots of learning rules what turns out that we can actually produce very intelligent system and this follows from uh so-called no freelance theorem that tells us basically uh that we have to have a very it says emit optimization algorithms are equivalent but their performance is leveraged across all possible problems but what it in practice means is that you cannot have uh very general algorithm that works in all situations and to be really efficient the algorithm either has to can be general and slow and efficient or it can be very specific for a specific situation but then it's going to be efficient so if you had take this architecture you have a lot of lots of specialized algorithms and that's how our brain works we have lots of specialist learning algorithms probably millions of them uh and that's that's the

useless picture of the brain that i was talking about now comes the question uh how do we create that how do we create millions of learning rules well it turns out that we are born when we are born we are we have something that we call genes and these genes contain a set of rules a smaller set of rules that actually know how to learn a new learning rules so if you have learned something about cars and you became expert in understanding cars you we were not born with the knowledge of cars but we are born with the knowledge on how to quickly learn all kinds of different things and it just happens so that we can learn from this knowledge a lot easily about cars now what is very important to understand about this knowledge in genes that this is the most important part of the whole thing this is the most intelligent part as i told you that in machine learning if you have algorithm that was created by really smart engineers at

google that really their intelligence of these engineers and google was compressed in this algorithm here in the genes is compressed the intelligence the experience of millions and billions of years of our ancestors who have tried to survive on this planet not only human but we also had ancestors as fish and algae everything all together was was condensed here and this is this really powerful thing that we have this is our easy mc square sort of intelligence and you can't match that with some engineering and machine learning it can be actually cannot be possibly replaced it cannot be possibly tricked away you cannot come up with some intelligent creative algorithm that would replace the need for this no way that's the that's the whole point so the only thing we could possibly do something like this we could build an agent that has this uh architecture with three these three levels of organization but at the beginning it would be some very empty without any knowledge it would be some sort of tabular asset now what

you have to do like our own evolution then you would have to evolve the genome of this of this agent and that's what we could do in the machines right but we don't have one billion years to do that right we would like to do it in one year five years ten years not one billion years and that's possible because it turns out that it's really hard to evolve something if you have to do it for the first time but it's much much faster if you have to evolve something that already exists in our own genes and you have to just transfer knowledge from our engines to the machine genome and i have actually come up with the whole method how can this principle be done and i named this method ai kindergarten because it would involve basically lots of interaction with it with machines and robots in a way that's very uh similar to to an actual kindergarten here you go uh this is an artist's illustration of what this may look like now

there are several important implications for that one is that there could never ever exist an ai that's more intelligent that has better smarter genome than us because all we could do is to transfer the knowledge that we have in our genes to them and only partly so they can never even reach our full intelligence they can only approach gradually to our intelligence for for ai to exceed our intelligence he would have to undergo its own evolution have been millions of years have appropriate evolutionary pressure and so on there is no way no trick that you could make this ai or that it could make itself smarter or develop somehow change itself to become smarter if it was possible then you would violate some theorems right and theorems shouldn't be violated right so that's one important implication the other important implication is that we could we can in principle develop quite an intelligent ai ai that is quite biological like quite similar to us gradually approaching our intelligence never reaching it but gradually approaching it and

the process by which we have to do it is that we have to follow biological evolution exactly how biological evolution went why because this is an experiment that has been already done and this experiment was successful and that's the only thing we have right now in front of us to see to copy to steal from right so we wouldn't need to do billions of years of separate evolution so the idea of ai kindergarten is that you would have some sort of uh central process where you would central server where you would evolve the genome of of an ai you would deliver this genome to the users like data mining people they would use it applied they would learn something from that they would feed back information to to the central server the center server would use this knowledge this experience to develop even better genome and this cycle would close it would repeat over and over and over again one important aspect of such organization of ai is that you don't you can build a

new knowledge on top of the other knowledge so you probably know if you use deep learning or something today if you train deep learning on one type of data and now you want to train it to another type of data you cannot transfer the knowledge if you if you train it to recognize pictures of birds and fruit and now you want to train it to recognize pictures of cars and i don't know what furniture there's no transfer you actually have to raise this and start all over well here it's not here this is not the case here you can actually accumulate knowledge on top of the previous knowledge exactly how our revolution did and exactly how our own development does right you learn some things as a child and you learn new things and you could accumulate that without having to erase the previous knowledge so at some point here in this example you would have a point where you could actually users would train recognition of little drawings but then by improving the genome

the robots would be able to draw not only to draw but also to recognize a car like we have have given example to you now uh how much time do i have five minutes okay so uh the last then the last thing i would like to explain are two more things one is about singularity and one is about how to actually begin building this so the beginning of of doing this and maybe the end of this or the non-existing end of this so how do you begin building uh such an intelligent system so what i think we could do is pretty easily code and develop the architecture for these three levels of organization that's pretty straightforward the problem is of course this genome part and we have to start with something very simple by definition right and i think that we could make within a year or two develop a genome for such for such an architecture that's already quite useful that you could deliver to various data mining scientists to use for for example

time series analysis right this product would be very different from what you have today this product would be already certain type of expert for time series and that in that example it would already have lots of experience with time series compress this experience in the gene in the genome and then once you present it with the time series it will do much better job than what you can do today it would already it will learn from smaller data sets it would be able to recognize when there is a change in the pattern changing the stochastics of a process and apply new new genome new knowledge it would be much much more efficient it would be much like biological systems much more similar to biological system and then you would start with something like that simple go back to a kindergarten develop more genes for more broader applications and so on and maybe in i don't know 20 30 50 years we would get close quite close to to something similar to human intelligence and never

quite there but maybe something quite close now about singularity and i said that this system cannot produce intelligence explosion right and um and then there are some theorems behind it that why the white cannot produce industrious explosives so this means that actually singularity as thought of exploding intelligence would never possibly happen and this is i think very good news for us because it's a good because there's no reason to fear it to run out of control and the second reason is that it's not really a great reason to fear of losing jobs of course this whole intelligence would be built to take over many jobs however since it never can really reach the level of intelligence of humans it can never really replace human right so it would be more like some really dumb machine dumb what we would say for what we would say today for some really dumb human or something like really smart animal right so basically you would have a lot of machines that can do lots of work but that

need a lot of supervision by human all all the time supervision so in in the world i don't know 50 years from now when we have such all over such machines basically everyone would have a job of more or less supervising such a machine everyone would be supervised everyone would have everyone would be a boss and i think that's a good thing right thank you very much thank you that's a question uh yeah it's happening right now of course all the time that machines are outperforming humans right so does handheld calculator multiplies faster than i do any that anyone here in this room does so it's not a problem that machine does not outperform human the problem is whether machine can reach human level medic human level intelligence right the agi can you build agili can you build a machine that's actually as smart as human more or less completely as a human that's what you cannot do and you ask that's what i'm saying you cannot do now you ask why in one sentence

it's because what makes our intelligence is a huge amount of work that has been done for billions of years that stored in our genes and you cannot do it in your lab

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.