Rise of AI Virtual Chat 2020 · Berlin

The Third Wave of AI

Kristian Kersting · Professor for AI and Machine Learning, TU Darmstadt

In this Rise of AI Virtual Chat from August 2020, host Fabian Westerheide talks with Kristian Kersting, professor for AI and machine learning at TU Darmstadt, about what Kersting calls the "third wave" of AI: combining reasoning, learning and acting through hybrid, knowledge-driven systems. Kersting explains GPT-3's excitement as the first sign of a system handling many tasks it was not explicitly trained for, describes deep learning as "differentiable programming," and argues the field needs a dedicated high-level programming language in which learning algorithms are first-class citizens, so domain experts like biologists do not need to specify network architectures themselves. He describes his own research combining probabilistic programming, statistical relational learning and deep networks, and his work leading the planned Hessian cross-university AI institute, which he calls in the talk the “Tensor Institute” (following, he explains, a naming progression from a “Parameter Institute” to the Vector Institute), bringing together universities across the state. On ethics, he argues bias findings in AI models mirror human bias rather than being an AI-specific problem, and rejects blaming AI for autonomous weapons rather than the humans and game theory behind weapons development. He is skeptical of the term AGI and says true intelligence means being a robust general problem solver, not an optimal one.

Key takeaways

  • Kersting says GPT-3's significance is handling many tasks it was not explicitly trained for, not any single new technique.
  • He argues deep learning is best understood as "differentiable programming," where gradients tune a sketched algorithm's parameters using data.
  • He calls for a dedicated high-level AI programming language where learning algorithms are first-class citizens, so non-programmers need not pick architectures.
  • He argues AI bias findings mirror human bias and are "a mirror of what we are doing," not a flaw specific to AI.
  • On autonomous weapons, he says people should blame the humans and industries building weapons, not only the AI used in them.

Detailed notes

Why GPT-3 excites researchers

Kersting says what stands out about GPT-3 is not any single trick but that, for the first time in the textual domain, a system appears to handle many different tasks, including ones it was not explicitly trained for, which he calls essential for a "truly smart system." He describes deep learning as "differentiable programming": rather than hand-coding every rule as in the first wave of AI, engineers sketch an algorithm with differentiable building blocks and let data and gradients tune its parameters, which he calls the second wave. He explains transformers, GPT-3's architecture, as roughly a differentiable version of nearest-neighbor search using a large dot product.

The third wave: reasoning, learning and acting together

Kersting frames the "third wave" as combining differentiable deep learning with symbolic, reasoning-based approaches, referencing Gary Marcus's paper "The Next Decade in AI" and Yoshua Bengio's talk on cognitive agents. He says his lab works on probabilistic programming and statistical relational learning, replacing parts of database-like tables with deep networks while keeping a gradient-based way to propagate information, building on work with colleagues from KU Leuven and companies including Uber. He argues systems like AlphaGo/Alpha Zero already hybridize search (Monte Carlo tree search) with neural evaluation functions, but that true systems combining reasoning, learning and acting are still early.

A high-level programming language for AI

Kersting argues AI needs a dedicated high-level programming language in which learning algorithms are first-class citizens, so a user can specify a goal, like classifying images and emailing a group, without choosing whether to use an SVM or a deep network. He compares current tools like TensorFlow to "assembler for AI" that is still hard to debug, and connects this to his lab's collaboration with biologists on plant phenotyping who do not want to specify network architectures themselves. He credits companies like Google, through investment in usable frameworks and advertising, with democratizing machine learning the way Linux and later usability efforts spread earlier technologies.

Bias, ethics and blaming AI for weapons

Kersting says a well-known study showing AI language models capture human stereotypes was an important finding, but calls it "a mirror of what we are doing" rather than a flaw specific to AI, and frames the debate over data versus algorithmic bias as central and unresolved. On autonomous weapons, he says he has signed a letter opposing AI in autonomous weapons, but argues people should also blame the humans and industries that build weapons and the game-theoretic logic of mutual deterrence, not treat AI as a convenient scapegoat. He also argues that fears about AGI "getting rid of humans" say more about human psychology than about what a truly intelligent system would do.

What counts as truly intelligent

Kersting says he dislikes the term AGI, since standard AI textbooks never distinguished general intelligence from AI in the first place, and argues machine intelligence should not aim to copy humans but can be usefully inspired by them, noting self-driving cars modeled purely on human driving would not reduce accidents. For him, true intelligence means being a general problem solver able to ask for help and adapt, not being optimal at everything; he argues reproduction-based optimization (evolutionary algorithms) trades optimality for robustness, citing a paper on "why sex is not good for optimization." Discussing his role leading Hessen's planned cross-university AI institute, which he calls the "Tensor Institute," he says a letter of interest from Canada's Vector Institute could enable collaboration with researchers including Geoffrey Hinton and Gary Marcus.

Q&A highlights

Asked where AI will be in 2030, Kersting says his hope is that AI becomes as broadly accepted as the car once became, that climate challenges are addressed with AI's help, and that AI and programming are taught in schools, calling this a matter of societal "enlightenment" as much as academic or industry progress, though he calls achieving it within ten years "challenging." Westerheide notes Kersting supervised the master's thesis of a student who later became known in Germany for criticizing politics and news, without naming him further in the transcript.

Quotes

It's amazing how much they can get out of this data and the huge model that they have.

Kristian Kersting

Failure is part of intelligence in a sense, and that's why we want to have the third way.

Kristian Kersting

We should not just be happy that there's a new player and therefore we can blame her, whatever you want to call AI.

Kristian Kersting

Questions answered in this talk

What does Kristian Kersting mean by the "third wave" of AI?

According to Kristian Kersting at this Rise of AI Virtual Chat, the third wave combines differentiable deep learning with symbolic, reasoning-based approaches so systems can reason, learn and act together, building on prior work by researchers such as Gary Marcus and Yoshua Bengio.

Why is GPT-3 significant according to Kristian Kersting?

Kersting explains that GPT-3 is notable because, for the first time in the textual domain, it appears able to handle many different tasks, including ones it was not explicitly trained for, which he sees as essential for a truly capable system.

Does Kristian Kersting think AI models are inherently biased?

Kersting says at this Rise of AI Virtual Chat that studies showing AI language models capture human stereotypes reflect a "mirror of what we are doing" as humans, so bias is a broader societal and data problem rather than one caused by AI itself.

Mentioned

a"": ""According to Kristian Kersting at this Rise of AI Virtual Chat

Keep exploring

More from Rise of AI Virtual Chat 2020

More on AGI & Future of Intelligence

Transcript

Auto-generated transcript, not edited. · 10,094 words

Show the full transcript

welcome to the next um reservation virtual chat my name is fabi Westerheide i'm an entrepreneur and investor with a passion for artificial intelligence and we are doing this because i love to have conversations with leading minds in ai and i want you to help to learn to understand and adapt artificial intelligence this is episode number 15. um wow it's number 15 and if you're live here introduce yourself in the chat if you want to check out the polls you can ask questions all the time i will collect them and read them out later if you listen to this in the podcast just enjoy if you're watching this on the video um the same and the topic today which we have is the next wave of ai um a little bit of a background deep learning came back it's like a renaissance um 2014 it created a certain hype but it also changed a lot i mean that because we speak today again about ai that's because deep learning came through and every you know today

we have a global ai industries government having strategies and investing billions venture capital is financing a lot of ai enterprises adapting ai i mean ai is real but what really ai is what can we expect where will it go well these are topics for today um in 2017 had professor hans oscar wright from dfke he gave a talk about super ai or as um you know strong intelligence on our conference Rise of AI 2017 and he already said this will take a lot of time because we have a lot of intelligence a lot of elements coming together and we just have a tiny part solved of it so it's a very complex topic and then i read the paper of gary mc gary marcos called the next decade in ai which was published in february this year and he talks about the next wave ai and he calls that we need hybrid knowledge driven reasoning-based approaches centered around cognitive models and that's not easy i mean it doesn't it's not even easy to pronounce it

to describe it and i think to fix it and then joshua banjo said on a conference i attended last year it's all about cognitive agents so what actually is cognitive agents how can we reach it what does it mean well that's why we have this topic here today and my guest today is professor christian casting and he is a professor for ai and machine learning at the teo darmstadt he was prior active for fraunhofer institute he was at mit he has a long list of awards uh from the more popularistic uh deutsche kahi price where he got a hundred thousand euros last year two best ai papers top 100 list etc um he's the designated director for the tensor institute which is a collection of over 12 ai institutes or universities uh in the region with 20 new professors he will mention this later he has published books and papers i counted over 318 papers so highly productive and creative just to read out one title it means discriminative non-parametric learning of aromatic circuits if

you haven't understand it you're like me i didn't get it either but he has published hundreds of those so he knows something and that's wonderful so let me bring him online there you are hey hello christian thanks for invitation i have the first question for you which is your most famous student you mentioned him is this guy right yeah i mean at least in germany at least in germany so this is so he got more famous because he was criticizing a little bit politics and news um still i heard he studied at your department he started well he studied in dortmund and he was back then already a very fascinating student in my opinion he did his master thesis with me and the actual story is that after the lectures we i i'm sorry i'm still one of the old guys smoking so we always had a cigarette together well i was smoking and we were just talking and so he was back then already a very clever person trying to listen having his own

opinion so it was a pleasure to get in touch with him and also supervising him a little bit so that was that was great and i think he's still doing a good job in a sense of independently of my own political reasoning i think it's good to get a little bit of discussion right and i i think it's interesting normally we have outsiders going into ai i mean a lot of consultants business people i mean i'm an outsider too i never studied informed computer science you know i come from the business side but then we have someone who studied machine learning and he went into media and music um it's very interesting and uh just just as a curiosity so um today we want to touch a couple of interesting topics and before we go deeper into the topic itself i would like to ask you describe a little bit your what do you do as being a professor for artificial intelligence because you told me corona was even more busy for you than before

yeah so maybe i i should be very honest here so the last half a year and actually going further back i was mainly business busy with what you already mentioned the tensor institute and just to be very clear here no decision yet it's just that the state of hessia they decided to establish or they want to establish an ai institute that is not just at one university but trying to really get together all universities and also applied universities in in hashem because um you know ai you can do fundamental research but you should also try to get this into industry into society you should really try to get more startups so most likely we need an ecosystem and that's the idea behind the tensor institute and tensor because after the parameter institute there was the vector or is the vector institute with jeff hinden and so we thought hey let's go some more dimension on top of it so it's now the tender institute but i should mention that if it comes the vector institute

also gave us a letter of interest so hopefully we can build up a strong collaboration also with jeff and all the other great people also gary marcus so now what i did i do well because of corona everyone likes now zoom and other online meeting platforms so it was more like having one meeting after the other a lot of political meetings but typically as an ai professor i would argue that you talk a lot to your students you try to develop algorithms you try to evaluate them and if you're working interdisciplinary you may also work together with people building robots and then really putting your algorithms into life in a sense into the real life and the real environment and typically you're frustrated as a professor because you have big ideas it doesn't work but that's also the motivation right so you you have this spirit of it has to work come on we have to get it right and then downstream in particular because of gary marcus because you mentioned him um we are

working also in a we also have a center for cognitive science which for me is the twin discipline of ai and so you also have a lot of chats with these people trying to understand what can we learn from for ai but what can they learn from ai and actually we try to not distinguish them we should just say look we are interested in understanding intelligence and you need both and some other disciplines as well so it was just a lot of meetings because of the positively decided uh tensor institute and for how much are you still uh you you mentioned before that your algorithms you have written yourself or your team has written are in plenty of devices and out there so just give an example like well we can find a tiny piece of your idea or code yeah so we were working early so that was all triggered when when i was a student um luke darratt was joining university of freiburg and i was doing my tour through all the the

labs asking what are good topics for master thesis and he said look i was attending a talk by daphne color that maybe some people know because stephanie was if i'm not sure i think she was professor at stanford and now she's really into business and into bio and medical informatics and trying to help developing new um drugs and everything and she was giving a talk about one of the big issues in ai how can you get methods that can deal with uncertainty together with methods that can deal with complexity and within the field of ai that means you're working with let's say probabilistic graphical models or you may use some deep networks that can also deal with probability distribution and you would like to get databases and ultimately ultimately logic together right so you want to have something like um a high level programming language that allows you to still deal with uncertainty and so he was saying look i didn't understand all the details of what she was saying but i think that's an

exciting topic let's do something on our own let's maybe even do it better and i thought that was wonderful right not doing exactly that but more open try to do your best so this is where where it started and um out of that also during my time at mit we developed also something that um fire dapper and there and stanford went into theory because if you have a dialogue system you would like to understand the intentions of the user and if you want to program that you don't want to do that on the very low level right you would like to provide high level ideas background knowledge about what could be the intention why is this person asking that and so they were using some parts of what we have done at least in the early versions of theory that's what people told me we never got patents on that we never got any money out of it but that's roughly it and so i guess at least on the first iphones many people have

used our okay so let's uh let's talk about the news which came out i mean i have the feeling that since deep learning we are like lacking the breakthrough so for me um alphago was one of those highlight moments and then we had a couple of but it's like there was nothing revolutionary out there so everyone's talking about gp t3 and i would like to ask you why are all those people out there excited about it i mean critic criticizing is always easy but what is exciting them yeah so so what is exciting if you grow up as an ai researcher in 80s 90s whatever you realize how how difficult it is to come up with a system that looks intelligent or smart right it's really really not easy now with this gpt3 what is happening is that you have the system where you can easily ask questions or you ask to complete sentences or to continue writing little stories and the output of the system is pretty amazing so we stay on the positive

side right so you could also say there's a little bit of cherry picking in there but it's simply amazing how much they can get out of this data and the huge model that they have but what does then look a little bit closer into what they have done actually um it's it's amazing because it also resonates well with old stories but just implement it in a new way in a more differentiable way we call it differentiable it's the you know if you want to be the cool kids on the block you don't want to call it deep neural learning but you call it differentiable programming because that's what is really at the heart of deep learning differential programming yeah let me explain it it's pretty easy so you you sketch your algorithm so and and that's also going back to the three ways um or the third wave in the first wave you are just writing down algorithm by hand right so you have to think about every eventuality you code it you test it

it's a lot of work if it works it's good because you fully understand what is going on but it's really hard now in the deep learning area what they realized is if we can get major programming blocks differentiable which means you can compute a gradient and you would like to compute a gradient because now you can sketch an algorithm and then using data the gradient will tell you how to change the sketch that you have provided how to instantiate it how to change a little bit these parameters to get it tailored to work what is written in the data or captured um so that was number two phase two that was number two now number three should be that you should be a little bit more human-like at least in terms of communication and reasoning capabilities that's also what gary is stressing and in particular you should be able a little bit like the human to recognize new situations and potentially even adapt to it you know if i if now something breaks down here

in the stream we will not simply continue we will simply also not simply break down we will try to find a solution to adapt to it and if it's just smiling and saying hey damn it we will meet again we will find somehow a way to deal with it and that's pretty hard with um still computer science methods and ai methods right we are good at specific tasks we are amazingly good and gpt 3 now coming back to that question at least for the textual domain is giving you the impression that the first time we may have a system that can deal with many many different tasks even tasks that you did not have in mind when training the system and that is something that seems to be very essential for a truly smart system right so if you you were talking about alphago alpha zero well that system was pretty amazing at chess and go but it's not good at i don't know um um cooking it's not good in in driving a car

right now we know this this problem of that it's all narrow ai or very focused one and transfer knowledge doesn't work but is gpt3 for what can i use it i only know it that people are writing blockers we will have to answer now right people just realized that it works well in asking several questions even about topics you have not in mind now how to monetize that how to turn that into other applications how to maybe even put that onto a robot big big questions but that will start now it's it's it's not that you know you instantly have is it only an nlp tool or is it also a tool for all other fields like which processing for example well it's well it's definitely not for image processing as it is right now people are trying of course now to use a similar architecture on the line um now also on images and soon you will also see that you have multi-modal version so you can have text and images and whatever um

so it's all about these transformers that's what how it's called right it's called a transformer network or layers depends on which perspective you want to take whether you take the rule network or layer words and the transformers if if we want to stay very simple it is a differentiable version of something like it well not exactly true what i'm saying now but just to give you the gist a little bit like a differentiable version of a nearest neighbor search not much more so nearest neighbor means given something you try to find the stuff that is closest to it one or many of them and now what they do there they have a big dot product implemented in a differentiable way i'm simplifying here because i think that the details are not so important and it's amazing how much you can do with that now you can transfer the idea to other domains but we still have not a single system that can deal really now with images that can even take actions or make actions

that interacts with the environment we don't have a single system that can listen to you maybe we now have a better idea how to set it up in the future but you still have to set it up and then you have all the little other interesting things but some people complain and say oh but how can we make sure that the system always does what it should do and not does anything crazy it's an issue on the other hand some some humans unfortunately also go mass in their life so right i mean what i want to say with that if we want to have an intelligent system that never fails that sounds to me like a contradiction yes failure is part of intelligence in a sense and that's why we want to have the third way because you want to learn from your failure right and i mean you may think of reinforcement learning but there might be also other ways because we have to get reasoning in there i mean we as humans you

know i'm not just doing simple reinforcement learning i'm also thinking about what i have done and i may skip some of the simple let's review and get the experience again and i'm projecting and i'm saying okay i will not because you talked about big breakthroughs um so first of all in other scientific disciplines it's not like you have every year a super breakthrough give us a little bit of time but i think you will see more breakthroughs at the moment where we get reasoning and learning together and there are some approaches already and it's now about scaling and making it more effective and and the reasoning part is important one for check and balance for due diligences for opening the black blocks we call it explainable ai and we had two talks about this to to cover um from your view from scientific view i mean you're doing ai longer than the hype exists i mean the hype is still was increasing but i would say most politicians didn't even know what agi means three

years ago yes and today most don't even know it yet um so for you was do you agree that deep learning was the thing which pushed it forward and opened the box of having more resources and everything um yeah it's a combination what was happening so i often hear that people say oh this is an old story there were already neural networks in the 40s and then um stages degrading you know already in the 60s 70s per se and then all of a sudden there was back propagation and then depending on whom you're listening to you know you've you had back progression already in the 70s 80s whatever it's all the old story well sure mathematics is also pretty old and the idea of intelligence goes back to the greek people and maybe even further so what was really different now is that also the hardware was changing and we were lucky enough that there were some people that understood that were not giving up to implement maybe even already existing algorithms in the beginning

on new hardware and all of a sudden you could do things that are that were well you didn't believe it was that easy before now that doesn't mean that the question of intelligence has been solved because we also know that humans can do it with so less data right so deep learning on average if we are not making use of pre-training and transfer learning and everything is pretty data hungry yes so we should try to understand how what can we learn for example from human in order to be more data efficient one question in terms of hardware tpu gpus they are not really energy efficient our brain is pretty energy efficient right so there are so many interesting research challenges still ahead so now anyhow let's go back and there was this breakthrough in make it happen and then another thing that i think is really important is that there were companies involved because why is that important first of all a company like google can afford to put engineers on the topic and develop

something like tenderflow they were not the first going for a framework for deep learning but they can do it in a way that it's maybe a little bit easier to use they have advertisement they can build up her brand and then all of a sudden that's for me the real difference to what was before you could do something like machine learning without first starting machine learning it was made easy and that's always a key to get something into the mess democratizing technology it needs to be usability needs to be very easy yes and still we have of course elsewhere to work a little bit more on on that part but i think that's really important and that's why um for for me and also for for many people in darmstadt we believe one of the big issues is now to get software engineering for ai because that's not existing right and if we now want to make it that um programmers can build these amazingly new systems we also have to support it and it

seems to be completely different i mean how do you debug a learning system right so um but so i want to say it's really something where you can get all of computer science and get again back together right because it's software engineering you may have databases you have hardware you have so many different sub disciplines it's amazing because you can talk to all of your colleagues i i agree from a niche topic ai it went into a mainstream topic which is now i call also ethics philosophy with a project management policy morals ethics i mean ethics belongs part of it yes so yeah the whole topic expanded is huge sure but just to be clear i mean if you look at uh the dark mouse workshop where where it was all invented in terms of at least the term ai and you had cognitive scientists involved you had philosophers being there so what i want to say is the origin it was always interdisciplinary but then somehow mainly in my opinion computer science was pushing

it over the years with all the big conferences and now some other people are coming back and i enjoy that um but it doesn't mean that you always have to talk to everyone right you sometimes also need so to say your way but definitely and in particular ethics it's a it's a wonderful and important problem um although i would argue that we need much more ethics from both sides so the the more classical one that is coming from human science maybe um but we need also people that understand what is technically at all possible and there in the us this is already much better i think in europe we and i mean we have super interesting super strong people but that is developing right now and i'm very happy that it's developing in particular in germany however we have to be careful that we are not discussing on the debating uh ethical ai by having this idea ai is already human-like and we use the human as a example for ai you know or as

a not example that um what do you say that i'm as a substance as a a little bit as a role model so i once saw something if if if ai is written on powerpoint you know it's it's fiction if a uh you know if machine learning is written in python it's called you know it's like expectations i know people out there writing on the slides you need ai to do this and we will come over the next virtual chats more on the application how to do it how to run it um with you would like to touch a little bit for example joshua borghart is in the tattoo and he wrote we need hierarchical planning honestly there are so many ideas out there which i don't fully understand so i don't understand the full problem deep learning i get it i mean that was one huge technology which then got more detailed which pushed it but i have the feeling we're now trying to figure out how to put more pieces together what's next

but it's also going in so many directions i've seen i mean the paper from gary marcus is only one of many what's coming next can you describe a little bit why we are kind of stuck it feels like we are stuck and maybe why and or why it only feels like and we are doing a lot of impact it's just but i don't think we are stuck sorry i mean this is like a little bit like people talking to me saying um the next ai winter is around the corner and it has started already no what has started is maybe sometimes people are frustrated because we have these super high expectations that we have human-like systems already but if you look what we are doing is there are so many success stories of making use of ai in real world medical domain we have done a lot in agriculture and if you think about climate change there's so much interesting um combinations of debate so there will not be an ai winter now in my

opinion because i ignore them i fully agree and i always say 99 of ai application is not done yet if we stop doing research we still have enough to do for the next 10 to 20 years i agree for me it was just the feeling on the researchers side that it's less less attention focused like like oh that's the new excitement like in our media awareness like jumping on the new things well i i think there are a lot of papers that should get more attention and that have the potential it's only the question of um if you're at a university whether you like to put a lot of attention on it because you are sometimes at least in europe also this more shy person you say hey if it's good it will survive it's a different culture sometimes you go to the us they hype it a little bit more maybe which i like i mean i i was at mit i also got the idea that you have to do advertisement but in

my opinion like gpt3 wow i mean you could say it's a breakthrough doesn't mean that we have arrived at agile but it's definitely something wow um i think all the stuff about um the ethical part i mean there was this very nice science article about now maybe already three years back about how they capture the stereotypes of us humans if you learn from text for me that was a very strong and important paper as well unfortunately it got a little bit discussed in in public in a weird way because it was now saying oh um machines are racist and machines um take over stereotypes but keep in mind it's a mirror of what we are doing right and and it's nice that we have now a tool to show that we are maybe not perfect so now it's the question and this is where again we need then the discussion with other disciplines what you make out of this result but it's not a in my opinion a problem per se of ai because it's

more a problem of what we are doing and this is a big debate currently going on between you know is the data bias are the algorithm bias how do we deal with biases can we get them unbiased at all super interesting super important discussion and and definitely for the field in my opinion breakthrough and explainable ai it's in that sense a breakthrough although there's so much we still have to do but i still recall when i was a phd student there was not much sound chance to work on explainable ai because people would say no you either know your idea doesn't work and the problem was a little bit different i i mean i agree um it expanded to many many subtopics and new topics and i mean the whole ai adoption is a huge new feel i mean we have plenty of homework to do um let's jump back to where we're developing on the research side because yeah let me you you raised this question hierarchical planning and why do we need all

these different things look um to a large extent ai not completely but there's a subpart that is also about engineering such systems now as an engineer i think i don't like um to just use deep learning the way we do it right now at least in the simplest case where you say here's a big monolithical big block of a neural network i need enough data i get lucky if it learns it or not as an engineer you know um i once was talking to her at arsenal he is a neuroscientist a pretty big guy there and he was saying look uh i'm a neuroscientist i have to understand the human brain but if i have to develop a robot that should also operate at night i could use infrared i don't have to now understand damn it how could i do it without infrared so that the human or the machine can operate at night so this is i think what you want as an engineer if you want to do planning and you have

a great planning algorithm for example for hierarchical planning if you could make it and use it that's awesome because then you know for that part what's doing now the challenge is and that's why i was talking about this programming for ai is how to combine the planner which is maybe a little bit more symbolic with this differentiable deep learning this is happening now there are papers trying to combine we try with the third wave we try to push what we call systems ai to live it like systems biology each cell is one of these ai algorithms but you can put them together and how do you do that what is what are the mathematical uh foundations for doing it can we still provide guarantees and if we have that that's cool because then you can put as much white boxes in there and only for the parts that you don't really know you you are happy to make use of the of the black box i think that's what you want to do as an

engineer and this is um how actually do you research i mean i'm curious how do you research this like do you do a lot of testing second if you publish it is it like that accessible for everyone i have the feeling that ai knowledge is more accessible than other technologies because it's less less based on patents and more papers about just the feeling i i have the feeling that especially china for example they participate a lot from our research and in other fields they couldn't do it that fast yeah i would argue that most of university people academia they don't care much about patents once in a while you do it if you're a researcher at microsoft research maybe also uh because we have one listening at uh dave kaii and other places you care about patents in europe sometimes we have an issue because if you want to go for a patent in the u.s you know it's an invest and if you're at an institution that is maybe not as rich you maybe

don't do it at my lab we we like open open source and open signs right we put our code as much as possible online and so that people can try um yeah and how do you now combine i call it old-fashioned ai and the new ai you know you call it symbolic you explore different ways one simple way and you talked about alphago already it's a very simple hybrid system right you have uh mcts or monte carlo tree search so you do a search but in a stochastic way and then in order to evaluate because you can't um check the whole search tree you you have at some point you you drop some of the subtrees and so on you make use of the networks to capture this evaluation function in the sense how promising something looks like okay it's a combination right um what we are currently doing is we come from what is called probabilistic programming and statistical relational learning so you have a little bit of a combination of logic and uncertainties

and now you can put some let's let's stay in the database analogy i think that's easier for for many many people so some of the tables you can now replace by your deep network and then the question is how can you now propagate information and it's again the question of the gradient but because we are making use of techniques that provides a gradient it's for now not a big issue so you can have a system where some parts are neural networks some parts are these arithmetic networks you mentioned in the beginning this is you can view it a little bit like a deep neural network but just for probabilities and um you don't have to use any analogy to a neural network but it's the very same idea that you would like to have a heat system in order to deal with joint distributions and then we have also the logical part that can deal with uncertainty um coming from statistical relational learning and we combine all of them based on work that else and

colleagues have done at ku leuven but also uber and so on so many of the companies they also understood that they need a programming language a high level programming language because otherwise some parts of ai will not scale not at least the way you would love to have it scale right what do you mean with programming language do we speak about python or do we speak about tools below um no no we ultimately we would like to have for example python or your c c plus plus f sharp whatever you prefer but we would like to have this language dedicated to ai so the first step would be that learning algorithms are first-class citizens of these languages so let me give you an example you can run machine learning in the following way you instruct the machine to and you say okay download images uh from there and classify into female and male and then send everyone who is a male sentence and an email so i have described something where you definitely need machine

learning without even calling it machine learning right even when specifying is that a support vector machine i'm using for classification is it a deep network or whatever and i think this level we have to reach in order to really democratize ai and the machine should make the decision should i use a deep network here should i use the sbm for that particular part because if you talk to well and i'm only talking about biologists now because we have this we have this collaboration on and deeply phenotyping so trying to understand plants and how they react to stress you know they they are not programmers they don't want to go even as deep as tensorflow and specify the architecture so essentially you could also say this is something like ultra ml or outer ai 2.0 right you need even the next level but maybe we need a little bit of a programming language because you still have to instruct the computer at some point i guess we will have natural language instructions for computers but that's

not there yet but that's why you need a high-level language and you can view it really for example python python is now a scripting language but you may also go for any other high-level programming language definitely i feel sometimes we are a little bit more like assembler for ai right now not exactly tensorflow managed to get away from the sm but it's still not that easy it's not that easy to debug if you're not an expert and you want now systems that tell you hey maybe you use the wrong architecture here hey maybe you don't want to use adam as an optimizer you want to use something else and hey maybe you can check here and i will do the decisions for you you don't even have to care evaluation and coming back to ethical part how much can we automize there so that we avoid that you do something wrong in the evaluation so many so many interesting things so much to do so much to do so i have two thoughts um first

we can get inspiration by human intelligence for ai but i don't believe we recreate human intelligence i personally believe machine intelligence will always be different and it will be surprisingly different and uniquely different like like biology itself has thousands of thousands of solutions for different problems or sometimes the same problem i just say we should stop saying agi is the goal and we want to have a human-like ai no it will be very different and beautiful and scary different at the same time but that's my view you mentioned this before like expectations shouldn't be should be matched to to realism yeah well first of all i don't like agi as a term yeah no so okay so i i grew up as a very standard boring ai person and if you read any introductionary textbook like norwich russell or whatever they don't distinguish between general intelligence or agi and ai because that's already part of ai so why do i now have to create a new term even for something that is already covered i

don't understand that but at some point you start to adapt and you say hey use that term if that helps you if you enjoy it fine with me now about the comparison so first of all i like if you say already human like because it doesn't mean it is like the human it means more like okay we may take inspiration from humans because ai can also be different i'm fully with you i mean imagine we are trying to get self-driving cars and our only goal is to be very much like exactly the humans terrible yeah we haven't no but we haven't achieved anything you will have the same number of accidents maybe you will have the same number of traffic jams so at some point being different from humans is good we also if you think of interpreting fmri or maybe at some point assisting in very um challenging operations um medical operation maybe you don't want to be human like you want to be very precise and reliable i'm sorry humans in that sense

are not as reliable but then if you really want to be have to have a truly intelligent system in the sense of a human so a general problem solver right general problems if you want to have that then maybe you can't be perfect everywhere maybe that's the trick actually to become a general problem solver who knows but now my main point was because you said you don't want to be like human but keep in mind cognitive science is making the assumption that we apply information theory and computer science to the brain and to the mind and to understand cognition so just to make it clear i mean when you develop a new algorithm that may also help at some point to understand better how we function and why we behave the way we behave so i i'm i would disagree with you i would argue okay machine do not have to work like humans but i see humans very much like biological machines they of course do not run on yeah but but as an

ai is called i talk about machines i don't care much about the so there are the three levels of david mar and you can debate them a little bit but it's first you would like to understand what is it computationally that we have to solve so you want to see mathematical formula now after you know what you have to compute you can ask how do i compute it that's the question for an algorithm and then we have this pseudo code algorithm and you can ask can i implement that on a standard computer for neuma can i implement that in a biological system so you should distinguish between them and for c a i is just about algorithm and the computations and sometimes you can also change now the implementation and of course that may help also impact on the other levels but per se come on we are interested in algorithms and we can abstract from the underlying architecture i like what you said because i i had this with eric von miller discussion that

we are biological machines i mean you can reprogram your brain we are working in patterns and i think a lot of ai is inspired by this form of intelligence but then with for example object recognition we discovered that they can see with infrared they can see way more data that can be faster i mean we currently have like an israel ai detecting movement patterns and sweating and they discover in china if you might have corona or in if you are a terrorist in israel so something human can't maybe see they get even more data so it's very interesting we also had this discussion that ai is mainly a tool and human plus ai are the most productive out there currently not for all fields and um that we go there so i i always say the moment we have a true ai i don't agi i don't even know if i want this because it won't stop learning you know it can some people out there have the fear that we can lose the control

like you know it can be your rough or amoc and we spoke about this weaponizing ai two weeks ago in a virtual chat where you have this risk out there too and it's interesting how they make no decisions automated currently everything is done by humans and ai has to wait for orders well but on the other hand for me and it's maybe now only thought provoking it's for me always interesting whether whenever you talk about agi and i'm using this only now because i'm talking to you the term then people say all of a sudden oh damn it because it truly is intelligent the first step it will do is to get rid of humans i don't understand it it might be also because it's truly intelligent that it understand only together we can save the planet but that option seems to be never discussed it's so clear that we have to get rid of humans and for me that is a projection of what we humans think what we would do if we have

general power and so again it's a little bit like with the uh with machines taking over our stereotypes and then we blame the machines i think this debate to a large extent is always actually about us and we use ai as a good excuse to talk about ourselves without naming ourselves so for me um that is not an implication and to be honest and eric horvitz is always saying that i think so far ai has if it all saved more lives than it killed and still we're talking much more about how where it could maybe kill sure and yes i have also signed as so many others the letter that we don't want to have ai and autonomous weapons and everything together but then to be honest and that is what i always want to ask elon musk and also i wanted to ask stephen hawking so why do they blame ai if they could also blame the weapon in first place the person who is building this weapon isn't he or she as much

in charge than the ai but all of a sudden we have accepted young weapons that's okay but why do i build weapons in first place i don't understand that but it's just me i had this discussion and we need weapons because it's uh um how do they say it's game theory yes and if one starts having weaponized ai everyone else needs to at least defend it's like like nuclear bombs it's this strategy game behind i understand that part but it's fun to see we accept that they are weapons but we say now it's really horrible if it's weapon plus ai and we start blaming ai for it but we should blame also the weapon and the weapon industry for it and we should blame us humans because we follow game theory here so i just want to say always please let us blame everyone it's not just ai and we should not just be happy that there's a new player and therefore we can blame her whatever you want to call ai so you just

said two things i noted first you say agi is more the idea of having a problem solver we humans are very good problem solvers we need little data to fix things and we need very little data to memorize it because our brain is by the way not active all the time it's just very focused and highly superficial but then you said speaking about truly intelligence what is for you truly intelligent because if i listen to you we are not good in math yes like calculating every you know calculator can be faster than us we are not good at driving we are very terrible a lot of things what is for you truly i can tell you for me true intelligence doesn't mean that we have invented the flying spaghetti monster that is controlling the world truly intelligent doesn't mean that you are perfect at everything for me for example an average human is truly intelligent already because for me that is the amazing part that you're a general problem solver and we have this capability

if i fail somewhere i can ask one of my colleagues one of my friends for help this is for me really smart smart doesn't mean for me always optimal it just means that over the long run of your life you do a pretty good job but that's my my view of what is and is this the main difference also to to the rest of biology nature because i have the feeling that a lot of animals they have solved problems but a lot by reproduction like by mutation reproduction very few animals are able to use tools few but some and that's like the huge gap between humans and the rest because we are using tools we're using language we're having paintings and everything still saying that we have a lot of intelligent animals out there i don't say they are not intelligent but after your definition they are not that truly intelligent i i don't know i mean if i think of apes some of the apes amazing right and there are the classical studies that

show if you take a five-year-old human they can't solve the problem and then you take an ape she is solving it instantly right there are so many i i mean i'm very i don't want to make any any commitment to saying that animals are not truly smart i think we also get more and more insights there and sure you may not have the same tools a bird has not really hands and now we start blaming the bird for not having having hands still they build their nest pretty amazing right so i mean and you grow up also as an ai person then you say there are different rates of intelligence right so if you understand what is an agent an a.i you also some people say at uh emirates something that is controlling temperature you could um argue it's an agent in the sense of ai it's maybe not the strongest agent we have a sequence of different agents and in the end you have maybe something like knowledge and reasoning and learning driven agents

okay but the other ones are still part of ai which is also times i think that is contributing sometimes to the confusion about ai because it feels like everything is ai and therefore it's not a good definition um but that's not true in my opinion it's really we want to understand what is intelligence and in order to understand intelligence you also have to understand what is not intelligent it's part of the game um so two thoughts first um there are i i watched on netflix the documentary explained and they had one about intelligences and they described i think five or six forms of intelligences which we human all have animals often have but it's like emotional intelligence it's like abstract intelligence like you know time and so forth so one thought is how can we implement this into ai um and the second is what human and all other forms out there even trees they have reproduction purpose a lot of everything in the universe is a lot about reproduction and growth and a lot

of energy goes into this so what we are missing is the motivation for ai i know that you can motivate ai by solving this problem and then you're giving it incentives to optimize on this but do you think we need inspiration there too to have a long purpose motivation for ai out there to so it starts evolving even more so let me let me clarify i think we are jumping between different levels of abstraction here so you're talking about reproduction reproduction is um one way of doing classical mathematical optimization okay right so you say i'm looking at these parameters now i have also another version of the parameters where these different individuals they perform in an environment we have specified an objective function and we can clarify which of them are good or bad in reality they die or not let's stay very simple right and the one that are dying definitely were not good enough so you know in evolution you go for the ones that survive in the one or the other way

so you can phrase that as a mathematical algorithm it's called evolutionary algorithms but you can also make a direct link to very standard gradient based optimization so there's no contradiction now additionally there is one paper by papa dimitry one of the super heroes in theoretical computer science and many other art parts it's a really an amazing person and he wrote i think in the journal of mathematical biology he wrote a paper with it's not the title but in the abstract or somewhere it it is essentially written like that why sex is not good for optimization okay and with reproducibility depending on what you're implementing which uh reproducing operators you're implementing so if you have a crossover where you say okay i'm taking two individuals and i'm now mixing their genes so a little bit like what we know from very classical school genetics what does it mean whenever you're close to the optimum that you want to reach you throw away half of it you're happy to take half of or not as good and

into your model right so that is why reproduction is much more about being robust you want to survive and be flexible but again you can formulate that as either an algorithm but if you want to reach the optimum maybe you want to use other optimization techniques uh growth babies yes regarding your time and you mentioned robust ai there too like yeah no so refraction you said it's not the optimal thing but it's robust so it's not the best result but it works yeah it's like in science as well once a while you have a superstar a genius and he will make this breakthrough but on average to survive as a scientist and as a scientific discipline you should be also robust and not collapse if there's someone saying oh ai is not good and then oops and we collapse all oh you need to be robust you have to deal with this little attacks it's also good because it it helps you to to reflect and adapt and to also check where you want to

go and i think we don't want to go into the regime where you enter maybe the super optimum of everything um and then maybe we're not happy with the optimum so i'm happy if i can sell the product that will survive long and for that you typically need robustness and in particular if we talk about agi because you don't know what questions people will ask the system if you think of a chatbot you know you want to be uh robust to all the stuff that can be entered there even not the intended input one question other question if you're driving if you want to have an autonomously driving car you may not when developing it think of all eventualities so again robustness means you're not failing or killing people only because there's a situation you are not really used to or is too far away from what you have seen and so in that sense robustness is important and again it's maybe not the optimal solution but it's a long debate what is optimum robust

because you can always take a different level and then the robustness is part of your optimization and then all of a sudden it is part of your objective yeah let's relax there you just want to have systems that are not going nuts something i would also like to have for society it's november is coming um so um regarding time i shortly summarize so what i took out is also that you are working others on this hybrid solutions where you combine different models as you call it the third wave to make it for my summary i like it to make it more usable yes like like one key thing is not to make to combine more skills to make it more modular but also to make it more usable is one goal so more people can use it well more people but you will be surprised how much more breakthroughs you will see if you can do reasoning learning acting all these things together if you can quickly combine reasoning and learning you will get even

better deep learning systems that you have that we have right now so it's both so let's jump last thing last thought 2030 in 10 years we know where we are today where could we be in 10 years and what is needed like high level so everyone out there knows what the milestones could be so for me where will we build i i think one of my dreams is that we have all accepted per se ai systems and it's a little bit like when cars were introduced there were beveled engines in the beginning now everyone drives a car i hope we will fix the climate things so i hope that we have all enough knowledge to understand that ai has so many positive aspects um that means we need also enlightenment not only in the academics not only in this industry but also at school in the general public about ai i mean we have to teach ai parts and also programming and other parts of computer science at school already if we have that then

i hope we will see that it actually helps to make the world a better place and that already in 10 years challenging yeah i understood maybe in 15. no no i i like the last sentence because we need more training for this more awareness and to make the world a better place i think this is a beautiful last sentence um christian i thank you for your time your insights your knowledge and everyone else stay in for a minute and i will talk again thank you for being here definitely please invite again and i hope i can also invite you to some of the tensor events if it gets accepted i i it will be pressing thumbs and i'm very optimistic we'll talk about this wonderful thank you thank you again for the invitation have a good day um so everyone else there um next virtual chat is next week 11 a.m again and it is with dr oli valttenger who is head of a.i siemens lab and we will talk about industrial applied ai um

today it was on purpose theoretical and high level we will talk about how can you apply ai also how can you apply it in the industry purpose i'm i'm very curious to see what applications are which is already working out there so join in sign up for it also i would like to paint out that we have um the summit in november 17th and 18th of november it's a hybrid event so there are roughly six tickets left to be on in person here it will be here in berlin there will be three stages and you can be there in person it's limited to 100 people but it's unlimited virtual so if you want to have the same experience like individual chat you know with swap cord and the networking you can buy a stream a networking pass or you can just watch for free but then you can comment and can't ask questions so get yourself a ticket for this too and that's it so stay home stay safe i hope to see you next

week again and thank you for tuning in this time have a great day

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.