good morning good afternoon to everyone out there in the world i've seen we have people from peru to sydney web americans who have europeans very few germans this time so very international crowd it's wonderful to have you here again for episode 14 of our Rise of AI virtual chat my name is i'm an entrepreneur and investor with a passion for artificial intelligence and i love to invite interesting people to this show to ask them all the questions i have in me so topic today is you see it on the shirt it's the future of warfare and ai and um just to give you a good example france has a national ai defense strategy where they i've i've read it this morning where they've written on dozens of pages how they use ai and technology for their national defense if you look for germany nothing like this exists so we already see the discrimination it's a relevant topic in china and the united states i think it's touching us all even if we don't serve in
the military it has an impact on us and in the future also if you want to prevent this so the best guest for this is dr rica franca who's already in the green room and i will have her live in a few minutes she has a phd from oxford and wrote about the use of drones by western armed forces she did her masters about the german military revolution which is needed and it's not coming in sao paulo and san gal she works today for the european council of foreign affairs so it's an ngo and a think tank which is mainly focused on foreign policy in europe and within this she's focused on defense and security and she has published a lot about future warfare drones ai and she has her own podcast so if you want to listen more to her if you want to dig more to this topic it's zika heights helper podcast i'm sure someone will post it in the chat and well let's bring her online hi hello a wonderful good
morning ah it's good so you're in london i'm in berlin and uh we talked today about europe ai military and you decided you want to have the hardest question at the beginning that's always good which is will ai and wars no is the very short answer um no i don't think so i mean this is really one of those things that you hear a lot there's always a new technology i mean in history there's always a new technology being invented and someone hopes like this really will mean that there will be no more wars um i mean we had this hope with nuclear weapons by the way um okay yeah yeah and you know that didn't happen um i mean with nuclear weapons the idea was that war would just be too terrible um for for it to happen i think with ai people who argue this and there aren't that many um would probably say that if you have a lot of ai in warfare it doesn't necessarily mean that it ends worse but
you would have less people or no people i mean it's basically this idea of let the machines fight it out um and as much as i like this scenario although it's a bit it's a bit tricky um i absolutely do not believe in it because what we have seen time after time with new technologies in warfare is that they basically come on top of everything that already um exists i mean sometimes it does replace certain things of course you know we don't need we don't use bone error anymore but but usually there's always this element of yes sure you have new technological capabilities but in the end you always have the kind of 18 year old soldier dying somewhere in the mud and this really is very important to me because i don't i don't want to create this this idea that you know aim warfare and cyber warfare can just mean that we don't have this anymore and it's all going to be great i i absolutely do not believe in this okay that's
um i thank you for this answer it's a good indication so let's jump in examples the easy one to explain every once on the same level we speak mainly about autonomous weapons and we have seen it there was a backlash in silicon valley i think amazon went out and microsoft went out of contracts they all don't want to do military defense so it is a topic people discuss it i think there are signatures by a hundred or one thousand companies and private people and researchers about we don't want to have autonomous weapons still they exist let's talk a little bit about real life cases yeah it's out there already so the first thing to say is that and as you know and all the the listeners probably know first of all even defining ai is a bit tricky right ai can mean a number of things and ai in the military realm also can mean a number of things and it really is a is a full range right so at the let's call it
the lower end you could have ai ai-enabled systems that basically just do like you know automation and then certain things are easier you know stabilizing a drone so it doesn't fly away in the wind stuff like that um and then at the other end of the the range you would have what a lot of people consider basically terminator weapons right a fully autonomous system that can engage targets on its own um and and this is this is the range that we have and the extreme cases are somewhat easy right because i think you can all agree you know the lowest level is perfectly fine it makes a lot of sense in the extreme level we probably don't don't want this but the question is you know where are we in the middle so that's the first thing to mention and the second thing is that you mentioned autonomous weapons right but this isn't the only use case for ai in the military realm you can have a lot of different ways of using ai in
in the military realm so intelligent surveillance and reconnaissance isr as we say in in the military right so how do you call it r i i s r intelligence surveillance and reconnaissance or i star also sometimes exists but anyway so this is basically you know surveillance um and here ai can do a lot of you know very useful things basically look through all this huge amount of data that that the military is collecting um shift through this data and find something um that's out of the ordinary um targets etc so the you just mentioned um uh silicon valley companies having issues with working with the pentagon right the one case that was a lot in the media was this project maven um a a project where google worked with the pentagon um in order to what to do what well to use ai to look through all the videos that u.s drones had collected and the idea really is you know to find to find something that's out of the ordinary find ied so so
improvise explosive devices find targets find a lot of things here we're not talking about killer robots we're not talking about armed systems this is really just you know surveillance but this is ai enabled so this is one example and you have ai you could have ai in logistics in cyber operations um in in forecasting in war games so ai in the military isn't just lethal autonomous weapons is palanti also involved with the military in the united states i i know that poland here um so that the data um uh broker or analysis company and they've been working a lot with border security in in the u.s um i don't know for certain that they're working with the the pentagon but um i i wouldn't be surprised um and in fact the pentagon in particular has been has been trying a lot to work with silicon valley companies which sidenote really shows us how important civilian companies are in this whole realm which is something that's relatively new for for uh let's discuss this it's like
so i feel i have a list from the french ai strategy for defense which is they say okay we use ai for communication uh collaborative combat that reminds me of my former portfolio company accelerated dynamics which had the swarm intelligence to control hundreds of drones and submarines to build mesh networks which adapt and that they never die which is an adaptive communication network which is flying or floating and it's always adapting we speak about submarines mines detection but also placement land robotics you know what we know from boston dynamics transport aircrafts connected soldiers which is combat suits and i think let me try i think it's it's there where is it that's the drone there is it right yes it's it's augmenting it's augmenting not replacing but augmenting the soldier you have a hut you see the data you get feeds in it is like combat suits exoskelets um we speak about satellites uh typical intelligence as you say like isr comments cyberspace and from your master thesis i mean nine years ago you already
wrote about this and you wrote about precision guided munition or stealth technologies um we all think it's science fiction but what i read in the news and i'm curious in this it's all real it's a lot out there not as fancy as we believe but more fancy than most people think yeah exactly exactly and that is my point um i always try to not just talk about military ai but about ai enable systems because ai isn't as technology as such right um and and this is what i meant earlier about this this full range um ai can mean a lot of things and ai enabled systems can do a lot of a lot of things and the level of uh of autonomy you give these systems really depends on well on the developer on on the the decision makers so there's a lot of possibility for us to shape how we use this but maybe maybe it would be useful um for the listeners to kind of have an overview of like why do we
generally want to use ai enabled systems in in the military realm and i would really kind of emphasize five points and this is from a military perspective right so you can say you don't like those but now you give us your five points because we have plenty of questions currently coming in which is very good so i will sort the questions and you just well five points okay okay so basically the five reasons why militaries pretty much all over the world really are interested in using ai to enable their weapon systems um the first one is speed right so if you have ai-enabled systems you can do a number of things faster why well because human beings just generally take more time especially looking through data and and deciding on certain things um computers can more or less you know analyze an unlimited number of beta and take a decision on it and speed of course is super important in warfare and especially in defense if you want a defense again defend against an attack
being able to do things quickly super important that's the first one the second one is stealth so the ability to hide yourself this is one that's interesting for me because i'm coming from the kind of drone world i've always been interested in in drones and the drones that we have at the moment are pretty vulnerable and pretty easy to detect i'm generalizing massively but kind of go with me and one of the reasons why they're easy to detect is that drones that we have at the moment have an uplink in the downlink so the the drone operator needs to send a signal to the drone um to tell it what to do and the drone sends back the data it discovered let me jump in here there's a not proven rumor but i believe it that isis used in the drone war um gps hijacking exactly to to to go against this up and cross link and there's a saying that once the drone had its goals it was able with image recognition to discover
the goal identify and make the kill decision of its own but no one is really saying this in public because this is against the law and even in the united states right yeah so we so so you mentioned an important point which is non-state actors increasingly using all these capabilities um and and um also being able to kind of jam them or head into them we had a situation years ago where hezbollah basically hacked into israeli drone fees and were able to see what they were seeing and so knew where the attacks were gonna come so um exactly so this is the uplink and the downlink um and the problem with this uplink and downlink is as you say you know this can be detected this can be hacked and there's also a certain element of lag time in there now if you give a drone a certain amount of autonomy which is ai enabled then you can make them more stealthy because you don't need this uplink and downlink anymore so that's that's kind
of second second advantage of why you would use ai in the military and the third one is is an easy one we're always striving towards is efficiency and cost reduction that's you know definitely debatable because it really depends on what ai system you're talking about a lot of them might be way more expensive than others but the idea of course is that if you can use ai you may need you know less people um and it just becomes more efficient faster on all of this related to this i just said it is the need for less people so right now actually drone operations are super people intensive um it's tricky to come to have the exact numbers but for you know use of an armed drones by the americans some say you need over 100 people involved and there yeah i mean again it depends on what you what you're comparing and processes it makes them very expensive and i think the price is we have less physical death as humans i mean the the
amount of soldiers dying goes down since second world war but the price is still going up so at the end you pay more per human surviving yeah i mean price the costs here are really tricky because here the big question is what you compare it to so a predator drone sure you can say it's expensive but of course it's much less expensive than a manned fighter i mean the current you know f-35 fighter that the us is developing and we're talking billions um drones on the way near but they also can't do exactly the same thing but so you may with ai you may need less people and also you may be able to take off some of the load from people and we're talking both physical we're just talking about you know um uh the infantry soldier having a boston dynamic robot just with your your gear you know whatever you need luggage you're crossing a certain country and they can carry 200 kilograms on top ex you know how you normally use small
vehicles or but it's adaptive yeah exactly so the physical thing is one bit but the cognitive part is also very important so again talking about data analysis at the moment you have quite a lot of people just looking at data and analyzing it and a lot of it is boring and dull and and it's it's tricky for humans to do this because we get bored looking at the same piece of land for like three hours um and so ai ai can really help to to lift this kind of cognitive load and then finally the fifth point and this for me is really the most important one and the most interesting one is that ai so goes the theory offers new military capabilities and this really is the holy grail of what the military wants right i mean of course you want you know stuff to be cheaper and easier and all of that sure you want to have a head start an advantage so no one is fighting you because they know they will lose
exactly you want something that others don't have and and new military capabilities enabled by ai is really what what especially the advanced militaries are looking at and here the one thing i would highlight and you already mentioned this earlier is swarming right having the ability of connecting several different units of military systems be it you know airborne drone or under drones or underwater drones all kinds of things being able to connect them in in a swarm uh can have enormous military capabilities you can overwhelm air defenses you can create well basically flying minefields to to occupy territory you can do all kinds of things and this is something where a lot of the the work is currently currently done because this could be a new military capability that really gives you this decisive advantage that you mentioned there is this book i think it was written by suarez or william hurtling kill decision recommended just describing state-of-the-art technology how they combined a drone swarm with um with chemistry so that they're able to smell you
so they identify the dna you know the the the smell the the chemistry which we all have uniquely and then they combine it with uh the algorithm of uh ends yes war ends and then they act like war ends and i had once at the conference professor tim landgraf who's researching swarm intelligence and animals to adapt this to ai so all these pieces are out there it's it's it's from my view very interesting because i see all these pieces but they are not yet and that's good perfectly combined so summary ai is super important for for military in the user factors we have a couple of questions now let me start with the one from henrik he says in my opinion military procurement at least in germany is not fast enough to keep up with the speed of ao innovation in the future are they any good examples from around the world where they do better at it yes and no so yes it's true that military procurement in the business there is kind of
notoriously slow and tricky and it takes a lot of time however this debate exists in every single country i mean it's the same in the us it's the same in israel countries that you know from our perspective are known to be super innovative and able to do stuff really quickly so i hear the same thing in in the pentagon and elsewhere where they say it all takes way too long um i think what this questions what this question shows is the civilian realm here is really important so let me take a step back what we what we had um basically since the 1950s is a situation where most of the technological advanced advantage um advances really came out of the state financed and very often military realm darpa who finance apple uh google self-driving cars i mean they all dodged gps the internet ziri by the way and so far they are all former um military strategic investments i call them yeah yeah no exactly and so this is especially true for the us but
it was also true um elsewhere where really you know for decades they're really cutting-edge stuff came out of out of military research or at least they financed research and this has changed recently and especially with ai so this is why we're always talking about silicon valley and pentagon going to silicon valley and talking to like google and facebook and all of this because this is really these are the the companies that build the cutting edge um ai and so what this means is that now the military has to has to buy from these civilian developers and and this already takes time and while this is happening civilian developers can can develop further um and it's much easier to bring stuff onto the civilian market and in the military market right because in the civilian market you can have a better test and you can put out you know one system that may not be perfect yet but it can be improved afterwards you really don't want to do this in the military right right you
want to do sure that stuff works you don't want to have a tesla auto pilot fail um i agree i agree exactly and i mean we still we still do this so we do feel in the military you do field drones you try them out and then you realize we had the situation in iraq where they tried a drone system and it didn't work because it was basically running on the same frequency that the iraqi telecommunications system like that that wasn't great um so this still happens but it just shows like how tricky this is and this has again this has been a problem that has played every single military it's true that the bundesvara struggles with this but yeah just to make the point this really is a problem everywhere and while you can streamline processes there is no there is no perfect solution no one does this so when when we were in this topic we discussed as a silicon valley has a cultural you know they take the hands of approach oh
we we are not a media company we are not you know we don't want to have any regulation responsibility for the world we're just tech we're just doing tech at the end tensorflow is running in drones yes and i'm pretty sure that an image classifier is somewhat invented it's running anywhere out there so at the end they use the open technologies even if you don't provide directly to military they use it how so your thesis is that more innovation comes from the private sector then it used to be in the past from the military so my question is where will we see innovation which we all expect because old economy they they won't get it they don't mvp they don't do lean they don't do testing they take forever we see with all the german european military projects that doesn't work i spoke with someone from the military said yeah we will have autonomous weapon systems by 2045.
i expect the singularity by then well okay but that's that's that's a bit of an extreme point of view yeah but this is my answer to him i'm like 2045 i'm uploaded yeah i don't need it then anymore it's like their timers on is not realistic yeah um so the first part of your question i think is important in that i worry sometimes about this idea or this yeah this idea from silicon valley that they they can say we don't build for the military i mean some of them do but there's this there's this idea right um we don't we don't build from the matter military this is all for good this is of the civilian realm um there you go i think you can't have this approach if you're building something that can clearly be used for military purposes and just for nefarious purposes in general it's not just armed forces by states um it's also you know we mentioned on state actors terrorist groups etc that can just repurpose these systems and it's
just not good enough to say that we are only building this good good so we aren't in any way responsible um i actually worry that this approach leads to developers in silicon valley and elsewhere not thinking enough about how you could use these systems for for military and other you know purposes so i think this is really important so all the startup founders out there have this in your mind exactly exactly exactly and it's absolutely fine to say we don't actually want to sell this to you know airbus and and general atomics like that's an okay position but saying you know we don't even want to think about this that's that's not good enough and actually really dangerous um about the kind of 2045 um thing i mean here again we're coming back to developing this stuff and preparing the stuff just takes time and and 2040 or 2045 is a good um date because this is also the date where one of the big ticket european items is supposed to to to fly and
to be used and that's fcas the future comet air system which is what i had in mind yeah exactly so that's and that's a european franco-german or franco-german spanish project of a fighter jet with drone swarms that are ai enabled and all these kind of things um and this is a big ticket european item we're talking you know a lot of money uh being invested and and this is interesting and so far as they're also they're basically trying to develop something which doesn't doesn't exist yet and the technology doesn't exist yet so this is why it takes so long um so i i yeah stuff stuff takes long i mean again and the the us f-35 also took a long time to develop and still has its issues so so i'm not i'm not as shocked as you are when it comes to that i i i i mean we both know otto yorn who was for four years the head of cyber in of of uh let's say innovation hub at buddhism and i
will have a virtual chat with him in a couple of weeks in september discussing this how you get more innovation in the not only military but in the public sector how can you source innovation how can you have open innovation because this needs to speed up us i mean if you're 10 years behind in your plan and the other one i mean we all have seen it if you have the nuclear bomb and no one else has it that's a huge head start but this comes now we have a couple of more questions so one question from joy joy wrote thought i would chime in with my questions fairly early how do you see ai impacting the various elements of the kill chain and how do you say ai combining with cyber electronic and even space based capabilities to achieve a combined effect in war yeah i think the cyber bit is important because i don't specifically work on on cyber related things but in one sentence i basically would say you know everything i
mentioned earlier about being you know it stopped being faster and and possibly cheaper and all of this this is all true for cyber even more because the the difference between the cyber world and the physical world is that in the cyber world you basically don't have physical limitations i mean other than stuff like how fast data can can uh go from a to b but but there are basically no physical limitations you don't actually need a let's say a drone swarm to take off and fly somewhere and and do that kind of stuff it is immediate and so um if you have ai enabled systems in the cyber realm all of this becomes even more important and immediate i think that's that's important to uh to say and with the kill chain i mean yes that this is this is these are basically the different elements of what i mentioned so data collection you can you can do with more ai enabled systems data analysis we're just talking about this you could do with ai
enable a system so that would be much faster finding targets would be um can be much faster or much easier with ai enabled system so really at every point of the of the so-called kill chain you can have ai the big question basically is where do you want it to stop because at the end of the kill chain of course is the kill so at the end of the kill chain is engaging a target and and possibly killing another human being and where where do you want ai to stop being the decisive effect and this is a really good lead to the next questions by bing zhang he says the true capabilities of ai and unmanned systems can only be realized through increasing the autonomy granted to those systems how can the cultural issue regarding divorcing the kill decision from a person be managed i mean it's tricky because i can you can definitely have uh systems where you use ai to do a number of things again repeating myself but data analysis command control
all of these things but then in the end you say because of cultural civilizationary rules and achievements and things that that we find important values we don't want the ai to take the final decision over life and death you can argue and you can't create a scenario where this slows stuff down that's true but there is basically there's a there's a question of what's what's more important um for you are there certain red lines you don't want to cross and i think there are good arguments tonight kind of not cross this final line where where it just as a computer who does the whole thing and then takes the kill decision um as well so i don't i mean i guess i guess the person who asked the question is right and that it's the the all encompassing um ai system would do all of this by itself um but i wouldn't say that you know you can't use ai properly unless you give it like full autonomy like that's just but the more you
give it the more it can it's faster it's you know at the end it's just a speed thing so you need to give it for maximum impact the most autonomy in steps yeah okay so this may be a good moment to talk about what we call manned unmanned teaming right um because there is a lot of research being done at the moment about how you can team up human beings and ai ai-enabled systems and one of the reasons why militaries but also outside of the military um they want to do this is exactly to have this kind of human control make sure that there's a human on the loop and that's a kind of you know value-based decision fair enough you can agree with it or not but the other reason why we're doing this is also because so far research has shown that manned unmanned teams are actually better at their jobs than just ais so you even have this in like chess right where we all thought that i mean the computer has
beat us you know ages ago but if you team up yeah as a kasparov lost 1997 against ibm blue then the teams were better i agree it's ai's augmenting us but at the end every chess computer on the cell phone is better than this so if it's cheap enough the technology and the learning is increments at the end only ai will be better but in the meantime this can be very long meantime the combination of both is the most proven and the best working for very long period exactly so i i wouldn't claim to know enough and i'm not sure to what extent anyone can really be certain about this but um so as i wouldn't claim that i could say that definitely at the end the ai is always going to be better i don't know but as you say at the moment in a lots of area we can show and prove that manned unmanned teams are actually better and this in a way is you know this is great news for all
of those that want the meaningful human control they want they want humans in the loop because then you you already have the human built in if you want and and can make sure that there isn't too much autonomy this leads me to the to the movie war games from the 17th which is a classical especially under the current ai idea let's think for a second so simulations are already run i watched last weekend with my wife and documentary about bunkers and they had a bunker where they showed all the nuclear bombs where they would hit cities in germany the fallout and where the movement would be so everything was hand drafted and on the depending on the celery they had to move everything to the ai makes running the simulation and i imagine one day ai will tell you it's worth fighting this or not or me even to ai's fight i mean we've seen this in the uk where russian military used a lot of ai to penetrate the infrastructure water electricity and meteor
it was all hacked it was all simultaneously hacked in the same day and just shut down for two days just to prove that they can do it and that was ai too yes i mean this in a way it brings us brings us back to this idea of let's just the robots fight it out or in this extreme that's just the computer let's let's have the computer calculate who would win right and no one has to fight anymore i mean this is this is great for a sci-fi movie it's just that and this is basically the the the material excellence right you don't even it's just literally just the material somewhere above of russia siberia thousands of drone clash and then it's done and it's like one day and we just read it in the news hey you're now part of the chinese empire okay and what do you do now because this this for me is exactly the point right um like like usual i accept the protectorate 90 would do it and you
end up being a terrorist or you could they call it guerrilla warfare warfare or something like this okay exactly exactly and the second point is important because what would of course happen i mean if imagine we could have this perfect simulation and china would say actually we are attacking the u.s now and the computer calculates and you know one side wins no way that the other side the loser would accept that outcome what would happen is that you fight back and this comes back to my point if you always have someone dying somewhere in the mud i mean in your scenario it would be the the guerrilla fighter um in mine it would probably first of all just no electricity no water no internet you know people died too in hospitals etc oh sure yeah i mean this yeah that definitely yeah um but yeah i don't i just i just don't buy this idea of um we can we can outsource we can outsource warfare to to computers or to robots and it's not
going to touch us anymore because this is just not how how human nature works and um i i don't i don't believe in that in that scenario and in that future so question by constantine um can you allow elaborate on the difference between military development and acquisition of ai compared to private civilian sector how can the military leverage the private sector's innovative potential and where's the limitation of the corporation and i mean you touched it before yeah but it's still fair because our innovation world is significant faster and cheaper yeah um i mean i guess i'm not sure whether i have a great answer for this because the answer basically is more cooperation um and and making sure that the military is connected to these civilian developers and know what they're doing and and um i think so there are a few things that the the military can do to make it easier to work with the civilian realm i mean one of the things for instance is that military um just just contact contracts
with the military are notoriously slow and tricky and it's pretty hard to get military money which often means that for especially you know startups and the like they just don't have the type they don't exactly they don't have the time to to to do this this application and to wait for it and then it takes ages and by the time you know they may have gone past been bought by google or whatever so um trying to to speed that up um would be would be good and this isn't again this isn't about like fielding stuff that's not ready yet this is literally the the first step just the time it takes to to come up with you know to agree to something being built so this is something i know that a lot of people are thinking about and i've heard this especially in the us but but i guess it's it's similar elsewhere as well um but yeah other other other than that um yeah it basically comes down to to working together more
to communicate more um to to keep an eye on on civilian um developers um possibly to be willing to buy more stuff off the shelf so to speak um but that can be tricky depending on who you buy this from so um i think because it's chinese produced and the german military ordered them but then the boss said hey i do you know what you do so they use paradrones and dgi drones too and they use dji drones and then you say do you need chinese dji drones for military operations just because you wanted a quick solution yes so that's exactly the example i i would have mentioned so dji is a chinese drone manufacturer civilian drone manufacturer and in fact not just the leading but the dominating drone manufacturers so 90 or so of of the civilian drones we have let's say in germany are most likely hmi produced um parrot by the way is french so my drone is a paradrone and it's french um so what what happened is that because dji
was so dominating and their drones are really good like they're just like technically really good um what happened is that a number of of branches of the military bought these systems to use them in military operations um the german navy the dutch navy the american marines like a number of actors um which is surprising for two reasons a because these were civilian systems bought for military operations which in and of itself yeah and second because these were chinese made and you can wonder whether the american military would have bought another chinese-made system for their their military operations probably not but here they did and what indeed happened is that after a while people started to realize that could be a problem because you know we're using these systems for surveillance and there is a big question for dji drones like where do they send the data to um metadata but also data in terms of you know the videos like is this being sent to china and if it's being sent to china can it
be used by you know the chinese military and so the the the um u.s marines decided to no longer use these systems and and yeah a number of armed forces followed this but this really shows like how tricky this is to to do it because in the end i mean these military tried to do the right thing they brought a great technological system to a comparatively you know low price especially for a military point of view and use them for the operation so they did everything right and in the end it was completely wrong yeah so just to show that ai is a tool private example on my wedding i had a parrot drone delivering the rings oh yes but my favorite part was even it was the imperial march by star wars the music background and then the drone came and then delivered the rings and i took them and then we did the rest um i loved i love gadgets i mean i mean i shouldn't laugh because when i submitted my phd
thesis i also filmed this with my drone so there you go yeah see okay another question from hendrick um where do you see the ethical differentiator between the deterministic pre-programmed kill decision of systems for example naval mines torpedoes or autonom automatic defense systems like the r ages combat system versus ai systems he then asked is it just our fear of losing control when a with ai because of the non-deterministic nature or something else i think it's interesting because a pre-programmed torpedo which is going after heat for example there is a there is a software in it yes so first of all so a number of the the systems that this refers to are in fact banned you know landmines for instance where you you know just put a nine and then whoever steps on it um these are internationally banned doesn't mean that they aren't used anymore but these are banned exactly because we feel that this is wrong um mainly because it doesn't differentiate between who steps on the mind right it doesn't just
only go off it's like random the kill decision is set and it doesn't differentiate between um who is getting killed which is why we have all these cases of you know child stepping on on these minds and getting blown up which is terrible so these so these are bands banned it is true that a number of other systems you know aren't banned where you have a a ready-made kill decision so let's take the underwater torpedoes yes you tell it you know go after um after whatever a heat source but of course this is a very limited scenario and that you still have let's say the the commander in the um in the submarine who is fighting a a known enemy somewhere there sends out the torpedo and says go for the for the heat source i mean they have a very good idea of what they are likely to to uh destroy things can go wrong absolutely but if things go wrong if whatever this torpedo goes up another heat source and it's exactly the
human took the took the decision to send out this torpedo and have a good idea of what it could do and so the responsibilities is with the human the moment you have kind of too much ai with too much autonomy there is a question of who is responsible um to what extent can you still put the responsibility on the person who send it out if they didn't just send it out and say you know this is the the target you should you should engage but but just kind of let it roam free and decide on the target um by itself i think this is this is a a very important question and then really the question is don't we if if we kill someone so we're really talking lethal system so against human beings and not against whatever radar systems if if another human being is killed don't we owe us to our kind of common humanity that someone feels responsible takes on this weight of having killed another human being and you can say
that this is whatever idealistic but this is one of the you know cultural or civilization narratory advances of of of our society and we really just want to give this up because it's you know easier and then faster and then and all of that and i wouldn't i wouldn't necessarily say that we should and this leads to the question from wolfgang asking what do you think about the idea of a political ban of autonomous lethal weapons wouldn't like this ever happened and my question is i know that a lot of people propose it but then when one or two actors don't act to it uh it's game theory so it only works if everyone is doing it or else someone else will have an advantage so this is a big discussion that we currently have particularly in geneva at the united nations within the framework of the convention on certain conventional weapons or ccw so for a few years now they've heard experts and they've been debating whether or not to ban or basically they've
been debating on whether or not to have the discussion on whether or not to ban um lethal autonomous weapon systems and my take on this is that i think it's extremely important and extremely good that we're having this debates in geneva because it allows us to define certain things and come to certain agreements personally i don't think that there will be a ban and if there will be a ban it will be so specific that it's really easy to find reports because banning stuff is actually really tricky unless it's something very distinct and the one thing that ai and and and autonomous weapons aren't is distinct it can be a number of things and so i i think that they couldn't come up with something that's very clear and and where you really ban what you what you wanna ban and i also don't think that countries will agree really agree to that but that doesn't mean that the debate is pointless on the contrary i think it's it's very important um when it comes
to game theory i mean you're right and so i mean the an american um i think it was the undersecretary um uh of of of defense or something other um not too long ago who basically said what you were just saying what you said like we don't want to build a terminator but if others do we will respond and of course that is the big danger and this is why i'm genuinely worried about arms race dynamics in this area of autonomous weapons and i say this as someone who's always said that there is no arms race than that dynamic when it comes to drones so i'm always a bit critical of what you call an arms race but but when it comes to autonomous weapons there's a cree there's a clear incentive to react if someone acts and this leads to the polls i have created in case you haven't checked them yet next to the chat there are questions there's a poll and i have asked how should ai and security and defense tools
be regulated and hundred percent i say there needs to be a high regulation control however it is legal so they say except this point you can't make it illegal because then you can't control it it has to be high area you know high control but it should be allowed and then i also ask the impact of autonomous weapon systems is and then 70 percent said how to predict instability and risk will increase yeah i agree i agree with the hard to predict um thing um i mean there's the the kind of positive view that if you increase for instance if you increase ai-enabled surveillance that could help decrease conflict because there are less misunderstandings right yes misunderstandings better reconnaissance and prevention i mean i have a portfolio company called aver they know they can create crimes or incidents even terrorism before it happens not on the singaporeans but in regions which is provenly we know that policy has prevented what 20 attacks they say sometimes in the in the tech scene um because they had
the data they could we we never know but they can prevent something before it explodes not always but often yeah that's very minority report oh yeah real but less human it's just software yeah um okay so that's that's one that's one scenario but the other is that too much reconnaissance too much surveillance could actually be really really counterproductive for international stability why well we are currently seeing there's this theory that that soonish it will be possible to turn the oceans transparent meaning that with a combination of sensors and and yeah basically ai and forecasting on all of that it will be possible to say where is what in the oceans i mean again sounds sounds great more rectangular more surveillance no interesting i thought they wouldn't know it no yeah but one of the things we don't know is where are the submarines from nuclear states that transport these nuclear states so-called second-strike capabilities so nuclear deterrence generally works that you you have a nuclear capability and you tell the others even if you attack
me and basically completely destroy me i'm still able to attack you back and completely destroy you don't don't the idea there exactly don't they're attacking us and one of the ways that a lot of countries are a lot of countries i mean the nuclear states are doing this is by placing nuclear weapons on submarines that are basically you know somewhere in the ocean so that if their country gets completely destroyed they can still have the second like the russia russia submarine which sank and they didn't tell anyone because they say that it might have been a nuclear submarine and all of them had to die right um that's a terrible um uh yeah example but yeah okay so but now combine this with what i just said about turning the the oceans transparent if you are able to turn the ocean's transparent you take the second straight hit strike capability from these nuclear states right which means that all of a sudden they can't claim anymore that you know if you attack me i can
strike back because now in one strike an enemy might be able to destroy both your country and your second strike capability and that one could argue really undermines international stability because you know as much as you might he might hate nuclear weapons and nucleated terrorists so far until now it has worked in the sense that because there was a second strike capability um people didn't attack these nuclear states so that's just kind of one of these unintended consequences that people like us that's interesting so i love to play um games like civilizations and there's always this mode of fog of war yes and it's preventing you to see the true strength of your enemy because you do you don't want to lose too much i mean you don't start a war if you win by 60 70 percent you start a war if your army is conquering the neighboring by 95 otherwise it's not worth it because you have more defense capital points in average but fork of war means i don't know the strengths
of the enemy so the more surveillance we have it's less harder to hide i mean we know all these articles about chinese military because they use google earth etc to identify and figure this out how they build it and etc so it's um it's it's interesting and you so so what again is your less so more surveillance more intelligence means more chance of war or i think yeah well the point is the point is it can go both ways the point is that there is no perfect answer the point is so so the fog of war of course is a concept that that's you know huge in in my area and one of the very first books i read on all of this is actually called lifting the fog of war and this is you know back in the 1990s i think or 1980s um where we had this this increasing digitization computerization of the battlefield exactly and the the hope at that point was that there would be this revolution in military affairs this
this i.t enabled revolution military affairs that would make it possible to lift up war now i would argue that there is always 404 it doesn't it really doesn't the fog of war isn't dependent on on surveillance capabilities it basically is the friction that happens if you know if things happen that you can't 100 predict so i would think that unit can never lift the fall of war but if we increase isr capabilities in a way that we just discussed um the point i was making is that it it really can go both ways and there are good arguments to say that you know if there is more transparency let's say as to the military capabilities and and things like that from everyone this can help decrease tension and this can help decrease conflicts i think that's a good argument the argument i was making is just that there are lots of unintended consequences um that we also need to take into consideration such as for instance undermining nuclear nuclear parents um we have only seven
minutes left and have two big topics still one is the key word surveillance capitalism yes and the other one is um at the end where are we in europe so let's see let's talk about europe i had diamond buzzer lager last week pro european party um we know germany is ignoring this topic we know france is going actively for it i'm a friend of the european army for costs and several reasons give us your opinion your opinion like where we're standing where problems what can be solved and will it be solved okay um so i don't believe in the european army um i think it's somewhat uh i i'm worried that the debate on the european army is wasting a bit of time and brain space of a lot of people um because i don't think we're gonna have anything resembling a european army until there are a united states of europe so basically what i'm saying is that you know we can have a european army if we become a united states of the
world what could lead to this very fast so if there is a crisis crisis always help to unite if it would be military conflict this could lead very fast but i agree currently there's no need for it to enforce this yeah yeah so okay so let's leave the european army um out of this because this is one of those pet peeves of mine um but okay let's let's look let's look at europe and kind of military eye oreo in general so the first thing to note is that one of my grievances in this whole area is that there is very little attention being paid to europe in this whole debate so when you look at the topic of military ai in the military realm what happens is that most of the research is focused on the united states and then quite some on china and then a little bit let's say on russia and a few others yeah yeah yeah israelis is always an interesting case um and the reason for that is that i
mean the us is is leading militarily just by the sheer size of its military um and it's definitely a technological leader so lots of interesting things are happening let's say at darpa and the chinese focus stems from the fact that you know the u.s is really interested in china and of course a lot of things are happening in china as well but this means that there is actually just very little attention being paid in very little research on what europe can do and i find this unjustifiable in the sense that a stuff is definitely happening in europe i mean we were talking about the future combat competitor system earlier we have at least two quite interesting autonomous um combat drone developments uh neuron and taranis by the french and by european consortium and by the the british we have a lot of data analysis so there are definitely things happening in europe that are worth looking at and secondly simply from a european point of view we should really be interested in this i mean
i find it really striking how how many european researchers primarily study let's say u.s development i'm not blaming them this is super interesting it's just that it's a bit it's not great for us if we don't really keep an eye on on this the tough question are we defensible currently without any i i i might see this is because the united states has this umbrella still about us and i call this at the beginning of start um it works but let's say america says oh trump is re-elected i don't care about europe are we defensible okay so this goes way beyond you know tech and warfare right so you're basically asking about kind of europeans defense capabilities the question is against whom and indeed in what scenario so at the moment european defense stands reasonably strong okay because no no but because of the nato system nato article 5 and indeed the u.s support if there was an attack against a european nato member the other european sorry the other nato members would come and
help or more or less contextually obliged to help although in different ways um and and all these nato countries together are pretty much able to defend against i mean more or less everything if you take out the united states if you know the us says we're leaving nato we don't care about europe anymore we don't care at all which is an extreme scenario i mean i'm i'm worried about the u.s turning away i think they will turn away but there's a difference between saying we don't want to do everything anymore and we don't care anymore if you get attacked at all but okay take this extreme then it of course depends not against you know who who is attacking let's take i don't know russia russia is potential i mean it's russia and china in the media only those two could be a threat i don't see russia as a threat okay but this is another um if you have russia attacking a european state if it's an eu state um there the eu actually
has the same article five that nato has this idea of common defense the eu also has this article 42 7 in the lisbon treaty which pretty much says the same if one member states attack the others are supposed to help again like helping can mean a lot of things but overall it means helping them so if an eu member state is attacked you would assume that the other eu member states would help if you have all the eu's military might against a russian attack it's it's definitely not clear you know that that the russian side would win um i would i would not claim this but again in the military realm it really depends on what exactly are you trying to do what exactly are you defending which type of attack do you mean um if russia just won't just if russia wanted to invade the baltics and would do so very fast the eu would definitely have issues um defending against this this quickly by the way nato would also have problems on the
other hand a lot of reports how they move and plan for this so because we're coming to the end um first learning for me is europe is defensible so we don't actually have the physical threat currently which is good no one has to uh be afraid for now that chinese drones are petrolling here in our streets soon or unless we buy them on our own terms yes exactly so yes a we already have these systems patrolling because we we bought them but also i mean i definitely in in the military realm you can never go that extreme because i it's you you can have plausible scenarios where you can have attacks from the outside that we aren't prepared for and aren't being able to defend against and let's be honest like european defense capabilities we aren't doing great i mean the buddhist way isn't doing great but even even the brits and the the french have their issues like let's not pretend that there are that there are no problems um i would i would
i wouldn't i am definitely calling for more uh defense capabilities um investing more in and capabilities in order to counter a threat just because i don't think that that we can't sleep at night at the moment because there may be an attack and tomorrow doesn't mean we should be complacent about this i think this is very important and important to make that one so my call to action out there is if you're an entrepreneur consider starting a not military but defense company if you want to um because it's a huge market and it's needed for european autonomy at the end if you're an investor consider it at least don't be so narrow-minded at the end it's important for the ecosystems to be refurbished um rika i thank you and i hope we see you again live for summit in november that would be fun yeah well thanks for for having me we had a you know huge discussion there's so many more things to say on all of these topics um the next time we're
gonna discuss whether whether russia or anyone else is a threat because i have views on that too yeah so thank you for this i say goodbye to you yes everyone else can stay for a line um well for you our next virtual chat is in two weeks and it will profes professor christian casting and we speak about the third wave of ai so we will speak very technical about ai and he was recommended to me by gary marcus um and we will talk about you know how can we build the next level of intelligent systems he calls it robust ai hybrid systems you know how to put all these pieces together and also i would like to point out we have a summit hybrid summit 17th and 18th of november here in berlin so we will try out have a limited 100 people physical here like here is in the studio and you know beyond with a couple of stages and eureka will be on one of the stages and but we will open it
we will stream everything we will have the same networking platform it will be accessible for everyone from all over the world so you don't have to fly to berlin to attend and to network and to learn and um that i would say is it for today it was as usual pleasure good questions very lively interaction i think again thank you for attending and if you're just listening in um it was great and there will more come check out her podcast if you want to have a deep dive thank you and goodbye