Rise of AI Summit 2020 · Berlin · Society & Strategy Day | Future Space

The JEDI Deepfake Grand Challenge and Frontiers in AI for Our Societies

André Loesekrug-Pietri · Executive Director, Joint European Disruptive Initiative (JEDI)

At the Rise of AI Summit 2020, Andre Loesekrug-Pietri, Executive Director of the Joint European Disruptive Initiative (JEDI), a body he describes as a European DARPA, argues that AI is both a threat to and a potential tool for democracy. Nine months earlier, around a hundred scientists, deep-tech founders and corporates, now several thousand across 29 countries, concluded that Europe cannot only play defense on technology and must invent breakthroughs, not just regulate. He argues AI-augmented decision-making, illustrated by aerospace predictive maintenance, could give democracies the long-term compass that authoritarian states already use, citing China's 2025 and 2049 plans against Europe's crisis-to-crisis management. He describes how fake news and platform-driven echo chambers undermine public debate, citing Cambridge Analytica and a claim that Donald Trump made 20,000 false statements in 1,300 days. He presents deepfakes, built on generative adversarial networks, as a frontier where detection is inherently hard, and announces JEDI's Deepfake Grand Challenge, targeting 99 percent detection faster than a video's own length. He also flags civilian surveillance uses of AI and proposes distributed face recognition and homomorphic encryption as privacy-preserving alternatives, arguing for transparency over bans.

Key takeaways

  • JEDI grew from about 100 to several thousand people across 29 countries in nine months, modeled on DARPA.
  • Loesekrug-Pietri cites a claim that Donald Trump made 20,000 false statements in 1,300 days.
  • JEDI's Deepfake Grand Challenge targets 99 percent detection accuracy faster than a video's own runtime.
  • He says roughly half of UN member states now actively use civilian AI surveillance technology.
  • He says 95 percent of deepfakes today are pornographic and increasingly linked to ransomware and blackmail.

Detailed notes

JEDI's mission: Europe cannot only play defense

Loesekrug-Pietri describes the Joint European Disruptive Initiative, founded nine months earlier by around a hundred scientists and now grown to several thousand people across 29 countries, including academia, the deep-tech ecosystem and large corporates. He frames it as a European equivalent of DARPA, built on the premise that Europe can tax and regulate more but will still lose if defense is its only strategy; the mission is to invent the next big technology, not merely protect against existing ones.

How AI reshapes decision-making

He argues AI is transforming how organizations and democracies make decisions, drawing on aerospace predictive maintenance, where AI cross-correlates thousands of data points per second that humans could never process by hand, as an analogy for the compass democracies currently lack. He contrasts Europe's pattern of moving from one crisis council and emergency recovery plan to the next with China's long-term planning, such as its 2025 and 2049 targets, warning that authoritarian states benefit from offering citizens a clear vision that fragmented Western societies increasingly lack.

Fake news, echo chambers and platform business models

Loesekrug-Pietri argues that fake news has never been more consequential, citing Cambridge Analytica and a claim that Donald Trump made 20,000 false statements over 1,300 days. He attributes this partly to platforms whose business model rewards engagement through algorithmically reinforced echo chambers, segmenting an open internet into groups that no longer talk to each other, which he says is the worst possible outcome for public debate, the core of democracy.

The Deepfake Grand Challenge

He describes deepfakes, built on generative adversarial networks, as one of AI's true frontiers because detection systems that flag a flaw prompt the generating system to correct itself. He criticizes leaving detection to platforms such as Facebook, which launched its own deepfake challenge, arguing platforms should not also decide what counts as true. He illustrates the economic stakes with hypothetical deepfakes of a bank CEO announcing a cash crisis or a luxury brand using banned materials, either able to destroy value before it can be disproven, and announces JEDI's own Deepfake Grand Challenge, with a global scientific committee and a prize for reaching 99 percent detection accuracy in less time than the video itself runs.

Surveillance, distributed recognition and encryption

He notes that roughly half of UN member states now actively use civilian AI surveillance technology, largely from US and Chinese vendors such as IBM, Palantir and Cisco, and gives examples from Sweden, where police use facial recognition without the regulator initially being informed, and France, where several cities accepted free camera infrastructure from a large Chinese vendor, with Saint-Etienne testing sound-based detection of violent acts. He argues the issue is not banning such technology but ensuring transparency, and proposes two technical responses: face recognition distributed across individual cameras using small data rather than centralized databases, modeled on the resilience of early internet architecture, and homomorphic encryption to let large data pools be used without merging them or exposing individuals to reverse engineering.

Q&A highlights

Asked whether the challenge can end the deepfake arms race, Loesekrug-Pietri says it cannot once and for all, comparing it to DARPA staff rotating out after two years to keep pace with fast-moving technology, and argues Europe needs pop-up regulations revisited every six to twelve months rather than rules meant to last twenty years. Asked how a single solution could work across a federalist EU, he says JEDI, structured as a foundation, will support multiple approaches that meet its criteria rather than fund one winner. Asked about deepfakes and blackmail, he says 95 percent of deepfakes today are pornographic and increasingly tied to ransomware targeting mostly female celebrities. Asked whether liberal democracies can compete with authoritarian efficiency, he says democracies need both long-term vision and agility, warning that what happened in the United States under four years of Trump could happen in Europe too.

Quotes

Democracy is not about regulation, democracy is about transparency and empowerment of the individual.

Andre Loesekrug-Pietri

We should not build the regulation that lasts 20 years, this is impossible. We need to have pop-up regulations that are as fast as the technology.

Andre Loesekrug-Pietri

If you talk diversity, if you talk about women empowerment, and if you're not talking deep fake, you're missing something very big.

Andre Loesekrug-Pietri

Questions answered in this talk

What percentage of deepfakes today are pornographic?

According to Andre Loesekrug-Pietri at the Rise of AI Summit 2020, about 95 percent of deepfakes today are pornographic, and they are increasingly connected to ransomware targeting mostly female celebrities.

What is JEDI's Deepfake Grand Challenge trying to achieve?

According to Andre Loesekrug-Pietri at the Rise of AI Summit 2020, the Joint European Disruptive Initiative's Deepfake Grand Challenge aims for 99 percent detection accuracy achieved in less time than the video itself takes to play.

How does Andre Loesekrug-Pietri think Europe should regulate AI?

According to Andre Loesekrug-Pietri at the Rise of AI Summit 2020, Europe should use pop-up regulations revisited every six to twelve months rather than rules meant to last twenty years, since technology moves faster than traditional regulation.

Mentioned

deepfakes, generative adversarial networks, echo chambers, homomorphic encryption, distributed face recognition, fake news

Keep exploring

More from André Loesekrug-Pietri

More from Rise of AI Summit 2020

More on Ethics & Trust

Transcript

Auto-generated transcript, not edited. · 4,818 words

Show the full transcript

and now to our next speaker in the age of baby yoda i think it takes guts to call an organization jedi it's the acronym for a joint european disruptive initiative think of it as the european darpa that wants to accelerate europe's leadership in disruptive innovations our jedi knight did not serve in the clone wars but i think he could have he's an aerospace engineer and an ace pilot a colonel with the french air force people's reserve he worked at airbus and in private equity and venture capital and he was special adviser to the french minister of defense please welcome the man who has everything except the lightsaber andrei luisiko pietre welcome great to have you how are you thank you we can hear you we can see you take it away thank you wonderful so thank you very much for this introduction i i listened very carefully to all the previous speakers and what is pretty amazing about what fabian what hans with chris said notice they all mentioned china and the topic i want

to talk to you about today is really this relationship between ai technology at large and democracy i mean i spent myself 10 years in china i'm not one of these anti-chinese at the contrary but it's when you are there that you realize how important the values and the system in which we live are and how much probably if we don't do something and something big and something bold very quickly this uh democracy is at risk and i'm not talking about the generation i'm talking about a couple of of years ahead and i think you heard these calls which which i thought were very uh pregnant the other thing as was just introduced indeed the word darpa appeared in all of these uh topics so what are we waiting for but well actually um nine months ago we decided with a group of hundred now we have several thousands of scientists academia people from the deep tech ecosystem many of uh jedis are in this very room or listening to us today and and also big

corporates to say we cannot just play defense in europe defense is not a strategy we need to tax probably more we need to regulate probably better but we are only in defense we will lose how can we invent the next big thing and look at it i mean i'm speaking to you unfortunately i mean berlin is my hometown uh but unfortunately i could not come because i'm currently based in paris so thank you to corona we're using zoom zoom exit didn't exist seven years ago so we are talking about an age where actually no position is here forever so it's all about inventing the next big thing and that's where darpa is is is a model for us and this is what we are currently doing in europe so let me delve into the topics that will be linked with democracy and and go a bit deeper on how ai really challenges democracy let's think about it what is democracy it's about core principles civil rights due process equality rule of law and freedoms they

are all challenged today by by ai but ai can be also a tool to accelerate them so let me go in detail on three specific points which are the core of all of our societies first an organization a startup a political body everything is about decision makings so let's think really hard how ai is completely changing decision making and corona is is an incredible case study of what can go well and what can go very very wrong think about do we do today policy makers really understand citizen do corporates executive understand their the team they're working are they able to attract the new generation think about how ai is able to set the agenda to understand the underlying trends the small trends also those who can potentially have a massive impact tomorrow we also can imagine in this information age and this acceleration age as a way to actually propose policies i mean i'm coming indeed from the aerospace industry there is one breakthrough that ai allowed but actually very basic big data analysis is

what we call predictive maintenance when you have engines collecting thousands of data every every single second it's impossible to actually cross correlate all the different factors ai is able to actually give a sense give this this this kind of compass that today we are all lacking we are all going from one day to another not understanding what will happen in a week in a month and not even saying a year if we don't have that especially in our democracies it's going to be increasingly the authoritarian states that for very bad reasons and and the more i i think about it the more i don't want these systems applied to us that will win because they will be able to give a vision think about the number of corporate executives that come back from china and they are all fascinated and there is a good reason for that by you know the china 2025 plan or the plan of the communist party for 2049 what do we do in europe we go from crisis council to

crisis council every six months there is another crisis emergency recovery plan we need to and that's where ai can help us i mean i'm not going to go into detail happy to share that presentation with those interested but there are a lot of use cases where that can be done and think about what has been uh said earlier on on the on the tracing apps where probably we don't understand how we can make use of it and and why is it so critical because simply we have entered an age where we gone beyond our cognitive capabilities this is the amount i mean we all talk about exponential exponential was a very technical or elitist word six months ago today everybody understand what an exponential is but that's what's happening with the number of data we're getting and so if we don't have ai to augment ourselves to be able to make better decisions better informed decision increasingly we will have the sentiment that somebody else is taking these decisions for us and i think this

graph shows probably materializes what what this exponential area is all about and it's all about empowerment this is probably this fragmentation of society i think fabian you spoke about that earlier the fragmentation is because you increasingly have large amounts of people be it you know the red states uh in in the us or or the extremes in in in our own societies that feel that they don't have any more control on their lives so what we need is not to put technology in the middle but have technology enable each and every of us to to really gain take back control and not in the way the breakfast city has started it's really about the individual and not about uh some some individual egoism but it's also challenging our democratic processes i mean fake news has been never as important as before this is a topic that most of you know very well but think about the boom we just experimented in the last u.s elections a few if you may a few data points here

i mean trump made and i don't know how you are able actually to do that that is pretty much an achievement in brackets 20 000 fake informations in 1 300 days it's incredible i mean we all remember the cambridge analytica play so we we are today in an age where these fake news generated automatically using echo chambers because this is the business model of the big platform so i'm not anti-platform because they have 30 market share in e-commerce or they have 90 um market share of the research i'm more and more against these platforms because they actually increase with the use of big data they use actually the worst in us which is this pleasure that we have to hear um information which is which is information that is closer to our beliefs and actually instead of having an open internet we're increasingly seeing this this segmentation into different echo chamber that don't talk to each other anymore and think about again what the democracy is all about a democracy is about the public debate

if this public debate is is hijacked or is it transformed with the help of ai then it's the worst that can happen to us i'm going to go very specifically on deep fakes which is a topic which is really one of the frontiers of ai it's about the the the generative adversarial network so it's one of the branch of artificial intelligence is basically systems that learn against each other and that make the detection of deep fake by definition extremely complicated because as soon as you have a small you know item or avatar that can actually make you understand that this video is actually not a true video the system actually auto corrects itself so today it's one of the frontiers and who again is putting massive efforts not the german government not the eu government they do a lot of speeches and i completely agree with hans these are individual great people i mean we probably have civil servants it's too easy to criticize the politicians that that really want to do good for most

of them but the system is today just incapable of just reacting and who again is trying to find a solution on the fix it's for example facebook that a couple of months ago launched a grand challenge on on deep fakes do we really want on top of their existing power uh platforms determine what is truth and what is not true i don't think so and i think it's important for the for the europeans and for the true democrats to react the other challenges is obviously about the the uh how how can we how can we make sure that that we don't achieve some kind of fragmentation of markets of trust how how is is ai by definition built uh with uh with this level of confidence that it's not biased by a gender or by a category of of people so all these and that's why it's a thin line it's not about anti or pro regulation it's about building the right ai from scratch and experiment as much as possible so i completely followed

the calls and saying if we if we if we continue like trying to find the big regulation and a perfect product the train will have departed with us us and we will not be able to shape the future that is really happening under our eyes and we should not underestimate how much this pandemic has laid bare and i'm a true believer in europe that actually as soon as there's a crisis every country reacted on itself a little bit like the ai strategies 27 countries 27 strategies there's what no connection whatsoever whatever we can hear the famous schulte starting between the french and the german is still in its infancy two years ago so this is a true call for action so we we really need to make that as as a tool that helps democracies and that it's not just uh undermining it's it's it's it's fundament last but not least another topic which is very well known but probably not in the dimensions and today we have probably half of the countries in the

u.n who are using actively civilians technologies using ai so you can have you have mostly uh us and chinese companies that are doing that you have the top exporters the ibm's the palantir the ciscos and we need to know what we want it's not about banning these companies obviously but it's we need much more and transparency of what is going on i think democracy is not about regulation democracy is about transparency and empowerment of the individual today i will give you an example which is happening in a french city you would say and i on purpose of course you have the us which is well known for movies like minority report it's actually happening more or less in baltimore or in some florida florida counties but think about probably the two countries that you would say are the most cautious about individual rights and i took sweden to france well in sweden the swedish police is today using uh ai in in face recognition to track uh to track uh personal data it's not that

it's bad it's about you know chasing criminals but interestingly at the beginning it was not something where the whether where the regulator was informed and in my in in my other country which is france you have several cities who today accepted the offer the famous offer you cannot refuse by by by a large chinese vendor uh offering basically to put free of charge the infrastructure of cameras etc and in sante tienda went even further they tried to use sounds in order to predict if um if there was a violent act being committed again the principle in itself is not reprehensible we are talking about the safety of a society but we are doing all these things without the transparency and without the enlightenment that i think all citizens need to deserve so again it's not about defense and not it's not about doing that in in in a secret place it's about uh taking back control and really empowering all of us to make the best out of it so what what can can we

do and now i go back to really what is our our mission our mission is not just to say there's a problem our mission is not about saying um uh we need to protect ourselves because the train will will will advance our mission is to make sure all of us it's not just about jedi it's about really a societal mission is make sure that we embed our values into the technologies of tomorrow that we invent the technologies that will have by design this humanistic individual rights approach these democratic rights that you can embed i will give you a few examples deep fix how can we make sure that we detect real time what is a fake video and what is a true video today some people some companies will claim they can detect with high accuracy you know it's about the false positives and the false negatives another term that has become a mainstream but today it's a real frontier we will launch specifically a challenge on that because it's not just about the political

system imagine a big brand where suddenly one of their i don't know they're the key people or the czo one of the big banks uh where you have a video of the ceo saying well actually our bank is is currently uh having cash problems this isn't a video impossible to to avoid that it becomes viral the next day you have a real life cash bank run imagine a big brand which is really also based on the trust and so then suddenly one of the big french luxury brands you have one of their morals who say well actually in in again in a deep fake video this brand is using uh you know animal material or doing tests it's hundreds of millions of euros that the next day can immediately be destroyed on the stock market because it would be impossible for for the company to to to say that it's actually not true because today these fake videos for the best of them are so so convincing so it's a it's an issue that that

has political but also massive economic and societal um consequences echo chamber this is a topic we use for several years but what have the europeans done in order to create another kind of business model we need to fight business model against business model these platforms that are only relying on this capacity of you know of of of self entertaining i mean that was the famous attribute of youtube i think youtube has now canceled it about you know a recommendation of the next video and once you you made it run you actually ended up very very quickly after five ten twenty videos to either conspiracy or violent um material so we need to find also something different on that on this challenge we're still looking for great ideas because a good darpa challenge or a good jedi challenge is about a clear problem to solve but also a very precise metric for success the metric is today not not that clear but then it's complete openness zero bureaucracy and immediate uh call for for excellence we

don't pay for for efforts we really want breakthroughs but we give all freedoms to researchers wherever they are face recognition another topic that is remember i mean you are all in berlin for those who are physically attending remember what happened i think now two and a half years ago two or three years ago on the on the christmas market after that you began to have first experimentation about face recognition on public spaces and who could criticize a politician to do that because if there is a terrorist attack and we could have prevented it with faith regulation obviously it's the thing to do but do we want a system with thousands of camera and a centralized database no we need to do like in the age of internet where basically arpaio came with this very simple but very powerful idea of the network is extremely resilient because it has no center can we do face recognition that is distributed in each of the cameras and where each of the camera can with small data so we're

not talking about big data anymore small data have the same vision and admission than than uh today the traditional algorithm running on massive that and last but not least we're working very hard again something that we need to go much more into detail on homomorphic encryption basically the idea here is can we use these massive data pools that have been described before and for uh and without having to merge them with all the privacy issues but use them as a single data pool but without with making sure that nobody can do the reverse engineering so let me give you a very precise example on the on the deep fake one so video please since i'm here i don't know if the video is running we're ready hey i think there was a video and we saw it for a couple of seconds then we're talking again it's starting again now i think today we eradicated aids president trump is a total and complete yeah i was just really surprised the more you express yourself the

more we own you fake news deep fake team i think you saw it already once so this is this is what our presentation ends we are launching that we have an amazing global uh scientific committee assembled the program manager uh a huge price uh for for the winners it's going to be a super challenging bar we're talking about 99 detection of deep fakes in in less time than the time of the video so that will be the precise metric for success and imagine uh if the europeans on this one challenging topic of guns can again take the lead on artificial intelligence and all the societal and economic implication it can have so um with that if you also feel as motivated as we are all i mean we have 4 000 today in europe in 29 countries uh if you want to help us craft the challenge if you want to support with your engineering scientific skills and if you want to be part of the team that is a bit distributed all over europe

uh feel free to reach out to us follow our twitter account and i'm super happy to answer any question you may have on this on this topic of technology and democracy thank you very much yes thank you thank you andre thank you for the impressive presentation and we have a couple more minutes for questions uh please do your audience if you like you can ask additional questions right now andre first question is will you solve or are you aiming to solve the problem of deep fakes with your challenge once and for all or will this be an ongoing competition between people creating and detecting deep fakes a quick anecdote what makes darpa so powerful and so different than other other organizations and we will have this we have the same functioning is that people working at darpa are there for two years exceptionally they have another two years why do they do that because first speed is of the essence i mean you would have had the same talk nine months ago it would not

have been very clear today we understand we are in a world where every day and every week counts otherwise you're running after the curve and the other thing you need to permanently technology is on the go that's what i meant with zoom and think also about tick tock tick tock the international version did not exist three years ago and that's the the real opportunity for the europeans not try to build a european google or european amazon but think about the next big thing and probably and so on your ass on a question on deep fakes it's going to be a battle that will need to be fought uh all the and that's why you need a little bit of revolutionary um mindset this is what we need we cannot have institutional organization and even regulation that's my only point about the regulation on europe we should not build the regulation that lasts 20 years this is impossible we need to have pop-up regulations that are as fast as the technology that we are trying to

and every six or 12 months revamp them but imagine the structural change in the mindsets of the people on the spree or in brussels or in paris for doing that so we need also probably a new generation of people who think like that right thank you very much we have a question from kai who first of all says amazing talk thanks he asked how realistic is one single solution to detect deep fakes or crime in general in a strongly federalist system like the eu or many of the member states and how about the political execution afterwards after the detection in terms of crime law enforcement what are your thoughts on this well again we are not an eu institution and we basically took the famous machiavelli talk is don't try to change organizations and that may be my call to hans and chris that we should talk more and fabian you cannot change people and that's probably what makes them beautiful is try to set something up next to it so we are structured as

a foundation and we think that it's with civil society working together with government and with ecosystem with the ecosystem that you are permanently on top of things so they will be not one size fits all right and by the way we can have we are not obliged like a traditional research program to fund one if nobody beats our criteria we will give out nothing right if three very different approaches like to a good math problem uh solve the problem we will support all three of them so we are very very pragmatic on that great thank you while you were talking about deep facts i was wondering about the business model uh of deep fakes and i thought that of course maybe you could do a deep fake about maybe a competing company but of course then you would obviously be one of the usual suspects um there is a probably a case to be made for political influence as you were also mentioning the whole fake news and fake facts debate but i was always

also thinking that um deep fakes might be the perfect opportunity for blackmail in the future is this something is that something that's already happening have you seen this in the wild already i mean if you look at it today 95 of deep fakes are porn and uh they are basically now the most uh perfection and the the perfected form of ransomware so you see increasingly ransomware which is connected with the potential deep fake and i obviously it targets celebrities today mostly female celebrities so it's and if you talk diversity if you talk about women empowerment and if you're not talking deep fake you're missing something very big because this is affecting one part of the society more than others and it will affect all the institutions in which we trust politicians banks financial institutions and basically all the big consumer brands they will be tomorrow if they are not yet affected by deep fakes because once one of these very professional defects goes out in the market and you have some social media wizard helping

make a buzz out of it it's quasi unstoppable and the the the the hit on the on the reputation will be unfortunately very very difficult to to catch up yeah so this will be the the business model for for for those who are inventing these deep fakes right thanks very much that makes makes a whole lot of sense to me there is one question that we i think can answer from alex which is rather a broader political question what chances do you give liberal democracies in a contest with authoritarians under the aspect of efficient organization of societies with ai tech can they the liberal democracies defend their lifestyle if we the the the paradox is that we live a an era where the two key success factors are one long-term thinking giving a vision because you don't motivate people by other things than giving them a perspective and on the other side in incredible agility the feeling we all have countries are behaving a bit differently is that we are lacking this compass and that's

where the authoritarian states for bad reasons are good because they have this ability to think long term because they don't have the the the election process and they are incredibly agile i mean for those who have spent time in in china it's very different than what you sometimes see in in in sudo expert saying it's everything vertical no china is the world of the gray zone well some people sometimes look at jack ma with and financial pass the gray zone a bit too much and then they get the hit but otherwise it's the world of experimentation so we need so i'm also pessimistic but on under today's liberal democracies think about the us we thought the us was the democracy that will never vacillate four years of trump four more years would have been probably more or less the end of what we know today that's why we are all more or less so relieved but hopefully we will take action because what happens in the us might very well happen in europe so we

need to change in the way that we give a vision again and we stop these six months urgency europe think which is a catastrophe we need to give ourselves what is what are the values that make us give this this perspective and that's where i think we have an incredible opportunity and on the aside be so much more agile and not wait three months to understand that there's a pandemic hitting us because that is probably the huge failure of our sisters not the individual people but this famous gordian knot that was described to us just before only thank you very very much i think you're very curious of the results of your challenge we're very much looking forward to that merci beaucoup phil dank thank you very much see you soon thank you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.