At Rise of AI 2022 in Berlin, Hans Uszkoreit compares AI development in the US, China and Europe and argues that Germany is falling further behind unless it changes how it funds applied research. He explains how large language models such as GPT-3 use self-attention to process long stretches of text at once and are trained through self-supervised learning, producing fluent, humanlike text while still failing at multi-step logical reasoning. Using a six-criteria survey, he shows the US leading in talent, research, development and hardware, China leading in adoption and data, and the EU trailing both. He credits the US lead in applications to DARPA-style funding, competitive parallel research contracts, infrastructure projects, procurement and sustained funding, contrasting this with German programs that fund topics only once and switch focus too quickly. He cites German-born researchers who achieved their breakthroughs after moving to the US, reviews Germany's underfunded national AI strategy, and presents the "Large European AI Models" initiative, backed by about 400 companies, as a step toward closing the gap.
Key takeaways
- GPT-3-style models use self-attention to process long stretches of text, up to about 2,000 words, and are trained through self-supervised, automatically labeled data.
- A six-criteria survey shows the US leading overall while China leads specifically on AI adoption and data.
- DARPA-style funding built the internet, GPS, Siri and autonomous-vehicle research through competing teams, procurement and sustained funding, not pure research grants.
- German-born researchers such as Sebastian Thrun and Franz Josef Och achieved their major breakthroughs mainly after moving to the US.
- Germany's 2018 AI strategy promised billions of euros but had produced only about one billion euros of actual spending by 2022.
Detailed notes
Large language models and self-attention
Uszkoreit explains that large language models such as GPT-3, BERT, Switch Transformer and Wu Dao are neural networks trained on billions or trillions of parameters that can translate, summarize, answer questions and generate text, and can accept inputs up to roughly 2,048 words. Their key mechanism is self-attention, which lets the system link related parts of a sentence or long text directly rather than processing word by word. Training relies on self-supervision: the system automatically generates labeled data, for example by deleting words or leaving sentences incomplete, then learns to fill in the gaps across billions of repetitions. He illustrates the result with a GPT-3 sample writing about Twitter in the style of the 19th-century author Jerome K. Jerome, and quotes OpenAI's Ilya Sutskever describing an impression of "real cognition" in the system, while noting it does not truly understand any sentence and cannot yet perform multi-step logical inference.
Comparing the US, China and the EU
Citing a Center for Data Innovation survey scoring the US, China and the EU across six criteria, talent, research, development, hardware, adoption and data, Uszkoreit says the US leads on talent, research, development and hardware, China leads on adoption and data, and the European Union, despite more than 500 million people, lags behind both overall, though it holds up better in enterprise and manufacturing AI. He describes AI as embedded in daily life in China: food delivered by robots, traffic tickets issued entirely by AI, AI-directed traffic control in Hangzhou, a fully artificial television news anchor, and driverless taxis already operating in Shenzhen. On research output, he notes China now publishes and files the most AI papers and patents, and while early conference citation counts lagged, Chinese journal papers are heavily cited, even as looser patent-filing incentives have produced many low-value patents.
How DARPA-style funding built US applications
Uszkoreit traces the US advantage in applied AI to funding built originally to win the Cold War space race, starting with ARPA/DARPA after the 1957 Sputnik shock. DARPA-funded work produced the early internet (ARPANET), the computer mouse, hypertext, GPS, the Siri assistant, machine translation and autonomous vehicles. He lists the ingredients he says Germany lacks: clear strategic technology goals, multiple competing funded teams on the same problem, large infrastructure projects, government procurement from small companies, sustained funding until goals are reached, and highly competent, independent program management by experts who rotate out rather than staying attached to specific labs.
Brain drain and Germany's underfunded strategy
Uszkoreit gives three US success stories rooted in German talent: Sebastian Thrun moved from Germany to Stanford and DARPA-funded autonomous-driving work before joining Google; Google Translate was started by a researcher from Uszkoreit's own group, who then hired Franz Josef Och, another researcher who had also worked on Uszkoreit's projects in Germany; and deep learning architectures developed in Germany by researchers including Jurgen Schmidhuber were commercialized mainly abroad. He notes Germany appears in international AI research collaboration data almost only with the US, reflecting weak cooperation elsewhere. On policy, he recounts Germany's November 2018 AI strategy promising three billion euros, a further two billion promised in 2020, but only about one billion euros actually spent by late 2021, compared with China's AI-first five-year plans. In response, the German AI Association launched the "Large European AI Models" initiative with about 400 supporting companies, including Aleph Alpha, working in three steps: funded pilot projects, compute infrastructure, and joint work with other European countries.
Q&A highlights
Asked how to stop the brain drain, Uszkoreit points to Germany's Bundesbesoldungsgesetz salary law, which ties research salaries to civil-service pay scales and prevents institutions from offering competitive compensation, alongside the need for genuinely competitive, comparison-based research funding rather than fixed multi-year project plans, and for accessible compute infrastructure so researchers are not blocked by limited access to high-performance computing centers. He says he believes Europe can still keep up with the US and China given these ingredients, since no law of nature guarantees continued American dominance.
Quotes
Our salaries have to be in line with the salaries of civil servants in the ministries, so you can never get the good people.
Hans Uszkoreit
There is no nature of law that says that the US has to be so much better than ourselves.
Hans Uszkoreit
Questions answered in this talk
Why has the US taken the lead in applied AI according to Hans Uszkoreit?
At Rise of AI 2022, Hans Uszkoreit attributes the US lead to DARPA-style funding that sets strategic goals, funds several competing teams on the same problem, builds infrastructure, procures from small companies, and sustains funding until goals are reached, an approach he says the German government has not replicated.
What makes large language models like GPT-3 different from earlier AI models?
According to Hans Uszkoreit at Rise of AI 2022, GPT-3 and similar models use self-attention to link related parts of long inputs, are trained with self-supervised, automatically labeled data at much larger scale, and can generate, translate, summarize and answer questions fluently, though they still fail at tasks requiring several logical reasoning steps.
How does China's AI adoption differ from Europe's?
Hans Uszkoreit explains at Rise of AI 2022 that AI is embedded in everyday life in China, from robot food delivery and AI-issued traffic tickets to driverless taxis in Shenzhen, while Europe, including Germany, lags in turning strong basic research into applications and in scaling AI in the market.