In the closing Meta Stage keynote at Rise of AI 2025 in Berlin, Prof. Dr. Hans Uszkoreit, Scientific Director at DFKI, argues that most of today's AI progress comes not from new algorithms but from what he calls "machine teaching,": feeding models more and better data. He reviews the shift from pre-2017 recurrent architectures to the transformer, describes current algorithm research as split between agent systems and next-generation architectures combining attention, memory and agency, then argues that since 2022 breakthroughs like chain-of-thought reasoning, DeepSeek's reinforcement-learning gains and reduced hallucination are really teaching effects, not algorithmic ones. He estimates humanity has produced roughly 150 million books, of which any single expert absorbs only a tiny fraction, and discusses lawsuits from authors like Sarah Silverman and Adam Nevill against Meta over training data. He describes a "knowledge paradox" where broad models beat generalist humans but lose to deep specialists, outlines ways to add domain knowledge, and closes by arguing that as intelligence becomes a cheap commodity, human value will shift toward handling dynamic, unforeseen situations and other people.
Key takeaways
- Uszkoreit argues most AI progress since 2022 comes from "machine teaching" (better data) rather than new algorithms.
- He estimates GPT-4's training data is roughly equivalent to 100 million books, and Llama 4's to over 200 million books.
- An individual expert absorbs at most roughly a hundred-thousandth to a three-hundred-thousandth of humanity's recorded knowledge, according to Uszkoreit.
- In his "knowledge paradox," broad AI models beat human generalists but still lose to genuine deep specialists in narrow fields.
- Uszkoreit argues human value increasingly lies in handling dynamic, unforeseen situations and other people, not tasks like diagnosis or report writing.
Detailed notes
Beyond the transformer: where algorithm research stands
Uszkoreit contrasts the major neural network architectures used before 2017, which powered systems like Alexa and Google Translate, with the transformer, which introduced multi-head self-attention and parallelization but initially lost the explicit memory that recurrent networks like LSTMs had. He says current algorithm research splits into two areas: the great majority of applied work goes into agent systems, where a non-AI orchestration layer calls one or more LLMs, databases, retrieval-augmented generation and knowledge graphs, since the models themselves have no agency or planning ability on their own; a much smaller group, including researchers at Meta, works on the next architecture generation, combining large-context self-attention with memory and agency to enable action models, situation models and eventually world models.
Why data, not algorithms, drives progress
Uszkoreit proposes a thought experiment: if algorithm research were banned tomorrow, AI would still keep improving, because he argues it is not new algorithms currently driving progress. He points to training data scale, noting GPT-4's training tokens are roughly equivalent to 100 million books even though far fewer actual books exist in the corpus, and that Llama 4's larger token count is roughly equivalent to over 200 million books, with Meta reportedly avoiding disclosing exact numbers due to ongoing lawsuits. He argues OpenAI's edge over Google came from collecting more data, not superior algorithms, and that this shift from "machine learning" (finding the best algorithm for given data) to "machine teaching" (how to feed, benchmark and test a fixed algorithm with data) explains gains such as chain-of-thought reasoning, DeepSeek's reinforcement-learning jump, and a discovered bug where reinforcement learning penalties on wrong-word proportion made models more verbose. He also cites related research, presented earlier that day, reframing most "hallucination" as missing knowledge rather than genuine fabrication, noting models never hallucinate a thirteenth month because the evidence against it is overwhelming.
The knowledge of civilization and its limits
Uszkoreit estimates, based on a Google Books count of 129 million books 14 years earlier plus roughly 2.7 to 3 million new books published each year since, that more than 150 million books have ever been published, representing the recorded knowledge of civilization. He illustrates its importance with a thought experiment: if children were raised by robots with no access to any of that knowledge, they would need a generation or more to reinvent language and civilization from scratch, referencing real-world cases where children spontaneously developed new sign languages. He notes an individual expert who reads widely absorbs at most roughly a hundred-thousandth, more realistically closer to a three-hundred-thousandth, of total human knowledge. He also recounts author lawsuits against Meta over training data, including from comedian Sarah Silverman and horror writer Adam Nevill, after a Meta employee reportedly claimed a single book improves model performance by only 0.06 percent, prompting Nevill to demand that share of the money invested in the model.
The knowledge paradox and domain adaptation
Uszkoreit describes a "knowledge paradox": broad AI systems beat human generalists on knowledge tests, including domain experts tested outside their narrow specialty, such as automotive engineers who don't know each other's sub-specialties, but still lose to genuine deep specialists and are not yet reliable enough for real tasks in a given field. He outlines several ways to add domain knowledge: supervised fine-tuning and reinforcement learning, which mainly refine behavior rather than teach new facts; continued pre-training, which risks catastrophic forgetting and is costly and risky since few providers expose checkpoints for it; and "domain adaptive pre-training," where a huge context window is used to teach a model a field temporarily, useful for confidential information since the model forgets afterward. He mentions advising and investing in a company doing this under the name "dynamic hybrid language model," combining prompting, ontologies and knowledge graphs, and suggests foundation model providers could sell this domain-adaptation service directly to large corporations as a future business line.
A world where intelligence is a commodity
Uszkoreit argues that if the full, curated knowledge of civilization were compressed into a reliable system, it would become the authoritative source everyone consults constantly, warning of a real risk that some countries would instead make such systems authoritarian by aligning them to a single state ideology rather than preserving multiple viewpoints. He predicts that because AI does not need to be physically embedded to be intelligent, in principle any device, down to a toaster, could become commodity-cheap and as capable as combined human knowledge, a revolution he stresses would not require any new algorithms. A consequence, he says, is that genuinely new knowledge not yet absorbed by any system will become immensely expensive, since it is the one thing left that is scarce.
What remains distinctly human
Uszkoreit closes by arguing that AI's inability to fully replace jobs shows we overestimated which human abilities were hardest to replicate, such as differential diagnosis, stylistic writing or composing reports, all of which machines can already do reasonably well. What machines still cannot do, he argues, is handle angry customers, manage exceptions, and navigate a genuinely dynamic and complex world. He illustrates this with a thought experiment: a system trained for ten years on everything a DAX 40 CEO read and decided still could not take over the role, because the next morning could bring new tariffs, strikes or shortages that no training data anticipated. He concludes that real human value lies in dealing with dynamic, unforeseen situations and other people, not in the tasks traditionally tested in school.
Quotes
It's actually the teaching, more data, not the new algorithms that make the progress.
Hans Uszkoreit
Every new knowledge will be immensely expensive.
Hans Uszkoreit
Our real value is in dealing with a dynamic and complex world and with other people.
Hans Uszkoreit
Questions answered in this talk
Is recent AI progress driven by new algorithms or by data?
According to Hans Uszkoreit at Rise of AI 2025, most progress since 2022 comes from "machine teaching," feeding models more and better data, rather than from new algorithms; he argues OpenAI's early advantage over Google came from gathering more training data rather than superior architecture.
How much of human knowledge does a single expert actually know?
According to Uszkoreit, an expert who reads widely absorbs at most roughly a hundred-thousandth, and more realistically closer to a three-hundred-thousandth, of the knowledge recorded in the more than 150 million books he estimates have ever been published.
What will remain uniquely human as AI becomes more capable?
According to Uszkoreit, human value will lie not in tasks once thought hardest, such as differential diagnosis or writing polished reports, but in handling dynamic, unforeseen situations and other people, illustrated by his argument that even a system trained for years on a CEO's decisions could not take over the role in an unpredictable world.