At Rise of AI 2023 in Berlin, Ralf Herbrich, professor at the University of Potsdam and managing director of the Hasso Plattner Institute, argues that AI's growing energy cost is an underexamined limit on its future, and that AI can also help solve the transition to renewable energy. He says the classic Turing test of matching human behavior has effectively been passed in narrow domains such as Go, image recognition and coherent text generation, but that a 'revised' Turing test factoring in energy cost is far from being met: training a robotic arm to grasp an object, he says, can cost roughly 2.8 gigawatt-hours, equivalent to about 200 human lifetimes of energy, and AI compute energy use has been doubling roughly every three months since 2012. He describes three research directions for more energy-efficient AI: data-efficient models and model pruning, approximate low-precision and error-tolerant computing, and analog computing, and separately describes how AI can improve renewable-energy prediction and battery management, including estimating a battery's degradation from limited sensor data.
Key takeaways
- Herbrich estimates that training a robotic arm to grasp an object can cost about 2.8 gigawatt-hours, roughly 200 human lifetimes of energy.
- AI compute energy use has doubled roughly every three months since 2012, a millionfold increase over a decade, per an OpenAI study he cites.
- Reducing numeric precision from 64-bit to 10-bit in a Gaussian process model saved a factor of five to ten in computation with little accuracy loss.
- His group can estimate a battery's internal degradation ('state of health') from about half an hour of streamed voltage and temperature data.
- About 12 million electric-vehicle batteries will retire over the next seven years while still retaining roughly 80% of capacity, a reuse opportunity for storage.
Detailed notes
AI has already passed a narrow Turing test
Herbrich defines AI as combining data science (digitally perceiving the world), machine learning (predicting the future from present and past data) and decision-making (acting on predictions). He argues the classic Turing test, whether an algorithm's behavior is indistinguishable from a human's, has effectively been passed in narrow domains: a machine learned to play Go by simulating future games in 2016, and machine-learning systems have matched or exceeded human accuracy in image recognition since 2015, and can now sustain 20 minutes of coherent speech.
The hidden energy cost of AI
Herbrich cites an OpenAI study finding that, for the same task, AI's energy usage has doubled roughly every three months since 2012, a millionfold increase over a decade. He gives the example of training a robotic arm to grasp an object, which he estimates costs about 2.8 gigawatt-hours of energy, compared with roughly 14 megawatt-hours for an 80-year human life, meaning the training uses the energy equivalent of about 200 human lifetimes. He also notes that GPUs are at least 10 to 100 times less energy-efficient at computation than the human brain, and that AI algorithms already account for about 4% of global energy use, a share he expects to rise as automation spreads.
AI as a participant in the energy market
Herbrich frames energy as a market with supply, storage and demand, each with a role for AI. On supply, he says the sun provides roughly 200 times the energy society needs, so the challenge is predicting where and when renewable sources like sunlight are available. On storage, batteries are managed by microprocessors whose control algorithms determine efficiency and lifespan. On demand, transportation, especially cars, is a major lever, and algorithms themselves already consume about 4% of world energy, a figure he expects to grow.
Three approaches to energy-efficient AI
Herbrich describes three research directions. First, approximate models: data-efficient algorithms that need an order of magnitude less training data (and therefore less energy), and model pruning, recently demonstrated by Meta, which compresses a trained model like GPT-4 by removing parameters after training, accepting slightly more error for far less storage. Second, approximate computation: using lower-precision number representations and hardware that tolerates errors, since machine learning, unlike accounting or databases, can cope with mistakes because it predicts an inherently uncertain future. In an experiment with Gaussian processes, his team found that reducing numeric precision from 64-bit to as low as 10-bit saved roughly a factor of five to ten in computation with almost no loss in prediction accuracy; a related idea, borrowed from how phones correct signal errors, triples or quadruples a computation and takes a majority vote to cut arithmetic error rates on unreliable hardware. Third, analog computing, which early vision-model results show can save close to two orders of magnitude in energy for prediction (though not training) at similar accuracy.
Batteries: physics, degradation and a second-life opportunity
Herbrich explains that a battery's usefulness for mobile applications depends on energy density, roughly 200 watt-hours per kilogram for lithium-ion, far below coal (about 6,500 Wh/kg) or gas (about 15,000 Wh/kg), which is why fossil fuels remain cheap to transport and use. Every charge cycle causes some capacity loss as fewer ions remain free to move. Because a typical lithium-ion battery streams voltage, temperature and current measurements roughly every ten seconds, and because battery physics and thermodynamics are well understood, Herbrich's group has used machine learning to estimate a battery's internal 'state of health' from only about half an hour of observed data, work demonstrated in simulation on packs of 14 cells. He also points to a concrete opportunity: around 12 million electric vehicles will have their batteries retire over the next seven years while still retaining about 80% of capacity, usable safely down to about 40%, representing a large amount of reusable storage the industry did not originally plan for.
Q&A highlights
Asked what matters to decision-makers besides energy, Herbrich and the audience raise trust, transparency, robustness and cost; Herbrich argues that the human cost of training and retaining staff able to maintain self-learning systems is often the larger and more real barrier to adoption than energy cost today. Asked whether finite raw materials (for chips and batteries) constrain the energy-efficiency methods he described, he says the larger constraint is the sunk cost already invested in existing CPU and GPU hardware, which makes solutions that work with existing chips more viable than abandoning them. On battery raw materials specifically, he points back to the roughly 12 million electric-vehicle batteries reaching end of vehicle life with substantial capacity remaining as an underused resource rather than a call for new mining.
Quotes
Energy efficient AI is a fairly new and emerging area because it didn't really make much sense to focus on it before it was clear that we can match human intelligence.
Ralf Herbrich
The reason why energy batteries play such a huge role now for cars and for other applications is because they're so dense.
Ralf Herbrich
Questions answered in this talk
How much energy does training an AI system actually use?
According to Ralf Herbrich at Rise of AI 2023, AI compute energy use has doubled roughly every three months since 2012, and training a robotic arm to grasp an object can cost about 2.8 gigawatt-hours, equivalent to roughly 200 human lifetimes of energy.
How can AI help extend the life of batteries?
According to Ralf Herbrich at Rise of AI 2023, machine learning can estimate a battery's internal degradation, or 'state of health,' from only about half an hour of streamed voltage, temperature and current data, which could help control batteries to last more charge cycles.
What are the main approaches to making AI more energy efficient?
According to Ralf Herbrich at Rise of AI 2023, the three main approaches are approximate models (data-efficient training and pruning trained models), approximate low-precision or error-tolerant computation, and analog computing for inference.