foreign networking so who have you had minimum has minimum one contact more two three plus five plus ten oh what great um Okay so we've learned a lot already about how we can develop in Germany and now we go on and expand that also in the next half to to Europe we will also listen to trust in AI this is like one of the main topics we all know and how we can also see a bit more the advantages in this field but first of all we also know that when it comes to technology we also must consider that we think responsibly about that and so it's about responsible AI it's about Green Tech it's about a sustainability and AI so how can we think about that when it comes to Ai and energy and I have the next speaker here with me he is Professor Dr haif hablish he is the professor of the University of Potsdam and managing director of the Hassel platina Institute and I would like to welcome you on stage
put your hands together let's go thank you good afternoon um I guess I have the honorable role to uh not make you fall asleep after lunch so I'll try my very best um what I wanted to talk a bit about today is uh in what our chair what I myself have after 25 years in the field of machine learning and AI devoted my attention to namely Ai and the role that energy plays so what I want to do see if that works start with a little bit of background why did I consider why do I consider this an important topic and then after painting the background um give an insight into what are the research topics on the the questions we're after when it comes to the role energy plays for AI and then some preliminary results some approaches that we've considered so you're going to see something that's sort of off the press but also what role can AI itself play when it comes to energy management there is some obvious ones um it
itself has it doesn't use too much but there is maybe some that are not so obvious so in terms of the background I often get asked um what is AI and when I started um you know the computer is the most powerful computer was in Intel 286 and you know the it was more a theory of what it could be back in 94. um but now what I would say is AI is really coming three areas together it's data science which is about how do we perceive the world digitally and we've made big leaps in the last 30 years uh you know all of you have phones so the location where you are the images around you everything can be sensed um in in huge amounts machine learning which is a core of making a system intelligent but what is really focused on it's it's methods that try to predict the future based on digital information about the now and the past so this is how you can explain advertising search robotics every application we
need AI to predict machine learning to predict what the Futures and then the last task is this of autonomously making decisions based on what we consider harmful or useful in in a decision taken and the possible future is that the system can foresee a little bit like we we plan and one test that is very classic has been around for 70 years to say is a system truly intelligent is the Turing test and I would argue that in the course of the last 20 years we really did pass this test exceeded it in certain domains so in 2016 by computation we were able to play the most difficult game by learning from past games simulating future games the game of goal by a machine in 2015 we've also achieved and by now supersede the accuracy by computational methods of machine learning and decision making of predicting what's on an image and you know I think this morning you've seen Fabian giving not giving the talk but having a system uh speak coherently for 20 minutes
that is the definition of a Turing test so what now and one of the interesting things is that these achievements come at a massive cost of energy to give you a sense there's a study done by openai that since 2012 for the same task not for the same data we've been doubling the usage of energy for this task in machine learning and in AI every three months so we're using a million times more energy for the same task like digit recognition image recognition than we did 10 years ago if you take the simple task of me grabbing this box cost me nothing in terms of my energy there is a robotic manipulator robotic arm can do this now which takes about 2.8 gigawatt hours of energy cost to train now you may wonder is that a lot or not give you a sense a human life in terms of the intelligence is 14 megawatt so this is 200 human lives of 80 years old of energy to use for this simple task of learning to
grasp things children need a lot less and if you want to just know you know most of this computation happens on gpus gpus itself are a factor of 10 at least if not 100 more expensive in their computation than the human brain is so clearly there is a mismatch and and this this was the motivation for me to go into this area and so when I looked at it I kind of thought okay energy is generally the most constraining thing we have on this Earth so how can AI play a role in it so I looked at it as a market view so energy is Supply that's predominantly it's actually only the Sun and luckily there is 200 times more energy coming to the Sun than the whole Human Society needs every day so we have enough of it but we need to know where it comes from so predicting where Sunshine is where the availability of renewable sources that's a problem AI can help because it's about prediction if you think about the marketplace
well if you wanted to match the supply with the demand we need to store it somewhere unless we can use it in the same space at the same time so another area that becomes incredibly important is storage solutions for energy and one of the things that's really changed our society is uh batteries that are extremely dense I'll talk about a bit more but these batteries they're controlled by a microprocessor so their lifetime their their efficiency is actually something that algorithms control today it's mainly ruled that's where they're not as efficient as they could be and so in a circular view it's something where I can really make a big difference in advancing our storage solutions and lastly the energy is used for a lot of uses and you see them predominance here Transportation this is why cars are such a big lever for society to look at and there's already quite a lot of AI in a car but the areas that I uh or we focus on is that algorithms themselves today contribute four
percent of energy usage of all worldwide energy around which is small there's still 96 others but as we keep putting Ai and more and more solutions to automate that number is only going to increase not go down if you don't change anything good so I think I hopefully made a bit of a point that energy is crucial if we really want to match human intelligence at the same energy level um but it's limited it's bounded so if we truly wanted to change our way of living and get help from algorithms we need to make them as efficient as we are not in terms of not only in terms of accuracy but also in terms of what they use and that's roughly speaking three kinds of methods that can be used some of them are very well studied for past 20 30 years some of them are relatively new because that problem is new there was no point as a society as a community thinking about it until we really matched the accuracy of human intelligence
so the first one is approximate models so what I mean by this is current models I mean chat gbt is something that has a hundred billion parameters that's just simply in terms of the computation and storage cost when we when a machine has to hold it is something that consumes energy and one approach to address this and it's been studied in the past but I think it becomes very relevant is how to approximate the models how do you use algorithms that don't need a whole crawl of the entire internet to learn the parameters but maybe an order of magnitude or two orders of magnitude less data so-called data efficient algorithms that typically uh known as algorithms that capture uncertainty also but they'll become hugely relevant for this because if you use the tenth of the data then obviously you can get away with a tenth of the energy for the same task so as long as you keep the accuracy the same something that's very novel and and I think meta recently just published an
example of this is metal model pruning so you train a model it's very predictive like for text like gpt4 but then you compress it so it makes a little bit more error for a lot less storage of the model so you remove parameters after the training I think that was just a couple of weeks ago or a couple of months ago released in the open source community and people are lots of researchers and Engineers around the world are demonstrating that you can get similar functionality with a lot smaller model it's a linear lever but a powerful one so that's one area there are the we focus on more is approximate computations and it's really motivated by looking at as as as humans and human intelligence as the Baseline because computers weren't conceived originally to make intelligent computations predictions they were conceived to help support the job of accounting the first computers in the 40s they were built as accounting machines and what was the most important property you have to have as an accounting machine
well it better not be wrong because nothing is worse than if I add my my actual expenditure and it's it's off by a few euros so the machines were principally designed to not make mistakes not a single smallest unit the transistor on a chip must make a mistake it's such such a crucial feature that on a spaceship every computer is tripled to reduce the error rate even further when it goes into outer space because the further you go into space the more cosmic rays do lead to to mistakes on the finest computation level humans however don't have that property we lose neurons all the time they're slow they're not accurate so clearly the the information processing in our in our brain isn't happening at a saturated level of a basically a switch a very small switch at a few nanometers but it's a switch so in the area of approximate Computing we're looking at two areas how can we maybe take the idea of representing numbers with lesser accuracy and how can we take this
idea of having basic building blocks like our transistors or different ones that take less energy but keep making mistakes they'd be horrible computational hardware for an operating system or for a database or for financial software but they're very viable ones because all the algorithms of machine learning and and decision making they naturally cope with uncertainty and algorithms of machine learning will always make a mistake because they predict the future they cannot be right an algorithm of accounting has to be right because it predicts the past what did I spend or a database and I'll give you an example here here's an algorithm class I'm not going to go into too much detail I promise um that is very data efficient it's known as gaussian processes think of it as a one layer neural network where the uncertainty of the prediction is tracked so when you take a very small data set for example these 11 points you're not only getting a prediction I think that every neural network would give you but the algorithm also
computes the uncertainty how little it knows about this prediction at every point so this can be visualized by the uncertainty in this range and as you can see seems a reasonable estimate but a little bit more fitting model that has the red one fits the actual data better but it has so much complexity that the model automatically knows to be less certain the further it goes out of where the actual yellow dots were so this is a data efficient algorithm for those interested they can look this up um called gaussian processes one of the experiments we've done is to say typically when we had looked into them until a couple of years ago we were always trying to use the highest Precision of of numbers to represent them which led to a quite costly computations and so in one experiment that we've recently done we've looked at how much savings could we have in the amount of computation so this translates directly to the energy consumes as the x-axis given we're willing to make some
mistakes in the some errors in the prediction by going from 64-bit numbers to 32 or down to 10 bit numbers to do this data efficient algorithm and as you can see the further we go down is the logarithmic axis on the X scale depending on which Optimizer we use we always save almost half an order of magnitude Vector 5 to factor 10 without almost any sacrifice in the bottom line so the training error the top lines are the generalization error the error of making in a prediction and that's very encouraging because as I said up until now we always as a scientist we always assume we have to have high accuracy for these volatile mathematical methods but turns out that's really not the case another example is as I was alluding to in a normal processor we do base everything on the idea that the chip makes the transistor the single operating unit that carries out operations like addition or multiplication make no mistakes and the if you actually take one of those modern chips
that has 10 or 20 billion transistors on them they're actually having what all they do is so this is every one of those symbols is a sequence there's a couple of functionality implemented by this transistor and all they do is adding numbers so your your CPU is pretty dumb your your central processing units or your GPU because all it does is it gets a number gets another number and adds them together now it does this a lot so a lot of the com almost all the computational energy of the algorithm goes into adding numbers this is how you add a number that just has two values zero one this is the car this is the wire wireframe uh the circuit diagram if you add two numbers zero one but you're willing to take the carryover from the previous position and this is the circuit diagram if you um if you add four bits so just numbers between 0 and 15.
so this is simple but what we've looked at is how stable are these circuits today if we introduce a small bit of error so what you see here is if we take this very dumb header and we add let's say 10 Arrow that's the x-axis we already get 30 errors on the output if we add a single bit a single digit and take the carry we get all the way to 50 error for 10 error on the input and if you add the simple number from 0 and 15 you have 90 error for 10 on the input so it's a very terrible Behavior so you couldn't without an algorithmic change you could possibly not use them even for machine learning so one of the ideas that we came up with is what similar to Communication in communication when you speak into a cell phone it's obviously getting Disturbed the signal from where you speak it and where it's received but what your what your phone does is something smart it takes the signal that it
records and it duplicates it it triples it it's so called it builds the so-called code with redundancy and then on the phone that you listen to it decodes what went over the wire and it does that so that while it travels through the air through the satellite and back it's okay to get errors in this in the communication Channel um we've got very used to we're accepting that somehow some smart algorithms make us hear sound regardless of how far it travels we want to use the same trick so we looked at can we use the same trick and we triple or quadruple the computation to minimize the error rate the edit difficulty in this setup is that um the the the decoding itself is a computation and we assume that everything computes wrong so even the decoding algorithm isn't accurate so we need to find an encoding and decoding where the computation of decoding and encoding introduces less errors than it saves in the first place and we came up with this uh so in
a theoretical study we've kind of analyzed when we quadruple every input signal and combine them with something that's a standard transistor type on a on a CPU we do reduce the error because it computes a majority function and you can see this here on the left you see um if you were to basically add numbers have been zero and 255 without any correction and the error rate keeps going up over time so from two percent three this is the beta it gets very bad so after 10 error adding two numbers I think this was 100 plus 88 um every every result is possible so in the middle graph we assumed we can artificially reduce the error rate for the so-called inputs that carry a lot of weight the the higher the high carries and it looks a lot better so obviously the middle one is really peaked around the right value but this we can't do artificially we can't reduce the error rates on the right we use this trick where we simply create triple
the computations and as you can see the middle and the right are very similar so this computational trick is really something by which it's a redundancy and we believe it's also something similar to the brain a redundancy in our computations that allow us to deal with these the noisiness and the error rate of our single compute units and the last block of of methods that people really look into for this energy efficient AI is to say we go away from um the the digitization so right now we we do all our computation on on digital circuitry where we basically really deal with two states only by by operating every piece of the device at what we call saturation Point lots of energy so it can either be on or off um analog computers are such a mechanism so you really use an amplifier effectively as your compute unit and if you do that and there's some early results for vision where you take an existing model and you use the uh you use the analog
nature of the transistor to save and Order almost like two orders of magnitude for the exact same accuracy in the prediction which will be it's a good Advance not for training but for prediction I do want to share in the last five minutes a bit of light on the other side this is energy and the role in place in AI algorithms but there's the other way as well as I said earlier lots of energy arrives here for us in principle we shouldn't have we shouldn't have an issue but we do we clearly burn fossil fuels more than we use Renewables because fossil fuels we can store them very well we can put them in a barrel and then we can put them in a bowl and we can get them anywhere this is a little harder if you have a PV panel and and get some power this way and so the solution for this is If You Could Only store energy that you can travel it through space and time um and this exists
it's not a new invention a battery is something that's almost 200 years old in terms of physics and the way it works um it has basically three components it's got the plus and the minus pole called the anode and the cathode something collects there's an overshot of of ions on the one side and a loss on the other and the differential is called a voltage and the two um one is called the anode and the two are separated and that's how every battery looks inside whether it's round or it's a poach and then when you charge it um the ions flow in one way and when you discharge your flow on the other and if these free moving ions would never react chemically with anything you could charge and discharge the battery forever however you can't do this because every time you run use a battery less of these red dots are free to move and we we know this as a capacity loss you use your phone for a year all of a sudden
it can only last for 20 hours rather than 24 hours um now I said the important thing is I can put energy into a barrel and I can move from A to B the key property of a battery for this is called energy density and what energy density really is is how much energy so typically measured in in watts per hour how much energy can I put in a given weight because if I if what I can store in a kilogram of weight is less energy than moving a kilogram of weight from A to B then I would use up all energy to move it it would be good as a static storage but be useless to move around so I the reason why energy batteries play such a huge role now for cars and for other applications is because they're so dense so the scale is roughly in what hours per kilogram so you can see here the lithium ion is around 200 and there's another measure called the volumetric density so this is
how much space it takes for an amount of energy is high enough now that it can carry the chassis of the car it can carry the people in the car and it can carry itself and that's useful but at some point obviously it loses enough capacity to do that and that's the point when you have to bring your car to the dealer and those batteries uh seem useless for driving but they're still useful for other applications because they're still 80 capacity in them now just to give you a reference figure because you may wonder how good is oil and coal and why have you used it well the units is the same so coal has about six and a half thousand watt hours per kilogram and uh and gas has about fifteen thousand so this is why obviously it's very cheap in energy terms to transport it and then still use it where it's where it's needed and it's much more costly to do this in a battery and so what we wanted to do
is we wanted to basically methodologically look use this fact that if only we could find out what the degree degradation of the battery is inside we can control it better to last more Cycles so instead of 2 000 Cycles it hopefully can last four or five or even ten thousand Cycles because the longer it lasts the more uh effective for the environment it is in the first place building a battery also takes CO2 only when you charge it thousands of times and and the good news is we don't have to we in contrast to advertising or search where we don't really know how do I predict what someone thinks when they click on an ad I know exactly what happens inside the battery in terms of physics and and thermodynamics so I have lots of data so a typical lithium-ion battery cell or pack that you have in your laptop or in a car streams at roughly the rate of once every 10 seconds a measurement of the capacity of the voltage a measurement of
the temperature as a response of the the current that is being used and that is controlled by a microcontroller and so we as a physical model that would say if we were to know what the residual capacity at each time T is it would tell us how much is the capacity in the next stage so this is your little this is your bar that shows you how full the the phone is it could also there's a physics model these are all known physics models what the voltage is on the output as a function of voltage and current and similarly if we already understand thermodynamics well enough to know how the heat distribution happens and we begin to understand how this aging how this this idea of the red dots clamping onto the separator or the anodes happen however the only things we can control is the current the only things we can observe or an algorithm can and we've digitized it is the voltage and the temperature the things you would like to know is
how degraded is the actual battery components inside and that's the that's the ingredients for machine learning problem and so we've kind of looked at this in a simulation to say can we estimate what really happens inside a battery in terms of this Soh the state of health only from observing half an hour of the battery in the laptop or in this case it was a car battery operating um which will become important if we want to control them better and show in contrast this is a method based in optimization that's pretty novel now people use it predominantly now for for optimizing the neural network architectures but it's equally useful as a technique for optimizing the lifetime of batteries in particular when you have because modern batteries are not one cell that would not be enough to drive a car they have hundreds or thousands of cells so here's a simulation where we put 14 of these cells together and re-estimated with the with the red bar what the blue bar where the real truth was
and um pretty promising results so to to wrap up I think the the one thing I would say is we've come a long way in the past 30 years in terms of AI because the original curing test of indistinguishable behavior of an algorithm to a human does is pretty much solved in in narrow domains such as speech Vision natural language processing the revised Turing test the one where we Factor the energy cost in basically achieve indistinguishable Behavior to human intelligence with the same amount of energy that the human operator takes that we're a long way away from and that to me is the interesting man if you really want to see AI being used in society well you know in every part of society because there's there's nothing by which we can make up energy we have to only use what's available to us um energy efficient AI is a fairly new and emerging area because it didn't really make much sense to focus on it before it was clear that we can match human
intelligence um it is one that's very interesting because it isn't just mathematics or isn't just computer science it's really the intersection of physics computer science neuroscience and information Theory and AI methods themselves uh shouldn't just be limited to e-commerce or to uh uh you know digital photography but I think there is a lot of a lot of potential for using it for components of the energy Market I presented one for storage solutions but I believe there's many more one key thing that is crucial is in this one we want to act so causality really to estimate what will drive a change in the ability to store or the ability to control a house is is crucial and it's um to me important research area thank you hi you've mentioned and splitted AI into three Fields so data science machine learning and decision making and I would like to go into the decision making thing because one aspect as we now saw is energy of course and maybe we can involve the audience and then you
you are the one who concludes that into the aspects you would say are important for a decision maker um in AI but maybe not to you you can just say it and just scream the the point so what do you think is an aspect for decision makers like beside energy when it comes to AI any other aspect or is it only energy quality something different trust yeah I think trust is a big one when you saw the homes because if you want to hand over our home this is one of the most private spaces if you don't trust this just I would not let it control the heat yeah nobody wants to talk about it but actually cost is probably the most uh the most important one for most decision makers yeah can I challenge him you're uh you do it at the end so we consider every aspect now and then you can say okay this was right and that was wrong and then you can conclude that another aspect transparency another one come
on privacy again robustness social connect okay so what do you think like we had trust transparency robustness um Quality yeah what do you think I think they're all extremely important as well in particular because we interact with these systems and if we if we don't they don't find adoption either I would like to challenge one comment on cost I think cost is a good one but my my challenge is where would you draw the difference between cost and energy because you have to pay for energy today and we can very much see when you double or triple it it becomes a factor so what costs could be there that are not related to energy water isn't that energy to cool it the water doesn't come cool oh assuming the transportation of the water for the cooling okay I'll take that what do you want to say to buy the computers first as a software as a service and then you have to pay the people who who install it and run it and that's difficult
right now to get them so they are more more expensive so I mean this is just what I encounter when I talk to decision makers that they are they are they're considering quality and costs couldn't thank you that I agree the human costs so to speak so the machine costs is mainly I think energy translated into how much it costs to build the thing because that was also driven by machine but the human cost is a very real one thank you but what do you think is the main cost in this uh so when you want to integrate AI into your business what are the highest costs I think today it's I mean it depends a bit but I think the energy one is not going to be the limiting one to today I think uh it's also part why I chose to like focus on in science I do think the um uh I think there was this concept of trust and transparency that to me is a big one and then to your
point the human cost what I've often encountered today is you you can build these systems that have components that learn from data and adopt themselves but if you don't if you can't guarantee that someone can maintain it as it runs 24 7. it's too risky for you to put that in and you'd rather go with a real based system where you can rehire your your next employee or because no one I mean you know no one works from 20 to 65 in the same company anymore so the the the the cost of training people to build and maintain these systems is a huge one to me so what would you say is the right decision maker for all these decisions to make in a business in the company the company yeah I think it's the product manager and engineering manager I yeah can they consider all these points they should they should I think they must in particular the trust one when it comes to Consumer technology where we use it can we have the
microphone I think this conversation deserves a different question because the alternative question is what's the cost if I don't do it yeah what's the cost for a company a mid-sized company if I don't today seriously consider the implementation of AI where am I with my company in five to ten years from now there's a high chance that they will be out of business that's an important analysis to make yeah what do you think what do you think what are the costs when we don't start now out of business out of Market well that's high cost as an individual company you face that decision as a society they'll be a company that does invest in this Automation and it'll be around that's your point right yeah yeah any other question here putting in there and you're building like a very Advanced battery technologies better neural networks like you also have to take into account that we have only finite resources yeah so we're building more chips we're building more batteries we have only finite amount of
chemical compounds where does it actually enter in this calculation will make a holistic economic argument about this let me try to recap there's sort of a fixed cost for every AI computation which is a need to have the machine first I need to manufacture it and then there's a variable one I operated so I talked a lot about the variable cost I'm operating it you're asking are we constrained by the fixed cost because the resources won't be sufficient um oh it's a good one I think it it enters in so far as the methods we should consider the reason I sort of started with the slide I took out how a CPU is structured I think it the the only viable next step for an alternative Hardware I see is something that leverages existing ones there is so much of existing CPUs and gpus available that if the only way to become energy efficient is to abandon them is is too costly in terms of the fixed cost for us as a society so where
I think it enters this when we there's always design constraints even in research so when we think about when I think about what constraints I should use is is how can I Implement for example you saw the GP the gaussian process approximation so we looked first at how to implement this on existing CPUs it's not that simple I didn't want to go in detail but if we don't then we the economic decision for everyone it has to implement it is to write off the the fixed cost investment in hardware and there's just too much sunk high cost uh sunk fixed cost right now so that's my design constraint Instagram I completely buy that like the question for me is if you're actually looking into Battery Technology you talked about energy density as well right we know that certain types of current uh battery technologies like for example lithium ion are not necessarily energy dense enough to really operate very heavy machinery so we need to find new compounds these new compounds are probably very highly
complex compounds yeah unless you find something like uh room temperature superconductivity that you can actually store these kind of things which might or might not happen but like these these design choices also matter with so how do we right now already account for the fact that we need to mine these very very costly compounds to build them together and do we actually how do we ensure that we have enough of them to actually satisfy the need that we need yeah I I can't answer that question one I can answer I find this related to this um and hope is I spoke about the the car batteries right now so there's give or take 12 million electrical vehicles on the roads today they're coming out of service in terms of the the the reach that they have so in the next seven years there are about 12 million uh cars where the batteries are already built now when we say they come out of service they basically can't reach the extent the mileage anymore every single
cell in these batteries in these better packs of a car still has roughly 80 percent of capacity left and it's very safe to run them at least until 40 capacity so there's a huge Economic Opportunity without building any new battery just reassembling them because right now when they started in 2012 the industry it did not foresee what to do with them when they've reached 80 percent so we run our phones until they're 40 but we can't run the car that way so my view is you know a lot of research should be put into and it's very urgent what are we going to do and then it's 12 now what we're going to do is this 12 million huge battery packs that can store a reasonable amount of renewable energy we need and we have today that it's a slightly different question but that's my answer I think you can discuss a bit further right about that I think um that it's really important to to see the decision-making points and aspects we have from
both sides because like I think here are many experts in the room um and there are decision makers on the other side so when we know what are the breaking points we can directly go into this direction and convince them so thank you so much for your talk a big thought so