foreign guest who really is at the Forefront of helping companies apply AI to their businesses newly found managing director of Deloitte digital please give a warm welcome to Susanna Fargo so okay so very hard to speak after it was an interesting talk but I will go for a little bit another topic and that's a topic that popped up in my in my environment summer last year and it has to do that already the bigger companies need to report the ESG and so the team I had at that time I was wondering how what can we do with our AI Solutions and I have one motivating slide for that and that shows I heard it already a little bit uh today in different presentations because right now we already see we all talking about Aviation we're talking about cars but you already see on the slide and it was in on MIT uh a couple of weeks ago that we already have more carbon footprint concerning digital Solutions then with Aviation right now and this is
the current status so if we look into the future and if we look only in 2022 we already see that we have more emissions with digital products then with cars and with Aviation and this is not the first step so it's really going exponential if I'm we had already today Quantum Computing so it takes energy if we think about iot if we think about everything we have in all our lives here and so that's one of the motivation to to take a closer look and what we see right now is that every everyone is talking about sustainability everyone is talking about applications they're looking at for example in my former job to predict for example optimized flight rules predict how many menus I need for the flights or predict what whatever or looking at energy I just went over to to Really optimizing energy but there are only a few things to look at what do we produce from Ai and from digital solution um concerning the digital footprint and I have here nice graphic
because if if you look at that it's not that you do it once you do it all the time and you're producing permanent emissions footprint or with your AI Solutions when you think about it and I I mentioned I started in summer so what you see and you you see is in several studies if you look at gdp3 and look at the energy we all they're already spanned for that one and I was really shocked when I saw it's really 63 homes energy so everything you do at home heating uh like surfing virtual conferences so it's it's an average of everything you you have in your home for a whole year so 63 homes or you can charge your phones 61 million pounds with things the energy only GDP 3 took for training this is not the usage it's only the training and if you look at the Bloom model it's a little bit less there so only three um around three our homes energy and about uh three million um smartphones and I have
to show that a little bit because everyone is talking about airplanes so airplanes is this little Mark here on on the lower side this is uh if you are traveling from New York to San Francisco and that's the emission there so you see here the effect and that was the motivation to look a little bit closer to that so everyone if we think about like AI training and so on everyone is thinking about model development everyone is talking about what kind of Hardware we are using what kind of ram we are using but if you do an AI project you have more and that's a little bit like here on the slide the iceberg that it's a little bit lower where we have everything about what kind of devices the employees are using like laptops phones then everything about data storing depending on the model what you're doing you're storing a lot of data you might travel it depends a little bit so you have the travel time and you you have everything about fixes
monitoring and so on so what we thought um is we take a closer look at the life cycle and we make it in the first stack relatively simple so as life cycle as basic we took the life cycle of of Amazon it doesn't matter so much what kind of life cycle so they are similar you all know that and we saw we taking one life cycle like preparing the business goals doing the formulation of the machine learning problem than looking for all the data doing the data quality doing the model development then the deployment and monitoring you all know that in the first step we saw okay we are looking at the first phase emission that means only the emission I directly need for that project so we are not looking for ESG you look for two and three so how much emissions do you need to produce your PC or how much emissions you need to produce a GPU so we didn't do that we just took like a very simple one row there
and we didn't look at retraining um and everything like that because it would make it complicated and what we took and that's a little bit more complicated tonight so we took a little example from our uh from our project so we did um in before it's not a very complex one a very easy one and you see it here so we thought okay we have two employees we have an image processing um problem there we have a data set of 50 gigabytes and we have two employees with a 50 watt um laptops there and then we said okay we have a GPU to train everything and it's about 350 watt so that is our basic so first we went to the usage and you are all in AI so you nearly know how much time it takes and we put it there the the time the employee takes so it's basically that top time then we have the data storage and we have the GPU storage and we did the estimation of the time of
the measurement of the time for each of these phases and no surprise you have first only like the employee's laptop then you have in the data processing you have the data storage we start for the first things using the laptop and then we have only half of the data and the next phase we have everything the laptop of the employee we have the data storing and we have the the kpu there during the deployment only less and for the monitoring that's very special because you have to see that it's only the monitoring for one month so it's a little bit different so we took that time we had everything from from the energy consumptions of the devices we had what we need to estimate what kind of energy the data storing needs and from that we calculated the energy consumption we needed for every phase of such a project um and then we took the public map of Germany how the electricity map and calculated how much carbon emission we produced in every phase of
that little project and the result you see on the on the right hand side over there it's not so surprising so mostly you have it during the data processing and the the huge one is during the data modeling just to to briefly say it so during the data processing for this little project 109 phones you can charge with that for the development you can drive 400 or 40 900 900 30 000 kilometers with an average car and the last one it's tiny again but you have to remember it's just for for a month so it's recurring every month so that's just some kpis and a small summary to make it very pragmatic if I looking at the different phases we started in the first phases you have only the PC in the in the example we hadn't any travel uh travel there you have the devices then in the next two phases you have basically the data storage and the computation Additionally you have PC and travel and devices and in the um deployment and
monitoring phase you have all the other things additionally the uses of the application and now the question is how to measure that and what we didn't found I can't I can tell you we didn't found it tool that is really measuring everything so that you just put the button and get all the energy your AI um application is producing but we found some solutions so the rest one I have is our own internal travel carbon footprint what we already have from our customer relation management system that we are taking the travel of every consultant and and calculating the um footprint we are producing with this kind of project and this is just an example like with a car from Frankfurt to Munich and from Nuremberg to Munich and the other way around by car and then by um by train um I found it quite interesting if you like to test a little bit but you probably know the uh the Deutsche Bahn carbon footprint calculator so you can put in the distance you are
traveling and you're you're getting the results you probably know that there are different apis you can use to really calculate the footprint of your travel the other one I have if you're in the cloud um and uh producing all the training in the cloud you already have from all the hyperscalers dashboards about what your company is using and you can you can look at it they are very similar so they are different measures they are different mesh dashboards but you already can see what what you are producing there then if you have already a project or you are planning a project what we found from a research paper is a little tiny estimator that's calculating only the footprint of the machine learning part It's relatively easy like my example so what you have to put in you have to put in the hardware type then you put in like I did it before the usage hours then you say what kind of cloud provider I'm using and the region are very compute um the the
model and just mentioning that I have it later on it really depends on the region and the time of the year how much uh emissions you are producing with your model training or usage of the model the advantages tool we found is called code carbon it's it's open source and what you can do you can integrate it into your Jupiter code code and then it's calculating everything on your laptop and everything you are doing during the training and uh it's it's provides you a dashboard where you can check how much energy how much emissions you're producing with your application so data storage is not integrated in that tool and the travel is not integrated into and not the the monitoring and the usage of the application but it gives you in the first step a quite good like measure uh easy to use to have first insights here and the last uh slide I have here is is more a little bit about what do we do with all the kpis because it's always nice
to to measure things so you have if you measure that the Baseline but nevertheless there are already a lot of studies papers uh advices what we should do if you're developing on AI Solutions and we put here the uh the the four M's of of Google on on that slide but it's basically similar in other studies first one is what kind of model what kind of solutions you are using um depending on how precise it need to be you always should check uh what we you really need to solve that business problem and you can really say for three to ten percent end of the the energy you're using then the type of machine you are using for the um overall training and everything so it's about it depends really on the problem it's about two to five percent um then the type um of cloud if you do it on premise it's not the recommendation but to do it in the cloud and then um what what country you are using and that's the
um the graphic on the left hand side here because it depends a little bit on the time of the year which is the optimal solutions to choose here and if you take for example the dark green line here that the U.S um central line it really drops down in May so you have much less carbon emission in May but it's going up again in August and November so you always have to be a little bit aware at what time you are doing the training uh and what kind of reaching you should use to train your model and um the last is a little bit guidelines from from from at this time um about how what we found with with Amazon it gives you direct guidelines recommendation in the Amazon and environment and it focused here only on Amazon but it's very detailed and that's the next step but to measure it is for me always the first step because if you then use this guidance lines then you can see what you're safe with all
your improvements and that's the end of my presentation thank you thank you very much Susan um so we have some time if you don't mind yeah I need to ask a few questions and please uh I'll open the floor as well if you have any of your own so I was personally uh quite shocked actually by some of that data hadn't quite appreciated by how much we do have a carbon footprint as a result my question is around who owns that problem or who do you believe should own that problem within an organization and if I think about my data scientists at work for me if I was to to curb what they would like to do and what they could use I would get a very big backlash from them and we're uh we're stopping Innovation as a result where do you what would you what would your advice be or where do you see that evolving going forward now that we even have kpis yeah it's a little bit like the ethical topic
yeah because uh some education and it always has has to be you have your business goals and then you have to see how much can I do and where I have to do that but what what we we see and we always already have it for big companies that you in your advisory and Reporting yearly reporting you already have the ESG there and the first questions is coming and I had it from some clients that the advisors are already asked how much are they producing which in Ai and it's not that you are not doing like fancy things but like I said in the last one you always at least should be aware that you can choose where you train your model to reduce it it will always cost energy yeah fantastic any other questions from anyone on the floor please I have a microphone at the front thank you hi I was wondering why um do we know why the CO2 grams emitted are lower or higher in some months for some countries and
not for others yeah it's it's because in summer you have for example solar panels then it goes down because it's Greener and then in winter it's go up so the the curves that are going up and down the most the the highest wave are normally like countries who are more in the north and not in the um I would say not in the tropical areas that's the reason why so you think we should train more models during the summer and the different countries no no no for example if you yeah you can trade it in summer or if you do it for example in Germany in in Winter you can say and you already can do that with your cloud provider say I want to train my model in I don't know shoot South Africa or Namibia or whatever where you still have and that's that's the reason uh to to take a closer look at this open app the electricity map where we really can see how much green energy every country is producing
right now thank you we have a second question here the gentleman in the black Jersey oh sorry I'll come back to you in a moment yeah thank you very much for this uh for the insights on the on the carbon emission um I want to ask you a question about like how to actually capture the full life cycle of of models because it's like not as simple as like oh I have a model I'm running in production I have to inference cost but I have also like the amortization cost of like training these models especially with those large language models it's becoming significantly more complicated because it's a huge invest in a carbon invest in training a big language model but then the inference costs are also not quite small but the use cases that you can cover are huge and you save a lot of uh carbon by using a large language model instead of training like small bespoke models over and over and over again like how do we actually start capturing this
kind of like non-optimal point if we just like optimize every single step of the of the curve it's actually sub-optimal as compared to like the full yeah yeah that's the reason why why we wanted to see the different phases and I agree with you if I if I have a very high end that's the reason why I pointed out the monitoring phase because we always have to see that we do it again and again and producing much more so you always have to to look at it from from the whole life cycle and during the operation and I agree with you this is very pragmatic it's a first view because if I'm going in every every every detail it's very very complicated so what we we did here is just to give a first glimpse of where you can do something and what you have to consider you might have to go really into more details but I'm not so sure if the effort we are then taking really gives such better results so something
like that is relatively easy and you get get a first Glimpse and it costs you not so much uh performance energy working hours thank you Susan so it's a start so I'm going to go back to the lovely day the blonde hair and then I'll move over Tuesday um so you talked about that um and I was also really shocked about that one graph you showed around the U.S households like yeah I think there's of course a stereotype that you think okay everybody like needs a lot of energy there my question is do you foresee and we had I think a like energy crisis right a couple of months ago I think everybody remembers at least in Germany um that there will be also maybe political intervention in terms of how much energy you could leverage for that and is there also a need for prediction so I understand that now it's a lot of this is what happened right when you flew from New York to San Francisco but shouldn't we not actually leverage
it even more and say this is what we actually calculate and we should not go over that if we have those goals and not just like pay for it right to like I mean if I book a flight to San Francisco and I just click on the beautiful Lufthansa button that makes me feel great but it actually doesn't solve the problem yeah yeah it doesn't solve the problem so for companies right now what we already have is that Enterprise companies have to really report the ESG and depending on that um you already get credits for Investments you have that and it it's quite obvious that we will get that in Europe for all companies so for the company side you are forced to do that for your private I am not so sure if if the governance will really go into such restrictions is it's more and that's the reason why I showed the Deutsche Bank calculator it's a little bit I believe it's it it's education and awareness and what we're seeing right now
is a lot of awareness especially uh for the younger person that they are more aware and not taking so much flights but even so they are surfing a lot I believe it's quite difficult to give from a from a governance side really overregulation there thank you gentlemen the final question Thomas no but I but I'm sensing a little bit of a I don't know if contradiction is the right word yeah but I I've seen these numbers before if you actually throw in the calculation of CO2 when it comes to Serious blockchain activities it's much much worse than this but my question is on one end you have the whole world chasing sustainability and Global you know CO2 reductions and whatever the goals are for 2020 you know 35 and whatever so here comes AI so who is losing and who is winning on one end we have you know worldwide uh goals for cot admission reduction since ChatGPT AI is really kicking into gear plus blockchain will be just around the corner one is a
gift one is a take I mean I don't see a solution you everybody by the end of the day is happy either AI needs to slow down which certainly is not going to happen but then the people who are after the sustainability and the green and you know all of that stuff it's like I am not sure it will add up yeah what's your opinion yeah I agree I agree it's very difficult because we see it in history that that people are using things without thinking about uh sustainability and stuff like that nevertheless I believe if you put too much burden on it then it's it's quieter it's quite difficult but to have at least some spotlights on it I believe we should do that especially when you on the first slide it I it's not for me it's from from colleagues from Switzerland and when I saw it myself I was really impressed and that impressed a lot of my colleagues and it's it's not that you can do nothing but but raise the
awareness a little bit like we raised the awareness in the companies for traveling by train instead of traveling by airplane for business trips so what I my wish was during the last days don't have music in hotels on on the floor and don't have in the hotels all the lights on and stuff like just just little signs um it's very difficult to solve everything but I I believe we we have to we have to take a little bit more effort to go really into green energy and I suppose the reality is that it's an ethical question that will be a maybe a generational change if anything but thank you for your question and thank you for a wonderful presentation