Rise of AI Conference 2022 · Berlin · Applied AI Stage

Supporting the AI Journey through 2030

Edmondo Orlotti · AI Business Development Lead, Hewlett Packard Enterprise

At Rise of AI 2022 in Berlin, Edmondo Orlotti, AI business development lead at Hewlett Packard Enterprise and a former Nvidia employee, argues that scaling AI into production is mostly a data and infrastructure problem, not a modeling problem. Citing an Ericsson Mobility Report forecast of 370 exabytes of data generated per month on wireless networks by 2027, he argues data volume is becoming unmanageable and that a cloud-only strategy cannot work, pushing HPE toward an edge-to-cloud approach. He illustrates this with HPE's partnership with the Mercedes Formula One team and an end-to-end platform built for Volvo Cars, through its subsidiary Zenseact, to collect, store and process the hundreds of petabytes of data needed to validate autonomous driving software. He describes tools HPE has built for production AI, including Determined AI for training large models at scale, swarm learning for training on sensitive data without moving it, and an MLOps platform for managing model life cycles, plus SmartSim, from its Cray acquisition, for embedding AI into traditional simulation. He closes by naming large language models and the metaverse as the trends he expects to most affect daily life.

Key takeaways

  • Orlotti cites an Ericsson Mobility Report forecast of 370 exabytes of wireless network data generated per month by 2027, arguing cloud-only data strategies cannot keep up.
  • HPE built an end-to-end platform for Volvo Cars, via Zenseact, to collect, store and process hundreds of petabytes of autonomous-driving data.
  • Orlotti says more than half of enterprise AI production problems come from data ingestion, aggregation and normalization, not modeling.
  • HPE's Determined AI acquisition, now open source as the HPE Machine Learning Development Environment, is used by Aleph Alpha to train large language models at scale.
  • HPE's swarm learning lets healthcare and banking customers train shared models without exchanging the underlying data, with results published in Nature.

Detailed notes

The data explosion and why cloud-only fails

Orlotti opens by arguing that AI and deep learning emerged from a need to process huge amounts of data, and that data volume is now getting out of control, citing an Ericsson Mobility Report forecast that wireless network data alone will reach 370 exabytes per month by 2027. He argues the common assumption of simply moving all data to the cloud will not hold, since the volume will make full data movement impractical, requiring processing wherever the data is generated and rethought data management and movement strategies, which pushes HPE toward an edge-to-cloud approach spanning the extreme edge, base stations, data centers and public cloud, driven by customer needs and the goal of minimizing data movement.

Case studies: Formula One and autonomous driving data

He describes HPE's long-running partnership with the Mercedes Formula One team, where AI was introduced across many aspects of the team's operations, and says HPE works with almost all Formula One teams. A second example is an end-to-end platform HPE built for Volvo Cars, through its Zenseact subsidiary, to collect, store and make available the hundreds of petabytes of data needed to develop and validate autonomous driving software within fixed roadmap timelines, built in close collaboration with the customer rather than by adapting an existing generic setup.

Making AI production-ready: infrastructure and Determined AI

Orlotti says most real-world AI challenges appear after a model is trained, in production pipelines where handling data from different sources accounts for more than half of the problems enterprises face. HPE addresses this with pay-per-use, as-a-service infrastructure spanning edge and data center, built on three efficiency pillars: compute (GPUs), storage for fast data access, and interconnect for moving data quickly. For training at scale, HPE's 2021 acquisition of the startup Determined AI provides software for scaling training across many GPUs and enabling multiple users and models to share the same infrastructure in a multi-tenant setup; it has since been released as open source under the HPE Machine Learning Development Environment name and is used by Germany's Aleph Alpha to train large language models.

Swarm learning and an MLOps platform for regulated data

For data that cannot be freely exchanged, such as patient records in healthcare or customer data in retail banking across countries, HPE has developed swarm learning, a variation of federated learning in which only trained model parameters, not the underlying data, are exchanged between sites. Orlotti says HPE published results from this work in Nature and cites healthcare and retail banking deployments, and notes a parallel European Commission effort to build normalized data spaces, including for healthcare, across member states. He also describes an HPE MLOps platform that manages the full life cycle of AI models, including versioning and improvement in a multi-tenant environment, used by a defense-sector customer running hundreds of users and models on the same platform.

AI meets HPC, and Q&A highlights

Orlotti notes a convergence between high-performance computing infrastructure and AI workloads, and describes SmartSim, a tool from HPE's Cray acquisition that embeds AI techniques into traditional simulation domains such as climate modeling, molecular dynamics and computational fluid dynamics to reach results faster. Asked about the most consequential future AI trend, he names large language models and the metaverse as the two developments he expects to most affect daily life and society, adding he is unsure how digitally-native generations will cope with the resulting changes. Asked about swarm learning's use in vehicles, he explains that next-generation cars will generate such large volumes of sensor and passenger data that moving it all elsewhere is impractical, making swarm learning a way to train models at the edge without transmitting the raw data.

Quotes

The amount of data that we have to deal with is simply getting out of control.

Edmondo Orlotti

AI is effective once you are able to apply and to scale it at large.

Edmondo Orlotti

Questions answered in this talk

How much data is expected to be generated per month by 2027?

According to Edmondo Orlotti at Rise of AI 2022, citing an Ericsson Mobility Report, data generated on wireless networks alone, mobile and fixed, will reach about 370 exabytes per month by 2027.

What is swarm learning and why does HPE use it?

According to Edmondo Orlotti at Rise of AI 2022, swarm learning is HPE's variation of federated learning that trains shared AI models by exchanging only the trained network parameters, not the underlying data, which lets healthcare providers and banks collaborate on models without moving sensitive or cross-border data.

What does the Determined AI software that HPE acquired do?

According to Edmondo Orlotti at Rise of AI 2022, Determined AI, acquired by HPE in 2021 and now open source as the HPE Machine Learning Development Environment, helps scale model training across large numbers of GPUs and lets multiple users and models share the same training infrastructure.

Mentioned

swarm learning, MLOps, edge-to-cloud, SmartSim, HPC-AI convergence, EU data spaces, large language models, metaverse

Keep exploring

More from Rise of AI Conference 2022

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,864 words

Show the full transcript

and yeah we are we are now really international um we have another speaker now uh from milan right uh italy uh edmondo or lottie i hope i pronounced that right he's also a former nvidia um colleague let me say that um and he's an ai business development lead now at one of our other partners hewlett packard packard enterprise and i would like to learn more about how do we turn this data into action and insight and a warm welcome to uh you know olatti welcome this is your stage thank you i don't have any i can use hands i mean that's italy usually use hands so should be easy so you can also use your hand exactly so well good afternoon it was asked before but can i ask to the people working in startups to raise their hands so quite a few still okay so i can claim uh that hp is the oldest startup in the market right so just 83 years of life and let's say we are a startup in the

mindset right so i was thinking okay how do i present hp in a way that doesn't look old right definitely i'm not a millennial i have kids who are millennials and and i already struggled to to understand them and and the i world i've been in this world since uh well 10 years basically since the deep learning popped up in the market and the the technology moves fast um and in hp we created the team that i belong to because we believe that turning artificial intelligence into something that can help customers enterprises in production is not such an easy thing and i'm going to show you a few slides or a few numbers um on on this specifically so there is one point that i always like to make no because ai and deep learning they came out of a need right they need to process a huge amount of data right and sometimes i have the feeling that we forget that in the market right we we got used to manipulate and manage massive

amounts of data um if we look at the trends what i uh usually say to my customers is that the data is simply the amount of data that we have to deal with is simply getting out of control so there is no way to handle the amount of data that is being generated this is increasing so fast all the forecasts they always increase the amount of data that actually will be generated and if i look uh if i look at the number on the extreme right here this comes out of the ericsson mobility report for 2027 it says that only the data generated on wireless networks mobile and fixed will amount to 370 exabyte per month so that's when let's say we like to say we have built the proper strategy because the usual thinking is okay we move the data in the cloud and in some ways we will manage that you can't the amount of data that you're going to deal with will require to process data wherever needed and wherever possible because

you will not be able to transport the data so data management data movement has to be rethink rethought so then if you have these type of problems then let's say how does hp cope with with these challenges we have a motto i say we we do believe as i said with the startup mentality we do believe that with technology we can change the way that people live and work but what does it mean how do we do that so what is the role of hp in a world like the world of ai that is moving fast and where us packer enterprise is usually perceived as the traditional system integrator for 480 so i show it with some examples so yap mentioned red bull first i can say that we started our partnership with the mercedes team a long time ago and this led to quite many successes but what we did there was to take their business challenges work with them and solve them and going uh i mean going into the details is is

impossible but we we covered a lot of the aspects uh where the introduction of ai uh really changed uh the landscape uh we did that for mercedes but we work with almost uh all the formula one teams in the world and and our focus is to make sure that what they have in mind can be turned into something into something real talking about something real another example uh which is an interesting example also in terms of data explosion and handling massive amounts of data we provide the end-to-end platform that volvo cars through zanzact is using to develop the autonomous driving features what does it mean it means that we are able to collect store and make available all the data that zendzak needs in order to develop and validate the software it seems trivial trivial but think about hundreds of petabytes of data that you need to validate and reprocess in a certain given time because you have roadmaps and so on in order to build this type of infrastructures what we do is to

embed with with the customer and define these pipelines together leveraging uh all the technologies available in the portfolio uh and and this is uh really this requires uh thinking out of the box uh this is where um we provide the advantage of designing the solution based on the customer requirements and not adapting what's existing to forcing forcing it to adapt on on something that is not fit for the job any reference to the public cloud is intentional so this is this is how we how we work then we have a strategy that spans from the edge to the cloud uh what it means it means that we can provide solutions on the edge the extreme edge even in the field or in the car or wherever uh or the almost edge like base stations uh cellular networks and and so on and we go up into the data center even in the public cloud so this edge to cloud approach is driven by the customer needs and the need of optimizing the data movements again

this is something i will keep on repeating artificial intelligence in production most of the challenges that we have experienced once you have your models trained and you have your setup most of the challenges are the implementation in the production pipelines with all the data you have right managing the data handling the data different sources different requirements is is a complex is a complex matter it may be boring i don't think so but it may sound boring not so exciting as developing an ai model but it is what makes the difference between a nice model that use only for nice demos or a model that you can effectively put in production so in order to do this that we have built a lot of tools including uh basically financial tools what does it mean it means that all these infrastructure that we can that we provide we can provide it on nss service a pay-per-use model so basically we have brought the model of the cloud into any type of solution whether this is on the

edge or in the in the data center we have obviously a strong heritage in the hardware portfolio we have basically almost any technology that we need available in in our portfolio so we have defined infrastructures from another perspective that are optimized for the different for the different needs and we do have the three elements uh that are usually determining the efficiency of of an ai solution the compute with the gpus but also extremely critical and very often underestimated the storage technologies how to get fast access to the data this is something that is usually really underestimated especially when the problem scales the last one extreme important is obviously also the interconnect so being able to move the data as fast as needed so then when it comes to developers of of ai then we have been working on the two main elements obviously on one side the development of the model and the training and on the other side on the deployment and the management of the entire life cycle for these ones we have

developed specific specific solutions but let me give you a little bit of example of an example of how we how we work on this so as i said we start from from what is the the customer business need and then what we do is uh to look first where the data is generated how injection works uh what what are the requirements there how do i aggregate date how do we normalize data we go through all these all these processes that in our experience they affect more than 50 of the problems that then enterprises have when they have to put ai in production so um then in the data center how are the models deployed how they are trained and and for this in particular we look into okay how is what is the scalability of of the models how can we run optimizations uh yap was referring to the high cost of data scientists uh before imagine that you are training a large language model that can i mean the training can last weeks or

if not month on hundreds of nodes right how do you make that efficient so that if anything happens you don't lose completely your job you you can run effectively on on on your infrastructure and now can you scale easily your model across let's say virtually any amount of resources you can have you can have available and then how you manage the deployment in production these are all aspects um that let's say for which we work closely with the customers then when it comes to trainings in particular we made an acquisition last year we acquired the startup called the term in the eye that a startup that has developed a software that helps facilitates the scaling of trainings and the management of training models with with several features in particular for scaling on any number of gpus so because we realized that um the models are getting larger so there is an increased demand for um large scale solutions in order to train to train models gpt3 is one usual example the other thing that the

this software that we acquired from the termini does is uh um it allows multiple users to leverage the same resources because you start maybe with one model then you start having more users more models all of them are using slightly different frameworks or different implementations how do you work in a true multi-talented environment and and this is what we fixed also with the software so we have then created a full stack that starts from the hardware up to the software infrastructure stack in order to make sure that all these technologies can be leveraged same goes for for the inferencing and the in this case particularly differencing the um there is a there is a a big motion of isvs and and companies that deliver all type all types of solutions and also looking into other techniques to adapt quick uh to adapt quickly and the way that models are are developed as well as some other some solutions for the um for the system management and for the fleet management that in particular enable the

deployment of trained models um across the edge and this is something that particularly in the telco industry is becoming a critical factor i was mentioning before the 370 exabyte of data that are generated per month the telco companies they cannot i mean get all this data to travel to central data centers there is a need to extract the value out of the data as soon as possible this could be down i mean if it's not possible on your phone already this will be done at least in the in the closest station from from the telco from the techo companies but then you need to have systems capable of running the inferencing at scale on tens and even hundreds of thousands of of edges so looking at this ai at scale problem the way that we that way that we perceive in in the market the trends that we see then um our target has been to develop as i said an optimized infrastructure that goes uh through the hardware the software stack and also the

service model so to make this available as a service in the in the seamless cloud experience we have then built a set of software tools that are part of our hp software stack that are let's say our contribution to the ai community uh most of these are open source and freely uh freely available for download in particular i was mentioning the the term in the ai software for training training at scale upper parameter optimization and and snapshots features you have in this in this tool that makes it i believe quite interesting for for anyone who's doing training at scale at some scale at least outside of of one box this software is available as open source it's been renamed hp machine learning development environment but this is let's say one one package there is uh um another technology app was mentioned in federated learning we are working heavily on a variation that is swarm learning um for uh for ai because we have seen big challenges when it comes to data that have let's say

strict requirements in terms of access like due to privacy for instance typically namely in healthcare in the in the fsi or even retail banking imagine i mean hospitals that cannot exchange data across each other's each other or even across countries i know that for germany but let's say there is now especially after the the pandemic there is this effort from the european commission to build data spaces to normalize data in certain fields and there's going to be a data space also for the healthcare this will allow to have normalization of healthcare data across the european countries and for some countries it's important because even in the country there is no normalization so but the problem still will be that it will be very difficult for countries to exchange the data itself right um it's very sensitive with technologies like swarm learning you can run your models without exchanging the data you only exchange the networks the trained networks and and you build a network that is the result of all the data coming out so

we got success on on this in the healthcare and also in the retail banking for pretty much the same reasons in the sense that if they want to build models on their customers crossing countries for for bank institutions is is a challenge but we have also another tool uh that is our envelopes platform it's called hp asmr platform emelops platform that is then managing the entire life cycle uh of a of um of an ai model uh so it takes care of uh um versioning uh takes care of deploying and and improving the models in a multi-tenant environment last one that i mentioned um one of the key aspects that we again it's always important to to consider is that ai deep learning in particular requires a lot of compute and this compute usually is provided by infrastructure that is hpc type so what's happening in the industry is that a lot of the hpc capacity that is available across the world is used also for ai workloads there is a sort of convergence between

hpc and artificial intelligence they try to leverage as much as possible the same infrastructure this is not an easy task because the two worlds they use completely different tools but there is also another trend on top of that or that which is the utilization of ai for traditional simulation that has now become i would say quite quite significant again we have we have another tool that comes out of our acquisition of crane which is called smartsim which is an environment built to leverage ai techniques for traditional simulation i will not dig that much more into into it i have a couple of examples so to mention a very interesting case a customer and the partner that we have in germany Aleph Alpha Aleph Alpha is i would say in this case a partner on the determined i software the development environment because they're using it and they're helping us to debug it and to improve it which is beneficial for for the entire community so that's an interesting example of a company that does require

training at scale for a large language model and and here the the training can actually run at any scale the limit is only the amount of gpus you can put in the data center to provide you other examples in this is a case in the defense sector talking about the mlops platform in this case we are talking about a user that has hundreds of users and tens if not hundreds of different models all of them in the need of using the the same platform so the envelopes platform allows them to keep the entire life cycle of all the models of all the users under control the as well learning that i was mentioning before we did get the paper published on on nature for the activities we have been running in on learning and in particular in the healthcare in the healthcare sector and as i said this is this is technology that is of interest in certain um areas of application particularly when the data cannot be exchanged around last last example just to

mention this ai in traditional simulation is particularly relevant because it allows you to get to results faster so we're going to see more and more application of ai techniques embedded into traditional simulation in in in different in different domains this one is in is in climate but you can imagine molecular dynamics as well cfd so all the all the environment of computer data engineering this will uh will help us on this so closing um on on my presentation the topics that we touch um are topics that are related to make sure that the customers really think out of the box on problems that are difficult to manage the traditional solutions they work up to a certain level our focus then is to help and enable the real changes in ai because ai is effective once you are able to apply and to scale it at large and and this is this is what we work on very often we do have to challenge our customers to to drop the legacy and think in a completely

different way when i go to a customer and they tell him hey you need to reduce your data before you can do anything with it then they start getting lost right because the traditional thinking even a lot of data scientists they just think okay i have all my data there and then i will process it i will do it this is not possible i mean it is possible in some in many cases but it's going to be less and less the standard and in order to do that you need to work with the technologies that are let's say part of our portfolio but in general technologies that are innovative technologies and that's why also we look into startups that can provide enabling technologies for the modern usage of ai at scale and that that closes my my presentation thank you ed monado really really well done um i have a bit a general question to you what do you think what will be the most future moving ai scenario well there are two trends that

are kind of scary especially if you have kids definitely the large language models and all the investments we are seeing in large language models they are going to affect our daily life for sure if they are not already doing that the impact of the evolution of these large language models is going to be huge the second element the second trend which is in a way related also to this large language model that scares me a bit also because i've been working in the graphics industry for a while is the metaverse because again with the with the large investments that are in there um then then it's coming in a way or another is coming and this is also going to change uh it's definitely going to change our um our society then i could mention other things like health care and so on and so on but yeah i think these two these two are going to have an impact on our daily life in ways that we are still not used to and and

for us let's say grown-up people will be able to handle that but we will have to think how the uh naturally digitally born people will cope with a completely different world yeah is there anyone else frightened by the or in the meter verse rise of your hand no one cool do you have any questions otherwise i also like your approach with the swamp thing i also know that um from new work do you see what are the what is the challenge in this field well the so it all relates to the fact that so i was mentioning that the privacy issues right so moving the data but we have other cases where basically the amount of data that is generated is uh so much that it's very difficult to move it if we think about the next generation of cars they're going to have quite significant computing capacity apart from the autonomous driving features there's going to be other computers in the cars that will do a lot of influencing on the amount of data

massive amount of data that will be generated by all the sensors and even by the passengers inside the car so swarm learning is a way to solve the challenge of sending this data somewhere else and then getting back the results because this would be unpractical so there are many uh many problems that was learning can solve especially when we think on training at the edge yes thank you so much for your presentation see you later and one more class you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.