At Rise of AI 2022 in Berlin, Edmondo Orlotti, AI business development lead at Hewlett Packard Enterprise and a former Nvidia employee, argues that scaling AI into production is mostly a data and infrastructure problem, not a modeling problem. Citing an Ericsson Mobility Report forecast of 370 exabytes of data generated per month on wireless networks by 2027, he argues data volume is becoming unmanageable and that a cloud-only strategy cannot work, pushing HPE toward an edge-to-cloud approach. He illustrates this with HPE's partnership with the Mercedes Formula One team and an end-to-end platform built for Volvo Cars, through its subsidiary Zenseact, to collect, store and process the hundreds of petabytes of data needed to validate autonomous driving software. He describes tools HPE has built for production AI, including Determined AI for training large models at scale, swarm learning for training on sensitive data without moving it, and an MLOps platform for managing model life cycles, plus SmartSim, from its Cray acquisition, for embedding AI into traditional simulation. He closes by naming large language models and the metaverse as the trends he expects to most affect daily life.
Key takeaways
- Orlotti cites an Ericsson Mobility Report forecast of 370 exabytes of wireless network data generated per month by 2027, arguing cloud-only data strategies cannot keep up.
- HPE built an end-to-end platform for Volvo Cars, via Zenseact, to collect, store and process hundreds of petabytes of autonomous-driving data.
- Orlotti says more than half of enterprise AI production problems come from data ingestion, aggregation and normalization, not modeling.
- HPE's Determined AI acquisition, now open source as the HPE Machine Learning Development Environment, is used by Aleph Alpha to train large language models at scale.
- HPE's swarm learning lets healthcare and banking customers train shared models without exchanging the underlying data, with results published in Nature.
Detailed notes
The data explosion and why cloud-only fails
Orlotti opens by arguing that AI and deep learning emerged from a need to process huge amounts of data, and that data volume is now getting out of control, citing an Ericsson Mobility Report forecast that wireless network data alone will reach 370 exabytes per month by 2027. He argues the common assumption of simply moving all data to the cloud will not hold, since the volume will make full data movement impractical, requiring processing wherever the data is generated and rethought data management and movement strategies, which pushes HPE toward an edge-to-cloud approach spanning the extreme edge, base stations, data centers and public cloud, driven by customer needs and the goal of minimizing data movement.
Case studies: Formula One and autonomous driving data
He describes HPE's long-running partnership with the Mercedes Formula One team, where AI was introduced across many aspects of the team's operations, and says HPE works with almost all Formula One teams. A second example is an end-to-end platform HPE built for Volvo Cars, through its Zenseact subsidiary, to collect, store and make available the hundreds of petabytes of data needed to develop and validate autonomous driving software within fixed roadmap timelines, built in close collaboration with the customer rather than by adapting an existing generic setup.
Making AI production-ready: infrastructure and Determined AI
Orlotti says most real-world AI challenges appear after a model is trained, in production pipelines where handling data from different sources accounts for more than half of the problems enterprises face. HPE addresses this with pay-per-use, as-a-service infrastructure spanning edge and data center, built on three efficiency pillars: compute (GPUs), storage for fast data access, and interconnect for moving data quickly. For training at scale, HPE's 2021 acquisition of the startup Determined AI provides software for scaling training across many GPUs and enabling multiple users and models to share the same infrastructure in a multi-tenant setup; it has since been released as open source under the HPE Machine Learning Development Environment name and is used by Germany's Aleph Alpha to train large language models.
Swarm learning and an MLOps platform for regulated data
For data that cannot be freely exchanged, such as patient records in healthcare or customer data in retail banking across countries, HPE has developed swarm learning, a variation of federated learning in which only trained model parameters, not the underlying data, are exchanged between sites. Orlotti says HPE published results from this work in Nature and cites healthcare and retail banking deployments, and notes a parallel European Commission effort to build normalized data spaces, including for healthcare, across member states. He also describes an HPE MLOps platform that manages the full life cycle of AI models, including versioning and improvement in a multi-tenant environment, used by a defense-sector customer running hundreds of users and models on the same platform.
AI meets HPC, and Q&A highlights
Orlotti notes a convergence between high-performance computing infrastructure and AI workloads, and describes SmartSim, a tool from HPE's Cray acquisition that embeds AI techniques into traditional simulation domains such as climate modeling, molecular dynamics and computational fluid dynamics to reach results faster. Asked about the most consequential future AI trend, he names large language models and the metaverse as the two developments he expects to most affect daily life and society, adding he is unsure how digitally-native generations will cope with the resulting changes. Asked about swarm learning's use in vehicles, he explains that next-generation cars will generate such large volumes of sensor and passenger data that moving it all elsewhere is impractical, making swarm learning a way to train models at the edge without transmitting the raw data.
Quotes
The amount of data that we have to deal with is simply getting out of control.
Edmondo Orlotti
AI is effective once you are able to apply and to scale it at large.
Edmondo Orlotti
Questions answered in this talk
How much data is expected to be generated per month by 2027?
According to Edmondo Orlotti at Rise of AI 2022, citing an Ericsson Mobility Report, data generated on wireless networks alone, mobile and fixed, will reach about 370 exabytes per month by 2027.
What is swarm learning and why does HPE use it?
According to Edmondo Orlotti at Rise of AI 2022, swarm learning is HPE's variation of federated learning that trains shared AI models by exchanging only the trained network parameters, not the underlying data, which lets healthcare providers and banks collaborate on models without moving sensitive or cross-border data.
What does the Determined AI software that HPE acquired do?
According to Edmondo Orlotti at Rise of AI 2022, Determined AI, acquired by HPE in 2021 and now open source as the HPE Machine Learning Development Environment, helps scale model training across large numbers of GPUs and lets multiple users and models share the same training infrastructure.