At Rise of AI 2024 in Berlin, Christian Winterfeldt, Senior Director of Data Center Sales Germany at Dell Technologies, argues that AI models should be brought to a company's data rather than the reverse, for latency, cost and sovereignty reasons. He says roughly one-third of worldwide AI compute already runs on Dell infrastructure, and contrasts triple-digit-billion infrastructure investments in North America and Asia-Pacific with more cautious, but leading-edge, German experimentation. He cites data leaks at Amazon and Samsung, hallucinated legal citations, and a biased AI photo-editing example as cautionary cases, and reports survey findings that most companies interviewed prefer running AI on-premises. He walks through Dell's approach: embedding AI in products, running AI on Dell's own portfolio, using AI internally, and partnering across an ecosystem including Nvidia. He describes moving from generic downloaded models toward retrieval-augmented, fine-tuned models built on a governed 'AI-ready data platform' and data lakehouse, and outlines Dell's staged consulting approach: focus area, use case selection, and right-sized infrastructure.
Key takeaways
- Winterfeldt says roughly one-third of worldwide AI data and compute already runs on Dell infrastructure.
- A Dell 'Innovation Catalyst' study found 43% of interviewed companies plan to run AI on-premises and 78% want that capability.
- He cites compute-consumption factors of 1 for a typical ERP app, 100 for traditional AI, and 1,000+ for generative AI.
- The average German company already has about 200 identified AI use cases it has not yet acted on, he says.
- A 1.7-trillion-parameter GPT model required 7,500 eight-GPU servers (60,000 GPUs) to train, he notes as a scale example.
Detailed notes
Bring the model to the data, not the data to the model
Winterfeldt opens with a proverb about the mountain and the wisdom needing to meet, arguing that companies should bring their AI model to their data rather than moving data to the model. He says Dell Technologies already carries roughly one-third of worldwide AI data and compute on its platforms and infrastructure. He contrasts compute demand across application types: a typical ERP or SAP application at compute-consumption factor one, traditional AI at factor 100, and generative AI at factor 1,000 or more. He notes triple-digit-billion-dollar infrastructure investments for large generative AI projects over the past two years, concentrated in North America and Asia-Pacific rather than Germany or Europe, but argues that in terms of customer experimentation, German and European adoption is leading edge.
What can go wrong: leaks, bias and hallucination
He lists cautionary examples from the past 18 months: Amazon and Samsung employees leaking sensitive company data by pasting it into public ChatGPT-style tools; a lawyer citing hallucinated legal precedents in court; and a student whose AI-edited headshot was lightened in skin tone and given blue eyes, illustrating bias in training data. He cites a Dell 'Innovation Catalyst' study in which 43% of companies interviewed said they intend to run AI on-premises rather than off-premises, and 78% said they want the capability to run AI models where their data resides. He gives technical reasons for this preference, including the roughly 130 millisecond latency for light to circle the globe, the cost of moving large data volumes, and the observation that most enterprise data is created at the edge, such as production lines, storage buildings and farms, and cites an estimate that three-quarters of created data will originate at the edge.
Where AI use cases stand today
Winterfeldt compares the current state of large language model adoption to the early days of email, thirty years before today's trillion-dollar internet industry of e-commerce, voice over IP and streaming, arguing enterprises are at a similarly early stage with AI. He describes Dell's approach as embedding AI into its own products, running roughly a third of worldwide AI infrastructure on Dell technology, applying AI internally across HR, legal, supply chain, sales and marketing during the company's fortieth year, and working through a large partner ecosystem including Nvidia.
From generic models to a governed, data-centric platform
He argues that downloading a generic retrained model from Hugging Face and running it behind a firewall delivers little value because generic and even larger models do not know a company's own business and do not necessarily improve business outcomes. He recommends model augmentation through retrieval-augmented generation and fine-tuning to bring a company's own data and culture into a model, describing this as the current investment sweet spot, with full model training remaining rare, especially in Europe. He describes Dell's 'AI-ready data platform', a data lakehouse that connects to existing tools such as Teradata and Power BI, plus another analytics/logging tool unclear in the transcript, and existing cloud data and ML workflows without duplicating data into a new silo, which he says can save data scientists up to 80% of the time otherwise spent locating the right data, illustrated with a bottling-line quality use case combining HR, sensor and outcome data.
Staged approach: focus area, use cases, right-sized infrastructure
He describes Dell's consulting sequence: first identifying a focus area such as HR, production or R&D; second, selecting the right use case, noting the average German company already has around 200 identified but unused AI use cases; and third, right-sizing the platform, since not every workload needs a four- or eight-way GPU cluster and some large models can run on a workstation or laptop. As an example of scale at the other extreme, he cites a 1.7-trillion-parameter GPT model requiring 7,500 eight-GPU servers, or 60,000 GPUs, to train. He describes Dell's 'AI Factory' work with Nvidia as managing the full model lifecycle through a pick-and-choose portal of microservices, and closes by noting Dell's European manufacturing, data sovereignty positioning, and a 1,200-person German-speaking support team based in Halle.
Quotes
It's more important to bring your wisdom to the mountain, so your AI model to the data, than vice versa.
Christian Winterfeldt
The bigger the model, it doesn't really necessarily help for your business outcome within your company.
Christian Winterfeldt
Questions answered in this talk
Why should companies run AI models on-premises instead of in the public cloud?
According to Christian Winterfeldt at Rise of AI 2024, moving models to where data already sits avoids the latency of moving data across the globe, the cost of transferring large volumes, and data-leak risks like those seen at Amazon and Samsung; in his cited survey, 78% of companies wanted the capability to run AI where their data resides.
How much AI infrastructure does Dell Technologies power worldwide?
According to Christian Winterfeldt at Rise of AI 2024, roughly one-third of worldwide AI data and compute already runs on Dell Technologies platforms and infrastructure.
What is the difference between using a generic downloaded AI model and a data-centric approach?
According to Christian Winterfeldt at Rise of AI 2024, a generic model downloaded and run behind a firewall does not know a company's business and adds little value, whereas augmenting a model with a company's own governed data through retrieval-augmented generation and fine-tuning is where most current investment and value creation happens.