Rise of AI Conference 2024 · Berlin · Applied AI Stage

AI & Data – Match Made in Heaven

Christian Winterfeldt · Dell Technologies

At Rise of AI 2024 in Berlin, Christian Winterfeldt, Senior Director of Data Center Sales Germany at Dell Technologies, argues that AI models should be brought to a company's data rather than the reverse, for latency, cost and sovereignty reasons. He says roughly one-third of worldwide AI compute already runs on Dell infrastructure, and contrasts triple-digit-billion infrastructure investments in North America and Asia-Pacific with more cautious, but leading-edge, German experimentation. He cites data leaks at Amazon and Samsung, hallucinated legal citations, and a biased AI photo-editing example as cautionary cases, and reports survey findings that most companies interviewed prefer running AI on-premises. He walks through Dell's approach: embedding AI in products, running AI on Dell's own portfolio, using AI internally, and partnering across an ecosystem including Nvidia. He describes moving from generic downloaded models toward retrieval-augmented, fine-tuned models built on a governed 'AI-ready data platform' and data lakehouse, and outlines Dell's staged consulting approach: focus area, use case selection, and right-sized infrastructure.

Key takeaways

  • Winterfeldt says roughly one-third of worldwide AI data and compute already runs on Dell infrastructure.
  • A Dell 'Innovation Catalyst' study found 43% of interviewed companies plan to run AI on-premises and 78% want that capability.
  • He cites compute-consumption factors of 1 for a typical ERP app, 100 for traditional AI, and 1,000+ for generative AI.
  • The average German company already has about 200 identified AI use cases it has not yet acted on, he says.
  • A 1.7-trillion-parameter GPT model required 7,500 eight-GPU servers (60,000 GPUs) to train, he notes as a scale example.

Detailed notes

Bring the model to the data, not the data to the model

Winterfeldt opens with a proverb about the mountain and the wisdom needing to meet, arguing that companies should bring their AI model to their data rather than moving data to the model. He says Dell Technologies already carries roughly one-third of worldwide AI data and compute on its platforms and infrastructure. He contrasts compute demand across application types: a typical ERP or SAP application at compute-consumption factor one, traditional AI at factor 100, and generative AI at factor 1,000 or more. He notes triple-digit-billion-dollar infrastructure investments for large generative AI projects over the past two years, concentrated in North America and Asia-Pacific rather than Germany or Europe, but argues that in terms of customer experimentation, German and European adoption is leading edge.

What can go wrong: leaks, bias and hallucination

He lists cautionary examples from the past 18 months: Amazon and Samsung employees leaking sensitive company data by pasting it into public ChatGPT-style tools; a lawyer citing hallucinated legal precedents in court; and a student whose AI-edited headshot was lightened in skin tone and given blue eyes, illustrating bias in training data. He cites a Dell 'Innovation Catalyst' study in which 43% of companies interviewed said they intend to run AI on-premises rather than off-premises, and 78% said they want the capability to run AI models where their data resides. He gives technical reasons for this preference, including the roughly 130 millisecond latency for light to circle the globe, the cost of moving large data volumes, and the observation that most enterprise data is created at the edge, such as production lines, storage buildings and farms, and cites an estimate that three-quarters of created data will originate at the edge.

Where AI use cases stand today

Winterfeldt compares the current state of large language model adoption to the early days of email, thirty years before today's trillion-dollar internet industry of e-commerce, voice over IP and streaming, arguing enterprises are at a similarly early stage with AI. He describes Dell's approach as embedding AI into its own products, running roughly a third of worldwide AI infrastructure on Dell technology, applying AI internally across HR, legal, supply chain, sales and marketing during the company's fortieth year, and working through a large partner ecosystem including Nvidia.

From generic models to a governed, data-centric platform

He argues that downloading a generic retrained model from Hugging Face and running it behind a firewall delivers little value because generic and even larger models do not know a company's own business and do not necessarily improve business outcomes. He recommends model augmentation through retrieval-augmented generation and fine-tuning to bring a company's own data and culture into a model, describing this as the current investment sweet spot, with full model training remaining rare, especially in Europe. He describes Dell's 'AI-ready data platform', a data lakehouse that connects to existing tools such as Teradata and Power BI, plus another analytics/logging tool unclear in the transcript, and existing cloud data and ML workflows without duplicating data into a new silo, which he says can save data scientists up to 80% of the time otherwise spent locating the right data, illustrated with a bottling-line quality use case combining HR, sensor and outcome data.

Staged approach: focus area, use cases, right-sized infrastructure

He describes Dell's consulting sequence: first identifying a focus area such as HR, production or R&D; second, selecting the right use case, noting the average German company already has around 200 identified but unused AI use cases; and third, right-sizing the platform, since not every workload needs a four- or eight-way GPU cluster and some large models can run on a workstation or laptop. As an example of scale at the other extreme, he cites a 1.7-trillion-parameter GPT model requiring 7,500 eight-GPU servers, or 60,000 GPUs, to train. He describes Dell's 'AI Factory' work with Nvidia as managing the full model lifecycle through a pick-and-choose portal of microservices, and closes by noting Dell's European manufacturing, data sovereignty positioning, and a 1,200-person German-speaking support team based in Halle.

Quotes

It's more important to bring your wisdom to the mountain, so your AI model to the data, than vice versa.

Christian Winterfeldt

The bigger the model, it doesn't really necessarily help for your business outcome within your company.

Christian Winterfeldt

Questions answered in this talk

Why should companies run AI models on-premises instead of in the public cloud?

According to Christian Winterfeldt at Rise of AI 2024, moving models to where data already sits avoids the latency of moving data across the globe, the cost of transferring large volumes, and data-leak risks like those seen at Amazon and Samsung; in his cited survey, 78% of companies wanted the capability to run AI where their data resides.

How much AI infrastructure does Dell Technologies power worldwide?

According to Christian Winterfeldt at Rise of AI 2024, roughly one-third of worldwide AI data and compute already runs on Dell Technologies platforms and infrastructure.

What is the difference between using a generic downloaded AI model and a data-centric approach?

According to Christian Winterfeldt at Rise of AI 2024, a generic model downloaded and run behind a firewall does not know a company's business and adds little value, whereas augmenting a model with a company's own governed data through retrieval-augmented generation and fine-tuning is where most current investment and value creation happens.

Mentioned

AI-ready data platform, Data lakehouse, Retrieval-augmented generation, Model fine-tuning, AI Factory, On-premises AI, Data sovereignty

Keep exploring

More from Christian Winterfeldt

More from Rise of AI Conference 2024

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,592 words

Show the full transcript

yeah well what a what a great event so um it was a great evening yesterday evening and I had lots of great discussions today and there there's a statement of our founder Michael Dell and he is saying if if you're the smartest person in the room leave the room and I can tell you I can stay here for quite a long time so that's it's a great audience so um I'd like to start with a proverb my maybe everybody might know this um if the um mored is not coming to the mountain the mountain needs to come to mored and if you're looking at this you so if you are need to move truckloads of mountains then to the next country to the next wisdom what happens then to this mountain if if there is some wisdom in some other country would you move your Mountain again so and you just realize it's not a religious discussion I don't want to start here please not it's more about your most important assets the mountain the

data and the wisdom you're are going to use your AI models and how um it's more important to bring your wisdom to the mountain so your AI model to the data than vice versa so Del Technologies um and I got lots of questions over the last uh hours what is Del technology actually doing are you really in that space and and probably onethird of the of the data and one one third of the compute you're already using is running in our our platforms and our infrastructure so we are in the middle of this and this is one of our strategic pillars we are using here so AI itself it's nothing new so since decades um uh data scientists are are working to make better outcomes making better decision um making detecting better patterns but the the real wave hit us two years ago and we saw lots of Investments over the last time and infrastructure matters in that space so the first time you can use human language as interaction with AI and that really

changed the market and why I'm saying infrastructure matters if you're looking at a typical application just like an Erp SCM sap whatsoever let's say this has the compute consumption of factor one if you're then looking to traditional AI this might have the compute compute uh consumption of factor 100 if you're then looking at generative AI this is then times thousand and more so you really see a high need of infrastructure and this needs to be handled and what I saw over the course course of the last two years uh never before we had that amount of investments in the world in technology we saw Triple digit investments in infrastructure to power big generative AI projects triple digit Investments a couple of those but you don't see them in Europe you just see them in North America you see them in apj so the world is creating new foundational models and these Investments are not here not here in in Germany and in Europe however if I'm looking at the experiments and I still call it

experiments what customers did over the course of the last two years they experimented with AI models they made Investments and the number of Investments made in Germany made in Europe This is Leading Edge so it always feels that we are far behind and doing something but I really see that German engineering is is advancing this technology and really using it and the adaption in Germany definitely is is there so um things can go wrong and this is also what we learned over the course of the last 18 month you saw lots of um examples where intellectual property um where were lost due to uh AI models which are public available so Amazon lost critical data just because employees put these data into chat gbt and others were able to look at the materials of um of uh of uh Amazon same thing at Samsung same thing with privacy issue every software can be buggy and so an jet gbt model can be buggy and may you may lose critical uh sensitive data and also um

you see that the data which is fueling your a model might be um uh biased so if you um have made bad decisions in a operation for a couple of time and using this data to feed your AI the uh Finance request from a customer might be directed just because his name and this is happening in mod AI already and also you shouldn't believe anything you read into jbt there is a famous example of that lawyer who just researches president cases and and bring them brought them to court and this was an excellent example of um yeah uh of um things which are not true and um if if you're using a Chad gbt and the system is just hallucinating so and another example um and I I am at student just asked to make her head shot more professional and ended up to be lighten up her skin and adding blue eyes so we have lots of things to learn and we are still learning and lots of companies learned about this and um

43% of the companies I interviewed and that Innovation Catalyst study said we are going to run our AI rather on premise than off Prem uh we are going to not allowing our employees to put data into public availabity chat models we want to run them on premise and exactly uh 80 78% of these customers said okay we are want to have these capabilities running AI models on premise where our data are and there are good reasons for for doing that bringing your AI to your data so whenever and the these are at at one point technical reasons so you see there are physical limits you can't uh exceed the speed of light so whenever data are moved from A to B it takes 130 milliseconds um for for light to go around the world so if you have lots of data on the one side if you have an application who is depending on these data and using lots of those there's latency so that's that's one technical thing the other thing whenever you're moving

data it costs you money it costs you lots of money so there are good reasons for having them also in your own data center or bringing eye where your data are and the other um experience or the other Insight we get that most of the customers are already ready to have ai in their Edge so that's the biggest use case we see um and G is saying that 75% of the data will be created in the edge what's the edge that's the place where you have your most valuable value creation within the company it's the production line it's a storage building it's the farm R of producing your crab so lots of data are produced in the edge and you will have inferencing and you will have a models somehow running in the edge of the Enterprise so let's go into use cases and there are and and to be honest there are just a shuffle uh just a couple of Shuffle ready use cases you can really use already in Enterprise there's lots of

experimentation and you see large language model among all sizes and we are just at the beginning of that Journey so one analogy if you looking at the uh uh the usage of internet today so you you have lots of uh e-commerce you have Voiceover IP you have streaming you have all these kind of stuff that's the trilli milon uh industry at this point in time on the other hand where we where have we been 30 years ago email so that that's what how it was starting and with large language models we are also at the email moment at this point and we are accelerating at PACE and we run this not only in large language model we are using large models um on scale and and we see that these models will run everywhere in your private data center in your hybrid environment at the edge in the cloud and also on your PC and these are also some use cases we see that you're using neuro processing engines within your enduser device and running

your models there so how is Dell Technologies is coming into this game so how we are applying AI first of all we are embedding this in our products we making products better we automate them more we making better predictions and also helping other oems making their products with um with AI better because we are excellent and and Engineering our own products second thing and that's the the most important part for us it's running AI on our portfolio as I said more than 30% are already of the worldwide AA Technologies already running on Del technology infrastructure and infrastructure is not a piece of cake it's it's just lots of products you need to interact with each other that's that's a big use case you really have to look at it other thing is using AI for us so Dell Technologies after we're ring 40 years this year we have the biggest Innovation um uh Innovation uh process happening right now in our company we're using AI a lots of process in lots of areas in HR

in legal in and in our supply chain helping sales helping marketing so we have lots of experience how to use it ourself we share this experience with our partners with our customers so that's a big Journey we are on ourself on it right now and the other thing is we don't have the mon Monopoly on Innovation this is just working with a huge ecosystem of Partners and we have one of the biggest AI ecosystems in the industry so let's go to the next level that's that's the wisdom we we need to use and there's a very simple equation I'd like to make things simple so what wisdom are you choosing for your cooperation you have to choose it based on what value are you willing to take out of this and what are you willing to invest and most customers uh made this already to just took an um retrained model um downloading it from hugging phase installing executing it behind your your um your firewall and you go for it so that's a very

simple approach but there is not a lot of value in this so these generic uh uh these generic models they don't know your company and even bigger mod models don't help you to make decisions better within your company they make may have might have better answers they have may be flowy languages but the bigger the model it doesn't really necessarily help for your business outcome within your company so you need to bring your data to your model and this is what happening with Technologies like R and others you heard already today and this is about model argumentation then you are opening up models fine-tune them adding other language bring the culture of your company into this model so this is the sweet spot where we see most companies right now are investing and we are heavily using model argumentation within our own uh own company and then model training uh there are just a few companies in the world who really making model uh training um and there are very few in in Europe sitting

so that that's the overview of uh of the wisdom you can use but more important um is basically coming from a code Centric model to a data Centric model because data are really the fuel who are who accelerating um this rocket so we see um lots of customers going to smaller models we're also doing this like mirandas falan choosing choosing that model but more important than the model is the data and there we are really looking into what kind of Highly quality labeled data we can use for this process and in the end how you are coming to these better labeled data and we are creating a framework a a platform for our customers to really have an and we call it D AI ready data platform and what we are doing here is connecting to open ecosystems be it we're using Analytics tool like Terra dat using using your select stream uh logging into a powerbi so using all the data sources you have today in your company adding um ml workflows you have

already in new company and using all the data you already have created in your Cloud ecosystem so it's not about copying data and creating another Silo it's about connecting to all these sources and make the work for your data scientists better and by using these platform data signs save up to 80% of their time really looking into hey what are what are the data I need for this use case so let's say you you're you're filling bottles and the the the the the pepper on on the bottle is not not always on the right place so what do you need to do is you need to have a small application you want to use AI to really identify okay what can I do better in this Edge use case so you need data Maybe from HR which people have been on the conveyor belt um what's the what are the sensors are saying who are actually on this machines uh what is the outcome what was the quality of the glass we got so far

so you have to connect lots of information and this platform is helping the data science to get the right right information from the right Department um having kind of a um governance model behind that and these most of these information are sitting still within your own company so we're building a data Lakehouse structuring these datas and this is the base for having a good workking Rec process and this is also powered by AI infrastructure so you really need to have a a good server technology be be underneath that you need to have a good storage technology under needs this to to make this working and um because infrastructure as I said is not a piece of cake we're helping our customers with validated design so this that's kind of a cookbook where you're you're starting your AI Journey how to create these capabilities within your own Data Center and these actually one of the capabilities customers need to search or need to need to create the capability of have kind of a cloud who's able

to provide large language models having this kind of data L having a Gateway uh uh um um to be to manage the access and really identify who's using what who's able to choose what choose what so creating these these um these these ability to do that so far we see that lots of customers are still experimenting they're still experimenting what's possible what we are doing and we are now getting into the next stage where we learn from this whole experimentation and customers who did didn't start yet they should start because the most important thing is to fail to really to understand what's going wrong in these projects how can I make this a rhythm of okay I have a business problem how can I fix this what are the data included what model should I use and and providing a platform to enable your data scientists your developers to go there so what's what's the approach where Del Technologies can help to go into this investment so this is a huge investment for most companies

and you need to WR sizes so first of all let's look at the focus area and what we are doing we are offering kind of um accelerator workshops we are defining the strategy we are helping on um on uh on matching the vision to the actual status quo looking in the data but the first step is really looking into the focus area so starting in HR starting in production starting an R&D and so on second level is really to look what are the use cases available or already and in average every company in Germany has around 200 use cases already available but they don't know how to use it is this use case really paying off is this use case really helping us in the future so therefore defining The Right Use case and then choosing the right wisdom choosing the patterns we're helping on this and then at the third Point deciding what's the right platform underneath so definitely you need the right platform not everything needs a um four uh or eight-way GPU

cluster you can also run some large language models on a workstation on a laptop and there you come into the situation of right sizing so for instance how massive this could be the GPT model with 1.7 trillion parameters it took 7,500 servers which 8 gpus which is 60,000 gpus to calculate this GPT model with 1.7 trillion parameters so that's a huge amount of instructure you need but you don't always do more entation model uh uh fine-tuning and and the creation of foundation model so in real life you just have a good platform a good Cloud environment which might be even smaller and this is how we are helping having validated design to really have the first um the the first into this environment what are we doing with um Nvidia so we are working very close with Nvidia and and building the fast path for creating your own AI Factory so having the capability to offer AI models offering data platforms to your uh to your business operations because most of the time these discussions

are not starting within the it they're starting within the business departments but these capabilities needs to be provided by uh companies it department so the I factory is basically everything about model life cycling so something my pre pre speaker just mentioned mention which model to use so you have a catalog of models you're working on the life cycle of models which model is helping you for which use case are you training the models tweaking them you are influencing these model using Rec so this is a pick and choose portal you're using you have Services um we are working with your data Sciences leveraging our experience of many projects helping you to get that way using the Nvidia Enterprise software underneath this to have a the these portal approach using microservices putting several llms into these microservices and around them on this environment and at the end you have the the right infrastructure underneath that and this infrastructure you are also consuming as a servers and you do it everywhere you need so if you're uh

creating lots of data in the cloud you have this in the cloud if you're doing this in your core data center you do it there and also in your Edge environment so we are helping you on every step of this Journey so data Del Technologies we have the platforms the expertise and lots of Partnerships to start this journey and within um consulant Workshop of half a day we are identifying where you are where's the strategy you're going forward with how can we help you on what step if you are far Advanced if you have already models evaluated if you have data signs and just you need to get the the data governance right okay let's start there if you just at the beginning and and create kind of um capabilities let's start with a strategy we are helping you with our data science we helping you with the implementation we're helping with the day two operation and ET and everything you see here this is technology also produced in Europe so we have another Point

here it's not only data sovereignty um in Europe you have all these infrastructure you saw we're producing these in three faps in uh in Europe we have also a big um support team sitting in holler so we are the only company who's providing German speaking uh support with 1,200 um employees out of H so this is a big added value for most of our our companies we're working with so I have lots of faith in in what we are doing in Germany the sheer number of experimental pcas things we are doing already in the German Market that's promising I see lots of big siiz customers I see lots of um governmental customers who are starting first PC uh these uh hidden Champions our mid-market customers so we saw lots of experimentation in the market and I'm pretty sure that this will scale and we as Germany are really gaining competitive Advantage out of this so happy to be part of this journey and thanks you very much

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.