Rise of AI Conference 2022 · Berlin · Applied AI Stage

Data Centric AI

Claudia Pohlink · Deutsche Bahn

At Rise of AI 2022 in Berlin, Claudia Pohlink, chief expert for data at Deutsche Bahn, argues that AI progress is bottlenecked by data, not models. Inspired by Andrew Ng's push for 'data-centric AI', she contrasts research, where nearly 99 percent of papers focus on improving models and only about 1 percent on improving data, with practice, where up to 60-80 percent of effort in AI use cases goes into data preparation, sourcing and access negotiation. She cites a Gartner estimate that poor data quality costs organizations around 13 million dollars a year on average, and notes demand for data engineers now outpaces demand for data scientists. Pohlink describes Deutsche Bahn's Data Intelligence Center, built three years earlier around a House of Data, a House of AI and a House of 3C (community, change, communication), and the company's 2020 board resolution making data accessible company-wide by default unless explicitly restricted. She walks through the group's roadmap of a central data catalog, data governance roles and a planned move toward data mesh architecture, and closes by warning against skipping data work in favor of 'sexy' model-centric AI projects.

Key takeaways

  • Pohlink cites that about 99 percent of AI research papers improve models while only about 1 percent focus on improving data.
  • She says up to 60-80 percent of real-world AI project effort goes into finding, accessing and preparing data.
  • Deutsche Bahn's 2020 board resolution made data accessible by default company-wide, reversing the usual permission model.
  • Deutsche Bahn organizes its data work into three units: House of Data, House of AI and House of 3C.
  • She warns that skipping data governance for 'sexy' model work risks projects being set back, citing a Gartner prediction for 2024.

Detailed notes

Why research and practice diverge on data

Pohlink frames current AI progress around 'ABC': algorithms, big data and computational power, but says data management itself has not improved much. She cites a striking gap: nearly 99 percent of recently published AI papers focus on improving performance by changing the model, versus roughly 1 percent that address the data. In practice, she says, up to 60-80 percent of effort in AI use cases goes into data preparation, a figure she traces to a 2014 New York Times report, and preparation includes finding data sources across a company and getting permission to use them, often taking six to twelve months in larger organizations.

The cost of poor data and the case for data-centric AI

She cites a Gartner estimate that poor data quality costs organizations an average of about 13 million dollars a year, and notes that demand for data engineers has outpaced demand for data scientists two to one, or by other estimates four or five to one. Referencing Andrew Ng's landing.ai work, she defines data-centric AI as 'AI is code plus data' and the discipline of systematically engineering the data used to build a system, illustrating it with a train-image labeling example where structured rules produce one consistent bounding box instead of many inconsistent ones.

Deutsche Bahn's Data Intelligence Center

Pohlink describes Deutsche Bahn's Data Intelligence Center, established three years earlier at headquarters and organized into three units: the House of Data, which she leads and which makes data visible and available; the House of AI, led by colleague Thomas Thiele, which enables machine learning use cases with business units; and the House of 3C (community, change, communication), responsible for making adoption sustainable. The three houses meet monthly or quarterly to share AI project learnings and data-access problems.

Governance timeline and the open-data resolution

She traces Deutsche Bahn's data journey: the Data Intelligence Center's founding, a 2020 move to a decentralized chief digital officer organization with data officers in each business unit, and new roles such as data steward and data owner. She highlights a 2020 board resolution making data accessible across the whole company by default, so that anyone needing to restrict specific sensitive data must justify the restriction, rather than the reverse. This year's roadmap centers on a central data catalog (with a new underlying technology she says no other data catalog supplier has changed before), data governance rules, and architecture work including data mesh.

Q&A highlights

Asked for her favorite career learning, Pohlink says her advice is continuous learning, describing her own path from studying data science into AI and later into quantum computing work at Deutsche Telekom. Asked about synthetic data, she says Deutsche Telekom began exploring converting real telecommunications data, including location and timestamp fields, into synthetic data around 2015-16 to avoid data-protection issues while preserving useful information, and calls it a likely future direction for exchanging data between departments.

Quotes

The challenge of AI in practice does not lie in the models or model architectures but predominantly in the data.

Claudia Pohlink

Data is for everyone.

Claudia Pohlink

Questions answered in this talk

What is data-centric AI?

According to Claudia Pohlink at Rise of AI 2022, data-centric AI is the discipline of systematically engineering the data used to build an AI system, shifting focus from changing models and algorithms to improving data consistency and quality, a concept she attributes to Andrew Ng.

How much of AI project effort goes into data work rather than modeling?

Claudia Pohlink states at Rise of AI 2022 that in practice up to 60-80 percent of the effort in AI use cases is spent on data preparation, sourcing and access, not on building or tuning models.

How did Deutsche Bahn change its approach to data access?

Pohlink explains at Rise of AI 2022 that Deutsche Bahn's board adopted a 2020 resolution making data accessible across the whole company by default, so that data owners must justify restricting access to sensitive data rather than employees having to justify requesting it.

Mentioned

data-centric AI, data governance, data catalog, data mesh, MLOps, synthetic data, chief digital officer organization

Keep exploring

More from Claudia Pohlink

More from Rise of AI Conference 2022

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,794 words

Show the full transcript

and with that i would like to welcome our first speaker Claudia Pohlink she's already next to me a big applause for her claudia is the chief expert in data at deutsche ban she's already here in red it's the sign again and she's a chairwoman of the bitcoin board in the group of a.i and last week she turned the 40 over 340 one week ago of our german's most inspiring women can you imagine that and now she's here and she is the expert to connect businesses and data science and therefore she will now speak about data science in ai welcome claudia thank you very much so it's really nice being on stage live again and a special thanks goes to veronica and fabian for made this extraordinary conference happen yeah it feels a bit weird because now you talk to real people i'm not used to it two years in the last two years so maybe it excuse some of my vocal mistakes let's see this speech will be about data centric ai and here's a

teaser it will be a low code presentation yeah so you will not see any algorithms or code and um this topic is inspired by andrew inc maybe most of you know him he's the founder of google brain cozera and also landing an ai and he's also one of my most ai influences i read and and and follow yeah and he had a presentation i think around one year ago about ml ops and he highlighted how important is for companies to focus on data instead of the models and algorithms and and i also observed that in my previous and also recent jobs in bigger companies like telecom and deutsche bahn so you see ai pocs and they are very sexy and most of the people are really interested in but when it comes to data like data management governance etl like extract transform load things like that i think the attention is quickly gone yeah so it's very hard also to keep or to get and keep ai projects up and running so what is the

reason for that yeah so that's why i would like to start with the current state of ai implementations and i think um fabian also mentioned that in this first speech here you see the recent evolution towards machine learning yeah and we say now we are in this trendy ai area because of the abc abc stands for algorithms big data and computational power i think most of you know in the last years there are a lot of models and and technologies um up and running and have been developed also in the past years like alphago gpg3 and all that stuff i think you know that yeah but you see this rights of ai exact in the last let's say 10 years yeah and when we have a look at big data not only the volume but also the technologies that you see there is a lot ongoing yeah the volume is increasing a lot yeah i don't need to explain that i know you know that and then we have the combination the let's say exponential

growth of the computational power but what happened in the area of data management we are including preparation and data quality and all that stuff i would say not that much and when we look on research and practice then there is a huge deviation of focus between theory and practice in research nearly 99 of recently published papers focus on improving performance by changing the model and only one percent in research deal with improving performance by addressing the data itself there's a clash yeah so this doesn't reflect the reality yeah when it comes to the practice by the way andrew ing compared that to cooking i really like that picture so in in the most of the ai application or in the use cases where up to eighty percent of the effort is focused on data preparation and i know this this discussion because this number went up or came up i think in 2014 in new york times they told us the data scientists have eighty percent to do with that preparation and only twenty percent

with um let's say screwing models and data itself we can discuss about the number but i would say let's let us agree it is maybe around 60 or 70 and then also preparation doesn't only mean cleaning it also means that you have to go through the whole company finding data sources asking people convincing people that they are that you are allowed to use the data and all that stuff so there are a lot of processes out there and it takes times a lot of time yeah in bigger companies we speak about six months 12 months yeah so this is not really effective and this is the fundament for efficient ai and this is basically what norris the algorithm yeah so for just imagine um that you need so much so much time to find the data and then it's the wrong data then you have this whole story from the beginning on again and again and again and just just to show you how this clash is also here in the room so please raise

up your hands who is working on ai all right and who's working on data governance okay this is good good answer yeah but normally i would say less people yeah okay i told about the gap what are the consequences of this gap and normally it is about resources money and time but it is not always about let's say bad consequences there there is also potential for improvement of course yeah on the left hand side um this is just um a chart about the complexity we have more data we have more parameters we have let's say large models yeah language models and of course you know the complexity is increasing and then on the other hand the performance is suffering this is clear i found two quotes which i really would like to to read the first one is from gartner every year poor data quality costs organizations an average around 13 million dollar and the second quote is from india it is about the comparison of data engineers and data scientists so the demand has

begun to outpace the demand for data scientists two to one o'reilly says four to one or even five to one so if you want to set up a new project you need five data engineers and one data scientist this is something what is in the reality yeah and then andrew ing he did a study in his ai community and this study showed that addressing the data could improve much more than just scrum screwing this model or changing or adjusting the model so there is unused potential in the data and also in regarding to performance yeah and then if quality is good less data is needed and just imagine bigger companies they can process a lot data but if you have smaller companies they have to calculate because it costs a lot if you have a lot of data to process yeah and then don't look only on the data there's this ecosystem which i mentioned the processes yeah so the regulations the law and so on which you also have in bigger companies so um

addressing data does not only refer to data quality but also the surrounding ecosystem and processes so is data centering ai now the new answer i'm not sure but we in deutsche bahn we tried to tackle this let's say topic this challenge what is that eccentric ai and just if you are interested in this topic go on landing dot ai there's a real very good overview and also listen to this presentation from android because it's very interesting yeah simply said ai is code plus data yeah and and data centric ai is for the performance for a better performance and the focus should be on data consistency and quality here's one example with trains let's have a look at this data labeling process on the left hand side you see the current state yeah so you see two pictures and if you draw or if people draw a bounding frame around the trains there are multiple answers possible you see that and the data centric approach would like to focus on data quality which means you have

to be more precise and you have to set up structured rules yeah so which means this picture is only possible in one way and you have to prepare that before yeah so data centric ai is the discipline of systematically engineering the data used to build in the eye system this slide is from a gartner summit in april this year and you see they're also dealing with that topic what needs to happen and just to give you an overview the traditional data management has to deal with more topics than just data and also the other way around yeah so call a data centric ai you can also call it ai centric data the topics have to be connected and i mean ai data labeling ai data enrichment and also things like scientific sorry um i need my glasses um i don't know the the right word but um you can read it so there's another quote yeah through 2025 80 of organizations seeking to scale digital business will fail because they do not take a modern

approach to data and analytics governance so you can look around and you see you have to focus more on the data itself and now i would like to give you a short overview of our data intelligence center so our journey with data in deutsche bahn or into japan the data intelligence center was established three years ago at the headquarters so we have three houses the house of data which i am responsible for the house of ai where you can also meet my colleague today here thomas thiele he is the leader of the house of ai and then we have the house of 3t which stands for community change and communication and you also see here the three different tasks the house of data is responsible for making data visible and available we work very closely together with our business units to also extract their demands on data and the house of ai is for pushing ai with using and also enabling use cases machine learning use cases together with the business units so to to

to make data also usable and then we have the responsibility of house of 3c they are responsible for making this whole effort sustainable because you can do a lot of machine learning pocs but you need to implement them sustainably yeah and we would say this is the journey you should also go when you have a bigger company and this is the heart of our data journey and of course the goal is to support deutsche ban to be more data driven how does that work in reality i will be very honest with you we are not perfect yeah so there is a lot of pressure of time of course and then you need to interact between these three houses but we are in one department in the data intelligence center and we have the chance to meet let's say monthly or quarterly where we update ourselves each other so we talk about our ai projects and the let's say challenges and learnings and put the challenges and learnings together or to the other house of the

data so they can ask us i have some problems with getting access to a specific kind of data please house of data help me and we are responsible for instance for the data catalog so we can try to figure out what is necessary to put this specific data in our data catalog that it's visible that it's accessible and so on yeah so we really try to work very close we are not at the end of course but i would say it is a good beginning here you see our let's say timeline for the last three years especially when it comes to house of data so as i said three years ago the data intelligence center was established um the data and also ai strategy is together and it's derived of course from the group strategy and the digital strategy in 2020 we established a decentralized cdo organization chief digital organization that means we have different data office in every business unit data officers not only the office but also the role data officer we are

trying to establish now new roles like a data steward the data owner and so on then there was a really nice decision at the board of directors in deutsche bahn because in 2020 they adopted a resolution it is called car 4b in german concern for stance pushless it is the resolution for wards and they said it is about added value from data and ai they decided that data is accessible at the whole company you don't need to ask you can get access everywhere every time except there is a specific kind of data which has to be let's say more sensitive we have to take care of it then just the owner of the data has to find reasons why it is not accessible so they turned it the other way around which i really like this approach because this is let's say a new and modern thinking data is for everyone yeah and then of course you have a lot of things to do to make this happen so that's why we established the data

catalog we set up new rules so we try to support the data offices that they are able to find the data owner so therefore you need guidelines so just a few days ago we published a new guideline to put them in a better position yeah and also we have to think from the customer side and our customer is the business unit so we have to explain them why the data catalog is so important and what their benefit is from the data catalog not only putting the data in the catalog but also getting new data other data so it's like a solidarity thinking very hard because um the the people don't have that culture yet yeah so have you have also to take your change people with you that they help you and support you with explaining and also convincing people yeah this is also a chart which i really like and this is from our data catalog supplier from intelligent because it describes the journey and you see the many milestones on the data journey

when just when it comes to data yeah so um you have two roads are getting into one road uh on on the hill you have the drm process um on your right hand side you have the data catalog on the left hand side the business part the business glossary you come to this one road i would say we are right now between data protection and data contracting in theory not in reality yet but we established the foundation for that but then it comes to data quality and this is a large or a huge need for the business units yeah so a data catalog is also just a tool this catalog can support the business units to make the quality more visible and maybe also to find out what are the reasons where i can screw a bit more a specific variable yeah this is our roadmap for this year so we have three let's say pillars or streams data catalog this is our main focus this year yeah so we got also a new technology

behind so we are now changing from one to another one i think we are very early with that because nobody uh changed already the technology behind the data catalog so it's also new for our new supplier but he is very important to make the data transparent through a central corporate data catalog and the definition of done yeah it would so in my view um data seekers experience added values to the catalogue are happy to come back again which is really hard yeah and the second and the third one is about data governance the rules establishing the organization setting up the the roles which i meant already mentioned and here i think it's more important to establish the guidelines here to also to implement all that stuff we created in theory and to turn that into the reality and the third one is about architecture and of course about this new approach about data mesh which i really like but this is not a topic for today maybe next time so here is um the definition

of done that our data management platform dmp is aligned with the customers our customers and offers a solution and access to valuable maintenance data maintenance data they are not customer data but maintenance data this is also what i would like to emphasize um this is the value chain when it comes to models ai models because the foundation is management data management yeah to gather govern and manage data then it comes to tools and processes yeah to improve the processes from the acquisition labeling up to the data preparation then we can talk about data quality for the entire life cycle for instance and then the sexy ai is starting yeah model development and also of course improving the performance the really important message is there is a foundation and this is much much bigger and there are the potential or the opportunities yeah so the conclusion yeah i think you all got my let's say message new models and new architectures generate major headlines and breakthroughs and research but rarely reach practice and i worked in

the research lab at telecom so i know that research is just the practice five in five to ten years so when i see now there's so less about data management i do have some fears so the challenge of ai in practice does not lie in the models or model architectures but predominantly in the data in real use cases almost every application must deal with completely new data this is also a topic yeah when you finish one project you have to go to the next one and you need new data and the story starts again so maybe the solution could be data centric ai and here's a short outlook on the left-hand side you see a bitcom publication about the machine learning trends in 2022.

maybe you want to have a look on it there's also a smart part about data centric ai and then again gartner they said by 2024 organizations that lack a sustainable data an analytics operation that was my tongue twister today operationalization framework will have their initiatives set back up to two years so my recommendation is don't tap in this fail of ai of sexy ai phase the data and put more effort in this whole process before so thanks for listening yes thank you that was really interesting um maybe we also have one questions from our online audience so they they asked who's the best uh contact person for a collaboration with you so i am available via linkedin so let's talk there yeah do you mean in the company in the i guess in the company was in the company yeah so of course um i have to listen to my customers because um i can put or i can set up this nice things like a data catalog but the user of the static data

catalog are the data owner the data user and also the data provider and i have to speak with them or i get into a dialogue with them very often so that i am building something what they can use yeah so it should be also accepted yeah this is of course one of my yeah most people or the contact persons i have to deal with all right and i have also written something down um because i as i saw your 40 over 40 posts on linkedin um i saw also some learnings you you earned in in the years and i wanted to know which one was your favorite yeah so um i studied data science i think um some for four five or four years ago and then i turned into their eye field and and had a department of ai for three years and i realized there's so much new knowledge out there yeah so i even started to establish quantum computing for deutsche telekom so i had to learn so much so my best

advice would be learning learning learning learning like a machine who agrees with learning learning learning is there any question uh in the room yes david he's from delight i asked him to ask a question okay yeah that was the term i used on my slides synthetic data so this is really a good approach i think yeah because um you can store synthetic data for years without having all these rule problems yeah so in telecom we started this approach i think in 2015 or 16 and tried to um turn the real data into synthetic data which was really complicated because when you have um telecommunication data then you have the longitude and edit you know that and then also the time stamps which is really hard to put that or to to to keep the real information in the data but i think this would be the future yeah because you can sort anywhere anyhow and if as long as you keep the real information yeah so and if you want to exchange between departments it's

easier to have synthetic data it could be like um a sample samples yeah i really like that yes yeah thank you for being here is that another uh but uh you can also uh talk afterwards you you are still here i think and then uh yeah you can all go into conversation with claudia thank you so much for being here big applause

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.