Rise of AI Summit 2020 · Berlin · Applied AI Day | Future Space

Scaling AI: Creating Value with the Application of AI

Andreas Liebl · UnternehmerTUM / appliedAI Initiative

At the Rise of AI Summit 2020 in Berlin, Andreas Liebl, Managing Director of UnternehmerTUM and the appliedAI Initiative, presents a maturity model for how companies scale artificial intelligence and explains why most struggle to get value from it. He cites an MIT and BCG study finding that 90% of companies have invested in AI but only 40% report any business gains. Liebl argues the core difficulty is that AI differs structurally from traditional software: it depends on data rather than code, on training rather than development, and on ongoing evaluation rather than one-time verification. He walks through appliedAI's four-stage maturity model, from experimenter through practitioner, professional, to shaper, and uses a computer-vision quality-control example to show how requirements change as a single use case scales into an enterprise-wide, board-level topic. He presents appliedAI's newly published AI Journey Map, a tool for structuring the challenges companies face at each stage, and argues that scaling AI ultimately depends less on technology than on organizational ambition and board-level commitment.

Key takeaways

  • Liebl cites an MIT/BCG study finding 90% of companies have invested in AI but only 40% report any business gains.
  • He argues AI differs structurally from software because it depends on data not code, on training not development, and on evaluation not one-time verification.
  • AppliedAI's maturity model has four stages: experimenter, practitioner, professional and shaper, and most German companies remain at the first two.
  • A computer-vision quality-control use case needs progressively more infrastructure, governance and liability handling as it scales from proof of concept to enterprise-wide use.
  • Liebl says about 90% of AI proofs of concept should be built with startups or outside providers rather than developed in-house.

Detailed notes

A four-stage AI maturity model

Liebl describes appliedAI's mission, as a non-profit tied to UnternehmerTUM, as helping companies and the whole country accelerate toward AI maturity. He lays out four stages: experimenter (running proofs of concept), practitioner (applying AI in two to three use cases but with low efficiency and replicability), professional (scaling AI across the organization to extract real value), and shaper (changing the industry's business models by becoming an AI-first company). He says most German companies, per surveys such as Bitkom's, sit at the not-started or experimental level, with very few reaching shaper status, which he frames as a problem for Germany and Europe's ambition to compete globally against the US and China.

Why scaling AI is structurally different from scaling software

Citing the MIT/BCG finding that only 40% of companies that invested in AI report business gains, Liebl identifies three structural reasons AI is harder to scale than conventional software: AI depends on data, not code, requiring reproducibility and auditability of data rather than just of programs; it relies on training rather than development, since uncertainty lies in finding a solution rather than in changing user requirements; and it requires ongoing evaluation and retraining rather than one-time verification, because of model drift. Citing a point he attributes to Andrew Ng, he argues that adding deep learning to an existing company does not, by itself, make it an AI company, any more than a shopping-mall website makes a retailer an internet company.

The strategy house and an AI-in-production example

Liebl structures appliedAI's approach as a 'house' covering vision and ambition, use cases, organizational enabling factors, and execution, applied differently at each maturity level. He illustrates this with a computer-vision quality-control case: at the proof-of-concept stage, the requirement is simply data and a working model to separate good and defective parts. At the practitioner stage, the challenge becomes system integration with cloud or on-premise infrastructure, a process for handling suspicious parts flagged by the model (including manual inspection and a feedback loop for retraining), and questions of liability if a defective part reaches a customer. At the professional stage, the same technology must be extended across many use cases (picking, logistics, robot supervision), which raises new issues of data pipelines, privacy and anonymization, bias, cybersecurity, auditability, benchmarking and system stability.

Five challenges from practitioner to professional

Liebl lists five challenges companies face moving from practitioner to professional AI use: rethinking AI business models rather than only products and processes; human-centered AI, since many companies have stopped building prototypes (some have 200-300) and now need to focus on process change, incentives and upskilling; moving from single use cases to use-case clusters and value clusters, because organizations can only manage a limited number of topics at once; building data pipelines; and moving from individual AI projects to an industrialization framework that scales AI reliably. He says appliedAI has published an AI Journey Map (available at appliedai.de) that structures these challenges across all four maturity levels to give companies a roadmap.

Ambition and board-level commitment

Liebl argues AI is ultimately a board-level topic, not one to be delegated only to experts, because the board must define the company's ambition for AI. AppliedAI published a report, 'Artificial Intelligence for Boards,' with input from more than 20 C-level executives on what starting to use AI meant for their companies. He cites a Chinese bank aiming for zero human operations as an example of an ambitious industry target, while noting that staying at the practitioner level is a legitimate choice for companies that do not aim to be global AI leaders.

Q&A highlights

Asked whether the scale of the AI journey map might discourage executives, Liebl says it reflects real, collected challenges from many companies and is meant to be realistic rather than encouraging or discouraging. Asked where the biggest problems occur on the journey, he points to the ambition/commitment gap at leadership level and to the lack of a working industrialization framework outside major tech platforms as appliedAI's main focus for the following year. On deciding which proofs of concept to pursue, he advises that in about 90% of cases companies should work with startups or other providers rather than build internally, reserving in-house development for the few use cases that touch core company knowledge. Asked about policy, he argues government programs focus too much on the earliest maturity levels and should instead help companies and startups reach the professional and shaper levels to build global AI champions.

Quotes

Our mission as applied AI, as a kind of a non-profit organization, is to help the whole country to get to an AI age by accelerating the journey to AI maturity.

Andreas Liebl

AI is not something that is done by the experts, but it's something that needs to be done at a board level.

Andreas Liebl

Questions answered in this talk

What is appliedAI's four-stage AI maturity model?

According to Andreas Liebl at Rise of AI Summit 2020, the four stages are experimenter (proofs of concept), practitioner (a few applied use cases), professional (AI scaled across the organization), and shaper (an AI-first company that changes its industry's business models).

Why do most companies fail to get business value from AI investment?

According to Andreas Liebl at Rise of AI Summit 2020, citing an MIT and BCG study, 90% of companies have invested in AI but only 40% report business gains, largely because AI requires treating data, training and evaluation differently from traditional software development.

Should companies build AI proofs of concept in-house or with outside partners?

According to Andreas Liebl at Rise of AI Summit 2020, appliedAI advises that in around 90% of cases companies should work with startups or other providers rather than build AI capability internally, reserving in-house development for use cases touching core company knowledge.

Mentioned

AI maturity model, proof of concept, AI Journey Map, industrialization framework, human-centered AI, data pipeline, use case clusters

Keep exploring

More from Rise of AI Summit 2020

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 4,852 words

Show the full transcript

our next speaker has set up the applied ai initiative untenima tomb it's one of the largest european initiatives in applied sciences he holds a phd at the entrepreneur research institute at the technical university of munich and he has a background working for mckinsey today he's the managing director at ontaniematum please welcome Andreas Liebl yes thank you for your applause yeah so thanks uh for having me here um now at the day for applied ai and i'm speaking here today for the applied ai initiative and um just want to give you a bit of a overview about what how we are thinking about scaling ai because that's ultimately what counts we need to get ai into application yesterday we heard a bit about the paper tigers in in germany we tend to talk about things but it's really about getting it into application so what are we doing so our mission as applied ai as a kind of a non-profit organization is to help the whole country to get to an ai age by accelerating the

journey to a ai maturity and the question is what is this i would talk a bit about this in a couple of minutes but that's actually our goal to really help a whole country the whole industry to get to that level and we do that by two different things on the one hand we work on a kind of a platform model so we try to help as many companies as possible and we just heard um erasmus with mirantix on kind of building ai first startups what we're trying to do with the kind of a second part is to work with the companies that tend or intend to become kind of ai first companies to help them to go there to really work on the core innovation topics of companies to become ai first companies and if we look at ai and kind of the ai journey of how you actually apply ai it's first of all really important that we consider the application of ai not as just one large thing that happens but actually

there are also maturity levels um you start um we heard about this from christian this morning um you start with doing experiments you start with doing pocs and that's actually the first stage so you get to know this technology but you actually tend to get stuck at that level it's not easy to get from this poc to the next level next level is practitioner practitioner meaning really applying ai into the first two to three applications really getting to understand what ai means for you and the third stage is then really professionalizing this because at the second one it's really hard it's really a lot of effort you need to put in it's less replicable so it's not really efficient and you won't get much value out of this so the third one is really about getting value out of what you're doing with ai scaling it globally within your organization and then there's a fourth pillar which is this the shape and shape is really meaning you change the way your industry thinks you build

new business models you really you you really become an ai first company so and if you look at the numbers that are um here um most of of the companies in germany and you have various um surveys on that from bitcom from from others you actually many of the companies here in in germany are at this not even started with ai or at this experimental level while there are very few companies here really on on the shaper level there are a couple of startups there but really on the majority of our industry we are um very much um in the beginning and that's the issue because also from an uh from a from a political side from a government side what we're doing in our ai strategy is helping to get companies used to ai to get them up to speed so that's really focusing on this not not started and experimental levels of what we're doing also from a political side but if you look at the ai strategy in germany in europe what

we want to become is global leaders we want to compete globally against china against u.s against other countries and we want to maintain a position where we have our hidden champions where we really want to be ai first companies so we need to get there and that's what's not happening right now and the question is how to get from these early stages um to these levels here the professional and shaper level so let's talk about this today um so what we're doing and i i said this we tried to help as many companies so we try to shift the curve right so there are many companies at this very first levels and our goal is to have as many as possible to become level three and four and what we do as applied ai we publish quite a bit we try to develop tools that help as many companies as possible very practical things that help companies to really become at this uh level three and four we do trainings we do the mentioned elements

of ai course in germany we built other courses like foundations of ai all three courses that we offer we have a ai lab we do kind of assessments conferences tools and so on but now really uh let's focus today when we talk about applied ai about scaling eye to get from this practitioner to the professional level and there are quite a few studies um this is one from mit and bcg i might just let's focus here on that on that lower part and that's kind of confirming what i just said of the 90 percent of the companies that have made some investments in ai only 40 percent really report business gains and business gains meaning some gain at all it's not really kind of drastic gains and and transformative activities and really changing the business but some gains at all so what you can see is most of the companies actually struggle to get value out of that and there are a couple of reasons and we heard a little bit today but kind of

we cluster it into into three different parts on the one hand we talk about data and data is not code and that is really really difficult to understand for many companies because you typically tend to write code you have software engineers and you actually get a program out of this now you have data as the main source that kind of builds your model and that means that you need to treat data in a way that it's reliable that you can actually scale with the data and i mean that that that that means that you actually need to talk about reproducibility auditability and all these things and that is based on data and not on code anymore so that's kind of the first really difference from um kind of traditional business and that makes it really for companies to struggle on on the ai journey the second one is that we talk about training and not development anymore so it's really a much more experimental approach you have uncertainties that lies in the solution space so

you don't really know where to get and what what you actually need to to do to to come to a specific solution while for until now the biggest uncertainty was changing user requirements so that changes that's the second part and the third one is that we go from a verification to an evaluation of the system so we need to retrain um it's not that you do updates that you do patches but or debugging but it's really a completely new training if you have model drifts and and all these things so you also have to take care of what's happening in the maintenance and and the long run of of actually using ai which is totally different from what has been done typically in in modern software development um and that leads to something that andrea um actually put quite nicely we learned in the internet area that if you take a shopping mall at a website it's not an internet company and today it's the same thing if you take any company and if you

add deep learning then it's not an ai company and that gets you if you do that that gets you to that maybe practitioner level you have one or two or three ai cases and that's it but you will never get to a professional company or this ai first these shaper companies you need to rethink how you do business so how we structure this is along a kind of a house you need to redefine your vision your ambition of what you want to do with ai you need to systematically talk and think about use cases about the whole organization the enabling factors um within the organization then you need to talk about execution and what you do different um what they just said about this kind of uh code and and data topics um trainings and then and so on so you need to think also about the different execution process it's not a classical development process of what you are used to do in companies so we take this type of structure and this type

of structure is actually valid for each of these maturity levels and the questions in each of those maturity levels change for all of these dimensions that are on this list so it changes for the organization you might have different structures in different parts of the maturity um also different people the ai technologies changes and the requirements changes and uh so on so what does that mean in practical terms and this is one example we just take a look at ai in production and you might start um with using a computer vision algorithm to um detect um errors and and and and and and like parts that are uh have some kind of uh scratches or or something within your production um system so the ultimate thing what you what what you need in the beginning is actually the data um good parts wrong parts you have to find and identify a good model to train the data with but that's all to prove that an ai system is capable of coming to a solution to

provide value is actually some data some model you do the training and you have this so that's your proof of concept in the next phase if you talk about applying ai in that specific case what you need to do is to need to think about how to integrate this and if you integrate this kind of solution in the whole um production system you need to have the system integration to your cloud infrastructure or your on-prem infrastructure that then then collects uh the data from your camera and then you have to have a process what uh very very decide what you do with the parts what you do with the output of the ai system and the output could be it's a correct part and it's suspicious it's something that might not be correct and the question is how you split between correct and and suspicious because you always get a probability and once you have these suspicious parts you have maybe a manual inspection and then this person then decides if that's still a correct

part of maybe it's something that actually has a scratch or something on on the part and and just puts it out of the production process and then you have the feedback loop um to the overall system to then maybe start retraining that from from time to time and the system then gives you the recommendation for for the overall process so that implies that you talk about a process that implies that you talk also about liability so who is liable if that part that comes out of that is actually something that causes you to have like major recalls in in like getting getting cars for example back because there's a really critical issue that is also a question of how you deal with that how you train these persons so it's more about the process of implementing the ai solution in your business and suddenly you don't talk in the kind of the strategy house here you don't talk only about data and and the ai technology but you talk about the execution you talk about

the people that are working on that the process is the governance structure and so that's the level two that's if you want to be practitioner if you talk about professional it's not feasible to just stay with a computer vision algorithm for one part in one plant to quality control but you now start to think about how can i use computer vision in my production overall because what i have it here um and maybe there are other use cases there's maybe kind of a quality control picking and logistics manual production processes and the supervision of the automation system and the the robots in the machine so you can use computer vision in more parts if you talk about this and if you have this larger vision of how to use computer vision in your production you then think about a data pipeline for all the pictures for all the videos you actually get and you talk about privacy um and and anonymization of data for all the recorded pictures and videos here you talk about the

correctness of the data bias also cyber security in this data pipeline and then on top on the model you then talk about auditability about benchmarks you need to know if the system is better than the one you had before about validation reproducibility explainability stability of the system because you don't want your um your factory to actually have you need to stop because your your system for example is not stable anymore so you somehow come from one case to talking about data pipelines talking about infrastructures that are on top of one case but it's applicable for many cases and then you talk about the whole structure and that is the the real challenge that you move away from ai as a technological question to ai as a company and organizational question and that and what i can only agree to to christian was he said that ai is a ceo topic or sea level topic because if you want to tackle ai at this professional level you need to really think about this whole structure and

that means that it's not only a tech topic anymore so how we deal with that we just published today a ai journey map you can find it online on our website apply the i dot d e and what we did is to first of all classify and structure all the challenges that typically companies face to get from the first level to the fourth level um in this ai majority on the ai journey to help you understand what's coming up and because that's what you ultimately need to understand is where are you at these kind of different levels and what are the next challenges you need to tackle in order to become more professional so that's the one the first step said we try to publish as much as possible in the next phase um we we further fill this out and we then provide solutions to all of these questions because ultimately it's about um then actually um helping companies to get um through all the different levels to mature on on this on this

journey but you i said you can find this online you can have a closer look um we did this exercise with a couple of companies and they like really liked it because it helps them to provide an agenda for next year they don't know what to do but if we have the structure and if you really thoroughly go through it you actually have a roadmap of how to mature systematically now talking about this what i said about this practitioner to professional level from our perspective there are a couple of points um that you would need to consider and and tackle and and kind of the challenges at that specific level the first one is you need to talk about this ai business models you don't talk only about this classical products and processes but you really need to rethink about what ai would mean at this larger scale if you apply it in your organization the second one is human-centered ai what we see in many companies is that they stop prototyping now they tell

us we have enough prototypes we have some some large companies have 200 300 prototypes and they stop developing prototypes they only focus on process changes incentive system for the people upskilling of people to really get to the application of ai so talk or think about the human that's behind the system that actually needs to apply ai the third one is from single cases to use case clusters and value clusters because ultimately there is a specific capacity in your organization of how many well different topics you can handle as a core team within the organization and you need to decide and re-evaluate what are really the core cases you need to focus on fourth one is uh getting to these data pipelines i quickly mentioned this and the fifth one is uh kind of really getting from ai projects to an industrialization framework that helps you scale ai drastically and reliably within the organizations and these questions are not easy questions there are no good best practices outside there have just been a couple of books

from google being published on on especially on the the latter two um points on the pipelines and the industrialization of how they do it but that's kind of questions and and challenges that all the companies face and that we hopefully also has applied i try to tackle together to then provide it again because ultimately we need to answer these questions to become professional users of ai technology and and i just emphasized it ai is not something that um is is done by the experts but it's something that needs to be done at a board level because ultimately the board needs to define the ambition of what you want to do with ai and most of the companies that we work with then also at some point struggle with sufficient understanding with sufficient commitment on the board level we just published this artificial intelligence for boards with more than 20 c levels that comment on what it meant for them as a company to really start using ai and this ambition is something that maybe is

a bit underestimated maybe it's it's not visionary enough actually there are quite a few companies in all different sectors that really drive ai and just due to the lack of time just want to emphasize the third first one so please if you don't know and financial google them and see what they are doing they have the goal of zero human operations they that's one of the largest banks in in china and they really kind of challenge all the processes in banking with the goal of having a fully ai driven uh bank and it's maybe one of the most advanced companies in the sector in the world there are quite some some articles about that but this is something what maybe is an ambition in in that sector and you have all the different other examples in all the other sectors so final point what's your ambition what's the ambition of you and your industry because that ultimately drives where uh um where you want to stay many companies want to say that practitioner level that's

also totally fine if we have the ambition to stay uh global tech leaders in the world to stay hidden champions we need to think about these two categories and that means for you as a board member as ac level that you have an ambition within your company because otherwise you won't get to that stage so thanks um from my side that was kind of a bit of an insight of of what we're doing as applied ai happy um to get in touch with you happy to follow this vision to really transform our economy in in germany and in europe because that's ultimately what we need to do what we heard yesterday that is essential for us thank you thank you very much andreas we don't have questions from the audience yet but we are very glad to take them i have a couple of questions i was impressed by this almost gigantic map called the the journey right um and i thought my that's a long journey do you think this is like encouraging to

cxos or rather off-putting saying well maybe then i don't need ai if it's that complicated well i i think it's it's necessary and it's realistic what we did in the last couple of years in in applied ai is actually collect all these different challenges from all these companies and if you don't well listen to talks on stage from these companies but if you talk to them in person they talk about these challenges and they tell you we are at that point and we don't know how to continue um so um it's from my perspective it's realistic um but um you you've seen also the numbers that is an explanation of why these numbers are so much on on the left side um of of this picture it is a challenge um and obviously well it's not easy because otherwise all the companies would be level 304 already sure we've seen a lot of different uh problems uh of course uh on the graphic ev everything looks the same has the same size the same font

so in which phase or or maybe where exactly on this map of a journey are the biggest problems or the the time where everybody thinks oh i'm not sure if we're going to make it as a company yeah i think so so on the one hand it's this this ambition and and commitment right um so on the very top um of this landscape um because um if we uh we heard about the uh from from christian about this kind of play money you get to to kind of experiment and and so on that's all sufficient to get to this first level to the experimenter but at some point there needs to be a commitment from the company to really start transforming to really think of ai first or professional use of ai and that's a different commitment that you have to take so that's a large struggle and then i would say on the execution side on this industrialization framework that's kind of our main focus also next year because i or we haven't seen

any working industrialization framework outside of the big digital tech players and this is one of the key challenges now for many of larger and smaller companies in germany that they need to get from this one or two working use cases in the organization to scaling it to professionalize it to make it reliable and stable and that from our perspective at least is one of the largest challenges because it affects the infrastructure on the software hardware side the data it affects the organization as a whole so that is something i would say is one of the largest challenges right i also remember the slide where the proof of concept was just the first of three steps or three phases um and we were talking about the graveyard before so um how do you advise people which concepts to prove at all and to decide after you have a proof of concept if you should go forward and implement everything and roll it down so what we do is we actually challenge companies quite a bit on

this core ai um what to do internally for themselves and what is something that they just do with partners so and then from our perspective 90 of the cases they should do with startups should do with any other providers and because it's not something they should do in the long run on their own they don't have people to do that they don't have the expertise and they simply don't understand this question of of retraining and technical depth and what's all associated with maintaining ai systems um so our first um discussion always is is that really something you need to do on your own if not there are so many good startups you can work with them and it's so easy to get these kind of first prototypes running and then also they help you to get into application then you get rid of most of the questions and challenges there then there are quite a few really essential ai applications in the company that really affect their core knowledge and where they say well that's

something we need to do internally right and and what we do now is to help them then mature on on that but it's kind of reducing the vast number of ai use cases to quite two or three or or even one um and then you really can focus on this one and have partners strong partners outside of this right and you also show the court i think it was good by andrew which said that having a company and adding a bit of machine learning doesn't make it an eye company but i think the truth is also that not every company has to transform into an ai company right exactly that's what i mean it's totally fair to have to stay at a practitioner level it's totally fair to have one or two um startups you work with to predictive maintenance and how do you deal with the fear that maybe it might not be enough like my you know the other company across the road if they transform themselves more than in 20 or 50

years we might not be on the market anymore i would actually be happy if that's the fear of most of the companies because right now many companies don't even think about they they think it's enough um they don't see that it's maybe not enough to get there so what i'm telling them is well you need to think up globally about the companies that are at this level three and four and you need to get there so i would be very happy if companies fear that what they're doing is not enough right we have one question from the audience from the perspective of policy makers where do you see the biggest leverage points to help companies advance through this journey yeah i only shortly mentioned this from my perspective what's the the kind of wrong a pro missing approach now is that right now policy only really focuses on the very first levels what i would like to have is a program that helps to scale ai with the goal of getting to this professional and

shaper level both on the startup side we have quite a few programs but they all focus on tech transfer and so on we don't have good programs to build global champions leaders these shaper companies on the startup side and we don't have systematic approaches that really help and accompany companies along the journey to also stay maybe global hidden champion or actually become hidden champions so on the both sides i like if we have the ambition to stay there as as germany as europe we should focus on these um two steps at least as a vision and with kind of programs from also supported by by by the government by the politics to really systematically get there and don't stop at the first level thanks very much i'm really impressed how uh how detailed your insights were i'm sure there's a lot more where this came from and i hope that everybody who is interested takes a look at it thank you very much andreas eva thank you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.