At Rise of AI 2023 in Berlin, Ronny Fehling, Partner and Vice President Artificial Intelligence at BCG X, argues that generative AI is a step change that complements rather than replaces traditional AI, and lays out how he guides C-suite executives through adopting it. He walks through a ladder of implementation options, from out-of-the-box chatbots and intelligent document processing to fine-tuning and building custom foundation models, warning that fine-tuning is expensive and prone to overfitting. He recommends mapping use cases by business impact and feasibility rather than starting with the easiest option, so unstructured company data, which he says makes up to 80 percent of enterprise data, can be turned into competitive advantage. The second half describes a project BCG X built for its own diversity, equity and inclusion data, using generative AI to quantify patterns in feedback text, such as women receiving more unconditional praise than men, and to measure the effect of recruiting initiatives. He closes by warning that competitors who move first on generative AI will gain efficiency advantages that are hard to reverse.
Key takeaways
- Fehling frames generative AI as complementary to traditional AI, not a replacement, extending semantic understanding of unstructured company data.
- He outlines a four-stage adoption path: out-of-the-box use, retrieval-style document processing, fine-tuning, and custom foundation models, each with rising cost and risk.
- BCG X prioritizes generative AI use cases by mapping business impact against implementation feasibility rather than starting with the easiest option.
- BCG X used generative AI on its own DEI data and found high-performing women left voluntarily at almost double the rate of men.
- The analysis showed women often received vague, unconditional feedback while men received specific, outcome-linked feedback tied to career advancement.
Detailed notes
Generative AI as a Step Change, Not a Replacement
Fehling explains BCG X as the applied AI and advanced analytics arm of BCG, focused on helping companies accelerate AI-driven transformation. He frames generative AI for C-suite audiences as a pre-trained model built on internet-scale data with self-supervised training, which gives it the power to generate novel content but also causes hallucinations that must be controlled inside companies. He notes the pace of change is breakneck, leaving executives uncertain which partners or vendors to trust. He gives commercial examples: Mattel using generative AI for novel toy designs, Procter & Gamble using GANs to generate and test new product formulations faster, and, he speculates, Nike possibly exploring an AI design partner for sneakers, though he says he is not sure that has happened. He stresses that generative AI is complementary to traditional AI, extending semantic understanding to help companies unlock unstructured data, which he says makes up to 80 percent of data inside companies and is typically untapped.
A Path From Foundation Models to Fine-Tuning
He describes a framework, developed partly with the Fraunhofer Institute, for how companies should approach implementation. Out-of-the-box use of foundation models works well for unstructured, general-language tasks such as customer service chatbots, but tends to be a supporting function rather than a strategic lever. The next stage uses an approach similar to LangChain, retrieving and ingesting only relevant company information before sending it to the foundation model, which controls hallucinations without retraining. Fine-tuning comes next but is expensive, fragile, and risks overfitting. Building a custom foundation model can cost up to 50 million dollars or more and is best suited to companies changing their business model entirely, citing BloombergGPT as an example of a company turning its own data into a new offering rather than just an internal tool.
Prioritizing Use Cases by Impact and Feasibility
Fehling describes starting engagements with an ideation phase that identifies where in a client's value chain generative AI creates the most impact, rather than defaulting to obvious use cases. His team maps candidate use cases on two dimensions: achievable impact, distinguishing what is realistic within three months versus a measurable profit-and-loss effect within six months, and feasibility or speed of implementation. This produces a shortlist of frontrunner use cases that show fast, visible results and set a roadmap for further investment.
Measuring DEI Gaps With Generative AI
Fehling describes a project that began with BCG's own diversity data, where the firm found a gap in perception between female and male employees despite its meritocratic culture. He argues most DEI initiatives fail because they rely on untested hypotheses rather than measurement of root causes. His team built a model that analyzes years of historical employee data, including qualitative inputs such as project feedback and promotion emails, turning them into quantitative measures using embeddings and large language models to detect nuanced language, such as mentions of personal traits. One finding: high-performing women left voluntarily at almost double the rate of men, and analysis showed women more often received vague, unconditional praise while men received specific, situation-action-outcome feedback tied to career impact. The team built an AI agent that flags this pattern to feedback writers in real time. In another case, they measured the impact of a recruiting initiative that moved a region from the bottom to the top tier for hiring and retaining women, and used consistent evaluation criteria to show the gain in female hires was a genuine net increase, not a lowering of the bar.
Closing Message and Q&A
Fehling's closing message is that generative AI lowers the barriers for new entrants and competitors, and that companies which delay risk falling behind on efficiency gains that compound over time. Asked about the biggest C-suite challenges, he says education is less about technology now and more about moving companies past isolated, non-strategic experiments toward identifying where generative AI creates real competitive advantage, citing Bloomberg's move to build a new core business from decades of accumulated data as an example of the bold approach he tries to guide clients toward.
Quotes
Up to 80% of the data in companies is unstructured, and it's untapped, typically.
Ronny Fehling
If you don't move, your competitors will.
Ronny Fehling
Generative AI is really a step change, but it's not replacing the traditional AI.
Ronny Fehling
Questions answered in this talk
Should companies fine-tune or build their own foundation model?
According to Ronny Fehling at Rise of AI 2023, fine-tuning is often expensive and fragile with a risk of overfitting, while building a custom foundation model can cost up to 50 million dollars or more and only makes sense when a company is changing its business model, as Bloomberg did with BloombergGPT.
How can generative AI help identify diversity and inclusion gaps in a company?
According to Ronny Fehling at Rise of AI 2023, BCG X used generative AI and large language models to quantify years of historical employee feedback text, revealing that high-performing women left voluntarily at almost double the rate of men and tended to receive vaguer, less career-specific feedback than men.
How should companies prioritize generative AI use cases?
According to Ronny Fehling at Rise of AI 2023, companies should map candidate use cases by achievable business impact within three to six months against feasibility of implementation, rather than simply picking the easiest option, to find frontrunner use cases with real profit-and-loss effect.