Rise of AI Conference 2019 · Berlin

The Democratization of AI

Dinakar Munagala · CEO at Thinci

At Rise of AI 2019 in Berlin, Dinakar Munagala, co-founder and CEO of Thinci, argues that four barriers are slowing AI adoption: cost, talent, time and data, and describes how new processor architectures and tooling could lower them and bring AI to the masses, much as cheap computing, operating systems and the internet drove the PC revolution and touch UIs and app stores drove the mobile revolution. He says building an AI application today needs cloud compute that gets expensive over months of training, or hardware costing up to 300,000 dollars, plus a large team of data engineers, data scientists and ML engineers, with 50 to 60 percent of time spent just collecting and cleaning data. He points to a global shortage of AI talent, citing a figure of about 300,000 AI researchers and practitioners worldwide against a McKinsey estimate of a quarter-million-person US data scientist shortfall by 2020. Munagala describes Thinci's programmable graph processor architecture, built to accelerate AI workloads more efficiently than CPUs or GPUs, and points to low-code tools, transfer learning, model marketplaces and synthetic datasets as ways to let non-experts, or "citizen data scientists," build AI applications.

Key takeaways

  • Munagala frames cost, talent, time and data as the four barriers currently blocking wider AI adoption.
  • He cites an estimate of only about 300,000 AI researchers and practitioners worldwide against a need for millions.
  • His company Thinci built a programmable graph streaming processor as a third hardware category alongside CPUs and GPUs for AI workloads.
  • He says his company spent 50 to 60 percent of its development time just collecting and cleaning data.
  • He describes deep learning cutting drug-molecule screening from about a month for 20-30 molecules to two hours for roughly 10,000.

Detailed notes

The barriers slowing AI adoption

Munagala says almost every company he visits wants to be first with AI but runs into the same barriers: cost, talent, time and data. He frames AI as the latest in a sequence of historical inflection points, after the industrial, electrical, PC and mobile/internet revolutions, each of which created new industries, jobs and wealth. He argues the PC revolution needed affordable powerful computing, an operating system and good tool chains, plus the internet for sharing work, while the mobile revolution needed a natural touch UI, a personal device and an app store model. He says AI needs the same ingredients: affordable, power-efficient compute for training and edge inference, easy application building, adequate data access, pre-built models and self-customized AI for a user's specific situation.

Why compute, talent and data are so expensive today

On compute, Munagala lists three options for building an AI application: renting cloud resources, which gets expensive over a roughly six-month training project; buying a dedicated box such as a DGX for 180,000 to 300,000 dollars; or buying a cheaper GPU, which then takes about three days to train even a simple model, before facing the separate challenge of power-efficient hardware for edge deployment. On talent, he cites Tencent's estimate of only about 300,000 AI researchers and practitioners worldwide against a need for millions, and McKinsey's prediction of a US shortfall of a quarter million data scientists by 2020, arguing that today's tools are built for AI experts rather than for closing that gap. On data, he says his own company spent 50 to 60 percent of its time on getting, cleaning and labeling data before training could even start, a pattern he says holds across companies he has spoken with. He estimates the full cycle of getting data, training a model and deploying it to hardware takes months to years and costs millions of dollars.

Graph processors as a new hardware category

Munagala argues that CPUs, which exploit instruction-level parallelism, and GPUs, built for thread-level parallelism, are not well matched to the graph-shaped workloads of neural networks and CNNs. He presents this as the next step in a decades-long pattern in which new processor architectures coincide with new programming approaches, and says the industry increasingly accepts the need for a third class of processor, alongside CPUs and GPUs, suited to AI workloads for the next 20 to 30 years. Thinci's graph streaming processor is presented as this kind of programmable, AI-specific architecture, positioned to cut cost and power consumption relative to existing hardware.

Tools, marketplaces and synthetic data lowering the skills barrier

To close the skills gap, Munagala points to a shift from Python- and R-based tooling toward languages such as C++ or Java that more of the worldwide developer population already knows, plus tools that let database administrators prepare training data and let business analysts use transfer learning without being data scientists. He describes this as enabling a "citizen data scientist" who can build an application with low-code or zero-code tools. On data, he expects model marketplaces and transfer learning to grow, including last-layer training that adapts an existing model, such as one built for face recognition elsewhere, to a new setting like an office with minimal extra training. He also points to synthetic data, citing autonomous driving companies that now do an estimated 70 to 80 percent of algorithm testing in simulated environments rather than years of real-world camera driving, with the remainder still needing real-world testing.

Expected benefits once AI becomes cheap and easy

Munagala argues that if compute and tooling reach something close to PC price points, developers will spend less time on infrastructure and more on major problems. He lists agriculture, citing better crop yield prediction and matching seed types to climate risks such as typhoons; education, including AI tutors for remote areas and curricula customized to a student's strengths and weaknesses; energy and utilities, through cross-country sharing of grid-efficiency learnings; and medicine, citing a claim that deep learning has cut the time to screen molecules from about a month for 20 to 30 molecules against 150,000 protein targets to about two hours for roughly 10,000 molecules against the full human proteome. He also mentions encountering three or four startups within six months working on early cancer prediction or disease diagnosis, and expects quality healthcare for uninsured and underinsured populations and lower insurance costs as further benefits.

Questions answered in this talk

What are the main barriers to AI adoption according to Dinakar Munagala?

According to Dinakar Munagala at Rise of AI 2019, the main barriers are cost of compute, shortage of skilled talent, the long time it takes to build an AI application, and scarcity of usable data.

What is a graph streaming processor?

According to Dinakar Munagala at Rise of AI 2019, a graph streaming processor is a programmable hardware architecture, built by his company Thinci, designed as a third category alongside CPUs and GPUs to more efficiently accelerate the graph-shaped workloads of neural networks.

How much time does data preparation take in building an AI application?

According to Dinakar Munagala at Rise of AI 2019, his company found that 50 to 60 percent of development time went into getting, cleaning and labeling data before training could even begin, a pattern he says holds across other companies too.

Mentioned

graph streaming processor, transfer learning, citizen data scientist, synthetic data, model marketplaces, edge inference, low-code

Keep exploring

More from Rise of AI Conference 2019

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,311 words

Show the full transcript

first of all that thanks to rise of AI for the opportunity to present I'm Dan Ackerman ogala co-founder and CEO of pinkeye we are an AI computing solutions company so what we have at the core of our innovation is a novel processor architecture which is suitable for AI class of all told CNN computer vision any general complex graph we process really well and and it's it's completely programmable and then software on top of it to enable different applications today we here to talk about as a young company how we see AI coming to the masses right and what are the some of the barriers for adoption and some of the things that are happening to overcome those barriers and and bytes K widespread the adoption and then the benefits that's the theme but on a lighter note to start off this is actually something quite common that we see in companies that almost any company we visit or even otherwise the companies want to be a first but they are barriers right they cost barriers

talent barriers preventing that the time for adoption the tools etc and then data so this is a common theme that we see so a I what's the big deal about here right as several of the people in today's conference mentioned this is a key inflection point and it's proliferating to different aspects of our lives everyday lives and historically if you look at it the last couple of hundred years they've been such key inflection points it was the Industrial Revolution followed by the electrical revolution followed by the PC revolution and then mobile and Internet and then now the AI revolution so at each such key inflection points what it coincided with is some new innovation you know coming to the table and as a result new industries were formed new jobs were created new wells were wealth was created new companies were formed so of the older ways of doing things got more efficient and were done away with so that was the general theme air will be no different from that we expect similar trends

to follow here as well and a lot of benefits to come about so let's just look at the in the more modern computing history right what we know of the PC and then the mobile and Internet era what were some of the common themes the the PC revolution happened because there was a cheap you know I should say cheap affordable computing rather an affordable but powerful computing machine that happened the concept of an operating system and and good tool chains around it right be it for proper processing Excel etc to to help create applications and productivity and the third one was internet right having being able to share we be connected with rest of the worldwide population and share your work if you look at the mobile revolution of the other hand some of the the cool things that happen that made it such a huge success was a really cool UI right was so natural touch easy to use and and the car and an app store model which was anything you want any

for any problem that's in your life you know caller call a cab or whatnot there's an app for everything so the concept of having a personal device close to you and have all these solutions to help you everyday life and and then of course connectivity right so those were the common themes that helped the PC and the mobile revolution we believe that similar trends are needed to happen are required for AI to come to the masses as well right so the concept of affordable but powerful compute for both for training as well as edge inference being powerful yet powered power efficient ease of building applications having adequate access to data having models which are pre-built and easy to use and then self customized AI applications in your in your forest Pacific circumstance so these are the kind of things that are needed really for the whole AI personal computing wave to happen but there are barriers as we feed today one of the the first barrier is cost the cost of compute is pretty high

and most of you here are either parts of an AI developing team or or a part of a company that's exposed to large servers that on which AI is developed so the cost of actually building and maintaining an AI solution is pretty high and that comes down to compute the second one is talent right there is shortage of talent very few people actually understand how to build an AI application completely compared to the worldwide developer population that is and data is the key ingredient to create any application but data is scarce right that's something that has to be solved and the time it takes to build an AI application is fairly long we're talking about months and so on so drilling down a little bit deeper into these the very first thing is compute so today if somebody wants to build an AI application there are three choices right a few choices I should say go to the cloud and you're paying per hour quickly the costs add up right if you if it takes

about six months to build an application renting cloud resources to to train it is pretty expensive a few companies can afford it but not not not for either independent developers or etc right so for example if you take the analogy of how the PC industry happened and lots of developers started coding for that to happen compute has to be a lot cheaper the second option is buy an expensive you know a desktop box could be a DG X it could be another brand but the point is that the costs are anywhere between 180 thousand to you know two hundred fifty to three hundred thousand dollars so how many even small or mid-sized companies or even individual teams can actually afford it right it's pretty expensive the third option is buy a cheaper GPU it's it's cost efficient but then as you train it takes a lot of time write a simple model will take about three days to train and once you do all this what wait when you deploy it right you need powerful

hardware powerful but power efficient hardware at the edge to deploy a solution that that's the challenge as well the second problem we we spoke about was skills and talented the people right so it so happens that AI came out of nowhere suddenly in an exponential growth therefore there is a shortage of skilled people who understand how to actually build air applications so currently the tools are targeted towards few people who are experts at AI right all the tools the tools are the ones which bridge the gap between having any expertise and building an application versus versus not having expertise about being able to build an application so the tools are inherently not the way you know they should be to help a a development the second one is in general even if you look at the worldwide AI population the 10 cent did a study that they were only about 300,000 AI researchers and practitioners on the old planet but what is required is like millions of them McKinsey predicts that by about 2020 for

us alone we'll be short by a quarter million data scientists so these are pretty big numbers right so how do you how do you actually solve this the gap between what a developer can do versus what is expected from the you know in order to build an AI application so how can tools help bridge that gap is the key question the third part is data right data is scarce data is hard to get we've seen it as a young company as we build some demo applications in our company 50 to 60 percent of our time was actually spent on getting data getting data good quality data cleaning it up labeling it and then presenting it to train and this is a common theme that we've seen to any company that we've spoken with 50 to 60 percent has just spent on getting the data even before they start training so this is a pretty big pay point if that has to be addressed so the combination of all of this translates to this so at

a high level there if there three phases right getting good good quality data and then training it and training a model for a particular problem that you want to solve and then deploying it to hardware in a very efficient manner it takes a pretty large team large and experienced a lot of expertise so you need data engineers data scientists ml engineers C++ developers UI and an IT to maintain the infrastructure so it's a it's a silo sizeable operation to actually truly build an AI application and the time it takes is about months or years and the cost of compute is pretty significant it's millions of dollars when you finally look at the cost of building an application making the infrastructure retraining etc etc or a period of a life cycle of a project so what are some of the the optimistic things that are happening in the industry to solve all these problems and truly lower the bar to make AI happen right in a democratic fashion the very first thing is looking at it

again bottoms up from a computer point of view the cost of compute and the efficiency of compute CPUs have been around for some time like 40 years plus and they exploited instruction level parallelism really efficiently then came along BSP which the data parallel really well and then GPU started off with thread level multiple threads and thread level parallelism and here we are in the next era of computing where you have a lot of tasks running in parallel layers and layers of noodle neural networks or cnn's with other kind of algorithms all running in parallel so hardware racket architectures which are able to exploit this parallelism and and do it in a very cost-efficient manner is is the answer that can actually help efficiency and reduce cost so so we call this essentially graphs streaming processors so this is a grouch art that shows different kinds of hardware architectures on the top right all the way from 1950 is still till now and different kinds of software programming approaches that went hand in hand with these

processor architectures over the years so the it's a busy chart but the the theme the message is this every time there is a new processor architecture it coincided with within a short window a new kind of programming approach here we are in 2019 and much of the problems in the AI world are graph kind of problems there many many tasks neural networks and whatnot and yet the the hardware that these are being ported on to because of the fact that the hard way that exists our CPUs and GPUs so therein lies some inefficiency right and more expensive more power consumption and so on so what's needed is a true architecture that can actually understand these graph languages in a nice programmable fashion and accelerate them well it is a pretty widely accepted fact now in the the technology community that besides the CPU and GPU there is the need for an additional type of processor that is suitable for AI kind of firk loads which are going to dominate for the next 20 30 years

so coming back to how do you address the gap of skills and and how does how do tools kind of rise to occasion to solve these problems right what you have on the top is a typical way of building an AI applications you know you need a combination of ML engineers data engineers and data scientists what you have at the bottom is some of these forces that are are happening in pockets to truly enable these kind of people to develop applications so moving from Python and are based approaches to either C++ or Java which is more developers are have been used to it more that you have a when you look at the world wide developer population much more of them actually are familiar with those so so that enables a regular developer to to build an AI application similarly database admins using people who can can can get data rich data and make it presentable to train and wouldn't it be cool if a business analyst was able to use transfer learning techniques to

train versus just data scientists being the only ones being able to train so as a result of all these forces what what ends up happening is if there's a way that almost there's no coding experience required right low code to zero code kind of experience required to create a nice AI application for a specific problem you know using transfer learning you know better tools and so on what it enables is the concept of a citizen data scientist you know somebody who's not really a data scientist but you know a person who can use intuitive tools to build any application one of the the the next key challenge is data we are seeing marketplaces happen that that's that's going to be big transfer learning is going to be big we see that the concept of last layer training right having taking a model from some views built let's say a model for face recognition on the other side of the planet and I customized it for specifically my office and and just to some minimal training

to recognize people coming into the office or onto a shop floor so these these kind of things are simple things and but yet they're powerful and they they're easy to to realize an application and having marketplaces freely distribute such models is is going to lower the barrier truly and being able to create good quality synthetic datasets synthetic data is already happening for example in in the world of autonomous driving several of the car OMS and companies what they're doing is instead of sticking cameras on cars and driving around the streets for let's say six years of testing time that time is come becoming shorter and shorter because much of the autonomous driving work is actually happening in the lab in a synthetic world beat freeway driving urban settings so so the whole idea is how do you do much of your testing maybe 70 80 percent of your testing your algorithms in a synthetic environment and then you still go you know for a real-world testing for the remaining 20% now autonomous driving is still

sort of a mission-critical set thing and it's it's you need real life testing quite a bit but what if you could have synthetic datasets to do training into you know several other areas that you want to deploy right that's that's going to be interesting and we are seeing efforts in that area so therefore with the combination of all of these things that are happening it's what you see is instead of needing a large and expensive team of people to build a complex AI applications were able to use really intelligent and intuitive tools to build an AI application developers who are not as a where can also build these AI applications what took about months and years to to build their applications and train and deploy is coming down to a few days to a month and the cost of compute and storage what those millions of dollars truly can come down so the goal is really if you if you're able to crack the PC price point right and because the world is used to

a pretty PC price point you know a few thousand dollars or maybe a few hundred dollars that kind of a price point and really have good powerful hardware to train as well as deploy that that's going to be an interesting point where mass adoption can happen so so once you have all this the the big benefits right is that instead of smart people trying to focus on hey what's my environment which version of Linux what is the tool chain what is the environment would look like and and trying to solve those problems they can instead focus on really important problems right solving climate change better health care and so on right the thinking about what the big problems and what are the outcomes that they need and trying to solve that problem so these are some of the big things benefits that can happen of course it's the listed pretty unless applying it for example to to agriculture and hunger right better prediction of crop yields understanding climate less bringing down insurance costs there's some

efforts that we saw where certain kinds of seeds are more suitable for areas which have let's say typhoons or any kind of natural weather risks so using those kind of using AI to understand what kind of crops can do better in those kind of climate and helping the farmers to grow that and thereby improving the productivity and reducing the risk students having access to AI and billing applications and understanding the whole AI sooner so that as they they get out of school they are able to work with all the AI J the changes are happening due to ai ai as teachers to remote areas right having high quality education for for kids in developed nations for example and and also in other countries the every student is is different so wouldn't it be nice if AI is able to customize education to each kids specific need so for example a kid is let's say good in math but weak in biology or vice-versa having an appropriate curriculum appropriate tests to help that kid to do

better energy and utilities for example learning that how efficiently grid can be managed in our different and applying that to another country right the sort of cross cross geography and and applying those learnings so that energy efficiency is really improved medicine is is one of the the big areas that be seeing a lot of development for example it just took about two years ago it took about a month to screen twenty to thirty molecules against 150 K protein targets and we are right now able to with deep learning bring the time down to just two R's and screen about ten thousand molecules like against the entire human proteome and drug discovery can be accelerated significantly because of this disease eradication I've come across at least about three or four startups in just about a span of six months who are either trying to predict cancer early or diagnose different disease and and and so on I wish they all succeed and big benefits do happen for mankind having quality health care for uninsured and underinsured

based on a nei based physician for example right and and give it you know have that happen for the whole population insurance costs come down you have overall better more healthy population so large-scale benefits can happen these are a few of them but really the possibilities are endless we're we're happy to be part of this journey and it's exciting thanks for the opportunity and open to any questions

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.