Rise of AI Conference 2018 · Berlin

I Want to Implement AI, Now What?

Rick Newman · IBM (Worldwide AI Sales Leader)

At Rise of AI 2018 in Berlin, Rick Newman, IBM's worldwide AI sales leader, argues that organizations should start AI projects with a narrow, proven use case rather than attempting to transform an entire company at once. He illustrates the power and risk of predictive models with an example, based on a real case, in which a bank used historical loan data to flag customers likely to divorce, and therefore likely to default, with about 80 percent certainty. Citing IDC and Gartner research, he says 80 percent of a data scientist's time typically goes into preparing data, and describes IBM tools built to automate that work, including one that let a non-specialist build a cancer-cell detection model reaching 95 percent accuracy in hours. He argues machine learning remains essential for traceability in regulated settings such as financial services, and describes IBM's NVLink-based Power CPUs and distributed deep learning cutting a breast-cancer research customer's training window from days to hours. He closes with an insurance example from his own life and a warning against proprietary, non-portable technology.

Key takeaways

  • Newman advises starting AI initiatives with a narrow, proven use case rather than an organization-wide transformation.
  • He cites IDC and Gartner figures that 80 percent of a data scientist's time goes into data preparation, not modeling.
  • A non-data-scientist colleague built a cancer-cell detection model reaching 95 percent accuracy in a few hours using an IBM tool.
  • IBM's NVLink-based Power CPUs and distributed deep learning cut one breast-cancer research customer's training window from days to about 11 hours.
  • Traceability back to a model's origin remains essential for compliance, which is why Newman says machine learning is 'not dead.'

Detailed notes

Start small and treat data science as a team sport

Newman's core advice is to begin AI projects with a proven, narrow use case rather than attempting to transform an entire organization at once, since broad AI transformation projects tend to fail. He illustrates the value and risk of predictive models with an example, based on a real case, in which a bank analyzed historical loan data and found that customers who divorced were far more likely to default, building a model that flags such customers with about 80 percent certainty; he stresses this kind of analysis has real social implications even though a loan officer would never say so directly. He calls data science 'a team sport' and warns that many self-described data scientists are unqualified, recommending buyers make sure any hire or consultant is genuinely skilled rather than, as he puts it, someone who was an Oracle database administrator the day before.

Where the time actually goes

Citing IDC and Gartner figures, Newman says 80 percent of a data scientist's time is typically spent preparing data rather than building models, which is why IBM has built automation tools, including one that uses Spark under the hood to normalize data. He describes a tool that let a non-data-scientist colleague build an image-recognition model detecting cancerous cells with 95 percent accuracy in a few hours instead of weeks, comparing that figure to benchmarks he cites for human oncologists: about 95 percent for a researcher focused solely on cell analysis, and about 80 percent for a practicing oncologist reviewing 20 to 30 cases a day.

Machine learning is not dead

Despite IBM's growing focus on deep learning tools, Newman insists machine learning still matters, particularly where a decision must be traceable back to its origin, such as when regulators investigate a specific financial trade for possible insider information; retrained models without a preserved snapshot of the prior version can make that traceability impossible.

Scaling training with distributed and large-memory systems

Newman describes an IBM announcement, made about three weeks before this talk at IBM Think in Las Vegas, showing logistic regression parallelized across GPUs without code changes and beating the previous performance record by 46 times, with a similar 45-times acceleration achieved for distributed deep learning, due to enter technical preview in June 2018. He also describes IBM's integration of NVIDIA's NVLink directly on its Power CPUs, giving access to a coherent memory space of up to two terabytes and, he says, roughly 95 percent scalability up to 256 GPUs; he cites one US breast cancer research customer whose training window fell from several days to around 11 hours as a result, without changing their code.

Two real-world impacts: insurance and cancer screening

Newman contrasts his own roughly 11-hour experience getting a car accident claim processed across two garages with a friend's insurer, where a photograph of the damage triggers a deep learning estimate and repair-shop recommendation within minutes, a difference he says will make him switch insurers. He also describes a breast-cancer customer aiming to give every specialist at least 15 minutes with a patient, versus the roughly three-minute US average, by using AI to let non-diagnostic patients be cleared by a nurse so specialists can focus on patients flagged with 92 percent certainty as likely positive. He closes by urging the audience to avoid building or depending on proprietary, non-portable technology stacks.

Quotes

No, machine learning is not dead.

Rick Newman

Avoid tinkering. Let people who are in the business of creating tools create the tools.

Rick Newman

Questions answered in this talk

How much of a data scientist's time is typically spent on data preparation?

According to Rick Newman at Rise of AI 2018, citing IDC and Gartner research, roughly 80 percent of a data scientist's time goes into preparing and normalizing data rather than building models.

How did IBM help a customer speed up breast cancer research training?

Rick Newman said at Rise of AI 2018 that IBM's NVLink-based Power CPUs and distributed deep learning tools cut one breast cancer research customer's multi-day GPU training window down to around 11 hours without any code changes.

Why does Rick Newman say machine learning is not dead despite the rise of deep learning?

At Rise of AI 2018, Rick Newman argues machine learning remains essential because some use cases, such as financial services compliance, require tracing a decision back to its origin, which is harder once deep learning models are retrained without a preserved snapshot.

Mentioned

Deep learning, Machine learning, NVLink, Hyperparameter tuning, Distributed deep learning, TensorFlow, PyTorch, Caffe

Keep exploring

More from Rise of AI Conference 2018

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,269 words

Show the full transcript

the first question is always can I master the clicker and the answer is yes that's good so what I want to talk to you guys a little bit about is the buzzword AI everyone's doing it no one's doing it every industry has solutions nobody has solutions but I'm going to do it I've got to do it right now that's the environment we all live in all the speeches I've heard today someone always makes reference to the same thing it's impossible to find data scientists it's even harder to find real ones but I'll talk about that a minute so what I would propose to you is if you decide as an organization that you're going to create some AI application start with something everyone else is already done pick something small and a common use case don't let yourselves or the organization you're with or of your consultant your customer fall into the trap of saying I'm changing my entire company by implementing AI it's a it's a recipe for failure every time I pick telco

just for fun I've listed some common use cases that are proven they either cut expense they allow you to sell to universe of one they give you more higher customer satisfaction you can if you're a different industry go on IBM's website go on and videos website in Google type in the industry AI you'll find press releases and use cases everywhere why start small one because everyone's afraid of AI Afraid of machine learning afraid of deep learning they don't know really what it does sometimes I think that those of us who are in this industry don't know what it really does but that's a different discussion you take small bites once you have the entire organization moving in the right direction now you can start looking at one of the big bites you read at the bottom a quote and it's an imaginary quote based on a real-life use case so if you can't see it in the back it says sorry Herr Newman your interest rate on your loan is going up two percent since

you scored high and are likely to divorce analysis now if you're seeing there with your partner significant other husband or wife and you think to yourself well wait a minute I'm not thinking about getting divorced this is something I need to know where did this come from well there's a financial institution a bank who decided that they had a problem with people defaulting or paying late on their loans I said okay well what's one let me analyze my historical data well as it turns out if you get a divorce especially in certain countries you have a much higher rate of defaulting on the loan and making late payments because now you've you know you've had two incomes now there's one typically the person who's responsible may not pay on time they said okay so we know this is a fact let me take my data which is where every good ml DL use case starts look at the data so I'm gonna look at the customers who defaulted or relate on loans well Wow a

lot of those people were divorced okay so let me subset that data put it off to the side these people paid on time and they stayed married let me look at everything from their historical trends on payments to the number of properties they own did they buy vehicles every piece of data that that Bank had on their customers that create a model that allows them to look at a customer and with somewhere around 80 percent certainty not very certain 80 percent certainty predictive you fit in a class or you're likely to get a divorce now you're never going to tell your loan officer to have this discussion however the social implications the company implications of doing this kind of analysis are profound when you think about this is the kind of information you can call from your records and predict about a hundred people in this room well we pick 20 of you likely and say yeah you don't get a great interest rate because my deep learning project just identified you 20 is likely

to be the 20 out of this room that will get a divorce in the next three years or the term of the loan so the next thing you have to do when you decide I'm gonna do AI you identified it simply use case now what you have to do is figure out who's going to be involved we say data science is a team sport deep learning and machine learning or a team sport lots of people get involved in your organization so the first thing I tell everyone is yes you have to find your data scientists find a consultant with data scientists grow your own data scientists and if you do hire a consulting firm or hire data scientists make sure they're a real one I have spoken to far too many people especially in the last 12 months it's actually gotten worse over the last year who say oh yeah I'm a data scientist and after about four or five minutes of conversation you figure out yesterday they were an Oracle DBA it's not quite

that bad but it's pretty close anyone can print a business card so then you have to say okay let's start with the data did I normalize the data am i confident in the data do I have one version of truth in my data lots of time is spent there the next is I've got to prepare it you'll see a later chart and I'll skip over it later because I've only got 14 minutes left 80% of a data sent is time if you look at IDC or Gartner 80% of their time spent on data prep I just said it's impossible to find data scientists by the way you pay them a lot the last thing you want them to do is spend time and data prep it's 80% of the average data scientist time then you build a model and I don't care what open source framework you're using if it's PI torch the no cafe tensorflow you spend the time there you train it you develop it to the point where you get a high

level of accuracy depending on the use case different levels of accuracy are enough a colleague used one of our tools now that's one of the things that we do is we produce tools and allow data scientists to be much more productive we also have a tool for those who use images that you'll see in a few minutes any movie pictures and allows non data scientists to create deep learning work he managed in a very short period of time hours not weeks or days to create a model that will detect the cancerous cell 95% of the time is that good enough well what I can tell you is an oncologist who's doing research and only studying cells is about 95% accurate an oncologist someone focused on cancer who only looks at 20 or 30 cases a day is about 80% accurate if I'm launching a satellite into space 95% accuracy is not enough so the use case determines how accurate you have to be you have to deploy your model your application developer is going to

call the API to use the model you're gonna look at how your data is changing and you're gonna start the cycle all over again so I said a moment ago whoops let's go back 180 percent of the time there's sometimes automation works and slides sometimes it doesn't 80% of your time is spent here there's a significant time spent getting up and running because you're talking about open source tools you've got to integrate them you got Testament against each other as soon as you finish testing someone released dot one six versus dot one five it breaks something else then you have to look at you know this CUDA you look at see you DNN and you're constantly creating this stack this is a place I would tell you lots of data scientists like to spend time can't tell you why but they like to spend time being the latest greatest on all the tools so one of the things we did to make data scientist more productive is we created a free to download supported binary

that integrates the stack we have data automation tools that allow you to use spark under the covers to normalize your data so you don't spend 80% of your time there we address the build tree and optimize as well as deployment and maintaining accuracy so our just raise your hand briefly how many of you are comfortable with the difference between machine learning and deep learning it's a good portion good when we started our products around this space we were all deep learning focused we've moved into machine learning tools probably over the last six months so now we've got a nice set of machine learning tools I posted something to my linked-in last Friday or Thursday that said no machine learning is not dead I would tell you that when I speak at least four people walk up every time and say so so when is ml dying when are we gonna stop with ml we're not stopping with machine learning having the ability to trace the outcome back to the origin is critical in certain use

cases compliance in the financial services industry why did I make a trade at 3:13 p.m. when the federal government comes in and says this looks like it was an illegal trade you may have had insider information what caused you to make that trade you better be able trace every time back to the the factors that led you to that decision if you've retrained your model in the mean time and having a snapshot of the previous model you have a problem so I mentioned that we have a tool that we have developed that allows non data scientists to create image based deep learning applications unfortunately there's not time to show you a full demo of the tool because it's a lot of fun this one is a view of what the tool looks like and how you use it you pick your own labels into a GUI you type a label name car person there's a great demo we show that identifies coca-cola Diet Coke Pepsi sprite the fact that lots of Coca Cola bottles have

people's names on them they don't say coca-cola and you simply put a box around it with a nice point click Mouse create your your label train against that data automatically label the additional images you put through the system at the end you look at your accuracy it monitors it for you it tells you whether you're trending up or down do you need more data introduced or not do you need to label more data yourself or not until you get to an accurate model yes there is cafe our tensorflow under the covers you don't look at that yes we're running spark jobs under the covers you don't ever know that it does hyper Prem they're tuning automatically you don't need to know that yes we do spark to do it the end result of which is a training model that you can deploy as an API into any application this is what our colleague did for fun with cancer cells in ours we've a common comment that we get from customers who have been playing with

this technology is well I spent three months and out of the box in three hours you were more accurate than I was it's a fantastic unique tool that came out of one of our labs that actually GA is in about two weeks but we've had an tech preview for months so what else are we doing put that to the side for data scientists we have a whole series of tools I'm not going to go through in detail you can read them a lot of you are very familiar with data science so you'll understand the importance of these that allow you to be more productive if you're doing AI work so the hyper parameter tuning having a GUI interface for you that shows accuracy of your model how the training is going stopping it before completion long training windows are common when you have complex data sets this allows you to monitor real time and then gives you advice are you doing overfitting do you have convergence of your of your data this tool and these

are some views of what the tool looks like comes together into a nice toolset that we call AI enterprise that's as I said a productivity tool machine learning I have to talk about this because we and only announced this about three weeks ago paralyzing ml code across GPUs without changing the code we announced this an event called IBM think in Las Vegas where we use this technology on a limited type of ML logistic regression to be specific where we showed that we could beat the previous record by 46 X you know I said we started with deep learning we've got similar tools for distribute deep learning that allow you to accelerate distribute deep learning by 45 times this is a unique case we are actually going to tech preview this technology in June and then sometime in the next six months it will become an actual product a moment on large models so most of you are familiar that a model has to fit within a GPU memory space 16 gig up until recently now

32 gig is supported by Nvidia and Danny from Nvidia who spoke earlier is somewhere in the back so yes he's got the green tie on we support of course both 16 and 32 gig however you know for those of you who are old like me you know I remember when no one was ever gonna use more than 640 K of memory for das that's absurd well why would you need more data space than that memory space you crazy people one of the things we announced last a year ago at GTC the us-based Nvidia event was large mall support without changing code the ability to support two terabyte data sets we use system memories of coherent memory space with the GPU memory because of our unique relationship with Nvidia we've actually put their technology of MB link on our chip it sits on our CPU now Intel could do that if they want to open up their chip architecture and have a nice discussion with Nvidia about how they could work together not typically until the

business model I don't expect it to happen soon but I would never speak for Intel what this allows you to do is a couple things one use much much denser data sources like high-definition images I typically would tell you that no one I've ever talked to is currently looking at a two terabyte data set 256 gig or so seems to be plenty right now that need will grow of course and we will grow with it as we grow system memory in our architecture this is distributed deep learning I talked about 95 percent scalability with 2 or 56 GPUs we can scale and take a 16 day training window that one customer I work with in the US that's doing research on breast cancer had a 9 day training window they did not purchase a cluster large enough to take it down to a few hours but they're now at 11 didn't change their code the technology does it for them so I've got four minutes left and I want to talk about a couple things

too in the discussion this is the most exciting space you can possibly be in right now use cases by industry are profound impacting both how businesses run their culture it has cultural impact on all of us it has the ability to change how we live my wife was recently in a car accident someone hit her from behind this person was not a licensed driver did not have car insurance and I would tell you it took I had to take the car to two locations because our insurance company demanded it they give different estimates it goes back to the insurance company they evaluate it and we were not at fault it doesn't matter so I would tell you I probably have about 11 hours into this discussion I have a friend who has a different insurance company who literally took a picture of the damage it's uploaded they have a deep learning application that analyzes the damage tells them how much it should cost to fix gives you a list of places to go fix who

will charge that amount and it's done within minutes you can guess once I'm done with this very long process of my wife and my seven-year-old who are both going through physical therapy once that's complete we will be switching insurance companies why would I spend that time deep learning can have a very positive impact the customer is talking about with a 90 day training window is doing breast cancer detection their goal is have every specialist spend at least 15 minutes with a patient in the US the average for a cancer specialist is about three minutes how will they do that because people who don't have breast cancer will not see the doctor the nurse will simply walk in and say just wanna let you know test was negative thank you for coming in that allows the specialist to come in to the patient who does have breast cancer at 92 percent certainty and say look odds are very high we would like to do a biopsy now what would you like to talk about are you

okay with this discussion you can't have a real human interaction in two to three minutes this type of technology fixes that problem radiologists are hard to find anywhere in the world having radiologists in rural hospitals have access to basically the best radiology knowledge in the world through an AI tool that changes how we live we all know about autonomous driving I have one minute and 12 seconds left what I'd like to leave you with is if you're in this space because you want to create an application you have created an application avoid tinkering let people who are in the business of creating tools create the tools don't get caught in proprietary it will kill you every time anyone can innovate and have a moment of time especially in this industry and this industry is riddled with companies that had fantastic innovations they led the world for a period of time and they no longer exist watch out for primary - I can't even say it proprietary with that are there any questions that last less

than 22 seconds if not I'll be at the IBM booth I'm happy to have a conversation with anyone who's interested and I appreciate the time today thank you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.