yeah hello everybody my name is Stefan I work for week you as a chief product officer and from a background as you heard I'm a machine learning guys or more on a technical side but I also studied business so I have these two legs business and tech and that that would be my first question to to the audience so that I have a little bit of a feeling who's who's sitting here so if you think of who's more on the business side could you maybe be so kind and and raise your hands okay and who would consider himself being more on the technical side maybe ok ok that's cool that so we have a 50/50 or maybe some even raise their hands twice so I will talk a little bit about something that I observe it's it's a journey how yeah some sort of intelligent processing enters in industry because I think it might be relevant for your industry I guess very few of you might be from advertising industry which is the current company
I work for but it's pretty much I think a pattern that I see in in in multiple industries so you all know this hype cycle if something is new then nobody wants to hear about it is super skeptical whatever talk to your parents about blockchain and you have this effect then half a year later everybody is on this super hype and everybody wants to invest all this money into some cryptic horns or everybody thinks all cars will be speaking next year and the fridges will talk to you and then after some time it becomes a little bit more realistic and I think in the company it's the same you have different people on the board you have a CTO or CEO CFO maybe different guys and they have a different position where they are on this hype cycle so at first maybe the tech guys try to convince the other guys on the board that they totally need to do something and later it's the same tech guys that try to convince the the CEO
that no they cannot do everything with learning an AI so this is what I see and so what is a realistic journey of something or realistic introduction of of technology and yeah this is what my talk is about at first I going to show you a doughnut it's it's the universal representation of data and for those who raise the hands to be more on the technical side I guess some of you have maybe a degree or studied whatever computer science machine learning pattern recognition artificial intelligence and if you did that I did that and I remember that the biggest problem was not the algorithms because they were all in the books but the biggest problem was data so if you wanted to work on any problem like you have a super idea how to improve the anti break system of a car where would you get break data from cars and and try to get it like try to write to BMW or so if they want to give you some some car data they
were probably not so you sit there and you are hungry for data this is pretty much the situation I know from every graduate student and yeah now I'm in advertising where the situation is a little bit different and I'm gonna quickly explain a little bit the very very core business principle and which is at the same time the the optimization criterion in that respect it's like what do we do and then we look at what how to optimize it pretty much the advertising business is very simple there's a person and we are in performance marketing so that means there's people that say I give you a dollar if you make someone install my crazy app or I give you a dollar if you make someone buy my just presented car buddy and so you say okay I I'll try to do that and you buy impressions but just this little banner slots - in the simplest case but you can also buy someone on YouTube - to to create a video or an influencer or
whatever but like at the end of the day you you you buy kind of impressions which is showing something to someone and the average price is maybe 50 cents for a thousand so CPM means for a thousand for for 50 cents you can say hello world to a thousand people and you do that for $1,000,000 because that's like your budget or what your cash flow allows you to spend and you buy 2 billion impressions and that's a realistic number maybe a realistic number for a day so maybe a typical company that is small or medium sized would spend a million dollar like like our company would spend a million dollar a day on buying impressions and they're not or money we do it for for other people and then let's say only 2.5 people click on the the thing so you get so you get a 50 million clicks 2% of those actually download the app so you end up with 1 million downloads and guess what you get 1 million dollar back so that
is pretty much the business and now you made it your breakeven you can do that if you don't pay taxes a pretty long time and now you want to improve that somehow and you say okay what can I do to be profitable to to make my business better and you typically in a small company like a 10 people company 15 people company you have people doing that manually and they will start yeah so your breakeven but also the nature of this is the Paradise yeah because you get from all these impressions you get like ground truth data you know who clicked and who not and you have a lot of data but you want to improve it and so at first in order to be preferable you can just go which is what we do you pick up the phone and you call the guy said dude I need two dollars instead of one dollar for install can we do that and then maybe you agree on one dollar 25 and in that moment you're
profitable next thing is you could buy cheaper impressions like cheaper cheaper places where you where you show that you could optimize your whatever you show to improve the percentages of people that actually click or you can make a better match like what you show to which person or if they go after clicking they go to the App Store you can also improve the descriptions and the images in the App Store to improve all that so that is pretty much the whole business if you if you do that you have a business you can go out and money and it's pretty cool the problem is if you scale it you need a lot of people to do that and and pretty much this is a typical example for many businesses we heard talk from telecom we heard a lot of other talks where there's a lot of repetitive tasks that involve a lot of manual work and and most of this can can maybe be done by by machines but how first of all congratulations if
you're at that point with the numbers I just told you you receive already a lot of data so you have between 10,000 and a million events per second which is all these showings of ads and and the information if someone clicked or downloaded and that that even if you compress it slightly but with all the additional data like where was it from which browser that he has like all the data that you receive you end up with terabytes of data per day so obviously you cannot take a PhD student or or a bachelor student with his notebook and ask him to run his machine learning algorithm on that data it doesn't work so what do you do and what I see is in all the situations where you have that many data abhi telecom or something else there's a journey that that goes across some stages and I would just like to talk about that and maybe let it discuss it with you maybe you have a business maybe you know where you are and
I think you should not skip a stage I think that is the most important thing some business are very manual and then there's this great idea yes let's do AI and they start at the end I think what what the telecom I think what is was it Yan right what the unjust said was that they started directly with natural language processing in order to make a bot talk to customers which is pretty much a pretty-pretty yeah that's a big problem at the end of what you can do so I think the first thing is to stop counting we use counting is very easy and you can just count I show this how many people click I show that how many people click where was it shown was it shown in this certain spot in all these places where you show something they have an ID and you can identify I mean you see okay all always this idea it never nobody ever clicks so you can you can start blacklisting which means I don't want
to show any more on this ID so obviously that is going to be millions of IDs that you need to review and you can use a very simple statistic and you just count and you say ok this is a good idea this is a bad ID in general in average and you optimize for that or you say oh I have for the same thing I have two banners like the red one and the blue one and I see which one performs better and you start the a/b testing which is the natural thing which is stage one which is very easy you don't need any PhD student nothing you can just use any person that has counted before and you can you can do that problem is you just look at one dimension so if you say if you look for example optimize for the price and you buy cheaper you typically the cheaper spots are also not in the nice places so maybe they're not even visible like maybe they are below the fold like
when the page appears or something or you a be test two banners and you find out that the one with a naked lady or the one with the win and I pad performs super good everybody clicks on it but obviously it has nothing to do with your product so you're just optimizing one dimension and this is the the biggest problem also in other industries when you go for that you you typically tackle only one dimension also if you do it manually if you let people do the work typically people optimize only for one damage they can say I look for the largest or best in this aspect but combining this banner at that time for this person that is pretty difficult but it has been researched centuries ago and you can use I think this thing yeah you can you can use machine learning and you learn on multiple various at the same time so machine learning is specialized for that and that is the first time when you really start getting some engineers in
the company building some databases and thinking what are the interesting variables because pretty much all of the time you have too many variables also like if you shown at I we receive I don't know thousand or 10,000 different variables like screen size operating system how many other apps the geolocation like whatever you want and with some of the data you can get additional data if you have the IP address and the time you can get the place and with the place and the time you can get the weather and you can you can pretty much use whatever you want and so it's important to learn which features are relevant so if you know the business you have been in it for 10 years you pretty much have a good guess and be like ok that's not important or this is important but there's also ways to do that automatically which is automatic feature learning methods and they should be applied at this stage it's a very good stage it's a stage as fun and that
has a really good impact on on revenue and also on scalability examples are all these recommenders that you know like you go on Amazon you buy something or even you login of course the system recognizes you and then gives you a recommendation what are the top 10 things that you should look at based on your selling history the problem is always the same you have you're asking a computer system to do a recommendation what will be the most likely thing that you buy the most likely banner that you click on the most likely insurance that will you will purchase and so on plus the next thing is you can provide this as a service as a company you can just say I have a recommender and I create a an API for that that means other people can connect to it so that means you do the work of recommending you observe and recommend and others say I have this offer wall on my cat page and I would like to know what I should
show on this cat page and you asked the computer and then the computer tells you which ads should be shown and that's also what you can buy off-the-shelf from Google like if you have Google advertisement on your on your web page it's pretty much your page ask Google what they should show they they put something on it and there's real time by programmatic buying which is a huge share of the market of how currently advertisements are being bought maybe if you don't know it it's it's an interesting this is actually like a stock exchange you have to imagine where well if you would all turn on your phones now and there would be let's say we all open a page where there's an advertisement you probably would be auctioned off in a on some sort of exchange where for each of you there would be like an ad request saying okay it's an iPhone it's an Android which version and so on and then other people can bid and say okay I bit one cent
to show him and I with two cents to show her an ad I and so on and that happens in real time so within like 50 to 200 milliseconds computers decide on if they want to buy an ad or not let us currently of how today the advertisement works that requires super huge amounts of data it requires machine learning it real-time everything you have only 20 milliseconds or so so sophisticated but to me it's not AI that is engineering it's machine learning maybe but it's not AI so I think the next thing is I have maybe maybe one thing on this thing what youtube or what I typically do or did wrong in the past and what I see others doing wrong is you model the reality like and then ideal world you know how it should be and then you let your algorithm run and then you realize that actually the world is different and then you try to find what is broken and one thing is for example that you have these Chinese
or Russian guys or also maybe Germans I don't know but that have these huge racks of phone and they don't do anything else but clicking adds the whole day like you can program them to just click and download ads and that's when you use pattern recognition because you don't want to pay for that you don't want to have that in your traffic so I think our company and we throw away 30 to 40% of traffic because it's just the empty spots that we we don't use because we consider it bad another problem with this recommenders you know the YouTube thing but it's the same on Facebook is that you get more of the same if you're a dominant Donald Trump fan for example you would think that everybody's a Donald Trump fan because once you liked it once the system thinks you will like it always and give you more of that because you're more likely to click you're more likely to like it you're more likely to share it so it's it is it
creates the problem that it puts us all in these bubbles and that's also the same for advertising and that's where we're far like AI is a completely different topic but there we we're not even able to after I advertise the the fridge that you always want it to you and you bought it you still for two weeks get the advertisement for fridges and you're like dude I just bought the fridge so why do I still get advertisement so there's a very basic promise that need to be solved anyways now III two and a half more minutes now III this is this is where we want to go and what does it mean so for me I is something where the problem is so complex or the behavior is so natural that it becomes human like or even over humanely I don't know what the word is for that that means to me a system needs to be able to observe and learn and to detect some patterns and then act on it which is pretty
much what a human would do and for our a thing it would be really I observe a lot of bad traffic coming from the source and the system texts it is sure and creates some action and maybe even sense the email to the the traffic source or notifies the the related account managers and acts like a human like like a real responsible person would do that isn't the place of managing this account or this traffic source and the next thing is and we heard it before bots yun talked about it from telecom and the other room was also talk about bots what is about a good part for me is i would like to have my account managers talk to my system but I don't need the natural language processing and that it wishes them a good day and so on it's really for me it's a tool that should do things that are difficult and that I want to do easier it's not like the biggest problem of Alexa is that you don't have
a use case like I pretty much asked her how the weather's and then I'm done with her so but you could really make more complex questions which normally a person would need to create an Excel and a pivot create a pivot table and and and work half an hour to answer a question but if you could put that questions in like what is the potential change in the revenues of hotel abroad of our top ten offers in the last month that can be understood that could be translated by the system into the corresponding database query and create a result that you know replaces half-an-hour of a bi person's work or if the system like before observed and acted it would just say ID deactivated the following offers because I believe something is wrong and I notified the partners and it just tells you that or writes you an email or calls you yeah and when we would get there and I'm done with my time I think the components are all there so you have
the components of natural language processing you can do pattern recognition you could also use image recognition in other fields and the point is to me more what do I want and where does it really add value and like this adds to me value but for example the the typical talking bot of my bank or insurance I don't like it because if I call my insurance I have a real problem that's so complicated that the five fa Q's didn't answer it and so that is when I the last person I want to talk to is a bot but like if I'm like ah now I now I have to create this complicated excellent get all the data that's what I would like to talk to a bot and be like dude get me the answer to this and this question and please know how we do it and that's that's it from my side unfortunately yeah I don't know if I if there's a opportunity for questions maybe one question thank you very much already