hello everybody good afternoon so I'm here today to talk to you a little bit about an oddly named but increasing in prominence type of machine learning or really improvement to machine learning called common sense if you've spent any time hanging out in the futures room in the other room you'll hear this Maureen that you know we've been we've been doing a lot with AI right it's it's really the time for AI now but there's a lot of issues with AI that are making it difficult a lot of people have talked about needing a ton of data to work with ai ai being a black box that's not explainable and you know AI not generalizing not being able to transfer knowledge across different things so what I'm gonna do today is talk a little bit about research that's been going on for a good number of years on how to try to fix that and bring it into the modern era of machine learning and then bring it into the enterprise and talk a little bit
about how it's actually changing everything from how warehouses work to how customer service works so I'm a computational linguist by training so this talk tends to skew if left to its own devices tends to skew a little bit more towards language than to other sorts of things and so a lot of my examples are in language and I do want to make it a little bit more broadly applicable but I'm very passionate about helping computers understand language more like a person and less like a machine so in that note and in the same way that that artificial intelligence has changed everything recently natural language processing has changed a lot as well so people probably remember and think about natural language processing they think about building rules and ontology Zand spending large amounts of time defining synonyms of things or a large amounts of training data to try to build probabilistic models and Allah that's changed very recently I think a lot of people here have talked about deep learning don't think anyone's really tried to
define it for people who you know maybe maybe haven't worked with it directly but we can really think about it as trying to understand and find high-level patterns and datasets and figure out how to extend those patterns outward so if you really don't know the technical side of it that's absolutely fine you can use this as sort of a working model and go from there so when people do deep learning on language they often work with a type of technology called a word embedding which is the idea that words can be almost these multi-dimensional representations in space and that's been something we've been doing for a long time and language it really became popular with Google's word Tyvek in 2013 but there's also a whole bunch of other packages out there Stanford's glom fast text etc that that use word embeddings so you know my open data project puts out one perplexingly called concept net number batch as well but embeddings are really what you're doing if you want to be doing word level natural
language processing over text and the problem that's come up with these sorts of things is that really deep learning needs a lot of data and I don't need to go into a large amount of depth on this because we've been talking about this all day training one of these systems takes hundreds of millions or billions of data points and that's a big issue and so there's often if you're trying to do many things you don't have that much data you couldn't fall back to the common crawl which is the web as a whole and that provides a lot of issues around things like Oh detecting let's say we're trying to detect faults in a factory system you know imagenet things like that are really interested in being able to figure out where cats are not where cracks are so that becomes a big issue same thing with word avec which is trained on the web as a whole and doesn't know that much about pharmacovigilance and so in places where we run into those kinds
of data issues people have really been trying to manufacture data and one of the big places we see that is in some of the big AI results so in order to be a human player at chess alphago had to play about 68 million games against itself and and so sort of the issue here is that it and everything it learned from doing go it can't take it and use to learn how to do chess right these things don't generalize that's a big issue so and that's especially an issue in the enterprise so you can't actually simulate text data so to make an intelligent agent to do translation to do all these kinds of things you can't simulate text data because if you had a simulation for text data you wouldn't need a simulation for text data so it's kind of this chicken and egg problem right you can't make up your call center calling itself sixty eight million times it just doesn't work that's just because language is special and different and when you manufacture
data for language you end up in situations where these all these interesting little tells and the computer will start picking up and working on those and when you put it out into the real world it doesn't work another place that's really like that is robotics so I don't know if people here are familiar with the problem that's called robotic picking at least in in the US it is which is probably most people think about the Amazon challenge that's out there right now around this where you have a robot it's in a warehouse it needs to pick up a can of soup and put it in a box and and give that box to a person who's gonna put a label on it and send it to you right this problem is really really hard right now and so in trying to solve this problem with deep learning a lot of companies have done things like I got enough arms full of robotic arms and farm and big piles of cannons and they've just had the
robots pick up the cans over and over and over again they find that they still can't generate enough data to actually generally solve the problem because of the lack of generalization in that kind of stuff so the reason why deep learnings has to have this problem and people stand on stage and talk about it being a black box that means infinite quantities of data is because really it needs to learn everything from scratch every time it does that so you don't do that I don't do that that's not how people work right and in order for us to progress from sort of these very specific how can we identify the tomatoes sorts of artificial intelligence to even more general ái that can be used across multiple different kinds of calls all that kind of stuff we really need to find some way to generalize and people do it all the time right you know we learn kids learning by who has kids know that kids learn from a very small number of examples and that
we learn exactly as one of the earlier speakers said we learned how not to do something from a very tiny number of examples and sometimes from no examples at all just from simulating in our head so the goal of what the type of machine learning that I do is to try to fix that and try to understand that and so first we have to sort of stop by saying how do people get so we generalize from knowing examples if I had if you had never seen a giraffe before and I asked you what is this you would probably look at and you do a couple of different things you'd sort of see the fuzz on the giraffe and you'd look at its eyes and that kind of stuff and you'd say ok it's probably an animal it you might be able to say it's a hoofed animal if you look at the feet things like that you can make generalizations by looking at examples you have and looking at properties the object has and when
we're looking at much more complicated concepts we tend to understand those concepts by making analogies to simpler concepts if you think about how people talk about things like water we make sorry if you think about how people talk about things like data and information we make analogies to water we talk about flow we talk about data like so we talk about all that kind of stuff and that's something that's actually really common in language we make analogies difficult we learn difficult concepts by making analogies to simpler concepts and those two things together taking knowledge that we know about the world in general and then using that base knowledge to be able to solve more complicated tasks is a type of machine learning called common sense reasoning it's not really a type of machine learning it's an addition to it the idea that you can take knowledge some kind of base knowledge base and add that base knowledge base even if it's about really general stuff into a machine learning system and that makes the machine
learning system more able to generalize and a little bit smarter so you know besides having a funny name never like to say at conferences that I did common sense I've been working in it for for almost 20 years now and when it was mostly not popular for most of that but recently it's been getting a lot of press by some of the stuff that's and results that we've been getting I'll go through some of that also Paul Allen who was co-founder of Microsoft gave a lot of money to the Allen Institute out in the Seattle to study common sense specifically because he believes it's the key to unlocking the next revolution in AI and what we really need to be able to do is take this knowledge that everybody has add it back into the Machine and then that makes the Machine a lot more flexible to be able to generalize to be able to jump to different things now of course people who are more technical in the audience are probably thinking oh you
need to add transfer learning on top of this yes but that's a whole different talk and I only have you know 20 minutes that I'm never going to be able to cover both common sense and transfer learning but I'm happy to talk later about transfer learning but the fact is that by building spaces that our understandings are more robust so they don't have to have as much data and they don't have to make the kind of same kinds of conclusions so to start to go back let's go back to the robotics situation so there are all these people competing for robotic picking and they're all got robots who are trying to generate data and it's not working so researchers at Princeton and Stanford started a project called Sheep net where they basically built models about small objects so those objects can be things like chairs or planes or cans or other sorts of things and they took that down and they infused it into their machine learning system that was working with this robotic process
and interestingly enough they started finding that their robots were much better picking things up when much less training data so it's the same kind of principle right it's the principle that even though the robot is picking up things that are not airplanes and not cars and things like that learning about the way the world works and having some basis with which to learn additional material speeds it up and makes it more flexible so at the City University of London going back to the game example they said well you know we're never going to teach a computer play chess that's not what we do here but we want to generalize it so we're going to teach a computer very basic ideas about games and then we're going to try to learn a more complicated game and that very much sped up by 5x the number of simulation sessions they needed to do to learn a new game and so this is new research that that just came out as well both of these either came out
in the last couple of years last year we got this this is actually Cambridge University but it's a guy who now has to spin out doing something different than this from Cambridge University but it's basically taking a look at adding opposites and similarities and how adding that information initially increases the ability for even a simple neural net to be able to do something like sentiment analysis so I run something called the common sense computing initiative which started in the late 90s using crowdsourcing to build what is the largest knowledge graph open knowledge graph so we're the first people to use crowdsourcing for AI and our idea was sort of hi how about we punish the powerboard people on the internet and make them help us train any eyes so we started building a knowledge base we work together with Oh a ton of people all over the world different universities and Japan and Korea and the Netherlands all different places to build knowledge graphs in these different languages and interlink them it's available online it's
concept net that IO these days are just Google concept net and we build in beddings and knowledge graphs that people can use and that's what concept net does we've spun out a company Goodin number of years ago in 2011 to be able to use transfer learning on top of that to help companies automate and optimize customer service using a data I'm not gonna sell you a data scarce environment because it can work there but in an enterprise data environment not in a tech company environment and so we've seen we've been doing a lot of work both with the common sense infused we're embedding technology and also both in the company and outside of the company benchmarking against a regular natural language so if you're interested in going and finding concept net everything we do works in German it works in ever European language I think actually cuts of net runs in every language that Wikipedia runs and that's sort of our bar and also as an open data project or a pre we left the
Wikimedia Foundation so we talk to them and that's kind of where our bar is if we add that kind of information if we add concept net into a word embeddings technology we find that we get really great results I'm not pulling up anything that is at least yet this is last year's seminal which is the ACL or the Association for computational linguistics benchmark and we're at the top right and so what we're doing here is we're adding in information knowledge graph information into a distributional semantics and betting system and using that to figure out the canonical tasks which is not that exciting that's used to benchmark languages analogies and how similar words are to each other and here we're doing word similarity in many different languages including German and really outperforming just native where embeddings technology and I can tell you that and here's this across languages so this is the thing where you have one word in one language and one more in another language and you have to say how similar they are
we also did that and that's also adding in the knowledge to really augment and scale up the deep learning in a way that that works really well some of our results for this year haven't come out yet you can say they're really interesting not just in our task but in other tasks in adding knowledge to to machine learning systems to really make a difference so keep watching this space this is only accelerating so what you do once you build something like that is you use a type of technology called transfer learning so the idea is I have a great general space that's fantastic I do pharmacovigilance I'm in financial services I do something really difficult how on earth can I the data is not out there I need to move this model and to mind of it and so transfer learning works best in models that are augmented with with knowledge it's the ability to take a model that was trained on a large amount of data take your tiny amount of data it's many
multiples less you know a couple thousand data points and move it into that domain and get the same kind of accuracy we're talking about a couple slides back so it looks like that and that's really the power of common sense is that it allows you to harness that to be able to move into these other domains a couple quick case studies although I'm running a little bit low on time let me pick which one I'm gonna do this one because it's a lot of fun so we work with supercell who is mobile gaming company some people here probably play clash of clans some people here probably play clash of clans way too much and if you've ever submitted a ticket through supercell you've interacted with what we do basically you can get the tickets coming in we look at that ticket we say and we have to use common sense and I'll explain that actually far explain what happens because the language that's used is very specific any time in mobile gaming and pretty much
in any industry there is so many words that are made up in meanings that are made up and I don't I'm not really somebody who plays a lot of mobile games um so I'm gonna use an example from web interface design which is that a hamburger is English slang for the menu bar in your mobile device and a carousel is English slang for the thing that that you move in the mobile device to sort of move back and forth this way or that way when you're swiping and both of those words have very different meanings anywhere else other than in mobile design and you need to be able to learn that kind of stuff and so gaming is like that a hundredfold right because people are making up words woodcutters are doing things they shouldn't be doing you know you can buy a bison all of these things right and in mobile gaming and so you need to be able to dynamically as they release new parts of the game learn and understand this new
stuff and so their goal is to take this and use it to optimize and automate customer service being able to tag who can go to self-help this isn't a chat bot this is hey how about you go to the part of the website that lets you reframe your password it's all that kind of stuff right and then if that's not what it is make sure it gets to the right it's tagged so that the routing system can send it to the right person and then with the stuff that we don't know what to do with like the stuff there's no business workflow for it make sure that we find all the problems that are emerging in that so that the folks who are writing the game can fix the bugs as fast as possible and also can prioritize the bugs to see which ones are going to cause them cause their KPI is the biggest issues if they don't fix them so that's the kind of a way that we see this actually working in
the enterprise outside of robotic picking as I said there's a lot more going on right now on the horizon for this kind of technology a lot of if you look at what's going to come out in the big conferences this year that are coming up a lot of people are using knowledge of one kind or another things like shape net things like what's being done at Cambridge things like what we're doing to augment and to scale and eventually something that we're working on has to peel away the black box nature of this kind of machine learning but this is the hopefully what we want to be able to do to both so I'm very interested in AI so I really want us to get to the next level I'm very interested in all that but in the same sense I really wanted to actually make a difference and I think they're both unlocked with the same key right I think we can work towards making a work towards really augmenting artificial intelligence by giving it
flexibility but I also think that allows it to actually work in the enterprise because without flexibility it becomes really hard to deploy and that becomes a big issue so we can we can get both things at the same time and that's really what my goal is so I'm pretty much out of time here I know we're about to have a coffee break I think so I'm gonna stick around for questions I'm English only but probably my colleague is going to join if you want to have a challenger all right thank you