Rise of AI Conference 2018 · Berlin

AI as platform

Ramin Assadollahi · CEO at ExB

At Rise of AI 2018 in Berlin, Ramin Assadollahi, CEO of ExB, argues that AI should be built as a cross-industry platform rather than as single-purpose solutions. He recounts inventing the next-word prediction technology used in mobile keyboards, sold to Nokia and scaled to 72 languages on Symbian Series 60 devices, and describes how ExB now applies one platform across health, mobility and finance, processing text, images and structured data in the cloud, on phones and offline in cars. He sets out three goals: reducing the cost per AI experiment, building data loops so systems keep learning from the errors a workflow corrects, and combining data modalities to gain performance and quality. Examples include extracting disease-related claims from medical publications, building a knowledge graph of automotive parts from technical documents, and a project for a German car manufacturer that predicts user behavior and answers questions about a car by mapping its internal states to language, which he calls referential semantics. He also describes closing data loops between vehicle design and after-sales repair reports, and warns that entity recognizers trained on one domain fail on another.

Key takeaways

  • Assadollahi says he invented the next-word prediction technology used in most phones, sold to Nokia's Symbian Series 60.
  • ExB runs one platform across health, mobility and finance, processing structured data, images and text in the cloud and on devices.
  • Combining document layout analysis with text analysis helped ExB win a hackathon on extracting information from documents.
  • Mapping a car's internal state to language creates referential semantics, letting a chatbot answer questions grounded in the vehicle's own condition.
  • A named-entity recognizer trained on one domain, such as news, performs poorly on another, such as medical text, so systems need domain adaptation.

Detailed notes

From predictive text to a cross-industry AI platform

Assadollahi opens by explaining that he invented the next-word prediction technology found in mobile keyboards, which he sold to Nokia; it became part of the Symbian Series 60 operating system and was scaled to 72 languages on-device. Building on that background, his company ExB now works along three axes, health, mobility and finance, with a single platform that understands structured data, images and text, and that runs in the cloud as well as on phones and offline in cars.

Three goals: cost per experiment, data loops, multimodality

He frames the talk around three points he wants the audience to take home. First, companies need to reduce the cost per AI experiment so they can prioritize which use cases to pursue. Second, whenever a workflow is replaced by AI, a data loop should be built in so that the corrections users make feed back into the system and let it keep learning. Third, combining data modalities such as text and image yields performance and quality gains, which he illustrates with a system that reads medical publications and identifies that aluminium chloride in antiperspirants is linked to learning and memory dysfunction classified as an Alzheimer's-like disease.

Knowledge graphs from text, images and structured data

ExB extracts entities and relations from unstructured and structured data alike to build a knowledge graph, for example reading automotive documents to identify that braking components are connected by force. Combining a deep-learning model for document layout with text analysis let the company win a hackathon by extracting information both from document text and from its position on the page.

Referential semantics: cars that talk about themselves

For a German car manufacturer, ExB built a system that analyzes structured data from the car or smartphone to predict what a user will do next, and a separate question-answering system that explains the car's functionalities. Combining the two lets the car answer not only text-based questions but also questions about its own state, such as which warning light is on, because the car's internal state is mapped to language; Assadollahi calls this referential semantics and compares it to embodiment in robotics. He argues this is what current chatbots and voice assistants such as Siri or Cortana lack, since they have no access to the car's state.

Domain adaptation and closing the data loop

He warns that a company-name recognizer trained on one type of text performs badly on medical text and vice versa, so systems must be adapted to a specific domain, and lowering that adaptation cost matters. He describes two data loops from car manufacturing: a predictive model trained on a pre-production cohort is rolled out to individual cars and adapted per user, with usage data fed back into the system; separately, engineering descriptions used to design car parts must be mapped to the language of after-sales garage repair reports so that design and after-sales are connected. He closes by noting that machine learning is only one layer of a sustainable AI platform, which also needs data evaluation and model versioning, and that reusing document-understanding resources across use cases lowers the overall cost of scaling AI within a company.

Quotes

This kind of next word prediction was invented by me, so it's in most of the phones today.

Ramin Assadollahi

Whenever you combine data modalities like text and image, then you will get performance and quality gains.

Ramin Assadollahi

If you have a robot that has sensor data and can reason about the outer world, this is something that is approaching more towards a real artificial intelligence.

Ramin Assadollahi

Questions answered in this talk

Who invented the next-word prediction technology used in mobile phone keyboards?

According to Ramin Assadollahi at Rise of AI 2018, he invented this next-word prediction technology himself and sold it to Nokia, where it became part of the Symbian Series 60 operating system and was scaled to 72 languages on-device.

What is a data loop in an AI platform?

According to Ramin Assadollahi at Rise of AI 2018, a data loop means that when AI replaces a workflow, the corrections and learnings from that process are fed back into the system so the engine keeps learning while it is being implemented.

How can a car's chatbot talk about its own state, such as a warning light?

According to Ramin Assadollahi at Rise of AI 2018, a car's structured internal state data can be mapped into a knowledge graph together with language, so the car can answer questions like which light is on; he calls this referential semantics.

Mentioned

next-word prediction, knowledge graph, multimodality, referential semantics, embodiment, data loop, domain adaptation, predictive interfaces

Keep exploring

More from Ramin Assadollahi

More from Rise of AI Conference 2018

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 2,737 words

Show the full transcript

good afternoon just a little bit of history about myself if you pull out your phone today and you have messaging service then you write a text you have a text field up there and your keyboard and in between you have something like hey hello good and you tap the word good then you get the next word day morning evening and you tap the word evening then you wrote two words with two taps instead of typing keep the keyboard this kind of next word prediction was invented by me so it's in most of the phones today we sold the technology to Nokia we were a part of the Symbian series 60 operation system and we scaled the technology to 72 languages on device what are we doing today we are active along the three different axes namely health mobility and Finance so that we are working really cross industry with one single platform we understand structured data images as well as text and our system runs in the cloud as well as on phones or in

cars offline and so from this perspective I want to give you some insights about how you can use multi modality across different industries to really apply AI so let me get straight to the end of the talk what I want to have you taking home as a message is the three topics I'm concerned with today namely how can we reduce the cost per se I experiment we are really in touch with many companies here in Germany really bringing AI to their industries their problem is how can I prioritize my AI experience to make something more efficient within the corporation and how do I understand how much money I can spend to get what kind of result so this kind of person prioritisation really implies the cost per experiment that I do and we plan to reduce the cost per experiment second thing is whenever you replace a workflow within a company then ideally you also introduce a data loop so you really want to take all of the learnings that a process makes and the

errors that you correct really feed back into the engine so that you basically can get the engine learning while you implement it that in your company and the third thing is our promise is that whenever you combine data modalities like text and image then you will get performance and quality gains so what we do is we understand text very deeply for example here you see how we understand medical publications so the machine was understanding that laser is really understanding that aluminium chloride which you have in your antiperspirants is inducing learning and memory dysfunctions which is classified as disease like Alzheimer's disease so it's really understanding a pretty complex sentence in a publication we can do the same thing across images building relations across stuff that you extract it from images or even structured data like canvas data or movement data from your smartphone whenever we extract these entities and their relations from the structured or unstructured data we can build a so-called knowledge graph to understand what the world that the data is actually reflecting

is keeping up and so reading text across for example automotive documents you see here all of that is basically moving forward components in a motor these are the braking components and it automatically extracted force as thing that is actually connecting these two and that's kind of an emerging knowledge that you gain from reading lots and lots of texts one of the things that we really found out that worked very well is whenever you actually combine two technologies so we are a text company by Nature but we also invested in understanding images so one of the things that we did is we combined a deep learning algorithm for understanding the layout information of documents and combine that with the text analysis to really win the aliens hackathon last year to extract information not only from the text that is in the documents but also the position within the document so that's one example of multi-modality when it comes to cars then one of the things that we see in the user interface is that the chat

BOTS or Cortana's that you have in the car become smarter and smarter they really establish one of the values for a car whenever you're sitting in when it's a self-driving car then most of the interaction that you will have in the future will be based on a dialogue system so one of the things that we see is that basically this interaction has to get more meaningful we have heard it via Chris Paul's and Peter just on the mainstage that the chat pods today are not very intelligent they don't know what they are talking about it's more or less something like a reflex where people write text that you can talk about but really the chat is not understanding what it is talking about so that in my eyes will be one of the next steps when you look at for example Siri and Cortana they are an isolated system they don't have access to the cars States or the data of the car so they cannot establish many of these semantics kind of data sources

and talk about that and so if you have a convergent representational framework then you can bring these two together how does that work basically this is what we've built for a German car manufacturer you can analyze the structured data within a car or the smartphone you can predict what the user will do next with in the car or with their smartphone we had these predictive interfaces for years in in smartphones we have built a general framework for understanding what will happen next in the user interface a second thing that we've built is a question-answering system that actually can explain the functionalities of the car so these are two different projects but if you combine these two data modalities namely structured data as well as text and bring them into correspondence within for example a knowledge graph then you have this kind of multi modality where actually you cannot only ask the car what is that lamp with a gas station on it that would be a text-based question but you could also ask what light

is lighting up there so the light that is lighting up is not referencing to something that the text-based system can understand but the car itself has kind of a knowledge about its States for example that it's not on a running low on gas so this kind of state can be mapped to something linguistic and suddenly the car can talk about itself and linguistics we call this referential semantics and this is basically the key critique that people on the mainstage had that a chatbot cannot talk about something that is happening in the real world in robotics we would call this embodiment so that you if you have a robot that has sensor data and can reason about the outer world this is something that is approaching more towards a real artificial intelligence so what we are building here with the predictive layer analyzing the states in the description structural description of a car and mapping this to the linguistics that is I think the way towards building chatbots that actually understand what they are talking about

a second thing in the automotive industry is you not only have the canvas data but you have the user behavior you have external context like all of the sensors that are looking out of the car plus you have the different interaction schemes that you can have with the user and the car manufacturers right now they are trying to source different innovations which are predictive or AI based but whenever you see the amount of data that you get and can get here in terms of dimensionality how many different factors of the outer world can I actually use to predict the user or be assistive and combine that with what you actually want to articulate on what kind of interface like talking or showing it or switching on something then you have a combinatorial explosion so all of these experiments that you can do basically they cost you something and you want to bring down the cost of generating a functionality based on AI so if you have a convergent knowledge graph that is combining this kind

of structured data as well as the different user interfaces then you get to a point where you easily can create more and more value based on one single platform so that's also one of the main points I think we have seen that in the talk before yes we know that chat BOTS work and they are valuable but scaling them towards really covering more and more knowledge about something is one of the things that that is really a matter of today so it's not really bringing the chat bot into a value proposition but really scaling it and mapping it to the reality what people are actually talking about and the second thing is how do I create active components that can predict or be assistive so we make all of these examples here which are real studies about data that we see in the world for example that food and music actually really correlate so bringing this kind of predictive analytics to a platform like a car will be actually pretty expensive if you have the

idea ok I'm looking at your Spotify data for example can I predict from that whether it's you're more a fan of Italian food or Japanese food so you wouldn't dare as a designer to invest in a POC or something like that to actually get this result but if the cost of an experiment to do so because you have a convergent network is very low then you can simply try that out on your crowd-sourced data yeah the second thing is whenever we are building systems and we have that through the day also with luminoso that you have to adapt the knowledge that you can extract for a domain let's say automotive the the cost for training it to the context that you are talking about a car or a hospital hospital or an insurance and so on this kind of cost has to get down because you have to adapt to the domain so we did an experiment here whenever you have a company recognizer in text you want to understand this is a company in

text like Apple or IBM or something like that and you train that on use then it's really bad on medical text and vice-versa so if a company is selling you a general kind of recognizer for companies it's probably not going to work on your text although it's generally really of high value so what you need is a a system that can adapt to exactly your needs your application scenario your domain or your industry and bringing down this cost is also one thing that is very important when you really bring a eye towards an industry or an application scenario exactly so talking about data loops we think and that's also our experience with car manufacturers here in Germany whenever you have a predictive model then usually you have something that you're using from a cohort of cars that are in pre-production they produce data then you can fit a computational model to do something predict a user behavior which is here in the middle then you can basically roll that out to a single car and

it will apply to a single user it will adapt to one of the groups that you actually have established through the initial data set and then whenever it adapts to a single user you can crowdsource that back to the system so that you have an iterative way of learning from your audience through sourcing the data as well as fitting single user contexts second data loop that is also from the car manufacturer whenever you actually design a part of the car then you have this kind of forum where you say okay the functionality of this part is such it's connected to these parts as well as whenever it breaks down what is the probability and how do I fix it this is a kind of engineering description of car parts that is used for constructing cars before they actually go out on the market but then when they are on the market you have these kind of procedures that you use for actually repairing the cars and basically you get garage report afterwards so that somebody

is writing okay I did this in that Diagnostics I found this and that is broken and I exchange this in that right and mapping these two languages together is a very important part when you're closing the data loop so you have designed the car basically you get it out on the field and it gets repaired and naturally you want to understand what kind of parts get broken through what kind of let's say environmental factors or heat or something like that and you want to bring back this kind of information into your design process so after sales and design will be connected but in fact these two groups really speak different languages so you need kind of a machine translation between the engineer and the guy in the garage that is producing their reports finally whenever you have an AI platform in your company and you used it to do a couple of different experiments there and you have established a couple of workflows that really work then our experience is whenever you have invested in

here to understand a type of document that is describing your workflow then you can reuse these resources for a different use case and whenever you reuse your use cases ideally even combined with the data loop then the representation of what is going on in your company will get sharpened and you can extend that to more and more use cases or workflows within your company so you're bringing down basically the overall overall cost when you're establishing different AI parts within your company so that's I think an important part when it comes to bringing AI to corporations we see that here in Germany that we have many industry companies that want to establish chief digital officer and he wants to establish AI and how to grow really this kind of potential that you have from a tensorflow workflow and deep learning whatever kind of single solution to really mature AI into your corporation that's I think a very important next step in the industry and so we think this kind of a multimodal platform is very helpful

so when you're looking at machine learning that's just one of the pieces here then you have this kind of interactive training or closing the data loop plus ideally which is very often not the case people don't actually evaluate their data so whenever you have an established procedure of correcting something then your problem is you don't actually know what you're in initial quality of your algorithm is then you establish some kind of machine learning but you don't know what the difference is how much better did I actually get but you need that whenever you're actually establishing a data loop because you want to see how your overall performance is increasing so it's not about you know getting tens of flow and doing some kind of deep learning you need the whole layer here of working with your machine learning models doing a versioning with them having different histories and so on to actually really build a sustainable AI module in your company this is what our system looks like you can basically visually combine different modules

you can analyze documents and structured data as well as medical data and structured data just as a little example and a couple of hackathons and challenges we formed and I still have 24 seconds thank you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.