Rise of AI Conference 2019 · Berlin · Unicorn Stage

What’s Next for AI

Florian Kuhlmann · CTO & Founder at Leverton

At Rise of AI 2019 in Berlin, Florian Kuhlmann, CTO and founder of Leverton, argues that today's narrow AI needs to take 'baby steps' toward more general capabilities before it can handle tasks that require reasoning and context. He describes Leverton's growth from two people to 90 across four offices, extracting information from legal and corporate documents with deep learning models trained daily in production. Using a hypothetical example of identifying twin brothers from a photo, and a real Leverton case of a two-column document with broken indentation, he shows that current deep learning excels only at single, narrow tasks and cannot combine visual, textual and semantic information to resolve ambiguity without heavy manual heuristics. He points to multitask learning, multimodal learning and differentiable programming as research directions that might narrow the gap toward artificial general intelligence, while stressing that businesses cannot wait for that goal. In the Q&A he discusses how Leverton stores extracted knowledge and says full natural-language understanding by machines remains far off.

Key takeaways

  • Kuhlmann says Leverton grew from two people to 90 across four offices extracting information from legal and corporate documents.
  • AI today excels only at narrow, single tasks and cannot combine reasoning across them, shown with a twin-identification example.
  • A real Leverton document example shows layout detection fails on a two-column format without feedback from a later semantic step.
  • He points to multitask learning, multimodal learning and Yann LeCun's differentiable programming as possible next steps toward general AI.
  • In Q&A he says Leverton stores extracted knowledge mainly in normal or graph databases, and doubts full natural-language understanding is near.

Detailed notes

Leverton's narrow-AI production experience

Kuhlmann says Leverton grew over seven years from two people to 90, across four offices with headquarters in Berlin, building a software-as-a-service application that extracts information from legal and corporate documents into structured databases such as ERP systems, semi-automatically with a review process for complex cases. The company has used deep learning for about six years, for OCR, document classification and information extraction, training thousands of models. He describes today's AI as excellent at narrow, single tasks such as classifying a document, finding a person in a picture or recognizing traffic signs, sometimes reaching superhuman performance, but unable to do anything beyond what it was specifically trained for.

The twins example: why narrow AI can't reason

To illustrate the gap toward artificial general intelligence, Kuhlmann uses the example of identifying two twin brothers in a photo. A face-recognition model alone cannot tell them apart; combining it with OCR to read a name on a t-shirt helps identify one twin, but concluding the other twin's identity requires reasoning that they are brothers, which current systems cannot do without extensive hand-built heuristics chaining several models together.

A real document that breaks layout detection

In an actual Leverton case, a lawyer-drafted document has broken indentation that, on inspection, turns out to be a two-column layout rather than a formatting error. Detecting this from visual features alone is not possible; it requires feeding back information from a later, semantic processing step to an earlier layout-detection step, which Kuhlmann says again demands complex heuristics for what is still just one edge case among many.

Possible next steps toward general AI

Kuhlmann outlines research directions Leverton is watching: multitask learning, where a network learns several tasks at once and each benefits from the others, though it remains narrow; multimodal learning, combining inputs such as image, text and audio, which needs well-aligned training data that is hard to produce; and differentiable programming, an approach he associates with Yann LeCun, in which the heuristics and code wrapped around neural networks become themselves learnable and differentiable. He notes tools like TensorFlow and PyTorch come close but do not yet fully support this. His closing thought is that turning hand-built heuristics into trainable neural-network components tends to improve performance, and doing this systematically might move AI closer to general intelligence.

Q&A highlights

Asked how extracted knowledge is stored, Kuhlmann says Leverton mostly uses normal databases, with some experience using ontology-based graph databases, but says reaching artificial general intelligence would likely require storing knowledge inside neural network embeddings rather than structured databases. Asked what is fully automatable today in document extraction, he says accuracy depends on how similar documents are to each other: near-total automation is possible for very similar documents, but complex, cross-referencing document series such as a lease with amendments and renewals still need careful review. Asked whether machines will achieve real natural-language understanding in the next few years, he says no, arguing that basic information extraction is possible but full understanding, including grammar and sentence dependencies, remains far away.

Quotes

AI of today is very good at very narrow tasks, very singular tasks dedicated to one specific workflow.

Florian Kuhlmann

We reach sometimes superhuman performance.

Florian Kuhlmann

Questions answered in this talk

Why can't today's AI reason about context, like identifying twins in a photo?

According to Florian Kuhlmann at Rise of AI 2019, narrow AI models each solve one isolated task, such as face recognition or OCR, and chaining them cannot supply the reasoning (for example, knowing two people are brothers) needed to resolve ambiguity without extensive manual heuristics.

What database does Leverton use to store information extracted from documents?

According to Florian Kuhlmann at Rise of AI 2019, Leverton mainly uses normal databases, with some experience using ontology-based graph databases, though he says a shift toward artificial general intelligence would likely require storing knowledge in neural network embeddings instead.

What are possible next steps toward artificial general intelligence?

According to Florian Kuhlmann at Rise of AI 2019, promising directions include multitask learning, multimodal learning combining image, text and audio, and differentiable programming, an approach associated with Yann LeCun, that makes the heuristics around neural networks themselves trainable.

Mentioned

deep learning, OCR, multitask learning, multimodal learning, differentiable programming, artificial general intelligence, narrow AI

Keep exploring

More from Rise of AI Conference 2019

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 2,594 words

Show the full transcript

yeah my name is Florian today my my talk is about what's next for I I was asked not to do a sales pitch so I want to size pitch and so I I give more like a like say business commercial perspective of what I think was next in an I so it's a bit of overlap with other talks in on the other stages but again with a more like commercial or business side focus on this but nonetheless I have this one slide about leviton so let me start with this I myself I found our in CEO of Leverton and we grew basically the last seven years from two people to 90 people now sitting around the world in four offices headquarters in Berlin I acquired many clients over the time huge corporates who are serving with our services we basically are building a software as a service application which is meant to extract information out of documents like legal documents corporate documents whatever we're trying to fill in automated way structured databases like ERP systems

with information which has hidden in documents it's semi-automatic so sometimes it's completely automated sometimes on complex documentation it's semi-automatic with a review process and so on and so forth that's all till now about 11 but my today's talk is a bit more in a general about AI we are using deep learning technologies which is the stupidest thing in it in AI since around 6 years so I would call us like early adaptors using a deep learning from many everything for OCR for document classification for the information extraction like literally we have like thousands of deep learning based models I train them on a daily basis and I use them in our production environment so we have lots of experience make long story short about deep learning and then I in general and as we all now a is of today is very good at like very narrow tasks like very singular tasks dedicated to one specific workflow classifying a document finding a person on a picture recognizing traffic signs ever translating a speech into text

and a single text and all of these are like very narrow these eyes can't do anything else or decide what they were trained for but on this I excel so we reach sometimes superhuman performance release we have many real-world applications as of today all of us are using these of course like Siri elixir whatever but yeah for the next step basically we need something else we own all this and hard to read but we are on the left side now with likely narrow intelligence artificial intelligence and we all know that like the the holy grail of AI will be the artificial general intelligence some call it the strong intelligence or whatever like in intelligence which is more mimicking how the human brain works like being very general have understanding of context of knowledge and so on and so forth but I need to run a business I can't wait for this because when will the senton no one else like some people think in five some intense I mean 50 some 100 some never something

we will be doomed as long as soon as we have this whatever I don't care for me important is the way to words this general intelligence like what are the next steps and I would call them baby steps because it's for saving right now we're making small steps one after the other improving but still like there's a long way ahead for sure oops sorry so I want to give you now two examples why we need more intelligence more smarter networks I have one completely artificial example which does not make much sense but it explains the underlying problems and then a real-world problem from the actual level use case where I explain you what we have word problem is so these two guys and most of you or some of you might know them these are Tyler and Cameron they had an idea in 2003 to build a social network to connect students at University they hired some developer was called Mark he built something like a few months later this guy Marc came up with

an like competition product called it Facebook and these two guys stood and then they got rich the other guy Marc got much more rich so what does it have to do with AI nothing I just need an example of two twins you have you have it so to build a legacy person classification algorithm if you would build something which we can do today of course like identifying a face on a picture and then saying this is Tyler this Cameron no issue but here in this example this will fail and so will we do we will also fail on this like no one of us can probably say this is Tyler this is Cameron or this is Tyler as it's Cameron no clue if you now give a small hint like say left guy wears today a t-shirt with his name Tyler it's easier who is this guy yeah Cameron correct so how could you deep learning algorithm today solve this it can't because we would need to do certain things several things we would need

to build a person identifier it would say this is Tyler or Cameron with some confidence same for him it can't read obviously any letters we could combine it with an OCR some we have two networks one for person one for OCR reading the text doesn't help us much and we need to build some heuristics combining as a result of the OCR with the image somehow okay cool maybe we do this with lots of code around this and then now we have something which identifies Tyler is very person still this doesn't know that this is Cameron because you know we need some reasoning you know these are twin brothers obviously one must be Cameron and Tyler this is Tyler so this is not Tyler this is Cameron even Morris sticks just for a small edge case lots of code lots of complexity and generally not feasible today as promised now a like a real-world example one of our huge funders of documents that we have drafted usually by lawyers oh yes a lawyers they didn't become

designers because they are not good at design but sometimes we see really examples where I just don't know what to say this is document you can't read it from behind but it doesn't matter so it's better if you can't read it the task at hand here is basically to find the correct reading one of the document because this is very important for like finding information in document what's the reading order obviously it's left to right and then you go line by line up to down easy but if you get somewhere around here you see there's something very wrong so the indentation seems to be wrong so the author might have missed to hit the tab key but if you start reading the text and you start reading this line here and we don't know but I can tell you it does make sense so the author didn't forget to add a tab key here he forget to add spacing so this is actually a two column layout we have here some information on the left

side which is a comment to something here on the right side to figure this out like only with our like visual perception nearly impossible and so for us like at 11:00 we're building lots of deep learning networks and all of them are like somehow executed sequentially so we have OCR we have layout recognition document classification information extraction and so on and so forth but this you cannot solve like in the layout detection which only relies on visual features it's not possible to somehow resolve this so you need some information from a much later step where you go into the semantics into a language model to understand this doesn't make sense if you read this and then you wouldn't go to back have a feedback loop to an earlier stage and say the allow detection Hey you up please redo and now again it gets very very complicated and very complex again need to build lots of heuristics and this is just again an edge so this brings me already to my nearly last slide so

what are some possible next baby steps and I cannot say like how to resolve these issues which I just explained because I don't know basically if you have ideas then please approach me after the talk and we can talk but yeah I'll give you some things but we are currently looking into what we're researching on what could be something as the next baby step so one interesting thing is multitask learning basically this basically means you're not only learning one task in like in a completely isolated way but you're started learning multiple tasks at once and then all of those like benefit from each other and with this you get to better new networks but in the end you still get better neural networks being good at a singular narrow task so nothing more general then there's something as multimodal learning which basically means means you combine like several different kind of input sources like image text audio and find all of these together and with this you basically end up in learning in a better

way understanding also a bit more of context I'm definitely but the issue is you need to have training data which is basically aligned and this can be very difficult to produce so at the moment we are stuck basically with combining neural networks in programmatic voice so they do lots of programming around these lots of heuristics lots of code I'm trying to handle edge cases it's what we're doing right now there's a bit of a smarter way around this so it's got different Schabel programming it's heavily propagated by laocoon for example who is one of the like main inventors of special deep learning topologies and he sais is next what it basically means that your heuristics which you're built around this and the parameters it's also learn about it's differentiable like a new light work and with this you cannot learn of only the new networks but also the parameters of the code around this sounds like a good idea but so far there's not really a real implementation of this like tensorflow and pi torch

and others come close to this but it's not really good but maybe this could be something but like the more I think about this and this now I come to an end here and the more I think about it the more we build these complex heuristics and for edge cases the more is something which comes basic from my experience like dealing with new networks in ten years and in a business case like six years whenever you have a very complex case with lots of heuristics if you manage somehow to put this new sticks into a new network and train this the new network will perform better at the end so my last thought for today is can we build something as neural networks which do these connection between different Avenue networks and maybe this is something which will lead us a bit more into the direction of artificial general intelligence it's a question if you have answers in please approach me thank you very much great thanks Florian do we have any questions in the

audience - Florian cool one here we need the microphone again because we record everything you will say Esther Moscow speaking the question is if you have extracted them knowledge so in which kind of a database format what you recommend storing the knowledge so this is a very good question in the end what most are doing right now they're either using a normal database database we are doing this I also have some experience with using anthologies which is a bit more like complex graph based database but in the end it's still the database or you can add some stuff on top of this this is very interesting but it's hard to combine actually later on with something as a new network so for our business purposes right now this is what we need like only a starter database but like if we want to go in this direction of artificial general intelligence I think we need something completely different I guess all the knowledge needs to be stored somewhere in neural networks in the like embeddings

in natural language processing or whatever like the network sims have contained the knowledge and not some structured database or whatever thank you any other question I would have one question something that always drives me is obviously we want to invest in companies that produce something for the customer so with your vast experience of extracting data from documents what is really possible today fully automated and where do you see it's probably not possible in the next years yeah this is a very good question actually it always down to like how similar are the documents to each other like if they're completely similar we are a ton of % for sure so there's no question about this if there are a bit similar we are so reaching near hundred percent if they are completely different then it gets more difficult it depends really on the like amount of differences you see between the documents like mostly in the layout not so much much in the language but in the layout this is very important what makes us

is very complex and we still need like a careful review process is if you not have only one document but if you serious of documents and all of them build on each other so you have like say a real estate lease and then you have an amendment another amendment some termination that are some renewal whatever and all the informations and like they overwrite and so there's a references between the documents and there becomes very very tricky we have I think really good at this and then the only ones we're providing this kind of information extraction on these complex documentation but still we are relying on revenue services later on to do some Corrections I would have another question if you guys don't have one there's n ru natural language understanding and I just learned it's not really about understanding it's really about extracting certain phrases content comparing that but do we do we have any chance understanding language with the machine and in the keys next years I don't think so no not not really

like some basic information yes but the full understanding I think we are far away from this because it's all done again - where do we store the knowledge what is the knowledge at all what we need to understand something in in in a text so we can understand like grammar something like this dependencies in sentences and stuff like this but a real full understanding we are far away from this you

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.