hello everybody yeah my name is Tobias I'm one of the cofounders of topple sends and yeah I want to explain to you how tapas ends uses yes sound and AI to come up with a very low-cost 3d perception system to understand our environment better and this introduction by a spa was pretty good because when thinking about me explaining what we're doing I I came up with these with these clips I hope the sound is on so maybe you know the movie The Dark Knight yeah so the Dark Knight in this scene Batman because he's very good sonar concept and applied it to every phone in the city but half the city feeling you saw now you can image all of that so what he was saying so what what did Batman do so he implemented a 3d sonar system inside every smartphone and through everybody has a smart phone in his pocket and through emitting sound he could see all the surroundings what this comes up to the scene quality isn't that good but I'm
still have an idea what's happening okay you got veal and alpha I'm Leone beta it's like classical he's easy to see everything team farewell team so you can see the waves and being emitted hopper calm down so why am i showing this so it's oh yeah nice introduction but the most important thing is okay if you've been seeing this movie okay this is totally unrealistic that this Batman this can never happen we have to develop this so that's that's the the trick so what I'm holding here in my hand is maybe not the smartphone technology so we're not at that level yet but in general we have exactly created what you can see here in the movie so we've invented the bat sensor so for the first time ever we can understand our surroundings like bats can perceive their environment through ultrasound oh no something a bit so here on the left you can see our initial sense we are just releasing to market in about two months and our technology is mainly based on
software algorithms and artificial intelligence so because what is basically happening maybe you know how bats perceive the environment and that is very similar to what our system is doing we are sending out high frequency sound we're receiving signals we process the sound and then we get a 3d data so every one of our systems generates a three dimensional point cloud as I think to talks ago we have seen you can do with a camera or lidar system the awesome thing about our technology is that we don't care about lighting condition it's lighting conditions at all it is a very robust system next big topic is it's the cheapest 3d sensing technology worldwide because we only need a very small processing chip to do all this 3d processing that's due to ya the special software and technology we worked on now for over eight years in total what is another factor the system is not to the long range a system is very good at short distance sensing so up to five meters since we're using
sound you cannot like look for 100 meters but you can sense up to 5 meters which gives you a lot of applications in the Meishan space and then yeah another important topic especially on the lower right hand side is our system only generates a super small amount of data so when we let the system run for about an hour we're in an area of 3 to 5 megabyte in data in 3d data our system is generating so now think of what our term yeah what artificial intelligence algorithms can do with this data you can do AI processing on the edge because the data amount is so small you don't have to upload it anywhere so where are we going with this technology Automation is happening everywhere I think that is pretty clear we have already talked about it now and seen some topics if you talk about the robotic space you can look at consumer robots you can talk about industrial robots and as well as in the in the automotive space so passenger cars
autonomous driving that's what I was asking a question a bit earlier but also commercial vehicles are automated every autonomous system needs to perceive its surroundings so 3d sensors and able these cars to drive and move and interact with the surroundings autonomously up to now the dominant systems which have been doing this task were mainly 3d cameras and 3d lidar systems so both of them optical based so sending out light or receiving light and then processing it and this is exactly what yeah we are changing now our system is using sound so totally different approach why is this important so why do we need this kind of technology out there on the market so these are some of the cases where a short distance sensing is failing so the existing systems don't work properly why is this the case the environm environmentally conditions are problematic the close distance 3d data isn't good yeah the systems are just too expensive to equip your object your autonomous system with a sensor yeah so several things lead to these
kinds of failures and they cause yeah deaths injuries a lot of physical damages so close distance sensing for autonomous technologies it's just really a gap that it hasn't been solved perfectly yeah and especially especially price is also from very expensive optical systems is keeping all these technologies from using good close to since we sensing and sure that is exactly what we solving with our 3d ultrasound sensing system so maybe here I want to show you give you an idea how is our system implemented into into a driving vehicle so for maybe you know your Park distance control sensors which you already have in the car yeah so they give you one dimensional ultrasound so only the distance with our processing we are able to generate 3d data yeah so we're send out sound we get the sound reflections back we process the data and we get three dimensional point clouds as you get with other systems so if a car is driving equipped with our seven sensors all the sensors fire and send out sound
we get the reflections and we generate a 3d point cloud of the surroundings this is still a simulation but in a second you can see like real life data mm-hmm so this is what it looks like when we are driving out on the street yeah so this is close to since up to 5 meters 3d data it's a bit difficult to interpret for the human eye the data is color coded so on the lower level you have blue then you go into green orange and red and what we are able to see what is very difficult for other systems is for example curbs yeah so um small things on the ground which is especially important for autonomous driving systems in the close distance maneuvering and parking space maybe you have already tested one of the autonomous parking functionalities and one of the more luxury type cars many of them don't work only working 50 percent yeah or yeah say no there's an error I cannot I cannot park that is mainly due to bad sensor
data yeah so and that is exactly what what we are providing maybe to jump one step into the robotic space I talked about vacuum cleaner robots or I have shown them on another image so this is an idea of how our system is perceiving the closest and surroundings in your home yeah so you can see the 3d point clouds of the sofa and yeah and compared to other systems we are very good at detecting glass for example yeah so that is also another thing that is difficult with optical systems so how does this system actually work yeah so I'm talking about 3d sensors but what's what's behind this thing here I'm holding your and so initially I talked about the bat echolocation our system is working very similar to it so I have a small speaker here we send out ultrasonic pulses of sound all the surrounding objects reflect the sound so in typical echo we have microphones in here and those microphones yeah listen to the reflections and then inside our processor all the
processing is happening yeah so we calculate the time of flight so how long does it take for the sound to travel from our system to the object and back and then our algorithms do all the 3d processing that we get 3d 3d data out of the small system so in the end on the hardware side our system is very simple you need our chip recalled the sound vision 1 you need three microphones yeah we just started a cooperation with Infineon because they also see the relevance okay now we can not only listen to our surroundings with microphones we can start seeing our surroundings and the major component company for transducers for senders so those are the major components one transducer through microphones and the chip sure a bit of PCB and housing and so on but that's it and you have a 3d sensor the magic is happening inside inside the sound vision one so now what are the software blocks where's lower and higher artificial intelligence happening inside our system so up to now
most of the data processing yeah is a lower level of artificial intelligence because we are yeah looking at the data that our system is receiving and finding different things so one thing is the echo location yeah so once we process the sound the sound data we need to find where are the echos because you have a noise noise level so there's lots of noise in in in such an audio signal you need to probe what what is a real echo what is a real echo I'm and I'm seeing it that relates to an object otherwise you have many false positives the next point is the timing analysis from this audio signal you need to super super precisely find out what the timing is yeah so you need very good analytics and processing to in the billionth of a second areas on the nanosecond area that we can process 3d data and then short there's all the 3d processing that we generate this 3d point cloud and now this is the next level where it becomes
really interesting so the 3d data analysis once we generate all this 3d data what can we do with it I mean that's where how we start understanding our surroundings if since we process the the 3d 3d data we are we are seeing and I want to quickly explain to you what has to happen to get we are highly reliable yeah deep learning machine vision data out of this to understand your surroundings that is a topic we are currently working on so we don't have the perfect deep learning neural net already installed and these are four very important prerequisites which we found out over the last one to two years which you have to have before you can really train a proper neural net so one thing is you need very reliable data so every sensor needs to give you the same data yeah if you have a very high discrepancy between the different sensor systems it becomes very difficult in the very precise state area so you need to be in the very low centimeter
if that even millimeter area per point you're receiving so you need to have a very high performance then you need a high density you have maybe seen the video and I've seen okay we have rather sparse point cloud that is more difficult to train a neural net on that and then if you have a very dense point cloud which is easier than to do with optical systems and then for sure if you have met all the other three topics you need large datasets generating more and more data the easier it becomes to train unit so what are the steps for us we were working on and have ahead of us to get to real intelligence what we think about when you think about artificial intelligence typically so we are like in the directly in the middle between step one and two right now so we have reliable sensor data now and we are able to detect patterns because objects yeah give us similar patterns and then on the next level we want to yeah start
understanding different objects in a static environment you place your object there you scan it I don't know thousand times a hundred times and replace it with a similar object with maybe different bit of a different shape and put in a static environment that's like the next tasks now when you think about all our vehicles we want to apply these sensors the next level is dynamic objects yeah so you have to move from static objects once you recognize them and have a very high probability of hitting the right targets with your neural net then you can move on to dynamic which makes it a lot more difficult because it's moving the data can look different and so answer brings an another level of complexity once you have solved that then you can go to the maximum level you need for autonomous driving yeah which is the real intelligence that you can start making predictions around around your environment so if you see a human and recognize it as a human can label it as a human
what is that person about to do yeah is this person moving at a quick speed slow speed is the person sitting somewhere is he standing and the same with cars and your other surroundings so that's like the maximum level we can go to and we are still in the in the middle of this this system so we don't want to keep everything to ourselves we are working on this but this is also yeah like an open shell to everybody who is interested in this kind of topic we are sending these sensors without all of these steps because we want to have spread it out and we want to democratize 3d sensing so everybody who is here or or interested in building up neural nets based on our data sets and then yeah please happy to talk afterwards that was quick yeah so here you can just see how many companies are interested in this technology and the capabilities what you can do with it so it's pretty it's pretty broad I mean we have many
customers on our list already many of them interested so there's a lot of things for us to do to service all these inquiries we are partnering with Nvidia for example for autonomous driving and neural nets working on that yeah I talked about in Rotten Infineon already and then some other partners the founders we are three founders my co-founders are the guys working on the technology which invented the sensing system myself and the business guy in the round and yeah represent the technology and then talk to customers and last but not least what is a road map for this year as ho yeah we're standing here today at rise of AI in about in in about two months in in July we are launching our first commercial product so now the test kit anymore but it like an openly available product which is mainly targeted at robotics mainly at robotics companies but can also be used in different in different areas then next level what we're currently working on is an autonomous driving development kit so
with several sensors which I'll a grenade the data that you can come up with something similar you have seen in the video and we're currently on a roadmap to start working on the commercialized automotive sensor and you talk about a sample as like the first step and that will be done by end of the year yeah on top of that we're currently starting to raise our series around the company has been financed by some Silicon Valley investors and German investors up to now with a total funding amount of around two million dollars yeah so if there any of it was in the room happy to follow up afterwards again oh yeah no I'm happy to take your questions Thanks all right then we have a first question Thank You Tuvia so the question is what does it actually have to do with machine learning I mean okay the applications you can build on top of it obviously but inside it doesn't look for me like any machine learning algorithm needed you can do this was
classic signal processing so as I said we're not in the we don't we haven't trained any big neural nets right now yeah so we're just starting off with that I think what is what is more what we want to bring over that we are building the cheapest 3d sensing technology where you can train which you can implement in almost every topic in smart building into consumer electronics into robots into cars where you can start training your nets on we are just starting with this and I think I yeah the the main point is that we are that we are delivering the basis now that we have reliable data but we have to do all these steps yeah with a typically are with a very new technology and I think just as you heard from Koenig's you need very good data yeah so about I think six to nine months ago we started really intensively working on this and came to the conclusion oh wow the data we thought was easy to handle with neural nets
is not that easy to handle so you have to take many prerequisites so it's 8090 percent before you can really then train your net so rather see it is an enabling technology for artificial intelligence at this point and we're moving on to the next level so we're not saying we are vai company and right now thanks a lot any other question from the audience cool there's one thank you very interesting product the question is do you see your product where do you see your product in the line of autonomous driving because soon a sensor you said yourself is good for five meters but if I'm driving I'm 5 meter I travel 5 meters in a half a second so if the sensor classifies a person the persons that if we talk about driving our data is being added on to the full sensory stack combined with lidar sensors camera sensors and so on that's why I said new field sensing and you currently don't have a proper technology which is able to cover that close
distance so at the close distance becomes important if you drive slower yeah so but every autonomous car has to start somewhere it has to stop somewhere and they're maneuvering situations and in all these situations you need full 360-degree coverage also of the clothes clothes to stand surroundings so it's a complementary technology to also the other technologies and not the single one yes please behind you is the mic Hey so famously Tesla is not using the lidar technology right they're using cameras for the self-driving but they are also using ultrasonic sensors or something they call ultrasonic sensors so is this something similar to what you are doing or oh yeah very interesting question so Tesla uses 12 1u ultrasound sensors I have to let's say due to NEA reasons I cannot talk about the details but let's say a company like Tesla is extremely interested in the level of data we can provide because that's what they need to properly have their autonomous driving features they are fully rolled out and this is one of the
missing important pieces right now yeah thanks for the talk and are you planning to expend like the frequency bands of that device to do like like real sound detection also passive sound detection sure so what does our system is pretty let's say Hardware agnostic it's really very software focused so we can use any kind of transducers so any kind of speaker and any kind of microphone and any type of frequency sure you have to adjust the hard way a little but in the back it's mainly adjusting the software so for sure you can also use it yeah maybe for hearing the sirens of a yeah of an ambulance in the city or like kids playing around so for these kinds of safety features and a lot of other topics I think are possible and and also from the images I saw they didn't look for humans they didn't look like they have much information rather than that there is something is that enough to do like some sort of image recognition image classification object classification
with that data that's exactly the part where it becomes tricky which is a tricky part for us what we found out that it's not we cannot just use typical approaches of optical data put it onto our data and it works but we have to like train new types of neural nets on that and have to see what are the specific data points or the specific patterns we can see in inside of that data and the longer you start looking at an object the more data we get the more shapes we get the more points we get the denser of the point cloud becomes but that is something you have to be to be executed on and see where we're going