Rise of AI Conference 2023 · Berlin

AI Enabling Automated Maintenance

Hayri Göcke · Gestalt Robotics

At Rise of AI 2023, Hayri Goecke, Head of Growth at Gestalt Robotics, presents how his company applies computer vision and machine learning to automate maintenance for safety-critical mobility operations, focused on the E-Check project with Deutsche Bahn. Founded in 2016 and grown to 60 people without venture capital, Gestalt Robotics builds software modules that run on hardware from vendor-agnostic partners across computer vision, mobile robotics, arm robotics and worker-assistance systems. Goecke explains that Deutsche Bahn's growing train fleet is roughly doubling maintenance needs while a new maintenance site costs about 400 million euros to build, so the company is instead digitizing five existing sites with a 32-camera gate that scans each ICE train's underside at about 5 km/h, producing roughly 6,500 images and 30 gigabytes of data per wagon. He describes the technical challenge of producing precise, reproducible images across different trains, the on-site edge computing and cloud processing behind the system, and the close but clearly defined supplier relationship with Deutsche Bahn, including regular scrum-based feedback.

Key takeaways

  • Gestalt Robotics has grown to 60 people without venture capital by building software modules for computer vision and machine learning that run on vendor-agnostic hardware.
  • Deutsche Bahn's E-Check project digitizes five existing maintenance sites instead of building new ones, which Goecke says cost about 400 million euros each.
  • The E-Check camera gate uses 32 cameras and speed-related mapping to produce about 6,500 reproducible images, roughly 30 gigabytes, per train wagon.
  • Processing combines 16 on-site edge computers with cloud computing, AI detection, human verification and supporting robotic tasks.
  • Goecke describes Gestalt Robotics' role with Deutsche Bahn as a clear supplier relationship, won through public tender, kept close through shared scrum events rather than joint ownership.

Detailed notes

Gestalt Robotics: a software company built on hardware

Goecke introduces Gestalt Robotics, founded in 2016 and grown to 60 people without venture capital funding, working with clients across mobility, discrete manufacturing, chemicals and pharma in Germany, elsewhere in Europe, and in the US and Canada. Although named for robotics, he stresses the company is software-driven, building networked and virtualized modules for computer vision and machine learning on the shop floor, and staying hardware-agnostic by working with partners such as UR robots or FANUC for the physical systems, alongside Verizon in the US for edge-computing deployments.

Four solution fields and early use cases

Goecke groups the company's work into computer vision, autonomous mobile robotics, arm robotics, and worker-assistance systems. Examples include 'navigate this,' a mobile robotics platform built with Verizon that uses 5G and edge computing so a camera-equipped robot can detect objects like pallets and feed that data into a factory's digital twin or SAP system; a laboratory automation system for a chemical client where two UR robots collaboratively open sample vials and use visual servoing to pipette liquid into small tubes; an augmented-reality assistant that guides a worker through tasks using a pre-scanned digital twin; and a quality gate developed together with Siemens that uses few-shot learning, trained on five to ten images, to check the configuration of assembled medium voltage devices, replacing two of the four eyes in a manual four-eye inspection process.

The E-Check challenge: maintaining a growing rail fleet

The main project, E-Check, addresses a capacity problem at Deutsche Bahn: Goecke says a fleet growth of 250 trains roughly doubles the maintenance need, while the workforce able to perform that maintenance is aging, and building a single new maintenance site costs around 400 million euros. Rather than build new sites, Deutsche Bahn is digitizing five existing ones, aiming for full surface inspection coverage of trains and a solution that works in both new-build and existing (brownfield) sites.

How the camera gate works

The core of E-Check is a camera gate with 32 cameras that scans an entire ICE train's underside as it drives through at roughly 5 km/h, producing about 6,500 images and 30 gigabytes of data per wagon. Goecke says the central technical challenge is producing precise, reproducible images, achieved through speed-related mapping that ensures the same parts are photographed consistently across different trains, plus coping with how rarely serious faults, such as a broken ICE wheel, actually occur in the data. Processing happens through 16 on-site edge computers feeding into the cloud, combined with AI detection, human verification, and supporting robotic tasks such as exchanging dirty and clean water on the train, handled by a robotics partner.

Rollout and the supplier relationship with Deutsche Bahn

A prototype camera gate is already installed in Munich, and Gestalt Robotics is rolling out to five sites across Germany with seven gates in total, some sites getting two. The project came from a public tender that Gestalt Robotics won together with a partner company, and Goecke describes the relationship as a clear supplier arrangement rather than shared ownership, though the teams work closely together, with Deutsche Bahn's own AI team participating in Gestalt Robotics' scrum events to give fast feedback. He says the company is working with a Deutsche Bahn subsidiary to turn E-Check into a broader 'smart assistant maintenance' product for high-speed rail operators internationally.

Q&A highlights

Asked what consistent core IP underlies the company's varied projects, Goecke points to the capability to set up and individualize such systems for a specific industry, underpinned by a shared MLOps pipeline used across projects, such as laboratory automation for any lab or the camera-gate approach for any high-speed rail operator. Asked to characterize the Deutsche Bahn relationship, he reiterates it is a clear supplier relationship won through public tender rather than a shared 'workbench,' but with tight day-to-day collaboration through joint scrum events so that feedback is immediate rather than delivered only after years of separate development.

Quotes

We are software driven, so what we built are technological modules that are highly networked and virtualized.

Hayri Goecke

A fleet growth of 250 trains doubles the maintenance need for Deutsche Bahn, roughly.

Hayri Goecke

For us, data acquisition is the key here, it's way more about acquiring the data and processing the data than in the end about the AI.

Hayri Goecke

Questions answered in this talk

What is the E-Check project with Deutsche Bahn?

According to Hayri Goecke at Rise of AI 2023, E-Check is a Gestalt Robotics project with Deutsche Bahn that digitizes train maintenance using a 32-camera gate that scans each ICE train's underside at about 5 km/h, producing roughly 6,500 images per wagon for AI-assisted inspection.

Why is Deutsche Bahn digitizing existing maintenance sites instead of building new ones?

According to Hayri Goecke at Rise of AI 2023, a growing fleet is roughly doubling Deutsche Bahn's maintenance needs, while a new maintenance site costs around 400 million euros to build, making digitizing five existing sites the more economical path.

What is Gestalt Robotics' relationship with Deutsche Bahn?

According to Hayri Goecke at Rise of AI 2023, Gestalt Robotics is a supplier to Deutsche Bahn, having won the E-Check contract through a public tender, working in tight collaboration through joint scrum events rather than as a co-owner of the project.

Mentioned

E-Check, camera gate, edge computing, few-shot learning, MLOps pipeline, digital twin, mobile robotics

Keep exploring

More from Rise of AI Conference 2023

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 4,286 words

Show the full transcript

foreign we have another two hours of some incredible content for you so no doubt there'll be people joining us uh throughout the course of this talk but since we're such a small audience we can keep it somewhat interactive if you would like so please feel free to ask questions throughout and I'm delighted to kick off this final section with hayride who's the strategy and growth manager at Gestalt robotics formerly from Siemens research your talk is going to be focused on how AI can help automate the maintenance of the safety and critical mobility in operations so very apt topic for our German manufacturing sector so welcome to the stage hey Ryan thank you so much hi everybody and welcome for my end to my talk I'm head of growth at Crystal Robotics and I will talk about how we enable OT partly automated maintenance for high-speed trains in Germany but also I'm going to introduce the company for a bit and I'm looking very much forward to having an open discussion with you in the meanwhile

so feel free to ask questions at any time in between so Gustav robotics was founded in 2016 by the three gentlemen on the left um it's uh Jen slamreich who was a professor at U Berlin for industry grade networks and clouds Thomas daffenbiel Ariana group rocket engineer and uh Oregon funky was uh doing research on computer vision at the German Institute for uh luftwaffe Aerospace we are now 60 people at this time all in all growing without a VC so we've been working with our clients and growing with our clients in the process we've been awarded by fabric desires which is a a German manufacturing award as a most Innovative startup last year we've been awarded by our customer Deutsche Bahn for the E-Check project which I'm going to be talking about in a bit and yeah the the company itself has its focus on software so we are even though we are named Crystal robotics we are not a hardware robotics company but we are software driven so what we built are technological modules

that are highly networked and virtualized and that enable computer vision and machine learning on the shop floor so I think the AI part is no news to the audience here but I think what's interesting is that we really work with Hardware with camera systems and we build entire Solutions around that so they are ready for production they're scalable and in the end adaptive to our needs this is what AI allows us these days so we've authored a d Norm very German very German thing to do for AI lifecycle and quality assessment and we hold a couple of patents in the in the field of virtualization and the field of Robotics um yes and the our idea is to work with our customers so our customers are mostly across Germany and um a bit in Europe and then we have also have some abroad in the US and Canada to work with them to jointly develop industry-specific use cases to solve their needs and our clients does come from a variety of sectors so we have

clients in Mobility such as Deutsche Barn such as Statler but we also have clients in the classical discrete manufacturing and in chemical and in Pharma in the Pharma space which is interesting because we apply very similar methods to solving their issues but always have to adapt and have to work with them to get to new ideas as I said we are a software company but we work with Hardware Partners we are vendor agnostics so the robots we use could be a UR could be a FANUC we don't care but we obviously have preferred Partners who we work with for integration which is especially drama they're going to get important later on we also work with Verizon in the US deploying mobile robots enabled by Edge Computing and uh with with others in the field we're active in research you mentioned that I worked with Siemens on the Berlin research Innovation ecosystem so we are as a company co-founder of the Vernon from Siemens Center here in Berlin promoting industry Academia and startup collaboration we are

doing around 10 research projects currently which are pretty wide across the field which enable us to develop ideas push them forward and then work those through with the customer get first customer interest um in the end yes we mainly work with very large corporations that have the funding to go from POC to MVP to a full scale solution but we are more and more also working with middle stancer with medium-sized companies especially in production because the solutions are becoming more and more standardized I'm going to talk about that a bit and more thus more and more accessible our goal is to bring forward the shop floor shop floor of the day so our solution Fields align four sectors first and foremost computer vision computer vision for us is the pure computer vision so what you would see here as well um the second part is autonomous Mobility mobile robotics we have a system there that I'm going to talk about in a bit intelligent robotics then is everything arm related so and then we have

the worker systems I'm going to also show you something there um that is mostly about enabling the workers to do their tasks more efficiently because Workforce is not going to end we're still going to need people especially in complex tasks but we can Aid them we can make it faster the training and those kind of things so some solutions we implemented I'm going to talk about three of them and one a lot in autonomous mobile monitoring we have implemented navigate this which is was our first product we developed it together with the Verizon and the US they asked us we have 5G now which wasn't right um 5G is not fully in fully there yet but it's coming more and more and it's coming more and more to the shop floor and the idea is what can you do with 5G you can virtualize and offload High resources and use the edge to run more powerful models uh for for mobile devices or for stationary decentral devices and this is what we do with navigates

this uh enabling uh quite interesting use cases second thing is with a customer in the chemical space mostly the laboratory again a space where we are very flexible where things come in that we don't really know they are in different vials I'm going to talk about that in a bit then we have the ARs assistance talked about that and then we also have different topics in quality and in process control where it's more about really getting into high volumes and understanding uh what's happening quite interesting case is the assembly gate where it's about few shot learning because assembly is a process especially in high value Goods that are highly customized where you need to have not as many samples as usually needed by Ai and yeah last not least I'm going to talk about E-Check a lot E-Check is the project we're doing with Deutsche Bahn the reason why we are 60 people um and our proudest moment so uh autonomous mobile monitoring um we call it navigates this navigates this as a platform enabling

5G and Edge computing for mobile robots so this is video was filmed in Van of Van Siemens Center um on where we have where we are not only a part of but where we're also renting some space to develop mobile robotic Solutions so obviously the system can navigate you utilize the entire resources from The Edge to be able to navigate using Machine Vision using um using also more advanced algorithms then we can follow a path but also what we can do then is we have a camera on top that can detect specific things easy easy models for example here we see a palette and then we can put it on the robots map and play that back into a manufacturer execution system an sap system so we can sort of feed data into the digital twin off of the factory and give them a more complete view of what's Happening uh what's happening on the shop floor you could imagine cases such as people lying around there such as a helmet being off all those

kind of of things would be quite interesting you could also think about way more advanced cases that are for example fences that are open doors that are open these kind of things also you can think if there is a door I should never get around it not right next to it so that it gets interesting mobile inspection is the last use case inspection use cases are things we are doing quite often but why not do it from a mobile robot if we have all the resources from The Edge available so Edge Computing um is is the enabler for this I'm just going to show it really briefly because we have quite a technical audience here as well I feel and so we are enabling our our system with Edge Computing so this basically just should show you that the entire resources of the mobile robotics tag are offloaded to the edge so the the robot itself is purely a receiver of commands and a sender of data and now if you think it one step

further you could also think why do I need a robot I could just place a camera somewhere and the camera could show could also put data into the system and we would have a more complete view especially in difficult to navigate around so you can enhance navigate this also with stationary cameras and build a complete set of your floor so enabling asset tracking enabling lost and found use cases enabling inspection use cases basically everything you can think in AI with the classical approaches you can think here in in a mobile mobile based approach so one step further the laboratory sample is something where I want to bring a bit of Hardware into the conversation here because we're talking so much about ai ai is very interesting but what can you do with AI if you combine it with Hardware well you can work in a laboratory really interestingly because we had the case our customer they get these you see it on the bottom bottom left of the pictures they get those samples they want

to test and they want to run them through their High throughput systems but the problem is the manufacturers there is no standards they bring them in many different files and so we developed a system that is independently gripping the parts that collaboratively opens them you see it in the middle picture so the two arms are working together then use this visual serving a nice technique to pinpoint and pipette is very small into this very small and Tiny tube and many more AI based skills that work on top of that so we have those two off-the-shelf robots that are Ur robots we have a bit Advanced grippers we put a bit of work into that and then we have software system that allows to add flexible skills flexible tasks to to the deal by the way to the I don't know the time with this turning watch so if you can just show me minutes I'll be I'll be fine um yeah so uh this is a really interesting use case it's a platform solution so

for any Laboratory um you could you could enhance it you could set it up but for now we are focusing on this how do you say material inbound process for uh for Laboratories so the next use case I'm going to thank you very much the next use case I'm going to talk about is adaptive AR assistance really briefly we have a worker here he got data from a digital twin we scanned it before whole process and the quite interesting thing here is how could you use AI well you could use a video stream to tell the worker where exactly the the data is he needs and what exactly he should do so we're giving them the task that what they should do on the on the AR base on the device uh yeah I'm just going to skip through a bit the last thing before we get to the magic is the quality gate we call it so when you have an assembly process they're very manual very prone to fail we developed this together

with Siemens so you have a scan of um this um it's a medium voltage device Let's uh and you want to know whether all parts are there and this is usually tasks that people need to do two people need to do because you need this 4i principle so we replace two of the four eyes with a robot with a few short learning technology so you need five five to ten pictures to train very fast and then you can check the configuration of of those devices yeah um and now I get to the last part that is E-Check E-Check is a project by Deutsche Bahn and the problem behind eCheck is sustainability we want to we talked about a lot about sustainability today well Deutsche Bahn is the most sustainable way to try to travel in Germany um but there is not enough trains Deutschland tact is not really working maintenance is a pain so a fleet growth of 250 trains doubles the maintenance need for Deutsche Bahn roughly I think it's a bit less than

than double but let's go double um and maintenance capacities are at their limits already so we have a high Personnel need for this and the Personnel is aging as well so Deutsche Bahn made this approach that they well they are building new sites actually they're not they just recently announced they're canceling one of their one of their sites but in general they're they're expanding their their maintenance time and they also want to digitize their maintenance and the interesting thing about this is really the cost when they when the article came out that they're canceling this site it's at 400 million of budget is canceled for this site so one side to inspect IC trains is 400 million we are digitizing five of them in Germany um so this project came to be through a POC that we did with them luckily a paid one we don't do free ones um so together with them we built this underfloor scan so the the robot drives underneath and scans uh critical Parts this was also few shot

learning very basic AI not spectacular but the task in the end is to scan the entire train so we want to have 100 surface again and also we want to have a solution that works both in the Green Field and the brownfields um well the solution for this is a camera gate together with AI processing in the back and then a human verification as well as robotic systems that assist and do manual tasks that are quite repetitive and not as interesting possibly but before we get to the solution we get maybe to some challenges I called it mittelstan Deutsche Bahn and the tech company and scrum well it's it's not super easy to work with such a large corporate as such a small company very limited resources and we built really an integrated team with Deutsche Bahn for our AI development but it also has large benefits because we get all the problems right away into our team and can work faster and can develop the solutions faster so um the project is still ongoing

we are not we are not crying yet but it's it's a very very interesting approach to work so closely drama our main partner general contractor is a is a middle standard from Bayern they are doing Zonda machine bow excellent company and Deutsche ban has a huge AI team within their own company and us together we are trying to solve this there is also technology related challenges so producing and precise and reproducible images is one major challenge I'm going to talk about how we solve that in a bit then the pure amount of data that we are producing we're scanning fully in ice train and drive through very slow drive through five kilometers hour roughly and we are producing 6500 images per wagon I think the data amount is something about 30 gigabytes we have very sparse errors so how often does an ice wheel really break not that often so to get to the errors we need to we need to wait a bit and then we also have the proof of equal safety so

the system we're working together with deutschevan on that the system has to be as secure as a person so the solution for the first part for the reproducible images is this camera gate this is the concept painting um we scan the entire thing it has 32 cameras and we have a speed related mapping of the pictures so we know exactly that we're taking the same pictures you're going to see a little bit and we um we map those pictures and train the AI later on so for us data acquisition is the key here it's way more about data and acquiring the data and processing the data than in the end about the AI important of course as well this is the way we process it a modular system just going briefly through it so we have all the cameras all the scanners and then in the Middle where we refuse it and send it out we have fully integrated into the Deutsche band system with with that when we um once we process it I'll

can I go back yes one important thing we are processing a large part of it on site so we have a um 16 Edge Computers right right there and then we put it to the cloud as well so it's also again a distributed system which is usually for industrial use cases the core to approach those things the user interfaces we are put giving the workers the workers remain there Deutsche band is not going to fire anybody because of us on the opposite they need the people and they need to make them more productive because they don't find the people to solve that house some are going to retirement some want to work less and so they they still need the people but we are giving them tasks that are also again located on the train where they should go what they should do or also just what they should check up on because obviously it's not going to be 100 reliable from the start we're trying to improve as we go on and also it's

going to be assisted so the foreign the head of staff can know what their people are doing and we have digitized the paper-based process in the end interesting about the um about the note the note was basically so the public tender process was basically everything they do before they want to do it digitally and we're in the process of working out of how to do that best so um last bit there is Parts where robotics help again this is done only in part by us drama plays a huge part there so mobile manipulation it's a mobile platform that goes there and puts the dirty water out and the clean water in obviously it's two systems one for dirty and one for clean um and also there is small scanners we're not really sure yet if we need them really but small scanners to understand um critical Parts like the wheels um or the actually where the wheels hang I'm not sure what the name is in German so where do we stand we've installed the

Prototype this is the actual drawing and this is the prototype in Munich so the Prototype is um not a freestanding camera guide but already allowed us to take pictures and we are going towards a germany-wide integration so what you see here are the sites where we're going to install it we have five sites across Germany each are going to be equipped with one camera gate and some are going to be equipped with two so seven camera guides in total this is what the construction looked like and on the right side you see the pictures and the interesting thing now is those pictures are not from one train but they are from very from various trains driving through the system and we're taking the same picture every time thus we can finally train AI and the summary this is the gate as it stands now completely in Cologne the benefits for rail operators are big they are getting freeing their personal from tedious tasks they're freeing their sites which they don't have to build so we

are allowing them to expand their fleets while not expanding their sights and they can make better predictions on their maintenance they can schedule better they um we have we are basically because it's a camera gate we are a manufacturer independent currently it's ice trains we are working together with the daughter of daughter company of Deutsche ban to turn E-Check into smart assistant maintenance and roll it out for high-speed trains across the world um infra views the company to name them and what you could do as an Outlook you could obviously track the vehicle life cycle and analyze how it is changing over time thank you very much um some final remarks Maybe we don't have a VC still interested in them possibly we are a very Tech heavy company so if you're Tech developers feel free to reach out even more interesting if you're a non-techy but want to work in robotics I'm really looking for people and if you're a potential customer please also be in touch thank you very much for the attention

and I'm open to your questions thank you very much hey I know stay right there we've got some time so I've got some questions for you and please feel free to put your hand up if you do have a question but given the VC piece and putting my investment hat on um I'm trying to understand word there's a lot of different projects that you do there and it feels very much like an agency or proof of Concepts that you then move forward into actual production you talked about some form of Ip is there a consistency in what you're doing across these projects that set your core IP yes um so at the at the core of it stands one the capability to set up those systems and to individualize them to Industries so for example the laboratory automation is something that could be used in any Laboratory um each shape could be used by any high-speed operator um and the core of many solutions is an mlops pipeline that we have in the back understood

so that's good to know and a final piece around your project with Deutsche Bank which I think is a fantastic one um what does that relationship look like and is it a sort of co-headed speed or you're an extension of their workbench it's beautiful to see the ecosystem coming together with a big corporate another metal stand and then a startup and maybe share some lessons on that because I think we could all learn on how to bring together different component skill sets to actually get a project like that off the ground so um it was a public tender we applied together with drama okay um we got it and so we are the supplier of the technology system E-Check as a project is a project a top project by Deutsche Bahn in which they are building also new maintenance sites in which they are sort of having their project site on the end there are more people than we are in their project we are working very closely with especially with their AI team to

build sort of the custom-made solution that they wish for and so it's it's not a workbench we are clear it's a clear supplier relationship that's also why supplier award um but we have them for example in our scrum events we have them in there and they give their feedback immediately because we don't want this we work for one year two years three years and then just hand it over and they say no yeah and that doesn't work and that will help your investment case no doubt so uh if there are any other questions from the floor because we are running to the full half an hour so anyway thank you very much for a very insightful talk thank you okay so on to our next speaker who I have to say is probably one of the coolest startups that I've seen

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.