Rise of AI Summit 2020 · Berlin · Applied AI Day | Future Space

AI Computing Power at the Edge: Industry Use Cases

Evgeniy Vlasov · Huawei Technologies

At Rise of AI Summit 2020 in Berlin, Evgeniy Vlasov, senior alliance and business development manager at Huawei Technologies, explains why AI processing is moving from data centers to the edge, close to where data is generated. He argues that classical CPUs are not built for AI workloads and that over 90 percent of edge AI use cases today are inference rather than training, so edge devices need dedicated chipsets and must balance computing power against energy consumption, especially for battery-powered robots and drones. He presents Huawei's Atlas edge AI stack, including the Atlas 200 module, Atlas 500 edge station and Atlas 300 inference card, built on Huawei's own chipsets and the open-source MindSpore framework, which also supports third-party hardware and frameworks such as TensorFlow and PyTorch. He then walks through concrete deployments: autonomous delivery vehicles from partner Neolix, computer-vision quality control in manufacturing, and fleet management for automated guided vehicles with partner Gestalt Robotics, arguing edge devices work alongside the cloud rather than replacing it.

Key takeaways

  • Over 90 percent of current edge AI use cases are inference rather than training, according to Vlasov.
  • Edge devices are meant to work together with the cloud, not replace it, for storing data and retraining models.
  • Huawei's MindSpore framework is open source and works with third-party hardware like NVIDIA and frameworks like TensorFlow and PyTorch.
  • AI-based computer vision quality control can raise manufacturing defect detection from under 90 percent to around 99 percent.
  • Running AI processing on a dedicated edge server instead of onboard the robot extends AGV battery life by about 8 to 25 percent.

Detailed notes

Why AI is moving to the edge

Vlasov describes how AI long stayed in data centers with sufficient compute power, but as software and hardware improved and became more affordable, AI began handling specialized tasks with lower resource needs. He attributes the shift to the edge to the rise of consumer devices (smartphones, smartwatches, smart home systems) that need AI capability without relying only on the cloud, improved wireless and fixed connectivity, and, in industry, a demand for the next level of process automation in manufacturing, autonomous driving, healthcare and smart cities. He defines edge AI as algorithms or applications processed locally on hardware close to the data source, avoiding the need to stream or store large volumes of data in the cloud, which matters for use cases needing high-speed processing with minimum latency, such as factory robots or drones. Edge does not require building an AI module into the device itself; it can also be an AI server in a factory network or an edge station near the data source.

Requirements for edge AI deployment

He notes AI computing needs dedicated chipsets, since classical CPUs can run AI algorithms only slowly. Of the two main milestones, training and inference, over 90 percent of edge AI cases are inference. Edge devices must balance computing power against energy consumption because many, such as drones, electric vehicles and robots, run on batteries, which limits how many AI scenarios they can run simultaneously without losing speed or efficiency. Edge and cloud work together rather than the edge replacing the cloud: data can be stored, analyzed and used to retrain models, which are then redeployed to multiple devices. He stresses the importance of a universal platform that lets developers build an AI application once and deploy it across different device topologies and scenarios, plus industry-specific SDKs for domains like robotics, manufacturing and computer vision to simplify development. Managing large fleets of AI-powered devices, for example in traffic monitoring or video surveillance, requires systems for device management and maintenance, data analytics, model retraining and redeployment.

Huawei's Atlas edge AI stack

Vlasov presents Huawei's Atlas AI computing solution, described as "all scenario, full stack": it covers public and private cloud, edge computing and industry-specific deployment scenarios, and spans the full stack from AI acceleration hardware based on Huawei's own Ascend chipsets through to application development tools. The stack includes Huawei's own MindSpore framework, which is open source on GitHub and, while originally built for Huawei's own chipsets, also works with third-party AI acceleration hardware such as NVIDIA's, and developers can alternatively use frameworks like TensorFlow, Caffe or PyTorch. On the hardware side he highlights the Atlas 200 AI module, roughly the size of half a credit card and consuming up to 8 watts, which can be integrated into cameras, drones or robots; the Atlas 500 edge station, a compact standalone device with Wi-Fi, LTE and Ethernet connectivity and passive cooling for harsh environments; and the Atlas 300 inference card, a PCI card based on four Ascend chips for installation in third-party servers.

Industry use cases

He describes Neolix, a strategic partner developing self-driving delivery vehicles used in hospitals, gated areas, parks and Winter Olympic facilities, with a modular design separating the autonomous driving platform from a customizable top layer for retail delivery, logistics or security patrol. Neolix vehicles use the Atlas 500 edge station for people and object detection, motion analysis and gesture recognition, such as stopping when a person waves. In manufacturing, edge servers run computer vision applications for real-time quality control, visual inspection and defect detection; Vlasov states traditional machine vision achieves under 90 percent detection accuracy, which AI-based computer vision can raise to around 99 percent while also increasing processing speed. For automated guided vehicle (AGV) fleets, built with partner Gestalt Robotics, he contrasts embedding AI acceleration directly into each robot with placing an edge server on the factory network to manage a mixed fleet, arguing the latter is more universal, modular in functionality and cost, and saves battery power, extending robot operating time between charges by roughly 8 to 25 percent depending on the robot.

Q&A highlights

Asked whether the Neolix vehicles are already deployed, Vlasov confirms the company has been running fleets for several years, though deployment is still limited by driving regulations in different regions. Asked who the main customers are, he says personally "everybody who is really interested in AI," but notes that in practice it depends on the country: in France and Germany manufacturing dominates demand, while in Spain and Italy the focus leans toward retail or medical applications, with the pandemic increasing partner work on AI in the medical sector. Asked about the roadmap, he points to further push toward edge-to-cloud collaboration where applicable, making devices more affordable in software and deployment, and more work on training rather than only inference, including developments related to video quality.

Quotes

For a long time, artificial intelligence stayed in data centers where there was sufficient computing power to perform processor demanding tasks.

Evgeniy Vlasov

My personal answer is everybody who are really interested in AI.

Evgeniy Vlasov

Questions answered in this talk

What is edge AI computing?

According to Evgeniy Vlasov at Rise of AI Summit 2020, edge AI refers to algorithms or applications processed locally on hardware located close to the data source, so data does not need to be streamed or stored in the cloud for analysis.

What hardware does Huawei offer for edge AI?

According to Vlasov, Huawei's Atlas line includes the Atlas 200 AI module (about half a credit card in size, up to 8 watts), the Atlas 500 edge station with Wi-Fi, LTE and Ethernet connectivity for harsh environments, and the Atlas 300 PCI inference card based on four Ascend chips.

How much can edge AI improve manufacturing quality control?

Vlasov states that traditional machine vision achieves under 90 percent defect detection accuracy, while AI-based computer vision algorithms can raise this to around 99 percent and increase processing speed.

Mentioned

edge computing, inference, cloud-edge collaboration, automated guided vehicles (AGVs), computer vision quality control

Keep exploring

More from Rise of AI Summit 2020

More on Enterprise AI & Adoption

Transcript

Auto-generated transcript, not edited. · 3,118 words

Show the full transcript

he will introduce us to industry use cases of ai computing power in edge he has a lot of experience in management business development and deep technical training today he's the senior alliance and business development manager at huawei technologies and i hope he already hears me evgeny vladlov are you there yes i'm here welcome great to have you um the floor is yours time for your talk thank you so much uh of course welcome to hawaii slots my name is evgeny as uh as you said i'm going to share some thoughts on ai computing in the age as well as some industry use cases hope is going to be quite interesting but before i start i would like to say thank you so much for our colleagues from rise of life for organizing such interesting events it's true and organizing it in so non-standard for all countries for all people uh time so let's start for a long time artificial intelligence state in data centers where there was sufficient computing power to perform processor demanding tasks

since that time ai made its way into software and hardware algorithms changed hardware modified become more affordable increased computing power starts performing specialized tasks and still start assuming quite low amount of uh so ai moved to the age of the networks uh you may ask what triggered that well there are several points there is a large amount of consumer devices appeared on the market during the last decade smartphones smart watches home smart systems and many others those devices increase the need in the eye in ai capabilities but it is saying draw their computing to the age on top of that connectivity developed a lot we have been widely using wireless and fixed networks so those devices can stay connected no matter where we are there is similar situation in different industries like manufacturing autonomous driving smart system healthcare smart city is healthcare but the tendency is slightly different while in the consumer business customers learn uei powered functionalities and benefits using those in industries there is a strong demand in the next level of processes

automation and optimization and it becomes life or better to say business necessity what is age computing especially in terms of er hei refers to the algorithms or wall applications processed locally on the hardware located close to the data source to the data service this means data analysis can be done without streaming big amount or storing big amount of data back to the cloud it's important um as the number of use cases which can't be handled via cloud is increasing requiring high speed uh processing with minimum latencies it can because uh factory robots many manufacturing live manufacturing processes application using drones or many others keep in mind when we speak about age it doesn't mean it doesn't necessarily mean building certain er acceleration module into a device like drone robot or vehicle it can be let's say an ai server installed in a factory network or an age station aih station installed close to a data service i just mentioned age devices do not need to stream or store data in the cloud do not keep

me wrong aih computing devices are not going to replace cloud but in fact those are going to work on works the work now together uh what's important for ai deployment in age uh let's keep in mind at first ai computing requires special cheap sets and hardware c classical cpus are simply not dedicated for uh such technology yeah they can accommodate and run uh ai algorithms but will do it on a very slow speed there are two main milestones for ai computing training and inference when we speak about ai computing engage over 90 percent of cases are related to inference so in this case what's important all scenario ideally aged devices as well as software should support as much use case scenarios as possible yeah h devices are very powerful but some of them are designed to have a good balance between uh computing power and energy consumption keep in mind that many devices still use batteries drones uh many electrical vehicles robots and so on such devices can limit use cases so better to say

running many ai scenarios in the same time hindstable computing power which can can accommodate high demanding ai applications having no confusion speed drops energy efficiency we describe that there is strong limitation for many age devices in terms of that cloud and age collaboration as mentioned age devices can work together with clouds there are different scenarios starting from storing valuable data indeed after ei analysis and lending with models update retraining against further deployment on the multiple devices if those working same network opening universal architecture it's quite important to have universal platform which allows to develop an application ai powered application once and deployed in on different devices considering different topologies and scenarios with minimum force and minimum changes industry specific there are there are various specific requirements for many use cases so industry-related sdks like the ones dedicated for robotics manufacturing computer vision and so on are pretty welcome that simplifies the development process of the applications what else management systems for example using cloud age collaboration scenario is one of the possible options many high

use cases really requires hundreds and thousands of ai powered devices to work together in the same system there are different examples it can be systems related to traffic monitoring or ipv surveillance or maintenance utilities or others such system includes many eye accelerator acceleration devices working together which should be at first managed and maintained so we are speaking about the following uh age device management maintenance uh then yeah valuable data processing analytics after running through ai algorithms so further processing afterwards and ai models updates or retraining if it's necessary and then it's further deployment on multiple devices uh most probably you heard that a couple of years ago huawei released the ai computing solution under the name atlas that means that we provide computing power hardware and software while our partners develop different applications for different versions there is a nice slogan in on top of the slide so what it really means all scenario means that the solution covers various high deployment scenarios including public or private cloud age computing industry related scenarios and so

on while full stack means that solution includes many different parts starting at first from the ai acceleration device based on devices based on our own chipset called atlas and covering uh two main milestones to main parts which is influencing training ken which stands for computer architecture for artificial networks can include cheap opera the cheaper operators library highly automated operation development tools and so on then uh framework uh huawei own framework called mindsport and it's actually universal i will say what does it mean and it's on the current department open source so you can find it on github or gt it's originally when i said universal it's originally designed for our own uh devices with own chipsets but the same framework can work with ai acceleration hardware from third-party vendors like nvidia for example on top of that uh our partners as an alternative to our own framework can use third party frameworks well known on the market like tensorflow cafe PyTorch and so on developing the applications over there and deploying using our tools with

minimum efforts indeed that depends therefore it depends on the use case but the efforts are really minimized and indeed this solution includes various tools for application development and making this process as more simple as it is possible so those tools include developments use industrial sdks pre-trained models pre-integrated solutions cloud services and so on and so on and so forth as mentioned based on our own chipset there is a range of ai accelerators available on under the brand atlas we are not going to touch the data center part or the part related to training as mentioned over 90 percent of high age use cases are related to inference colleagues please play the video the first video so please keep in mind this video is made a bit earlier than we are talking right now so at least couple of devices like one of the servers and reps for data settings so what do we have uh over here atlas 200 ai module with a size of a half credit card which provides a fantastic computing power

and still consume up to maximum 8 watts of power this module can be integrated in different devices like cameras drones robots or any other devices atlas 500 erih station the standalone device of a very compact size in high computing power the device has wi-fi lte and ethernet connectivity on board and can work in harsh environments thanks to the passive cooling system the next is quite classical in terms of form factor device it's a pci card atlas 300 inference card which is based on the four ascent chips and provide much more computing power normally those cards can be installed in third-party servers all the servers from huawei like adults 500 pro h server or the ones dedicated for data centers there are plenty of use cases for ai computing engage video analytics speech and text recognition quality control various manufacturing processes medical image analysis and so on i would like to share a couple of use cases from our side colleagues if you may play the second video that would be fantastic so new alex is

the company developing self-driving delivery vehicles this kenny is one of the by the strategic partner its vehicles are operating in hospitals bonded areas parks winter olympic facilities or other places vehicles have modular design the bottom layer is a autonomous driving platform while the top one is dedicated for different services which allows to customize vehicle for various applications like smart retail delivery vehicles logistics or security it can be a security patrol vehicle the legal deals with different road conditions providing high safety on the road to interact with different people analyze behavior of uh of people uh and provide feedback depending on for example about purchases uh people interaction with this vehicle and so on uh atlas 500 station is used in nuex vehicles powering different ai operations like people and object detection motion analysis uh people behavioral gestures for example the person or people can wave the hand and the vehicle will stop where and when it's necessary interaction feedback analysis depending on the main function and many others the reason of choosing such a stage

station is uh pretty straightforward uh using the device with full functionality on boards higher computing power and wireless connectivity as well as possibility to use such device in harsh environments uh colleagues play the lessons uh the next uh use case is manufacturing actually it's a very very light large area and the number of sub use cases in this vert is really enormous and in many cases are still not 100 discovered i mean possibilities but as mentioned there is a strong demand in the next level of a process automation and optimization this related to many use cases robots material flow warehouses automation digital twins safety and so on and so forth the devices like age servers process various i power it applications uh for example computer vision applications for real time quality control during manufacturing process visual control products defect detection mist of damaged components and so on to give you an example detection rate using traditional uh ways machine vision applications is less than 90 and variated quite a lot but it can be increased

up to 99 or even a bit higher with some digits after after chroma using the high computer vision algorithms in the same time processing speed during this inspection increased several times another example uh is related to intra-logistics what i mean is uh the autonomous navigation system for what systems better to say for automated automated guided vehicles agvs like the ones developed by our partner gestalt robotics actually there is several ways to build tnai acceleration modules in integrated whether to say i capabilities in mobile the first one is building ei acceleration module into a robot which has which has some disadvantages the second is to put mobile robot mitigation management system um in the factory network somewhere on the on the server and connect the fleet of agvs mobile robots back to a factory network using uh wireless connectivity the second option provide more benefits along life cycle first it makes platform more universal allowing to work with different agent vendors in the same time on mixed fleets uh an ngv management system can be uh

can be made developed by modular way providing different functionalities with different costs so if the customer do not need this or that functionality it's not going to be charged or it can broaden ai functionalities range especially related to high performance ai keep in mind that an h server meaning that this device is still located in the edge but it can provide much more computing power so more functionalities more speed more progress and so on faster deployments of mobile robots that's definitely easier with the second option system redundancy yes for sure longer operating time which is quite critical remember those mobile properties their robots are not permanently connected to any kind of power source so the use battery is according to our partners the power saving is quite significant because robots are not going to consume the power for algorithms processing and the difference is depending on the robot engine the function is from 8 to 25 which is significant so those devices those robots can drive longer providing the main function with the longer time

between charges and deploying software or firmwares on the fly including ai models updates retraining fine tuning if it's necessary of different software parts and so on that's it on my side colleagues we can speak ages the topic of er deep learning is so much interesting it's that there are so much details so each campaign each person who are deeply in this area can can share a lot of experience so you're welcome to ask questions if i may answer i'll do that if not we will answer in the chat box of uh swagger yes first let me tell you uh thank you for your talk uh and i'm not sure if it's typical german but i fell especially in love with neolix your uh your automatic automated car system is this something that is on the road already or is this a glimpse into the future right now uh if you're talking about new alex it is on the roads uh the company started deploying the fleet of those robots uh quite intensively several years ago

indeed the company appeared on the market for quite a long time so it's better to say new business for for new orleans and indeed uh it can be really massively deployed everywhere because there are certain limitations including related to the driving regulations and so on so the number of places where the vehicles like like that can be deployed and allowed to be deployed is limited but it's already on road it's a massive deployment according to my knowledge i checked that before presenting it right thank you we have a question from the audience uh who's your main client or who are your main clients to buy these solutions uh do you cater to ceos or rather to developers uh you can have my personal answer and the company answer my personal answer is everybody who are really interested in ai but if i would be a bit more serious that actually depends on the country better to say because each country even if we talk about europe have some kind of specifics yeah for example if

we take uh france and germany it's more manufacturing it's a lot of there is a lot of companies related to this process better to say or to this domain in some other countries like spain italy or others uh the main focus is towards let's say retail or uh medical you know the situation right now with uh pandemic so there is a lot of work done by our partners towards uh using ai functionality capabilities in medical sector so that actually depends on the country but there are main milestones in verticals where we play it's just mentioned right thank you and now we were of course uh or you were of course talking about edge solutions already but i'm curious what do you see um what would will your next steps be during the next couple of years do you have a roadmap for the products you showed us is there a next version already on your mind or in development uh there are ple plenty of products those are related to something i can share right

now there are some very interesting developments uh but just to give you the feeling at first uh the strong push for age to cloud collaboration when it's necessary it's not just typically push this area but when and where it's applicable that's the first point secondly making the devices more affordable on the software side on uh on deployment side and so on then if we talk about uh training during this session i've spoken mainly about inference because it's the main part of age computing for the moment in terms of ai but there are a lot of steps uh done and going to be done in terms of of training even some extra parts related to video infidelity and so on so something which is quite exciting but this way for me personally right thank you very much uh thank you very much i very much appreciate your keynote afghani and i would love to see one or two of the neolinks at maybe the next rise of ai thank you very much you can see it

not not only uh from the customer but uh there is an open lab in unix or there are plenty of other interesting uh use cases and devices and projects and so on so you can visit and see it in germany great thank you bye bye take care thank you so much stay safe and stay healthy

Rise of AI 2027 · Berlin · May 2027

Be in the room for the next one.

Ten years of talks started as conversations in one curated room in Berlin. The next edition brings 300 selected leaders together in May 2027.