Design And Implementation Of An Integrated Platform For Development And Deployment Of Edge Intelligent Applications

With more and more EI applications deployed at the network edge,deep-learningbased streaming data processing will be the most important computing task of edge nodes.However,at current stage we are faced with dual difficulties,i.e.,not only lacking of a unified framework for efficient and flexible development of edge intelligent applications,but also lacking of effective methods to efficiently deploy them on CPUGPU heterogeneous nodes.This article proposes SmartPipe,a platform based on CBD(Component-Based Development)for fast development and efficient deployment of edge intelligent applications on edge nodes.SmartPipe provides a fast development method that allows developers to use templates to customize their own components,migrate open source projects quickly by reusing the code and running environments of native projects,and define application logics by simply building a directed acyclic graph between components.SmartPipe designs a fully asynchronous platform framework for efficient inferencing deployment.It decomposes processes into finer-grained functions according to the dominant resource used,which not only decouples the allocation of different types of platform resources but also allows developers to build application pipelines that are easy to scale resources.On this basis,it employs techniques such as platform-and application-aware resource allocation,globally optimized task combination and placement,non-preemptive and non-blocking task scheduling,etc.,to fully utilize platform resources while guarantee application performance.Taking a license plate detection and recognition application as an example,this article also illustrates how to use SmartPipe to develop and deploy a edge intelligent application,and evaluates the design and implementation of SmartPipe in detail.Experiments using real traffic video show that a single GPU card(Nvidia RTX 2080Ti)can support up to 7-way video input with 82%GPU utilization rate in throughput preference scenario,and gain an average end-to-end delay of 0.728s in delay preference scenario.This article developed some of the same applications as MediaPipe and compared the operating efficiency on edge nodes.In the comparative experiment of the single-person pose estimation application,the real-time processing frame rate was increased by 1.64 x compared with MediaPipe in the delay priority mode,and the real-time processing frame rate was increased by 13.13 x in the throughput priority mode;32.61 and 108.70 x higher real-time processing frame rate.