Real Time CNN Based Detection And Tracking Of Unmanned Aerial Vehicles(UAVs) Using Pan-Tilt-Zoom(PTZ) Camera

The commercial Unmanned aerial vehicles(UAV s),specifically civilian drones,were remarkably increased in the last two decades because of their application areas.Also,be-cause of the low price and the fast growth of technology,the devices highly available in the public market.This condition triggered the security concerns to the public as well to the government because these tiny devices intentionally or unintentionally can cause serious hazards.In recent years,the academic researcher and industry have been proposing several solutions to protect critical locations.The researcher and industry recommended a variety of solutions such as using radar,acoustics,lidar,radio frequency(RF)signal analysis,and com-puter vision-based to detect and track a drone.From the recommended approach,recently,computer vision widely used in identifying and tracking drones because of its robustness and effectiveness.However,due to challenges like high computational time,a variation of illumination,shadow inferences,the smallness of objects,background inferences,scale vari-ation,occlusion,in-plane,out-plane rotation,camera motion,etc.it is difficult to identify,detect and track a drone effectively.In this thesis,the system present,autonomous drone detection,and tracking system that uses an infrared ray(IR)camera mounted on a rotating turret.In order to implement a real-time convolutional neural network(CNN)based detec-tion and tracking of drone using the pan-tilt-zoom camera,we utilized two parallel working algorithms.The parallel algorithms used offered the ability of the system to be flexible and independent.The parallel algorithm divided into the detection algorithm and the tracking algorithm.In the detection algorithm,detection,identification,and localization of drone using deep learning techniques are applied.Currently,the deep learning approach is an effective and robust method in the computer vision area,which overcomes the drawbacks of radar,acoustics,lidar,RF,and classical image processing technique.The deep learning approach employed in the detection algorithm uses the input image to find the drone in the scene.The deep learning approach is designed by the utilization of CNN with the image processing techniques in the sequence manner.The CNN used in the method is designed to be effective and robust regarding the computational time,performance,and accuracy while detecting the drones.In this work,the utilized deep learning algorithm uses two different input sizes.The first deep learning uses full-frame size and process using the CenterHedNet.Once the drone is detected,then the region of interest obtained from the previous state used as scaling and cropping boundary region for the next frame,and in the next frame,the CenterHedNet skipped,and the frame scaled then cropped according to the region of interest obtained from CenterHedNet.The cropped frame further resized to the fixed ratio for the CropNet,which is the second deep learning framework.The region of interest further uses the deep learning approach to localize the drone from the sequence of images without the utilization of CenterHedNet.CenterHedNet and CropNet use the keypoint detection to identify and detect a drone;however,as we discussed,the input for the network is varied.Making this will enable us to focus on the region of the target only.In the tracking algorithm,based on the measured state received from the detection side,a Kalman filter is employed to provide a coarse prediction of the drone state and track the drone by controlling the two-wheeled rotation motor.The drone controller algorithm toward center(TC)is used to make sure that the drone center is always at the center of the camera image plane.The camera used in the system serves as a feedback sensor to guide the rotating turret regularly towards the drones.Overall,the system reproduces the eye-tracking ability of the human being,when people tend to focus on moving objects within the nearest distance of their view before he/she took any action.At the system design,the system carefully designed to work in real-time and detect small targets from a far distance.In order to boost the computational time and make to work in real-time,the method utilized a free anchor box detection algorithm,which effectively increases the processing speed of the detection,the implemented system run at 66 FPS at the average speed.Furthermore,to detect the small target,the method utilized the region of interest approach;the system can identify 4 x 4-pixel size of the drone.