Design Of Real-time Vision System Applied In Flapping-wing Micro Air Vehicle

In the past ten years,Flapping-wing Micro Air Vehicle(FWMAV)has gradually become a promising research direction in the field of aircraft and robots.However,as for real-time vision system of FWMAV,complete theory and technical route have not yet been established around the world.There is still plenty of room for innovation in algorithm design,hardware module selection,etc.The real-time vision system which includes digital video stabilization,person detection,obstacle avoidance and hardware module design is designed for the FWMAV developed by Shanghai Jiao Tong University.First of all,considering the characteristics of high-frequency flutter of FMAVs,a digital video stabilization algorithm based on Kalman and lowpass hybrid filtering is proposed in this thesis.Corner detection and LK optical flow method are used in this algorithm to calculate and filter the affine transformation matrix of two consecutive frames.Experiment shows that the algorithm diminishes respectively 66.56% and 73.15% of the aircraft’s horizontal and vertical flutter,and the processing speed of each frame is about 6ms.Secondly,as frames captured by FMAVs usually are noisy,lowbrightness and motion-blurred,an improved YOLOv3 person detection algorithm is proposed in this thesis.Constructing FMAV dataset,improving anchor parameters,improving loss functions,pre-training and fine-tuning are used in the algorithm.Experiment shows that the algorithm achieves an precision rate of 90.5% and a recall rate of 85.2% in self-built test dataset.At the same time,the processing speed of each frame is about 15.4ms.Thirdly,a real-time obstacle recognition algorithm based on ORB key points is proposed for the actual situation that FMAV can only take off with one single camera.The scale expansion algorithm proposed in this thesis is used to estimate the distance of FMAV to obstacles,and the frame-skip matching algorithm proposed in this thesis is used to ensure the real-time performance of the algorithm.A roll control signal is sent to the control mechanism when the distance is 1～2m.Experiment shows that the success rate of obstacle detection is over 86%,and the processing speed of each frame image is within 1～10ms.Finally,hardware of FMAV including micro camera,5.8GHz video transmission module and wireless video transmission receiving module is completed.Overall tests including system transmission test,function test and time-cost test are accomplished.Experiment shows that the system transmission delay is about 20.4ms;the video stabilization effectively filters high-frequency flutter in horizontal and vertical directions;the average correct rate of person detection reaches 90.28%;the average success rate of obstacle recognition reaches 85 %;The overall time cost of algorithms is within 30 ms,which meets the design requirements of real-time vision systems.