Research On Key Technology Of Electronic Image Stabilization Based On Composite Motion Estimation

Electronic image stabilization based on composite motion estimation refers to a method of implementing image stabilition in combination with an inertial measurement IMU and a camera.A single camera can not accurately estimate motion in scenes where there are insufficient features or where the texture is not obvious so that video may not be well stabilized,an IMU used can effectively avoid this problem.In order to overcome the technology limitations of the single calibration method and achieve better and more accurate motion estimation,in this paper,a new method based on neural network is proposed for the relative pose calibration of IMU-Camera in electronic image stabilization applications.In the algorithm,according to IMU inertial data and true camera attitude,the mapping function between inertial measurement sensor and camera is fitted by self-learning.Then camera pose can be estimated by inertial data and composite motion estimation can be relized.For the purpose of reducing additional camera attitude measurement equipment,training data is collected in scenes with rich features,and the camera pose estimated by extracting feature points is taken as supposed camera pose.Secondly,regard the optimal smooth trajectory problem as a least-squares fitting problem and smooth the Euler angles to remove high-frequency random jitter.Rotate the spherical image towards the oppsite direction according to the rotate matrix causing jitter,then a stable video sequence is finally obtained.The experimental results show that the IMU-Camera calibration method based on BP neural network presented in this paper can accurately reflect camera pose from IMU data.By smoothing camera trajectory obtained by IMU inertial data according to trained BP neural network and motion compensation,the random jitter is effectively suppressed.The proposed composite motion estimation algorithm can overcome the scene limitation and achieve electronic image stabilization with high robustness and practical value.