Research And Implementation Of Spatial Positioning System Based On Multi-modality

Currently most smart devices need to be combined with the working environment in which the device is located when running the application.In order to adapt the device to the environment and execute work order accurately,it's necessary to ensure that the device can be accurately positioned anytime,anywhere.The position of device in the environment including the position and the orientation of the device can be confirmed by positioning.The positioning problem can be classified into three types according to the operating state of the device in the environment.When running in an unknown environment,the device needs to solve the problem of global positioning.Because the initial pose of the device in the environment remain unknown,it is necessary to obtain environmental data through the sensor to position itself.When the device is running,it usually needs to solve the problem of pose tracking,since the initial pose of the device has been confirmed in the established map,the device only needs to compensate the pose error by using the environmental data acquired by sensors.When the device has confirmed its own pose,directly changing the position of the device without pre-processing will lead to more complicated positioning,this problem is called machine abduction.All of the above are problems that need to be solved for current smart device positioning.Aiming at the above problems,this thesis studies and implements two sets of different spatial pose estimation systems for intelligent mobile robots and mixed reality devices,which can locate their position and attitude in space with higher precision.The pose estimation system of the mobile robot achieves an accurate estimation of the pose of the device by combining the operational data acquired by the inertial navigation system(INS)with the pose data output by the visual odometer.The current common visual positioning system can use a visual odometer to estimate the motion trajectory of the carrier according to the adjacent image frames acquired by thecamera and restore the spatial structure of the scene,but the visual odometer only estimates the camera position by calculating the image change at the adjacent time.And the visual sensor itself is susceptible to device capability,ambient lighting changes,and device motion speed,therefore cumulative drift is inevitable when using a visual positioning system to position the device carrier.The inertial navigation system can measure the ratio of the carrier to the sensor by the inertial measurement unit.The accelerometer and the gyroscope equipped with the inertial navigation system can accurately obtain the motion acceleration of the carrier and the tilt angle of the moving direction of the device.The visual and inertial coupling positioning system implemented in the thesis adds the camera pose estimated by the visual odometer and the motion data acquired by the inertial navigation system to the state vector for calculating the positioning pose,The multi-modal positioning system can acquire multi-dimensional motion data to accurately estimate the pose of the device and optimize the effect of device positioning.By comparing the pose error results of the method of combining inertial navigation with vision with the results of the pose error of only relying on single sensor output,the experimental results show that the method of inertial vision combined with the method of single vision sensor can output pose result more accurately.A system for locating real objects implemented by a mixed reality intelligent terminal is a positioning system for locating a real object by a mixed reality device.Since the real object itself does not necessarily have a feature that can be quickly recognized,this thesis proposes to fix the identifiable identifier on the surface of the real object.The method enables the visual sensor to detect and locate the identification of the real object,thereby estimating the coordinate position of the real object in the virtual reality device coordinate system.In order to achieve accurate positioning of real objects,two different identification positioning methods are adopted in this thesis.One of the identifications that is easy to be detected but not suitable for real-time positioning is used as the basis for positioning system to confirm the detection area of the target object,and the another identification with abundant spatial features is used for the calculation of the pose of the target object by the positioning system.Real-time tracking system based on multi-modal information can be realized by detecting and identifying different kinds of identifiers,this system can solve the global positioning problem of real objects in the environment coordinate system established by the mixed reality device.The experimental result indicates that the mixed reality device positioning system implemented in this thesis can accurately locate the target object in the real environment in real time.