Research On 3D Reconstruction And AR Tracking Registration System Based On Furniture Image

In the post-epidemic era,virtual reality technology has become a hot spot for social research.With the popularization of the concept of "metaverse",Augmented Reality(AR)technology is slowly applied to people’s daily life as its underlying technology support,and the integration of virtual world and real world information is widely used in military,medical and educational fields.The integration of virtual world and real world information is widely used in military,medical and education fields.In recent years,the rise of the Internet e-commerce industry has made it more and more common for people to buy goods without leaving home,but due to the relatively large size of physical furniture,it usually requires offline shopping in stores.If purchased to the home is not suitable also need to replace and other complicated processes,which not only increases the merchants to replace the furniture required labor and freight costs,and greatly reduces the user shopping experience.Most of the AR furniture purchasing software on the market requires 3D modeling of furniture in advance,and hundreds of furniture in different shapes and sizes require a lot of labor and financial resources.In this paper,we develop an AR home environment system,which can reconstruct the 3D model of furniture on shopping websites in real time,and use AR 3D registration technology to display and control the furniture model on mobile.The main research work of this dissertation are as follows.(1)The 3D reconstruction algorithm based on a single furniture image is studied.For furniture images containing complex backgrounds,background removal and instance segmentation are performed,and a Bayesian classifier is added to the traditional foreground background separation technique to improve it,and a Bayesian Classifier and Grab Cut(BCGC)algorithm is proposed to obtain furniture images with clean backgrounds.The reconstruction part is based on the classical occupancy network reconstruction algorithm and proposes a fine occupancy network(PONet)algorithm,adding camera position information as input,improving its encoder and decoder modules,adding side branch probability prediction and designing a probability mixer to mix the main branch and side branch probabilities.The evaluation index of the model is set and compared with the state-of-the-art reconstruction algorithm,which can reconstruct the structural details of furniture pictures and improve the accuracy of the model.(2)The 3D registration algorithm based on AR technology is studied.For the traditional 3D registration model is prone to the problems of model position shift and inaccurate matching with the scene,the AKAZE-based tracking registration algorithm is improved,and a multiple filtering matching points based on AKAZE(Multiple Filtering-AKAZE,MF-AKAZE)is proposed.By constructing the scale space,FREAK(Fast Retina Keypoint)is used in the description of feature points for effective description,FLANN(Fast Library for Approimate Nearest Neighbors)is used to coarse match the feature points,and the matching results are further filtered to increase the matching accuracy based on the improved RANSAC(Random Sample Consensus)filtering mis-matching algorithm.The spatial coordinate conversion relationship is obtained by estimating the camera pose,and adding light information to the model to increase the sense of shadow to strengthen the effectiveness of the fusion of reality and imagination.(3)The AR home environment system based on 3D reconstruction of images in mobile is designed based on Python and C# programming language.The main interface of the system was designed using Python language,and the functions of importing furniture pictures,image segmentation and display,and 3D reconstruction and display were realized on the PC side.The reconstructed model Obj format file is imported into Unity3 D,the model is controlled by gestures based on ray collision detection technology,the interactive system of model parameter measurement and display is designed and developed,the model can be changed arbitrarily according to user requirements and the furniture model parameters are obtained for online procurement,the personalized furniture model is reconstructed in real time from furniture shopping websites and the furniture is visualized and interacted with by AR The feasibility of the system was verified.