Research On Part Recognition And Assembly Monitoring Based On Depth Image

In order to better adapt to the new situation of manufacturing industry and meet the growing needs of consumers,the traditional assembly workshop urgently needs an intelligent assembly monitoring system,which can intelligently identify the parts of the assembly and monitor the assembly process.Aiming at the problems of parts identification and assembly monitoring in mechanical product assembly a method of parts recognition and assembly monitoring based on depth image is proposed.Compared with ordinary color images,depth images not only can detect three-dimensional information of scenes,but also have strong ability to resist environmental factors such as illumination interference,chrominance interference and shadow interference,so they are concerned by computer vision.Compared with ordinary color images,depth images can not only detect the three-dimensional information of the scene,but also have strong ability to resist the interference of environmental factors such as illumination,chroma and shadow.Therefore,the depth image is concerned in the field of computer vision.The main tasks are as follows:(1)The assembly depth image marker sample library is constructed.The depth image labeling sample library mainly includes synthetic depth image labeling sample library and real depth image labeling sample library.When constructing the label sample library of synthetic depth images,firstly,the three-dimensional model of the assembly is built,and the parts are labeled with different colors;then,the depth image of the model and the corresponding color label image are synthesized by artificial synthesis method;finally,the model depth images and color label images from different perspectives are obtained by rotating the model,and the label sample library of synthetic depth images is constructed.When constructing the sample database of real depth image,firstly,the true depth image of assembly is collected by depth sensor,and the Cavity filling and smoothing are processed;then,the parts of the assembly are marked by artificial marking;finally,the real depth images and color label images from different angles of view are arranged to construct the real depth image tag sample library.(2)The PX-LBP operator is proposed.Aiming at the problem that classical LBP operators can not be well integrated with pixel classification methods,an improved LBP operator named PX-LBP operator is proposed.Compared with the classical LBPoperator,the PX-LBP operator increases the number of initial pixels,the number of neighborhoods of each central pixel and the number of LBPs generated by each neighborhood.Finally,pixel classification based on PX-LBP operator is realized.(3)The depth difference feature extraction algorithm is improved.Aiming at the problem of poor adaptive ability of classical depth difference feature extraction algorithm to offset vectors of foreground edge pixels,the edge factor is introduced to improve the classical depth difference feature extraction algorithm.Compared with the classical depth difference feature extraction algorithm,the improved depth difference feature extraction algorithm has significantly improved the pixel recognition rate of the real depth image,and improved the denoising function.(4)Pixel classification of depth image of assembly is realized.PX-LBP features and depth difference features of depth images are extracted respectively.The random forest classifier constructed by random forest correlation algorithm in OpenCV machine learning library is selected for pixel classification.Determine the relevant parameters through experiments and output pixel classification to predict images.The pixel classification of depth image of assembly is realized.Finally,PX-LBP features and depth difference features are compared from the aspects of feature acquisition efficiency,image recognition efficiency and pixel classification accuracy.(5)A method of parts recognition and assembly monitoring based on depth image is proposed.Comparing the pixel prediction image with the color label image,the RGB value of each part in the color label image is obtained according to the corresponding RGB value of each part in the color label image.The RGB value of each pixel in the image predicted by pixels is analyzed,and the corresponding relationship between RGB value and each part of the assembly is combined to realize the recognition of each part of the assembly.Comparing the state pixel prediction image with the correct assembly pixel prediction image,the pixel coincidence rate and the pixel reduction rate of the state pixel prediction image are calculated and analyzed.The assembly monitoring function is realized.The experimental results show that the proposed method not only has high accuracy,but also has a certain degree of real-time and robustness.It has a certain application value in the field of assembly maintenance guidance,assembly monitoring and automatic assembly.