Penetration State Recognition Based On Deep Learning And Weld Pool Image Pairs

Gas metal arc welding(GMAW)is characterized by high efficiency,low cost,and easy automation.However,detecting the penetration status in GMAW is challenging due to its dynamic and multi-factor interference welding process.Developing new,adaptable GMAW penetration detection theories and technologies is an urgent need for promoting the further application of robots and automated welding.This thesis proposes a passive stereo vision sensing and deep learning-based method for recognizing the penetration status,which simulates the welder’s binocular vision and experience to identify the penetration status based on the three-dimensional information of the weld pool surface.The Research focuses on the acquisition of three-dimensional information from the weld pool surface,end-to-end depth estimation of the weld pool surface,and penetration state recognition based on deep learning neural network.A passive stereo vision system based on single camera with biprism was designed,and the calibration method for the camera’s internal and external parameters was studied.The Zhang Zhengyou method was adopted to obtain the visual system’s internal and external parameter matrix.Virtual stereoscopic image pairs of weld pool surface were collected for CMT bead on plate welding and V-groove butt condition welding with different welding parameters.A accurate and reliable stereo matching data set containing disparity of the weld pool surface is constructed.To solve the problem of stereo matching caused by the lack of texture on the weld pool surface,a globally optimized variational stereo matching algorithm was introduced.By establishing the feasibility functional of the energy function with gray-scale difference data items and spatial continuity constraint items,a complete dense disparity map of weld pool surface containing abundant rich-details was obtained by iterative solution.The error analysis was carried out on a self-made non-standard concave surface,and the 3D reconstruction results showed that the relative errors of width and depth are less than 3.16% and 4.82%,respectively,which met the test requirements.The recognition model of weld penetration state based on weld pool surface depth information was established.In order to simplify the difficulty of model training,end-to-end deep learning stereo matching network and classification network were trained respectively.Pyramid stereo matching network(PSMNet)was trained and optimized on a network data set Monkaa and self-made weld pool data set.Based on the output of stereo matching network,a classification data set is constructed by stratified sampling method,the output of stereo matching network is taken as the input of classification network.In order to improve the model’s industrial applicability,five classic convolutional neural networks were evaluated and selected based on the consideration of model performance and complexity.The optimal classifier selected in this thesis was an 18-layer residual convolutional neural network(ResNet).Throuth the output disparity of the weld pool surface from PSMNet to drive the training and optimization of ResNet,the classification accuracy of the model on the test set reached 99.6%,and the four penetration states including partial penetration,full penetration,over penetration and burn through was achieved.The deep learning interpretability strategy was used to comprehensively evaluate the model performance and visually analyze the decision logic of the proposed method.This thesis provides a new welding penetration detection method,which lays a foundation for the application of stereo vision and deep learning neural network in welding engineering.