| The 3D shape reconstruction method based on multi-depth-of-field image sequences uses images with different depth-of-field and image self-contained scene information to jointly derive depth cues,and its high reconstruction efficiency and low hardware cost are beneficial to realize multi-scene applications of 3D reconstruction.However,due to the complex depth-cue relationship of existing multi-depth-of-field image datasets,the 3D shape reconstruction models could not maintain the performance under the unknown scene conditions.In addition,the existing convolutional neural network model couldn’t effectively fuse the global spatio-temporal information of multi-depth-of-field image sequences,and couldn’t accurately capture the transfer information of focus between image frames and the change of focus area of single frame images.To address the problems of data sets and network models,this paper conducted the following studies on the 3D shape reconstruction.(1)The existing 3D shape reconstruction dataset was difficult to build a unified dataset due to the heterogeneity of image acquisition methods,imaging principles and application scenarios,and the mixing of many depth cues.The convolutional neural network model,as a data-driven method,rely heavily on large-scale labeled data to establish causality,which is difficult to adapt to 3D shape reconstruction in multiple scenes.To address this problem,the proposed dataset had stripped the strong correlation between the actual semantic information of the image and the depth information of the scene,and guided the highly robust convolutional neural network model to learn 3D shape reconstruction.The dataset was simulated by combining the scene image with texture-rich features and the 3D morphological map with depth homogeneity and step coexistence into the point diffusion model,and the unique morphological kernel function was proposed to increase the diversity and multidimensionality of the depth information contained in the image sequence.The proposed multi-depth-of-field image dataset not only provided a fundamental test dataset,but also helped to realize the cross-scene and cross-domain application of convolutional neural network model training.Based on this dataset,this paper proposed a local-global coupled 3D shape reconstruction model.In the traditional3 D shape reconstruction test dataset,the network model constructed based on the proposed dataset outperforms the existing traditional model in terms of focus measurement,weak texture regions and sparse sampling;in the 3D shape reconstruction network model comparison experiments,the proposed network model outperforms the existing network model in terms of depth accuracy.(2)The core essence of the convolutional neural network-based 3D shape reconstruction model for accurate prediction of 3D shape in a scene lies in accurately capturing the subtle changes of focus between image sequences and edge identification of focus regions within the image.In response to the inability of existing models to effectively fuse the global spatio-temporal features of multi-depth-of-field image sequences,this paper proposes a global spatio-temporal feature coupling model to extract local and global spatio-temporal feature information for adaptive fusion,and design a multi-level depth-focused volume module to accurately identify the transition information of depth-of-field.The backbone of the model with U-shaped structure takes into account the efficiency while ensuring the accuracy,and proposed to use 3D-Swin Transformer module to constitute the global feature attention.First,the 3D-Conv Ne Xt module and 3D convolutional downsampling layer are interpolated to capture multi-scale local spatio-temporal features;meanwhile,the 3D-Swin Transformer module captures the global correlations of local spatio-temporal features of multi-depth-of-field image sequences;subsequently,the local spatio-temporal features and global correlations were fused into global spatio-temporal features by adaptive parameter layer,which leads to the generation of focus volume;finally,the focused volume extracts the sequence weight information by deep attention and retains the transition information between focused and defocused.The effectiveness of the proposed model is verified by three parts of ablation experiment,comparison experiment and sparsity experiment,and the results show that the global spatio-temporal feature coupling model proposed in this paper outperforms the state-of-the-art Ai FDepth Net deep learning model and the RR-SFF conventional model,and the proposed model can retain more depth-of-field transition relationships.In summary,this paper constructed a multi-depth-of-field 3D shape reconstruction dataset with cross-scene adaptability and a global spatio-temporal feature coupling model with global focus information aggregation capability to achieve high-precision 3D shape reconstruction,respectively.The validity of the proposed dataset and model was verified through experimental comparison and analysis. |
