Research On Scene Layout Algorithm Based On Deep Learning

In recent years,with the rapid development of VR/AR applications,while enjoying the sensory shock brought by the virtual world,the demand for automatic creation of high-precision virtual scenes has become increasingly significant.Nowadays,most virtual scene layouts are hand-made by professional designers based on experience,which is often accompanied by extremely high labor costs.Therefore,it is of great significance to design a layout algorithms that can quickly generate high-quality 2D or 3D scenes.For the scene layout problem,there are two major types of layout algorithms,rule-based and data-driven.The scope of this article belongs to the latter,that is,data-driven algorithms directly use neural networks to learn the potential distribution of the dataset,which has better scalability and lower labor cost due to the power of the network.However,some existing methods produce unrealistic scenes when solving complex layout problems due to the uneven distribution of datasets or poor network structure.In order to solve these problems,this paper designs a new deep learning framework from the perspective of network process,and also combines the idea of constrained optimization of traditional algorithms to further optimize the complex situation that the network cannot solve.The specific contributions are as follows:1.We propose a new indoor scene layout generation method NTFO(Non-autoregressive Transformers with Fine-grained Optimization),which redesigns the network architecture to solve the error chain propagation problem generated by previous autoregressive methods and improves the rationality of indoor scene layout.The core idea is based on the parallelism of Transformer Encoder,which converts the sub-scenes generated during the iterative process from input to high-dimensional constraints in a non-autoregressive way,and we design a new fine-grained optimization method to solve the problem of unrealistic scenes due to the uneven distribution of the dataset.The fine-grained optimization method constructs a cost function for the scene based on the a priori of the dataset and the geometric properties of the layout elements,and finally optimizes it using the L-BFGS algorithm.The experimental results show that our method has significant improvement in metrics compared with the currently existing algorithms,and also outperforms the existing algorithms in terms of visual quality from the visualization results,especially in solving complex indoor scene layout problems.2.We propose the MA＿Transformer(Multi-Attribute Transformer)network based on NTFO to solve the image scene layout problem.Most of the existing image scene layout algorithms extract the layout feature with a single encoder and a single decoder,such a design would ignore the variability between different attribute distributions.Our core idea is to improve the traditional encoding and decoding structure of image scene layout algorithm by proposing a multi-attribute transformer network,where each class of properties of the layout object is learned with a specialized encoder and decoder.Experiments show that our method outperforms existing image scene layout generation methods in terms of both visual effects and quantitative metrics.