Investigation Of Deep Learning For Image And Video Capturing And Editing

With the development of the Internet and social networks,multimedia formats such as image and video have filled the whole network.The main reason is that they have a greater amount of information and a more vivid way of presentation compared with plain text.More and more users share their daily life and travel experiences through image and video on the Internet.In general,users spend a lot of time and energy on boring and repetitive work in the process of capturing and editing materials.For example,exposure adjustment,image cropping,video clipping,post-effects and so on.Therefore,it is urgent to use artificial intelligence technology to assist users to capture and edit image and video and free users from boring operations.Based on this,this paper applied the most advanced deep learning technology in the field of machine learning to study the relevant issues in the process of image and video capturing and editing.In addition,the paper proposes innovative work and verifies its effectiveness in practical scenarios.Specifically,three representative issues in the field of image and video capturing and editing are selected in this paper which are exposure control problem in the process of image and video capturing,image inpainting and object removal problem in the process of image editing,video highlight detection problem in the process of video editing.Through the analysis of the difficulties in the above problems,combined with the deep learning technology,reinforcement learning technology,generative adversarial training technology and unsupervised learning technology,which are widely used in the field of computer vision and computer graphics,this paper proposes systemic-level solutions to different problems.Therefore,this paper has strong practical significance and practical value.Firstly,for the exposure control problem in the process of image and video capturing,compared with the traditional point-based,center-based and matrix-based metering algorithms,this paper proposes a two-branch network structure including attention branch and exposure branch,which controls the exposure importance and exposure adjustment value of local areas respectively.Through the above network structure,our system can more smoothly predict and control the exposure reference in the scene,thus greatly improving the exposure ability of the system.The above system has been further verified on the public dataset and achieved excellent results.Secondly,through further analysis and research on user data in the above-mentioned exposure control system,the personalized requirements of exposure control are found.To meet the above requirements,this paper proposes a personalized exposure control system based on deep reinforcement learning.Through the application of the deep reinforcement learning technology in our system,the problem of data label collection in the process of model training is overcome,and the training process of our system is greatly simplified.In addition,in the case of using limited data labels,this paper achieves results close to the supervised learning method.The above exposure control system provides great convenience for users to capture images and videos.Such excellent multimedia materials obtained through capturing laid a solid foundation for the later editing work.On the basis of the above,this paper proposes an image inpainting system based on smooth partial convolution and adaptive feature fusion for image inpainting and object removal in the process of image editing.Through the smooth partial convolution,the missing part in the image is completed successively from outside to inside according to the corresponding mask.In addition,through the adaptive feature fusion technology in the decoder,different levels of features are fused,and the completed image is finally obtained.In particular,this technique can be further applied to one of the most important problems in the image editing process which is the object removal problem.It refers to the removal of redundant people or objects in the scene through a user-defined mask.Our system has been verified on the existing public dataset and achieved state-of-the-art performance.In addition,it is further tested in the actual scene and has achieved good inpainting results.Finally,in terms of the video highlight detection problem in the video editing process,this paper proposes an unsupervised learning system based on the auto-encoder.The system analyzes the difficulties in practical application,overcomes the difficulties in data label collection,and proposes a method based on unsupervised learning technology.In this paper,the training data of our system are expanded simply and efficiently by a web crawler,and the scale of training data is raised by an order of magnitude easily.On the above basis,aiming at the video feature encoding problem,this paper proposes a feature encoding method based on the combination of 3D convolution and long shortterm memory cell.It encodes the dependencies between video frames from different granularity and achieves excellent results on large datasets.In particular,the proposed system is also compared with the existing supervised learning methods on public datasets and achieves similar results in some categories.In conclusion,this paper successfully applied the deep learning technology to the field of image and video capturing and editing.In view of the different problems,this paper proposes innovative and effective solutions.The proposed system is also tested in the actual scenario and achieves state-of-the-art results.It has strong scientific research innovation and practical application value.