Research On Mobile Phone Video Super-resolution Method Based On Deep Learning

With the popularity of smartphone devices,mobile phones have gradually become an essential tool for people.The popularity of short videos and other information streaming methods has made videos and images the mainstream way of disseminating information on mobile phones.Higher resolution images and videos make for a better viewing experience.However,due to the high cost of hardware for mobile phones,it is difficult to achieve a better viewing experience simply by upgrading the hardware.In recent years,various high-performance video super-resolution methods have been proposed.However,these methods are cumbersome to deploy on mobile phones due to limitations in power consumption and computing power.Therefore,it is essential to design a new super-resolution network based on mobile phone performance as well as algorithms.Super-resolution methods are usually divided into image super-resolution methods and video super-resolution methods.The biggest difference between the two is whether or not they make use of the timing information between video frames.Some previous methods usually use optical flow etc.to directly compute the timing information between two frames,or implicitly model the timing information between adjacent frames using deformable convolution,3D convolution etc.However,all these methods impose significant additional energy consumption on the network and unconventional convolution such as deformable convolution is difficult to computationally accelerate on a mobile phone chip.This paper has the following three main contributions.First,for the super-resolution task on the mobile phone side,we design an energy-efficient super-resolution network(EESRNet)on the mobile phone side,which is obtained by removing the remaining connections in the anchor-based planar network(ABPN).Secondly,for the mobile-side video super-resolution task,we use a lightweight structure to generate hidden features to preserve the temporal information between adjacent frames.Combining EESRNet with the temporal information,we propose an energy-efficient super-resolution network(TEESRNet)that can efficiently utilize video temporal information with low energy consumption.Finally,in order to obtain a network with lower energy consumption,we find a streamlined version of EESRNet through model tailoring and extensive experimental comparisons,which enables the lowest energy consumption while maintaining good super-resolution recovery performance.Experiments show that the delay of EESRNet is reduced by more than40% compared to ABPN.In addition,the PSNR of TEESRNet is improved by 0.24 db and 1.19 db compared to EESRNet and RRN,respectively,while still maintaining the real-time performance(<30ms).