High-quality Reconstruction Of Visual Content For Asynchronous Address Events

"The Event-based Camera " was born based on a bionic sense named "Dynamic Vision Sensor(DVS)",it has a microsecond response time,only to record the location of the brightness of the sight in change,each position output an "event",which contains the address coordinates of each event as u,representing the coordinates(x,y),brightness information as p and time information as t,whihch is expressed as(u,t,p),multiple locations created "events stream" which is output asynchronously.Thus,DVS only view the location of the light intensity changed then output the event information,and not like a traditional camera whether sight in light intensity change,exposure the image frames rich in the strength of the background information timely,the whole frame contains all the data,so DVS can capture a moving object using less data,the response speed of the microsecond level also made it has a very high time resolution,but throw the static background where the light intensity doesn't change.In addition,the dynamic range of DVS(140d B)is more than two times higher than that of the traditional camera(60d B),which can sense the change of light intensity under exposure and dark light and output the corresponding event information.Therefore,compared with traditional cameras,DVS has the advantages of high time resolution and high dynamic range(HDR),low delay,no motion blur.But,because the asynchronous output of the camera is event stream over a period of time,rather than the traditional image record image frames,so the existing image processing algorithms cannot be applied directly,to processe the event stream directly is quite difficult,so the way of reconstruction algorithm can be the solution of the problem by reconstructing from the event stream to the traditional image,the reconstruction of the existing algorithms are based on a priori assumptions statistical reconstruction which was carried out on the event stream,another kind is using the deep learning network translate the asynchronous event stream to the clear image intensity directly.At present,for the reconstruction of event,the mainstream idea is to use deep learning networks such as LSTM,GAN etc.translating asynchronous event stream to intensity image frame directly.But the reconstruction algorithm does not take into account of that the DVS events under dark-light containing more random noise and detail losses,thus affecting the quality of reconstruction and for the above consideration,new reconstruction algorithms are designed for the detail reconstruction of event stream,and makes the dark-light event light can also be translate as clear intensity image,achieve the goal of high quality event stream cross-domain reconstruction,the main work and improvement are as follows:A detail enhanced domain adaptive network is proposed to achieve reconstruction of the event stream in this article,the network separate the share characteristics(e.g.,scene structure)and domain specific characteristics(such as texture for details),for the dark domain branch details enhancement module is designed for the specific characteristics,which enhance the dark-light events in details,to complete the clear cross-domain reconstruction for event stream.Then,a network model based on generative adversarial network is proposed for event stream reconstruction which uses multi-scale feature extraction,attention mechanism and gradient supervision to pay attention to the local details of the reconstructed image.At the same time,the reconstructed image is locally segmented,and the local reconstruction details are constrained by discriminator,so that the details and texture information of the generated image are more complete.In order to evaluate the above methods in this article,a binocular camera made by Celex-V event camera and traditional camera was used to capture the event stream reconstruction database.The reconstruction database was captured in various natural scenes and indoor scenes,which is a large-scale data set of sunlight/dark-light event streams and traditional intensity images.Experiments on this data set show that the above method has better performance to complete the cross-domain reconstruction for event stream.