Denoising And Image Reconstruction For Event Cameras In Spatial And Temporal Domain

The event camera is a new type of visual sensor with high time resolution,high dynamic range,and low power consumption.Therefore,it has extremely high application value in the fields of computer vision,robotics,and automatic driving.Different from the imaging process of the traditional camera,the event camera asynchronously triggers the event by sensing brightness changes in the scene,and outputs a sparse event stream.However,these event streams can only reflect changes of the intensity,lack the texture and details of the scene,and are susceptible to noise interference.As a result,most traditional image processing methods cannot be directly applied to process event streams.Therefore,the research on denoising and image reconstruction for event camera is very important for the application of event camera in subsequent actual scenes.The event camera outputs a stream of events with pixel space,time,and polarity information by sensing changes in brightness.The event stream is continuously distributed in time and is similar to the edge structures of the scene in the spatial domain.This structural characteristic can be used to solve the denoising problem of the event stream.In addition,the event can represent the change of the intensity between consecutive image frames.The fusion of the continuity of events in the spatio-temporal domain with the visual information provided by traditional intensity images,is extremely useful for high-quality,high-frame-rate reconstruction of event cameras.Therefore,based on the spatio-temporal characteristics of the event,this thesis separately conducts research on denoising and image reconstruction for event cameras.Aiming at the noise interference in the event stream,this thesis designs an event stream denoising algorithm based on the low-dimensional manifold constraint.The denoising algorithm exploits a graph to approximate the manifold structure of the spatiotemporal distribution of events and defines the manifold constraint on the graph to smooth the noise of the event stream.Aiming at the problems of motion blur and noise in image reconstruction based on event camera,this thesis proposes two intensity image reconstruction algorithms based on smooth constraints.Firstly,the algorithm introduces a total variation regularization as the image spatial smoothing constraint on the basis of establishing a relationship model between intensity images and event streams,and then proposes a reconstruction model based on the total variation regularization for reconstruction.Finally,based on the proposed variational reconstruction model,this thesis fuses the high-resolution time information of the event stream and uses timestamps of events to define the event manifold,then extends the variational reconstruction model to the variational model on the spatio-temporal constraint of event manifold to reconstruct the intensity image.Experiments of denoising and intensity image reconstruction for event cameras are conducted on synthetic dataset and real dataset.The experimental results prove the effectiveness and robustness of the proposed algorithms under high-speed and highdynamic scene conditions.