Perceptual Quality Evaluation And Optimization For Multi-Exposure Image Fusion

In recent years,with the advent of artificial intelligence(AI)and 5G,con-sumers are no longer satisfied with the visual quality experience brought by tradi-tional imaging systems.They prefer to immerse the full luminance levels of many realistic natural scenes - high dynamic range(HDR).The emergence of the next-generation computational imaging technology based on HDR has largely made up the shortcomings,and becoming one of the hot research topics in both academia and industry.However,how to realize HDR imaging simply and efficiently and ensure the imaging quality has always been a tough point in research.Currently,due to the bottlenecks of camera components,HDR imaging is achieved by fusing multi-exposure images,which can not only ensure the structural details of under-exposed and overe-exposed regions,but also provide richer color information.On the one hand,for the same natural scene,with the change of the multi-exposure image fu-sion(MEF)algorithm,the quality of the fused result will vary greatly.On the other hand,for different natural scenes,a single MEF algorithm is difficult to ensure that all fusion results can present the best visual results.Therefore,the quality evaluation for MEF has an important academic value.This paper starts with the construction of MEF database,conducting subjective quality assessment experiments,and ana-lyzes the factors that affect the quality of MEF and its action methods in order to build an objective quality evaluation model.Based on the objective quality evalua-tion model,we design a MEF algorithm that can ensure the quality and improve the fusion efficiency.The specific research content is as follows:(1)A common approach to HDR imaging is to capture multiple images of different exposures followed by MEF in either radiance or intensity domain.A predominant problem of this approach is the introduction of the ghosting artifacts,which occur frequently in dynamic scenes with camera and object motion.While many MEF approaches(often referred to as deghosting algorithms)have been pro-posed for reduced ghosting artifacts and improved visual quality,little work has been dedicated to perceptual evaluation of their deghosting results.Here we first construct a database that contains 20 multi-exposure sequences of dynamic scenes and their corresponding fused images by nine MEF algorithms.We then carry out a subjective experiment to evaluate fused image quality,and find that none of existing objective quality models for MEF provides accurate quality predictions.Motivated by this,we develop an objective quality model for MEF of dynamic scenes.(2)We propose a fast MEF method for static image sequences of arbitrary spatial resolution and exposure number.We feed a low-resolution version of the input sequence to a fully convolutional network for weight map prediction.We then jointly upsample the weight maps using a guided filter.The final image is computed by a weighted fusion.Unlike conventional MEF methods,MEF-Net is trained end-to-end by optimizing the perceptually motivated MEF structural similarity index over a database of training sequences at full resolution.