High Dynamic Range and 3D Video Communications for Handheld Devices

Handheld devices present a compelling opportunity for ubiquitous video communications. However, inexpensive camera sensors and two-dimensional displays contribute to flat, poorly exposed faces and an unnatural appearance. Our research identifies two requirements of video communications on a handheld device to achieve high quality, natural interaction: high dynamic range (HDR) video and 3D display.;Small, inexpensive sensors cannot adequately capture the full brightness range when exposed to uncontrolled lighting environments. Furthermore, HDR video techniques for handheld devices must not increase camera hardware costs or size. Therefore, our research investigates novel post-processing methods to increase the dynamic range of videos captured using alternating short and long exposures. Past software-based methods failed to eliminate all ghosting artifacts introduced by scene and camera motion. Our research contributions include hierarchical block-based motion estimation (ME) between differently exposed frames, bi-directional ME and pixel refinement within poorly registered regions, and spatially adaptive filtering to smooth radiances using reliable edges, producing high quality HDR video for scenes with both moving cameras and fast local motion.;Handheld devices with "glasses-free" autostereoscopic displays now enable 3D video communications. 3D can enhance realism and enrich the user experience, yet it must be employed without visual discomfort. Display-camera geometry and recent "zone of comfort" research underscore the challenge in creating perceptually satisfying and comfortable 3D content for handheld devices. Shift-convergence can minimize disparities within the region of interest, yet this can lead to uncomfortably large disparities in the background. Reducing camera separation, in an attempt to minimize all disparities, may lead to a flat appearance that does not aid realism. Our research explores the unique stereoscopic challenges posed by handheld 3D video communications. We derive camera separations for optimal viewing comfort and realism. A novel bi-layer disparity remapping technique aligns the front of a user's face onto the screen for comfort, while limiting background disparities. Finally, we evaluate results and recommendations with a user study on a current 3D handheld device.