A Real-Time Seamlessly Immersive Display System, Based On Video Panorama

Wide field-of-view (FOV) is necessary for many industrial applications, such as air traffic control, large vehicle driving and navigation. In this thesis, I introduce a real time immersive display system. It captures live videos from the cameras and recreates an immersive environment. It immerses the viewer with a panoramic view of the environment. The design goals of our system are real-time, live, low-cost, and scalable. The system consists of three stages:initialization, real-time and projection stages. In the first stage, I detect automatically robust features in the initial frame of each camera and find the corresponding points between them. Then the matched point's pairs are employed to compute the perspective matrix which describes the geometric relationship of the adjacent views.In real time stage, to reduce the computation, parameters for stitching are determined once during the system initialization. I register the frame sequence of different cameras on the same plane using the perspective matrix and synthesize the overlapped region using a nonlinear blending method in real time. Then, I stitch multiple video streams captured from ordinary charged couple device (CCD) to generate a panoramic video. To avoid being blocked by the supporting frame, we allow a flexible placement of cameras. This approach trades the accuracy of the generated panoramic image for a larger FOV. The panoramic video is presented on an immersive display which covers the FOV of the viewer. In this way, the narrow fields of each camera are displayed together as one wide scene. Due to the existence of seam artifacts, exposure differences and other misalignments along overlapped region, least square fitting technique being used to achieve wide seamlessly immersive image mosaic.