Rate-distortion optimal video summarization and video retrieval based on principal component space geometry matching

The demand for video summarization originates from a viewing time constraint as well as bit budget constraint from communication and storage limitations, in security, military, and entertainment applications. In this work we formulate and solve the video summarization problems as rate-distortion optimization problems. Effective new summarization distortion metric is developed. Several optimal algorithms are presented along with some effective heuristic solutions.; The problem of video retrieval arises from video content sharing and communication. The same piece of visual information can exists in many different spatial and temporal scales, as well as corrupted by spatial and temporal noise from coding and communication loss. There is a need to find matching video clips from a given video clip. In this work, video sequences are mapped into traces in the principal component space of video frames. Efficient and robust algorithms are presented for video segmentation and retrieval under distortion, scale variation, and frame drop distortions. Under the proposed retrieval framework, efficient indexing of video traces in a high dimensional space is also investigated.