| This thesis introduces principled, broadly applicable, and efficient patch-based models for data processing applications. Recently, "epitomes" were introduced as patch-based probability models that are learned by compiling together a large number of examples of patches from input images. This thesis describes how epitomes can be used to model video data and a significant computational speedup is introduced that can be incorporated into the epitome inference and learning algorithm. In the case of videos, epitomes are estimated so as to model most of the small space-time cubes from the input data. Then, the epitome can be used for various modelling and reconstruction tasks, of which we show results for video super-resolution, video interpolation, and object removal. Besides computational efficiency, an interesting advantage of the epitome as a representation is that it can be reliably estimated even from videos with large amounts of missing data. This ability is illustrated on the task of reconstructing the dropped frames in a video broadcast using only the degraded video. Further, a new patch-based model is introduced, that when applied to epitomes, accounts for the varying geometric configurations of object features. The power of this model is illustrated on tasks such as multiple object registration and detection and missing data interpolation, including a difficult task of photograph relighting. |
