| In this dissertation, we present a unified framework for sports highlights extraction from baseball, golf, and soccer video in a hierarchical fashion. First, the framework extracts highlight candidates from a video of one of these three different sports without knowing exactly which sport it is. Then, the framework decides the type of the sports according to the most frequent highlight candidates. Next, it rejects many nonhighlights from these candidates. Last but not least, it groups the rest of the candidates into finer-resolution highlights. With a summary that consists of most of the highlights, efficient video browsing can be supported.;The first part of the thesis presents our methods to locate key audio "objects" (audio markers) and key visual "objects" (visual markers) that are indicative of the events of interest in the video. Examples of these audio-visual markers are the applause, cheering sound from the audio signal, and the squatting baseball catcher, the golfer trying to hit the golf ball, and the soccer goalposts from the video signal.;The second part of the thesis presents three algorithms to model finer-resolution sports highlights after associating each visual marker with an audio marker. The first algorithm uses the K-means clustering algorithm to cluster color/motion features. The second algorithm models these highlights with hidden Markov models on the visual features. The third algorithm does the modeling with coupled hidden Markov models using the audio features and color/motion features.;The last part of the thesis presents our work on "single-channel audio source separation" using generative probabilistic models. The approach can be potentially very useful as a pre-processing step for the key audio "objects" detection. |
