Constraint-aware computational adaptation framework to support realtime multimedia applications

This thesis develops a framework for constraint-aware computational adaptation. This problem is important since in multimedia systems, many aspects (e.g. available computational resources, operating conditions including user profile) are not fixed and a generic, computationally scalable framework for multimedia content processing is needed. This thesis provides a computational framework for adaptation of information processing operators including linear transforms as well as adaptation of mixed reality systems. The adaptation problem is generalized as a conditional optimization problem and the proposed framework is applied on two real applications---(a) adaptive linear transform approximation including the Discrete Cosine Transform (DCT) and the Fast Fourier Transform (FFT) and (b) adaptive training using mixed reality rehabilitation system for stroke rehabilitation.;In the adaptive linear transform approximation problem, this thesis develops a framework to systematically trade-off computational complexity with output distortion, in linear transforms. There are three novel ideas---(a) a fast basis projection approximation framework, (b) an algorithm to estimate the complexity distortion function and search for optimal transform approximation and (c) the determination of optimal operating points on complexity distortion function and a meta-data embedding algorithm for images. This approach is applied on FFT and DCT approximation for images. The results validate the theoretical approach by showing that the computational complexity can be reduced significantly while minimizing distortion.;For the adaptive training using mixed reality rehabilitation system, this thesis proposes novel approaches for three key sub-problems in the automatic system adaptation: (a) designing a mixed reality rehabilitation system, (b) proposing a mixture-of-experts based Dynamic Decision Network to model the relationship between system changes and subject's functional task performance and (c) a computational evaluation framework for assessing subject's kinematic movement. The solutions can be generalized to any motor-learning system that aims to transfer task specific knowledge to users.;The adaptive rehabilitation is conducted on stroke patients using the mixed reality rehabilitation system. The results show that the mixed reality rehabilitation system is well designed to communicate key aspects of reaching motion to subjects, that the mixture-of-experts model predicts patient performance and recommends adaptation decision accurately, and that the computational evaluation agrees with the therapist's annotations.