Research On Image Comprehension Technology For Intelligent Glasses

With the miniaturization trend of devices used for computation, storage, communication and display, many wearable computing systems, such as Google Glass, have begun their journey into commercial use. These wearable devices are bound to substitute for cell phones to become indispensable for everyone, due to their portability and flexibility. Being a daily used machine gives it the advantage of collecting personal information over all other devices, and thus provides the potential personalized auxiliary that does not exist before. Despite of good news above, there are still some bottlenecks in collecting and processing information of user activities and environment. Aimed at this issue, this thesis researches on computer vision understanding techniques, and makes some adjustments and improvements based on traditional techniques of image segmentation, objects detection and hands recognition to fit them into the smart glass background, and achieves three results as follows:First, image segmentation is one of the foundations of image content understanding. Traditional methods usually focus on the segmentation of one single image. However, with a smart glass, long videos can be recorded, which makes it possible to improve the performance of segmentation by exploiting the continuous movement information. Based on this motivation, by combining segmentation techniques with metric learning, this thesis proposes an aggregating method to segment images from continuous videos. Results show that our method achieves significant improvement.Second, object detection is a basic task of image understanding. Traditional methods usually detect objects via template matching, which requires templates of objects given by human experts. Such methods cost a lot of human efforts, and cannot detect objects without templates. To overcome this challenge, this thesis proposes a novel algorithm ChamferMI for object detection. Based on Chamfer matching and multi-instance representation, ChamferMI extracts good templates automatically for any user-specified object, and can be widely used with high detection rate.Third, currently, human-computer interaction in smart glass is mainly implemented with voice recognition and touch sensors. But it’s more natural and effective to interact with smart glass via hand gesture. Traditional hand recognition methods are usually designed under the third-person view, where the hands and torso are visible. However, in the environment of smart glass, only hands are partially visible from the first-person view, which makes the traditional methods less applicable. This thesis designs a real-time method to recognize hands from the first-person view. By combining skin-color model and depth filter technique, our method achieves high performance on hands recognition, and can be employed as a key step of gesture based HCI for smart glasses.