| An Ambient Intelligence Environment is meant to sense and respond to the presence of people, using its embedded technology. In order to effectively sense the activities and intentions of its inhabitants, such an environment needs to utilize information captured from multiple sensors and modalities. By doing so, the interaction becomes more natural as well as accurate and robust. We have focused on 3 aspects of such an environment, using speech, localization and robotics. Speech is one of the most natural forms of communication for humans. Therefore, it can be used as one of the main information sources for deriving the intentions and needs of a person. In our work, we have extended the traditional speech recognition paradigm by introducing 3 dimensional visual articulation information for recognizing spoken words. The development of our system included the capture of a novel dataset, implementation and extended testing under a variety of audio-visual noise types, demonstrating the usefulness of 3D visual information for this task. Additionally, person localization and identification is of paramount importance in a smart environment, since by knowing each person's location, her/is actions can be derived and abnormal patterns can be recognized. Our implementation conducts person identification by means of RFID. Furthermore, three types of input are combined for multi-person localization, namely, skeletal tracking, audio localization and RFID signal strength. The system was deployed and tested in our simulated assistive apartment exhibiting high accuracy. Finally, every domestic environment changes dynamically over time, creating the need for altering the position, orientation and type of sensors used within it. In our approach, we developed a framework of sensor bearing robots with the ability to relocate automatically to compensate for such a dynamic environment. Their positioning is done in such a way so as to maximize coverage. Navigation is carried out using visual information and autonomous placement uses a decentralized algorithm. |
