Auditory Interface And Its Application Development Platform

Traditional WIMP user interfaces tend to make users' visual modal overload, and they can not be used by visually disabled people. New user interfaces have been developed to change this situation. Auditory User Interface (AUI) is the most useful and compelling one of them.Sound is a multidimensional medium, the fundamental dimensions of sound are pitch, timbre, loudness, duration and direction, further more, sound has multiple effects, these effects roughly are cocktail party effect, streaming, echo and reverb. The many dimensions and effects of sound make auditory interface design very flexible. There are mainly two kind of auditory interface: speech auditory interface and non-speech auditory interface, in recent years, the auditory interface developers tend to use non-speech sound and speech together. Non-speech sound can be everyday sounds (auditory icon) or musical sounds (Earcon), Earcons can easily create hierarchy or compound earcons. Developers tend to take this advantage and choose earcons over the others.At present, the lack of systematic guide lines and application program interface confined the effectiveness and popularization of auditory interface. This article sum up a few general concepts on how to design and organize structured musical earcons. We then developed a platform and describe function of each part in detail. This platform mainly consists of basic and complex dialogue auditory widgets which can be used for the design of dedicated auditory applications. These components are extensible and universal, and have no special requirement for the developer's music knowledge background. Based on this platform, we developed a prototype梞obile telephone for blind users. User can control this mobile telephone with keyboard, mouse and speech command, together, separately or alternately, the auditory feedback are structured musical Earcons and speech, which make the interaction much natural and effective. The mobile can also be used by normal users under special situation such as user's hands were occupied.