Content-based Music Information Retrieval Via Humming/Singing

With the development of digital music technique and the increasing of the storage capacity of mobile devices, how to search the intended music information becomes more and more important. Nowadays, people often find the intended songs by performing a text-based query according to the metadata of the songs, such as the title, name, artist, etc. This method is based on the assumption that short tags or key words are available to identify files in the database. Such an assumption requests that people should make the property description and classify the information into categories in advance. However, this manual work can not meet the demand of the rapid growing of the music data. Moreover, the manual description is error-prone due to the highly subjective nature of personal understanding and the difficulties of the perceptive description of music content, and users may forget the text description information. A convenient and friendly music retrieval system is eagerly desired for the mobile devices.Content-based retrieval techniques allow users to find the intended song without the text description. It searches the music information by the melody, lyrics, pitch and so on. Query by humming/singing (QBHS) is one of the techniques that allows users to find a song even they merely know a part of the melody. Our research work is mainly focused on QBHS, which can be divided into three modules: feature extraction of the query audio, melody representation of the music in database, and the melody similarity comparison. Content-based music information retrieval becomes popular in the search field. However, most research work is based on the symbolic music data such as MIDI, few work has been done on the retrieval of audio music data such as WAV and MP3. In this article, we design a query by humming/singing music retrieval system for mobile devices that can search both kinds of music data, and makes the system more practicable. We proposed a novel pitch detection algoirhtm to extract the feature of humming/singing signal, and improve the noise robustness and research efficiency of the MIDI-based system. As for the retrieval of polyphonic music objects (MP3/Wav), we proposed a baseline system, which employs computational auditory scene analysis technique, music onset detection, GMM-based singing voice detection, and HMM-based pitch tracking scheme, etc. We provide exhaustive experimental results to show the effectiveness and feasibility of the proposed system.