Research On Lyrics Extraction And Recognition For Chinese Numbered-Notation Score

The numbered musical notation(NMN) is a musical notation system widely used among the Chinese people, with the technology of Optical Music Recognition(OMR) being well developed, the OMR of NMN scores has gradually attracted some scholars, but few scholars had detailedly studied for a lyrics information extraction in NMN scores at the present time. This paper has detailedly studied the lyrics extraction and classification from Chinese Numbered musical notation(CNMN) scores by integrating computer image processing technology, music recognition technology and knowledge of CNMN. It is a work of great significance for reducing the manual workload of inputting Chinese lyrics, automatically singing by computers, lyrics search, and promotion and dissemination of CNMN scores.The paper includes three stages of study and experiment as following:(1) During the image segmentation stage, it firstly analyzes the structural feature of CNMN scores, then it presents a segmentation algorithm to separate lyric regions from note and title regions in the scores utilizing the projection and high-level domain knowledge of CNMN. And it uses the randomly collected100CNMN score images to do experiment and analysis, the result of experiments show that the average accuracy of lyrics segmentation is99.11%, so the proposed segmentation algorithm is effective.(2) During the lyrics extraction stage, it firstly uses the clockwise spiral outward extended Bounding-box algorithm based on the object seed to extract all connected-components of lyrics from the lyric regions in the CNMN scores, then according to high-level domain knowledge of CNMN scores, it designs a structure of the vertical line for dilation operator to merger the-bounding-boxes of upper-lower structure lyrics in the lyric region. Meanwhile, the paper proposes the sliding-window algorithm to merge the bounding-boxes of left-right structure lyrics, it solves the problem of the wrong separation of left-right lyrics and the removing of punctuation symbols and is suitable for the lyrics of CNMN scores with horizontal intervals of various sizes. Lastly, it uses the randomly collected100CNMN score images to do experiments and analysis, the result of experiments show that the average accuracy of lyrics extraction is94.86%, so the proposed lyrics extraction algorithm is effective.(3) During the lyrics recognition stage, according to the feature of the large-scale set and complex structure of Chinese lyrics, it designs the method of two levels classification based on RBF neural network and BP neural network. First of all, it uses gross meshed feature of lyrics and RBF network to classify roughly the large-scale lyric set, and the large-scale lyric set was decomposed into many smaller subsets. Then it uses half-breakthrough of strokes of lyrics and BP network as fine classification to recognize lyrics. Lastly, it uses the randomly collected100CNMN score images to do experiments and analysis, the result of experiments show that the average accuracy of lyrics recognition is68.50%, so the method of two levels classification based on RBF neural network and BP neural network is feasible.This paper has presented a series of methods in the stages of image preprocessing, image segmentation, lyrics extraction, and lyrics recognition for CNMN scores, it is a new application and exploration of OMR and promotes the development of Chinese musical art and arts technology.