Key Technologies On Structural Feature Based Music Resizing

With the rapid development of Internet and multimedia technologies, large scale me-dia entities (e.g. image, music and video) are available on the Internet. Among all typesof media, music always works in temporal constraint environments. In other words, peo-ple want to change the time length of a music piece to a preferred length so that it enablesto fit other media. Example scenarios include background music production for anima-tion and movie where the music track has to be the same length as the length of video,radio advertisement production where the music track must fit the length of speech, aswell as slides show preparation where the length of music should depends on the num-ber of slides. All of the above applications request to modify the time length of a musictrack while preserving the acoustic quality of the resized music. Existing music editingtechnologies, e.g., time stretching and resampling, failed to generate acceptable output,once the diference between the user preferred length and the original length of the musictrack is slightly large. To cope with the above challenge, we propose a new technologycalled music resizing in this thesis. The goal of music resizing is to change the length ofa music track to a user preferred length using a series of basic operations automatically,e.g. compression, prolonging, cropping and insertion. The primary contribution of thisthesis is summarized as follows.1. A novel lyrics density based non-homogeneous music resizing is proposed.Firstly, a lyrics-based density model is presented, which takes advantage of lyrics to ana-lyze the musical structure. This density model can be used to describe the compression-resistance for diferent parts of a song. Secondly, two music resizing scheduling algo-rithms, LDF and LDGF, are developed to schedule the compression over diferent partsof a piece of music. Finally, both subjective and objective experiments are conducted,which prove that LyDAR can efectively and efciently generate compressed versions ofsongs with good quality.2. We present a new stretch-resistance measurement by mixing lyrics density andcontent-based audio features for accurate quality degradation estimation. Based on thestretch-resistance measurement, we further develop a content-base music stretching algo-rithm to improve the quality of the stretched music track. Experimental results demon- strate the efectiveness of both the stretch-resistance measurement and the stretching al-gorithm.3. To relieve the efects of cropping and insertion operations on the continuity ofmelody and content in a music track, we design the contiguity-preservative croppingand insertion algorithms which eliminate the abrupt changes at the joint between twosegments after removing/inserting segments between them. Comprehensive user studyshow that the contiguity-preservative cropping and insertion algorithms can generatehigh-quality music versions and, meanwhile, decrease the quality degradation leaded bystretching. As a result, the capability of the music resizing technology was enhance ef-fectively.4. We formalize the problem of music resizing, and generalize five basic musicediting operations needed for music resizing, which are stretching, cropping, insertion,insertion and repeating. Furthermore, we propose a generic music resizing frameworkwhich consists of two phase, i.e., music structure analysis and music synthesis. Theproposed music resizing framework is able to reconstruct a music track and automaticallygenerate a new version with exactly the same length as users demand.