Video Object Extraction And Its DSP Realization Based On High-Order Statistic Characteristics Of Multi Frame Difference

Object-based Image compression is one of the research hotspots in low bitrate video compression technology in recent years. By extracting moving objectswhose shapes are discretionary, we can rebuild the image preferably and improvethe coding efficiency. The most important characteristic of MPEG-4 which is thesecond generation video coding standard is using video objects to describe thecontents and codes, which needs to extract video objects firstly. In this standard,the extraction of video objects is still an open part. The advantage anddisadvantage of the extraction algorithm is important to MPEG-4 coding quality.Video segmentation plays an important role in the application of multimedia,and has a promising prospect in video coding, video browsing and multimediainteracting. It is also a key technology in computer vision. Due to the complexityof video contents, there isn't universal segmentation arithmetic in the world. Sowe should put more effort to improve it.In the paper, the key techniques, which are involved in MPEG-4, are studied.At the same time, the extraction algorithm of video, oriented by MPEG-4, isdiscussed. The main work consists of two parts: one is video extraction methodbased on MPEG-4, which discusses a typical algorithm of obtaining VOP fromnatural video sequences and proposes an automatic extraction method based onspatial-temporal information. In temporal domain, the high-order moment ofdifferences among multi-frames is utilized to detect moving areas, while in spatialdomain, edge detection is used. The other is to study the architecture and developenvironment of Blackfin533 DSP of ADI Corporation. On this platform, theextraction of moving objects is implemented.In temporal domain, the change detection method is used. Change detectionis to obtain the change information according to the difference between images ofdifferent period. It is one of the general methods of moving object detection. Theresearch topic of this paper is moving object extraction, that is, how to extract thewhole moving object from image sequences.The general methods of moving object detection can be classified into twokinds: the method based on characteristic and the method based on gray. Themethod based on characteristic is to detect moving objects according to imagecharacteristic, which is mostly used in the situation that the object is bigger andthe characteristic extraction is easier. The method based on gray is to detectmoving objects generally according to the changes of image gray, and the generalmethods used are frame difference, background subtraction, and optical flow field.In this paper, the frame difference is adopted. Because this paper aims to the videowhere quantity of motion is smaller, we choose the five-frame difference to getmoving information instead of generally used two-frame difference. If we adoptedtwo-frame difference, there may be holes in moving objects because two-framedifference doesn't always get enough information. According to the experiments,five-frame difference can provide satisfying performance.In spatial domain, Roberts arithmetic operator is used during edge detectionwhich is based on the gray information of a single frame. The combination ofspatial extraction and temporal extraction employs spatial and temporalmovement detection to maximum advantage. In temporal domain, we usecontinuous multi-frame images to find out moving areas and corresponding maskimages, which provide cursory binary mask. Spatial edge detection providesaccurate boundaries of VOP. The combination of the two methods overcomes theshortcomings that the sizes of the extraction boundaries based on multi-framedifference are larger than that of actual moving objects. At the same time, theimprecise extractions, which are caused by covering and noise, are avoided. Sothe accurate boundaries of moving objects are obtained.The detailed algorithm is as follow:As to still background sequences, in the continuous five-framesAccumulative-Difference images, we adopt the detection method of Higher-orderstatistics of stochastic signal to confirm automatically the foreground andbackground of video sequences and get the mask of moving objects. Because ofthe influence of outer interference, there is noise in video images. Afterframe-difference images are binarized, there still exist many useless noise spots inthe background and target object. In this paper, the mathematical morphology isused to process the binary images extracted. Basic algorithms of mathematicalmorphology include dilation, erosion, open, close, and so on. Combining openwith close can form a morphology noise filter. After noise spots are filtered, theremay be holes in binary mask. Aiming at this case, we apply one row and onecolumn scanning to fill the inner holes. And with morphology and median filterwe can smooth boundaries, remove short branches, remedy the narrow gaps ofboundaries.In the paper, we adopt summing up continuous five-frame frame-difference,so the sizes of the binary mask of moving areas calculated are larger than that ofthe actual objects. The bigger the displacement of the moving objects between twoframes is, the bigger the difference is. So the moving objects extracted containbackground information. And we can revise the edges of the moving objectsextracted using Roberts edge detection algorithm. Because the amount of videodata is very huge, we use up sampling and down sampling to increase theoperation rate of the algorithm that is decreasing and increasing the pixel ofimages. Before images are processed, we use down sampling to decrease the pixelof video images, and use the data having down sampled to process multi-framedifference. Then the mask obtained is on one hand up sampled to recover originalpixel to confirm the gray information of VOP, on the other hand the binary maskwhich is down sampled is used to confirm chromatism data. The gray informationand chromatic data are reorganized into video stream according to the formatYUV4:2:0, then we accomplish the extraction of color video moving objects.For the implementation of the extraction algorithm of moving objects invideo sequences, besides the software implementation on computer, the extractionalgorithm for the moving objects can be implemented on hardware with thedevelopment of high performance universal chips, which lay a foundation on theimplementation of real-time extraction of VOP. In the paper, we select theBlackfin 533 of 16-bit fixed point DSP-Blackfin 53X series which ismanufactured by Analog Device Corporation to realize the whole flow of movingobjects extraction. The algorithm in this paper is implemented on the developboarded—SADSP-BF533EZ-KIT Lite, then we can debug the system byconnecting the JTAG and the Evaluation module.The algorithm mentioned above can separate the moving target form stillbackground preferably, which is practicable. But the inexperience with DSP andless comprehension to the core of the chip make the real-time algorithm ofmoving objects extraction not ideal. Besides, the algorithm has much localization,such as not dealing with the light, moving objects only limited to one person andso on, all that need to be improved in the future.