Color Constancy Algorithms For Background Recognition In Video Surveillance

With the developments of digital colour image device and computer technology, video surveillance has been widely used for many applications, such as network video-conference, safe protection of many important places, and so on. However, the traditional video survellance can not be able to make any response to some abnormities, but just record and transfer the current video images, so it may not be really useful for the purpose of surveillance. For these reasons, intelligence and automation for a video surveillance system have been promising and valuable research directions.The final goal of video surveillance is to detect and track the targets automatically. To achieve this goal, there are commonly two approaches: one is color matching, the other is background subtraction. Color matching is an approach to recognize and track the targets by directly matching the colour feature, and background subtraction is to detect and track the targets by removing the background pixels of the image sequence. Both methods are needed to keep the colour recognition stable as well as real-time. However, as the effects of illumination, noise and the small motions on the background, the surface colour of the background is often varying, thus even affect the stableness and adaptation of the colour recognition system seriously.As video surveillance itself is a computer vision system, we introduce the theory of color constancy which has been addressed in human vision by many researchers into the context of video surveillance, and based on the widely studying of the previous colour constancy algorithm in computer vision, we dedicate to the background colour constancy of video surveillance in order to discount the effects of non-uniform lighting on diffuse and specular reflection background respectively, and the effects of strong noises, slowly varying illumination for background modeling. Based on these efforts, we have achieved the corrective estimation of the background colour regardless of the variation of the incident illumination to some extent.Most of the previous color constancy algorithms have assumed that the illumination incident on the surface is uniform, and they only account for the variation of the illuminant chromaticity. However, it is not applicable for the context of video surveillance with point source, since in this situation the lightness on the background is often non-uniform. Thus, for a video surveillance system with point source, the first problem for colour matching method is how to discount the non-uniform dirtribution of the lightness incident on the surface, and the traditional colour constancy algorithms can not be used directly in this context.On the other hand, for background subtraction of a video surveillance system, the first problem is how to discount the effects of heavy noises and small motions on the background model, and the second is how to make the background model itself adaptive to the slowly varying illumination.From the above, our work makes five contributions which include:(1) A new illumination & hue color estimation which can be applicable for diffuse background reflection is proposed. The algorithm dedicates on colour constancy for background recognition in video surveillance which is lighted non-uniformly. The algorithm has two steps. In the first step, the colour estimation based on illumination compensation is derived from the Lambertian Law and colour perception model. In the second step, the illumination & hue colour estimation is implemented by transforming the results of the colour estimation based illumination compensation from RGB to HSV space. As the estimation algorithm combines the physical properties of the reflection model with those of the HSV space, it seems to compensate the effects of the non-uniform lighting and the shadows on the background recognition.(2) Another colour estimation algorithm based on the colour invariance of diffuse reflection is proposed which also can be applicable for diffuse background recognition. This algorithm can be used to estimate the diffuse background colour by the combination of the colour Gaussian model with the colour invariance of diffuse reflection which is derived from the diffuse reflection model under non-uniform lighting of point source. Stableness and discriminative power of the color estimation is experimentally investigated. Extensive background segmentation results show that this color estimation can be successful in discounting the effects of non-uniform lighting. As it is well-founded on the physics of diffuse reflection as well as on the measurement of statistics, the proposed color estimation algorithm is considered superior to the previous one.(3) a new colour estimation based on the Phong illumination model is proposed which can be applicable for the colour constancy of specular background affected by both the non-uniform lighting and specular effect. This colour estimation has two steps. In the first step, based on the colour invariance of diffuse reflection, the diffuse factor of the model was estimated. In the second step, the specular factors and specular exponent of the specular component was derived by using estimated diffuse factor and differerial functions. The experimental results show that this algorithm can be robust with respect to not only the non-uniform lighting but also the specular effects on the background.(4) A self-adaptive region Gaussian background model which is applicable for background subtraction of video surveillance is proposed. Due to the heavy noises and the slowly varying illumination and the small motions on the background, the background model often contain many small and uniformly distributed regions which be called disturbing regions by us. In order to remove these disturbing regions, a modified connective regions detection algorithm is proposed and a disturbing region Gaussian background is modeled. And then the self-adaptation of the background model can be achieved by using Kalman filter. The experimental results show that this model can be not only robust to the effects of the noise and the small sudden motions of the background scene, but also adaptive to the slow variation of illumuination.(5) In order to verify the usefulness of the background colour constancy algorithm, we developed two video surveillance systems which use colour matching and background substraction respectively as the methods to achieve target recognition and tracking. One is a virtual ball system, the other is a high-way multi-vehicle surveillance prototype.The virtual ball system is a human-computer system which combines video surveillance with virtual-reality. The system can be not only useful for entertainment, but also valuable for computer vision research. In order to verify the usefulness of the colour invariant estimation in real application, the estimation was used in background colour recognition around virtual balls which often affected by the non-uniform lighting.The practical results show that the algorithm make the system can be not only accurate and real-time, but also robust for the scene under point source. This system has been worked very well in Shandong Science and Technology Museum.The high-way multi-vehicle surveillance prototype system includes three modules: one is background modeling, the other two are target detection and tracking respectively. The self-adaptive region Gaussian model was used as background model for discounting the effects of disturbing regions and slow variation of illumination, and the conntecting-region detection algorithm and Kalman filter are used for target detection and tracking respectively. The experimental results on real high-way video show that this background model can make the system robust to the effects of heavy noises, small motions on the background as well as be self-adaptive to the slowly varying illumination.In summary, the colour constancy of video surveillance is a complex and key issue for a video surveillance system. In this dissertation, we have mainly discussed the colour constancy problems of the diffuse background with non-uniform lighting, the specular background with the specular effects as well as with the non-uniform lighting, and the background model with the effects of heavy noise, small motions on the background and slow variation of illumination.On the foundation of the achieved research results, we will expect to solve the more complex colour constancy problems for video surveillance system and exploit them in more applications in the future.