Object segmentation and image labeling by learning from examples

We propose a system that employs low-level image segmentation followed by color and 2-D shape matching to automatically group those low-level segments into objects based on their similarity to a set of example object templates presented by the user. A hierarchical content tree data structure is used for each database image. The “learning” phase refers to labeling of combinations of low-level regions that have resulted in successful color and/or 2-D shape matches with the example template(s). These combinations are labeled as “object nodes” in the hierarchical content tree. Once learning is performed, the speed of second-time retrieval of learned objects in the database increases significantly. To reduce the computation involved in the initial search of any object, we propose to utilize a threshold-free multi-level image segmentation approach, where each of database images is pre-segmented into a hierarchical uniformity tree. The hierarchical uniformity tree is then used to seek and label objects that are similar to an example object presented by the user. Significant reduction in computation complexity for the initial learning of an object is achieved since the algorithm employs “coarse” to “fine” search in a top-down fashion, thereby avoiding unnecessary tests of combinations in finer levels. We also propose a new partial shape recognition algorithm by sub-matrix matching using a proximity-based shape representation. The method is translation, rotation, scale and reflection invariant. Applications of the proposed partial matching technique include recognition of partially occluded objects in images as well as significant acceleration of recognition/matching of non-occluded objects. The speed up in the latter application comes from the fact that we can now search only those combinations of regions in the neighborhood of potential partial matches as soon as they are identified, as opposed to all combinations of regions. At last, we present a novel “dynamic learning” approach to automatically improve object segmentation and labeling without user intervention, as new examples become available. We also introduce a new shape similarity metric called Normalized Area of Symmetric Differences, which has desired properties for use in the proposed “dynamic learning” scheme, and is more robust against boundary noise that results from automatic image segmentation.