Active foveal object recognition system

This dissertation presents a software prototype of an active foveal object recognition and tracking system. It explores the use of an active object recognition framework which employs a rectilinear foveal vision sensor structure for robust, reliable, and efficient performance in the 2-D object recognition, 3-D object recognition, and 3-D tracking settings. The study uses the polygon data structure, which corresponds to the available subset of the traditional image pyramid when a rectilinear exponential focal plane array is used. The polygon is used to represent image data captured from the foveal vision sensor efficiently and permit traditional shift-invariant data processing independently and in parallel at each polygon level.;The investigation starts with recognition in the 2-D setting, for which the bounding contour feature set of an object is represented in a multiresolution fashion. A prediction-correction next-look generation algorithm is developed to determine a sequence of saccades to search for a recognizable object. Validation of the initial hypothesis is accomplished by the partial match of a subset of detail features around the contour. Overlap and occlusion are treated by shape decomposition into convex components and matching is done on convex parts.;The study next turns to the recognition of 3-D objects from active image sequences. Given view ambiguity among different object classes, a smart view planning module is introduced to purposively select the optimal next viewpoint to resolve the problem. A motion control scheme for the foveal vision sensor is developed to fixate the object while moving. Feature integration is implemented not only within the same view of the object but among different views.;Tracking systems need both high resolution in a small target region of interest, and a large field of view (FOV) to prevent loss of track. For this, a foveal imaging system is naturally well suited, combining as it does a high resolution region in the fovea with gradually reduced resolution outside the fovea. A foveal active vision system capable of detecting an independently moving object, and maintaining the object centered in the FOV is therefore next addressed. Independent motion detection is launched in the peripheral region using hierarchical motion model analysis while identifying and compensating for ego-motion of the vision sensor. Then via one or more saccades, the object is captured at the fovea of the sensor and maintained tracking afterwards.;Results of this design approach are shown for sets of test objects cast into virtual 2-D and 3-D environments for purposes of active foveal recognition and tracking. These results encourage consideration of such a scheme for robotic or autonomous vehicle navigation applications.