Effective multi-sample fusion methodology for improving biometric identification of individuals

Sequences of video frames on a person's activity are often available. The individual samples are acquired by possibly different cameras or even a single camera capturing multiple poses. Fingerprint probes can be also available in multiplicity. For example, people may be asked to provide multiple fingerprint scans from the same sensor or multiple sensors for redundancy and quality control. The data from the multiple samples and views can be fused for reliable authentication of individuals. The fusion of matching scores between the probes themselves rather than probes against enrolled templates is a novel idea which will be explored and integrated in our fusion framework. This dissertation takes a fresh view of biometric fusion where multiple samples, rather than simply the multiple modalities, is the prime focus.;A second area of examination will be the structure of the fusion functions which have hitherto been largely pre-determined (e.g., average of matching scores) rather than "learnt" from data. The fixed functions do not factor parameters associated with a particular sensor or a particular time frame (e.g., quality). They also fail to take advantage of vital auxiliary information such as the correspondence between matching results output by different sensors, or the magnitude of changes in the scans related to different time frames. Even the few fusion approaches that have attempted to account for the above factors have been limited to very specific matching algorithms and therefore do not scale. We will explore the use of trainable fusion functions that will combine the results of separate time frames by taking advantage of the relationships between the matching scores assigned to different enrolled persons, as well as, auxiliary information specific for each biometric modality. This represents a transformational shift in biometric fusion research and calls for biometric matchers to output additional information (fingerprint quality, rotation and pivot points of two matched fingerprints, and correspondence between landmark features etc.) for subsequent use by fusion algorithms. The dependence information will be derived either from the statistics of the available matching score sets and the matching model state information that is already available for specific biometric modalities. We will explore suitable matching model state information and construct theoretically optimal fusion given the statistical information about the differences in the available matching scores.