Research On Classifier Design Based On Pattern Representation And Pattern Source

Pattern recognition (PR) mainly learns how to make computers hold the ability of human recognition. It has been applied in many fields such as artificial intelligence, machine learning, and iatrology. Both classifiers and patterns have become focuses of PR Learning. This thesis proposes several novel classifiers that can utilize some prior knowledge from patterns, and presents a novel classifier design approach on top of matrix-based pattern representation and pattern source infomation. The contributions of this thesis are that:(1) This thesis designs a classifier method suitable for matrix pattern representation. The proposed method can directly classify patterns represented with non-vector (matrix) and thus induces a new classifier design idea that is different from the conventional vector-pattern-oriented one. The new classifier model can not only deal with matrix patterns, but also make the conventional vector-pattern-oriented classifier model as a special instance. This thesis selects one of the existing vector-pattern-oriented linear classifiers and matricizes it so as to construct a corresponding matrix-pattern-oriented linear classifier that would improve classification generalization. This new design idea helps: i) explore how pattern representation (matrix and vector) affects the performance of classifiers; ii) decrease the storing space; iii) provide a novel way of constructing learning model.(2) This thesis proposes two matrix-pattern-oriented classifier algorithms: i) Matrix-pattern-oriented Least Squares Support Vector Machine (MatLSSVM); ii) Matrix-pattern-oriented Modification of Ho-Kashyap algorithm with Squared approximation of the misclassification errors (MatMHKS). The experimental results have shown that the proposed classifiers have a supervised advantage in terms of both classification and storing space to their corresponding vector-pattern-oriented ones on several datasets especially images.(3) This thesis designs an improved matrix-pattern-oriented classifier method on top of AdaBoost so as to partially avoid the matricization-dependent problem on single matrix-pattern-oriented classifier. In particular, the thesis presents a new algorithm named AdaMatLSSVM by combining AdaBoosting with MatLSSVM with an improved classification perfomance. (4) This thesis proposes a so-called fully matricized approach, i.e., the matrix-pattern-oriented feature extraction followed by the matrix-pattern-oriented classifier design (MatFE+MatCD). To comprehensively validate the strategy of MatFE+MatCD, this thesis considers all the four possible combinations of feature extraction (FE) and classifier design (CD) in the manner of matrix and vector respectively. The experiments on the combinations have shown that: i) the designed fully matricized approach (MatFE+MatCD) has an effective and efficient performance on those patterns such as images with the prior structural knowledge; ii) the matrix gives us an alternative option for pattern representation in feature extraction and classifier designs.(5) This thesis develops a more generalized and effective multi-view learning (MVL) framework on top of pattern source information: i) this thesis introduces a method to create multiple views from a single view for MVL. This is important because while the existing MVL has been shown to be effective, it relies heavily on the natural separability of the feature set into two independent components. In many settings, there may not be any natural way to partition the feature space, and the existing MVL framework may therefore not be applicable. In such scenarios, the proposed approaches in this thesis can potentially create multiple independent views from a single view and then learn from these views simultaneously; ii) instead of training classifiers on different views iteratively bootstrapping each other, this thesis adopts a joint learning approach that minimizes disagreement across the classifications using multiple views. There are similarities with ensemble learning, where predictions from different classifiers over a single view are combined, but, again, the critical difference is in the joint optimization; iii) the proposed MVL is a wrapper technique that is not restricted to any certain classifiers as the kernelization technique does not limite to any linear algorithms; iv) according to so-acquired data for an object that can be sorted into single-view data and multi-view data, correspondingly, this thesis sorts learning machines into the single-view machine with only one machine architecture and the multi-view machine with multiple architectures, and first gives four learning models: the single-view machine on the single-view data, the single-view machine on the multi-view data, the multi-view machine on the single-view data, and the multi-view machine on the multi-view data. Then this thesis focuses on the multi-view machine on the single-view data, and proposes five different classifier algorithms: a) MultiK-MHKS; b) MultiV-KMHKS; c) MVNA-KMHKS; d) MultiV-MHKS; e) MVDR. The experimental results validate the effectiveness of the proposed classifiers.