Multi-target Regression With Target-Specific And Inter-Target Correlations

Multi-target regression(MTR)was an extension of traditional regression model designed to predict multiple continuous variables by a common set of inputs.Its main challenges are: first,modeling the complex relationships between the inputs and corresponding outputs;second,modeling the complex relationships between the outputs.To address the first challenge,the specific features could be built for each target to predict different outputs through different inputs.To address the second challenge,the correlations between outputs could be considered by stacked single-target algorithm or regressor chains algorithm.However,the existing MTR algorithms rarely considered the both aspects at the same time.Therefore,this thesis proposes two MTR algorithms that combine target-specific features and inter-target correlations.The main contents of this thesis are as follows:1.MTR algorithm via K-Means and stacked single-target is proposed.On the one hand,K-Means is used to construct target-specific features: first,the different distributions of each target in the input space are mined through K-Means binning.Then the clustering centers are obtained by clustering analysis again on the binned input spaces.Finally,the distances between the inputs and the centers are the target-specific features.On the other hand,stacked single-target is used to consider the correlations between targets: generally,the stacked single-target algorithm included two-layers.And all target predictions obtained by the first-layer were stacked for the inputs in the second-layer.However,the predictions of low correlation would drastically degrade performance.Therefore,this thesis improves the traditional stacked single-target algorithm,in which the second layer only stacks the predictions highly correlated.2.MTR algorithm via variational auto-encoder(VAE)and max-correlation chain is proposed.On the one hand,VAE is used to construct target-specific features: first,the matrix formed by the input space and the current target is transposed.And the representative points of the current target are obtained through VAE.At last the distances between the inputs and the representative points are the target-specific features.On the other hand,the max-correlation chain is used to consider the correlations between targets: the traditional regressor chains algorithm had the defects of large randomness and high complexity.But a single target chain named the max-correlation chain can be formed by arranging the targets through global Pearson correlation and local correlation.The max-correlation chain is more stable and less complex than traditional regressor chains.This thesis compares 4 classic MTR algorithms in rencent 5 years on 18 commonly used MTR datasets.The experimental results fully prove the advantages of the algorithms in this thesis,and demonstrate the combination of target-specific features and inter-target correlations is of great significance for MTR.