Medical Image Classification By Using Deep Learning Models

Medical image classification is receiving increasing attention in the domain of computer aided diagnosis(CAD),which aims to improve the diagnostic rate with the analysis and calculation through computer programs.It is very challenging due to the two aspects.First,the insufficiency of training data limits the success of classification algorithms,as there is usually a small dataset in most medical imaging research,and this relates to the work required in acquiring the image data and then in annotation.Second,the accuracy of medical image classification suffers from the inter-class similarity and intra-class variation.Classifying imaging modalities or benign and malignant in medical images is much more complicated than classifying objects,animals or scenes in natural images.Visual confusion makes it hard even for a human to distinguish the fine-grained visual appearances without the expertise.To overcome these challenges,a number of investigators have reported solutions,which can be briefly grouped into two categories: handcrafted feature based and deep learning based methods.The former manually extract features based on the experience and classify by using a classifier,such as support vector machine and random forest.The latter provide a uniform feature extraction-classification framework to free users from troublesome handcrafted feature extraction,while suffer from overfitting when the training dataset is not large enough.In this paper,deep learning based algorithms have been proposed to improve the accuracy for medical image classification.The main contributions of this paper are summarized as follows:(1)To address the small-sample learning problem,we propose a combined deep and handcrafted visual feature(CDHVF)algorithm to classify diagnostic medical images and illustrations in the biomedical literature.We fine tune three pre-trained deep convolutional neural networks(DCNNs)to extract deep features and estimate the bag-of-feature and local binary patterns(LBP)as handcrafted visual features.We combine them as super features to train an ensemble back-propagation neural network(BPNN)classifier.Our results on the Image CLEF 2016 dataset indicate that handcrafted features complement the image representation learned by DCNNs on small training datasets and improve accuracy in certain medical image classification problems.(2)To strengthen the attention ability on semantically meaningful regions,we propose an attention residual learning convolutional neural network(ARL-CNN)for small-sample skin lesion classification in dermoscopy images,which is composed of multiple ARL blocks,a global average pooling layer,and a classification layer.Each ARL block jointly uses the residual learning and a novel attention learning strategies to improve its ability to discriminative representation.Our results on the ISIC-skin 2017 dataset indicate that the proposed ARL-CNN model can adaptively focus on the discriminative parts of skin lesions,and thus achieve the state-of-the-art performance in skin lesion classification.(3)To address the intra-class variation and inter-class similarity challenges,we propose a synergic deep learning(SDL)model,which not only uses dual DCNNs but also enables them to mutually learn from each other.Specifically,we concatenate the image representation learned by both DCNNs as the input of a synergic network,which has a fully connected structure and predicts whether the pair of input images belong to the same class.We train the SDL model in the end-to-end manner under the supervision of the classification error in each DCNN and the synergic error.We evaluated our SDL model on the ISIC 2016 Skin Lesion Classification dataset and achieved the state-of-the-art performance.