Application Of Multi-instance Multi-label Learning In Biological Image Recognition

Multi-instance multi-label learning is widespread in the field of biomedical imaging,and it is of great significance to explore universal solutions to this type of problem.We selected the issue of subcellular localization based on immunohistochemical images and discussed the automatic annotation of fruit fly embryo development images.We expect our solution to provide a reference for solving multi-instance multi-label learning problems of other biomedical scenarios.The tissue atlas of the human protein atlas(HPA)houses immunohistochemistry(IHC)images visualizing the protein distribution from the tissue level down to the cell level,which provide an important resource to study human spatial proteome.Especially the differential localization analysis across normal and cancer tissues lead to new cancer biomarkers.We propose a deep multi-instance multi-label model,Im PLoc,which employ a deep CNNbased feature extractor to represent image features,and design a multi-head self-attention encoder to aggregate multiple feature vectors for subsequent prediction.The experimental results show that Im PLoc achieves significant enhancement on the prediction accuracy compared with the current computational methods.We further apply Im PLoc to a test set of 889 proteins with images from both normal and cancer tissues,and obtain 8 differentially localized proteins with a significance level of 0.05.With the rise of image-based transcriptomics,spatial gene expression data has become increasingly important for understanding gene regulations from the tissue level down to the cell level.Especially,the gene expression images of Drosophila embryos provide a new data source in the study of Drosophila embryogenesis.We propose a new method called Fly IT which employs an image tiling strategy to get a combined image feature representation for each gene and a new loss function to deal with the class imbalance problem.The new loss function differentiates the difficulty based on whether the classification result of each label is correct,and increases the contribution of the difficult samples to the overall loss.The experimental results on the Fly Express database show that both the image tiling strategy and the deep architecture lead to the great enhancement of the annotation performance.