| Understanding human visual attention is important for multimedia applications.Many studies have attempted to learn from eye-tracking data and build computational saliency prediction models.Since the Internet era,web pages,as the core application on the Internet,have become the most common and important visual objects in people's lives.The visual saliency prediction of the webpage can reduce the manual test in the webpage design process for reducing the labor cost,and can also assist the webpage retrieval technology to extract the core content of the webpage for improving the retrieval performance.Therefore,the research on the visual saliency prediction model for webpage images has practical significance and application values.However,there are currently a small number of related researches devoted to the visual saliency prediction of web pages.The existing work is mainly focused on exploring better feature representations and using support vector machines to make saliency predictions without considering two key features of the problem:(1)Web pages contain more diverse content elements than natural images.Different elements usually have different effects on the human visual system.(2)The layout of web pages is greatly affecting the area of interest of human eye on the webpage image,and different webpage layouts will result in different reading modes which lead to different visual position bias.In addition,because the labeling cost of visual saliency prediction is very high,the scale of the existing labeled data set is very small,and the published FiWI data set contains only 149 images,which limits the ability of deep learning models.Based on the above two key features of the visual saliency prediction problem and the limitation of data size,this paper proposes a novel visual saliency model for web page image.The whole model is based on deep neural network design and can be used for end-to-end training and inference.Aiming at the above key points(1),considering that the image and text are the main elements in the webpage,this paper proposes an element-sensitive webpage saliency model.In this paper,a multi-discriminative region detection module and a text region detection module are proposed to locate the semantic image region and the region area in the webpage image respectively.The feature representations extracted by the two modules are more in line with the human attention mechanism,and additional prior knowledge is also introduced into the model.Aiming at the above key points(2),this paper proposes a visual saliency model framework with position bias learning,and proposes the use of variational autoencoder and Gaussian mixture model to model the visual position bias caused by web page layout as probability distribution.After the modeling of the position bias is completed,a position bias learning network is trained to predict the position bias,so that the position bias features affected by the webpage layout are introduced into the model to improve the prediction performance of the model.Aiming at the limitation of the data size,this paper proposes a semi-supervised learning algorithm based on auxiliary task for the visual saliency model of the webpage,and proposes to use the text region detection task and the discriminative region detection task as the auxiliary tasks.These auxiliary tasks have high correlation with the saliency prediction task.Thus,the model can use a large number of webpage images without visually significant annotations for semi-supervised training by using these auxiliary tasks,avoiding the over-fitting to the small dataset,and improve the prediction performance.This paper experiments on the published FiWI dataset to verify the effectiveness of the proposed model and algorithms.The semi-supervised element sensitive saliency model with position bias learning proposed in this paper outperforms other existing stateof-the-art visual saliency models in many performance indicators,and obtains the best web page saliency prediction at present. |
PDF Full Text Request