| Following gaze in the third-person view is very useful in many tasks: understand what action people are doing,predict what they will do next,and find which part in the scene attracts attention most.However,few works have been conducted on this topic in the community.In this paper,we present a novel framework to deal with the gaze following problem.Here,the problem is divided into two cases,namely front/side face and back face.The case front face or side face is when the face is can be detected,and the case back face is when the face is can not detected.Head pose estimation is employed to estimate gaze in the case of front/side face.An end-to-end head pose estimation method based on deep learning is used.Object detection is used to refine the gaze direction by selecting the object that intersects with gaze.For some small objects that may not intersect with the gaze is solved by searching within a certain range.For some larger objects,the center position may be too far from the gaze and is solved by setting a threshold.The algorithm effectively improves the accuracy.On the GazeFollow dataset,there is a 2.5 degree improvement in the case of front or side face.In the case of the back side,since the face is not visible at this time,it is difficult to estimate the head pose.It is proposed to combine the pose information of the human body to estimate the gaze direction.A deep convolutional neural network containing a saliency path,a head pose path,and a human pose path is designed to estimate the gaze.It is proposed that in the head pose path,the head feature map is moved to the position of the head in the entire image by using the method of convolving the head feature map with the head position,so that the subsequent dot multiplication operation is more effective.A spatially smooth cross entropy loss function is used to make the network easier to train.Experiments have shown that the proposed method has one degree less error in the GazeFollow data set. |
