RGB Image-Based Pupillary Diameter Tracking with Deep Convolutional Neural Network

Pupillary diameter monitoring has proven successful at objectively measuring cognitive load. This work presents three robust RGB video based pupillary diameter trackers and compares them for measuring cognitive load using commodity cameras. We investigate the use of modified starburst algorithm from previous work and propose two algorithms: 2-Level Snakuscules and a convolutional neural network which we call PupilNet. Our results show PupilNet outperforms other algorithms in measuring pupil dilation, is robust to various lighting conditions, and robust to different eye colors. In addition, this work explores various deep learning architecture for extracting the pupillary response as well as understand the network's behavior with three visualization techniques: feature and filter map visualization, gradient ascent, and occluded heat map. The study shows promising results for extracting the pupillary response as objective cognitive load measurement.