Measuring task relatedness for selective multiple task learning in artificial neural networks

The selective transfer of task knowledge is studied in the context of Multiple Task Learning (MTL) neural networks. Given a consolidated MTL network of secondary task representations and a new primary task, a method of measuring task relatedness based on the sharing of common feature representation is derived. The existing consolidated representation is frozen and an estimate for the primary task hidden-to-output weights is generated. Three potential measures of relatedness are compared to determine their feasibility. Those secondary tasks most related to the primary task are then used to transfer knowledge to the primary task in a new network. Results indicate that the employment of a measure of relatedness from one representation in a separate representation is questionable but also indicate that the measures of relatedness do facilitate a positive transfer of knowledge to the primary task.