Value Function Approximation-Based Q-Routing For Optical Network-on-Chip

Silicon photonics-based optical network-on-chip(No C)has been widely proposed as an emerging on-chip communication architecture for chip multiprocessors.However,in the presence of on-chip temperature variations,the thermal sensitivity of silicon photonic devices will result in significant thermally-induced optical power loss.In order to alleviate the thermal issue,we propose an approximate Q-routing based on value function approximation of Q-learning.In traditional table-based Q-routing,each node in the network needs to maintain a Q-table,and the table size increases with the increasing network size.In order to take advantages of the optimization effect of Q-routing but without the high overhead of Qtable,we propose a table-free approximate Q-routing,which is able to find a suboptimal low-loss path in the presence of on-chip temperature variations.Based on the linear function approximation method,the proposed approximate Q-routing does not require Q-tables in each node.To verify the optimization effect of the proposed approximate Q-routing,we use an 8x8mesh-based optical No C as a case study,and conduct network simulations under a set of synthetic traffic patterns as well as real applications.Simulation results show that as compared to the traditional table-based Qrouting,the proposed approximate Q-routing can converge even faster,and its optimization effect is very close to that of best optimization effect achieved by the traditional table-based Q-routing.We further implement the proposed approximate Q-routing in a 16x16 mesh-based optical No C and simulation results show that it is still of faster convergence speed.