| There are many cases where it is not possible to program a robot with precise instructions. The environment may be unknown, or the programmer may not even know the best way in which to solve a problem. In cases such as these, intelligent machine learning is useful in order to provide the robot, or agent, with a policy, a set schema for determining choices based on inputs.;The two primary method groups of machine learning are Supervised Learning, methods by which the supervisor provides training data in order to help the agent learn, and Reinforcement Learning, which requires only a set of rewards for certain choices. Of the three categories of Reinforcement Learning, Dynamic Programming, Monte Carlo, and Temporal Difference, the Temporal Difference method known as Q-Learning was chosen.;Q-Learning is a Markov method which uses a weighted decision table to determine the best choice for any given set of sensor inputs. The values in this Q-table are calculated using the Q-formula, which weighs the expected value of a decision based on the known reward and uses a discounting factor to give more recent choices a greater effect on the values than older choices. The Q-table also allowed the learning to be modular, as a learning agent would only need the file containing the table to be able to use the learned policy generated by a different agent.;Because of the large number of iterations required for Q-Learning to reach an optimal policy, a simulator was required. This simulator provided a means by which the agent could learn behaviors without the need to worry about such things as parts wearing down or an untrained robot colliding with a wall.;After a policy was found in simulation, the Q-table was transferred into Koolio, a refrigerator robot, to allow it to navigate the hallways with the experience gathered in simulation. This Q-table was then further refined through more learning on the real robot. |
