Automated Negotiation Research In E-commerce With Learning Mechanism

As online trading becomes more common, a large number of electronic commerce services are being developed on Internet. Currently, with Agent-based e-commerce research becoming hotspot, soft Agent technique is considered a very useful tool to develop online business, especially its characteristic of Autonomy, Personalization, Social Ability and Intelligence, which can effectively meet the needs of being flexible in online trading. However, current e-commerce has done very little towards automating the way we do business. In particular, e-commerce has done little to automate decision-making processes humans typically get involved in as they conduct business transactions, such as "automated negotiation". How to apply the advanced Agent technique into automated e-negotiation has been a mainstream in both economics and computer science domain.Agent-based automated negotiation is a decision-making process in incomplete-information environment among individual Agents whose behaviors are competitive, as well as cooperative. Agent can get more information by offer exchange process. An offer is a complete solution which is currently preferred by an Agent given its preferences, constraints and the negotiation history of offers and counteroffers. An agreement takes place when a particular offer is accepted by all negotiation parties. Since MAS is an open and dynamic system, negotiation process should adapt the change of such dynamic environment. Theories analysis show that, if learning mechanism can be embedded into multi-agent based negotiation, which makes every Agent adjust its behaviors by learning, they will achieve the negotiation aims effectively. The problem of how to combine machine learning theories into automated negotiation system has got more attention recently. My paper is just based on the points mentioned above for further expansion in many related fields.The main topic of my paper is the research on how to use online learning to improve the negotiation efficiency in bilateral multi-issue automated negotiation.Paper outlines the current research status at home and abroad, as well as some relative theories. By presenting negotiation protocol and detailed describing of negotiation flow, based on multi-attribute utility theory and sequential decision making process, we establish a formalized negotiation model for multi-issue automated negotiation in e-commerce and add learning ability into our model. Elaborate process descriptions of evaluating offers, belief revision and proposing counteroffers are presented, in particular, we analyze the use of Bayesian learning and reinforcement learning in negotiation process, restructuring the traditional Q-learning into a dynamic Q-leaming algorithm by introducing current beliefs and recency exploration bonus. In addition, paper discusses the way to reuse bilateral automated negotiation methods into multilateral negotiation.At last, simple experimental module is presented to test the effects of main parameters in learning algorithm, beliefs learning and dynamic Q-learning. Results show that our online learning mechanism is effective.