Towards The Evaluation And Improvement Of Bug Localization Techniques

Bug localization plays a key role in bug fixing process and is also a very challeng-ing task.Up till now,much research efforts have been made to develop bug localization techniques.However,these techniques are rarely adopted in practice.To understand the reasons behind this phenomenon,we conducted a study to understand how practitioners from both industry and open source community perceived the research in bug localiza-tion related techniques.During the study,some practitioners questioned the reliabil-ity of existing bug localization techniques.To gain some insights into this problem,we performed a comprehensive evaluation of main-stream information-retrieval(IR)based file-level bug localization techniques,focusing on their strength/shortages,limi-tations,and substitutability.We further proposed a technique which exploited their sub-stitutability and significantly improved the performance of existing techniques.Mean-while,some other practitioners expressed their needs in method-level bug localization techniques.Towards this,we developed a method-level bug localization technique based on word embedding and code embedding techniques.This technique mainly explored the potential of word embedding and code embedding techniques in repre-senting the semantics of bug reports and method code,which was largely ignored by existing studies.Our major contributions are as follows.1.Practitioners'perceptions of research in bug localization related techniques.To understand why many bug localization related techniques proposed by the research community have not been adopted in practice,we performed an evaluation on these techniques from practitioners' perspectives.Through the evaluation,we gained some insights into the values and potential problems of existing research in bug localization related techniques,as well as practitioners'opinions on improving them.Specifically,we first summarized the categories of bug localization related techniques through lit-erature review.Then,we conducted a survey among practitioners and follow-up inter-views with some of them to understand practitioners' perceptions of research in those bug localization related techniques,with a focus on their values and potential prob-lems.We found that on the whole,practitioners were positive about the research in bug localization related techniques(including facilitating bug fixing and helping make a better schedule,etc.).Some practitioners questioned the reliability of existing tech-niques.Some practitioners also provided some advices in improving bug localization related techniques,including integrating other kinds of information(e.g.,screenshots and developers'schedule)and explaining prediction results,etc.2.A comprehensive evaluation and improvement of IR-based file-level bug local-ization techniques.To help practitioners understand the reliability of bug localization techniques,and to provide some guidances in improving such techniques,we conducted a comprehensive evaluation on IR-based bug localization techniques(a main-stream category of bug localization techniques).Meanwhile,based on the evaluation findings,we proposed a new bug localization technique which greatly improved the performance of existing bug localization techniques.Specifically,we did the first comprehensive analysis on three state-of-the-art IR-based file-level bug localization techniques,with a focus on their strength/shortages on unbiased benchmark datasets,their substitutabil-ity,and limitations in locating certain kinds of bugs.Upon the finding that none of three techniques could be replaced by any other two techniques,we proposed a new bug localization framework called CombineBL which combined three evaluated tech-niques.Experimental results on six Java projects showed that,on average,CombineBL could outperform any of three studied techniques by 6.28%-10.5%,4.87%-9.86%and 7.05%-12.15%in terms of of Accuracy@1,MAP and MRR,respectively.3.A method-level bug localization technique based on word embedding and code embedding techniques.To address the problem that the semantics of bug reports and method code are not fully represented by existing studies,we proposed a technique which used word embedding and code embedding techniques to better represent the semantics of bug reports and method code.Experimental results showed that our ap-proach is helpful in facilitating the performance of method-level bug localization tech-niques.Specifically,Our proposed technique used different feature spaces to represent bug reports and method code,i.e.,word embedding to represent bug reports and code embedding to represent code.With experiments on five Java projects,our technique could make correct recommendations within the top 10 ranked methods for 10%to 33.33%bug reports.This to some extent,indicated that it is favorable to use word embedding and code embedding in retrieving the semantics of bug reports and method code.The findings of our work provided a good guidance for future research in bug localization related techniques.We also plan to study some interesting but challenging problems identified in this work in the future.