| During the process of software development,testing,maintenance and usage,many software bugs will be found.Developers usually use bug tracking systems to manage software bugs,but due to the highly parallel nature of the development process or some components are used in different products,and the same bugs are easily submitted multiple times by different people,which leads to a large number of duplicate reports in the bug tracking system and affects the subsequent bug assignment and repair.Therefore,researchers have proposed various methods for automatic duplicate bug report detection.Previous studies have considered duplicate bug report detection as a retrieval or classification task.The retrieval task kind of duplicate bug report detection method based on information retrieval generates a list of suspected candidate duplicate bug reports by measuring the similarity between query reports and historical defect reports.The duplicate bug report detection method for classification task mainly uses a deep learning model based on neural network to extract relevant semantic features of bug reports and feeds the generated feature vector into a classifier to predict whether a pair of defect reports is duplicate.0)(8(67)7)and MAP is mainly used as an evaluation metric in the retrieval task,while the evaluation metrics in the classification task are accuracy,precision,recall and F1-score,so a fair and reasonable analysis and comparison of duplicate bug report detection methods in both types of tasks cannot be made directly.In addition,the existing studies usually use different projects of bug reports and different evaluation time periods to evaluate the performance of detection methods,and even use the wrong evaluation methods,resulting in less scientific and reasonable evaluation results,and even performance overestimation problems.In addition,previous studies have used old datasets that are ten years old,and there is no in-depth study on the performance of the generalization of the repeated bug report detection method to the latest time period dataset.In order to solve the above problem,this paper first investigates the effectiveness of state-of-the-art deep learning models in the classification task of duplicate bug report detection.A large number of experiments show that the BERT model trained on a large-scale dataset based on Pairwise idea outperforms the SOTA method in previous studies and achieves the best classification results.Based on this finding,this paper trains the BERT model to generate semantic vectors for defect reports using a streaming data incremental update model approach,and generates a list of candidate repeat bug report recommendations by measuring the cosine similarity between bug reports.Extensive empirical studies over the entire lifecycle of three bug report projects(each randomly divided into 100 time periods and covering the entire project lifecycle)in OpenOffice,Eclipse,and Mozilla show that bug report semantic vectors generated using only the BERT model are not effective for duplicate bug report detection in retrieval tasks,and their performance is much lower than that of the information retrieval-based approach.Considering that both information retrieval-based methods and deep learning-based methods use only part of the information in the reports when performing duplicate bug report detection,this paper proposes two duplicate bug report detection methods that combine deep learning and information retrieval.One is a two-stage retrieval method,which uses information retrieval methods to get a list of candidate duplicate reports in the first stage,and then calculates the semantic similarity between bug reports by deep learning models,and then sorts them to get a recommended list of candidate duplicate reports in the second stage.Another is the REP-Semantic method,which can more comprehensively use the lexical and semantic information of text fields in bug reports to measure the similarity between the text fields of bug reports,and simultaneously gather the category field information to retrieve the duplicate bug reports more accurately.Experimental results on three projects,OpenOffice,Eclipse,and Mozilla,show that the secondary retrieval approach does not improve the performance of duplicate bug report detection.In contrast,extensive empirical studies throughout the lifecycle of the three projects show that the REP-Semantic approach obtains a list of candidate duplicate reports by combining the lexical,semantic,and category similarities of the reports as a similarity score ranking at the time of retrieval,and its performance is improved by 7.167%-13.853%compared to the information retrieval approach,i.e.,the REP-Semantic approach is more effective for duplicate bug report detection is more effective.In addition,considering that the dataset used in the previous study is ten years old,this paper crawls the bug reports of three projects for the last ten years.The experiments on the latest time period dataset show that the REP-Semantic method proposed in this paper can still achieve better results than the information retrieval method,proving that the REP-Semantic method has certain generalization ability on the latest time period dataset.Considering that both information retrieval-based methods and deep learning-based methods use only part of the information in the report when performing duplicate bug report detection,this paper proposes two duplicate bug report detection methods that combine deep learning and information retrieval.One is a two-stage retrieval method,which uses information retrieval methods to get a list of candidate duplicate reports in the first stage,and then calculates the semantic similarity between bug reports by deep learning models,and then sorts them to get a recommended list of candidate duplicate reports in the second stage.The other is the REP-Semantic method,which can more comprehensively use the lexical and semantic information of text fields in bug reports to measure the similarity between text fields of bug reports,and at the same time combine the category field information to retrieve duplicate bug reports more accurately.The experimental results on three projects,OpenOffice,Eclipse and Mozilla,show that the two-level retrieval approach does not improve the performance of duplicate bug report detection.In contrast,extensive empirical studies throughout the lifecycle of the three projects show that the REP-Semantic method obtains a list of candidate duplicate reports by combining the lexical,semantic,and category similarities of the reports at retrieval as a similarity score ranking,and its detection performance on three metrics,RecallTop5,RecallTop10,and MAP,has 9.920%-13.853%,9.042%-11.982%and 7.167%-10.135%improvement,i.e.,the REP-Semantic method is more effective for duplicate bug report detection.In addition,considering that the dataset used in the previous study is ten years old,this paper crawls the bug reports of the last ten years for the three projects.The experiments on the latest time period dataset show that the REP-Semantic method proposed in this paper can still achieve better results than the information retrieval method,proving that the REP-Semantic method has certain generalization ability on the latest time period dataset. |
PDF Full Text Request