Evaluate Quality-In-Use Of Software Through User Feedback

Quality in use(QU)evaluates software quality from users' perspective.Traditional method to evaluate QU usually needs experts to build model,and then the model is analyzed by people.Such methods have three backwards.1)Need a lot of manual efforts,and hard to apply to large scale engineering.2)The subjective of people will cause deviation.3)One model cannot apply to different category of software.As the huge quantity and explosive growth of software and users,the QU evaluation on hundreds of thousands of software should be more intelligent and automatic.This paper proposes a novel approach to define QU model by analyzing user reviews with user opinions and emotions,and evaluates the QU of software.The work focus on two issues:1)We use NLP method and Naive Bayes classifier to filter out non-informative reviews like gibberish and advertisement etc.Then,we extract aspects from reviews,and complement hidden aspects.Next,we use topic model to cluster aspects into different topics.At last,we summarize topics into characteristics of QU model and compute the weight of each characteristic.2)We propose an algorithm to match aspects and QU model.Through analyzing the probability distribution of aspects and the document which the aspects in,matching aspects with characteristics in QU model.We also apply sentiment analysis method.Using aspects in reviews as smallest unit,and recurrent neural network to analyze sentiment of aspects and evaluate QU of software.Wilson Interval is applied to punish software with insufficient reviews.We have evaluated our method on ten software genres on SourceForge,with 260 thousands software and 190 thousands reviews.The evaluation results show that QU model constructed by our method is more reasonable.And our method can output QU model to evaluate different software genre according to input.When software has sufficient reviews,our method has over 80% precision.Even when software has insufficient reviews,it has over 70% precision.This paper makes the following contributions: 1)QU model is constructed according to user reviews automatically,which requires less human efforts and makes software QU model more satisfies users' requirements.As the QU model constructed by our method is based on input,so this model can be adopted on different category of software with different input.2)Instead of documents or sentences,we use aspects as the smallest analysis unit,which can discover more details in the document.Meanwhile,we use probability rather than traditional classifier to map aspects to model,which makes accuracy much higher than traditional classifier.3)The method proposes in this paper can be used whether user reviews are sufficient or not,reduces the deviation caused by the number of reviews.