| This thesis describes the investigation conducted to discover if software developers exhibit cognitive biases while searching for source code on the Web. A cognitive bias is a person's tendency to make errors in judgment based on his processing of information and application of knowledge. We attempted to gain a deeper understanding of the search process followed by developers by examining if they suffered from cognitive biases. Biases investigated are order bias, conjunction bias and exposure bias. This investigation is a re-analysis of data from a laboratory experiment where twenty four participants were each given a search scenario and asked to perform the search using five different code search engines. They were allowed to use the engines in any order and they had to judge the relevance of first ten matches returned. The dependent variables in our analysis were time spent on the search engine, number of queries entered in a search engine, number of terms in a query, percentage of the matches clicked and precision of the first ten matches returned. The independent variables in this analysis were order in which an engine was used, motivation and size of the search target (came from the scenario assigned to the subjects), frequency of searching source code and number of years of experience (which came from the background questionnaire filled by the subjects). Background information together with the scenario was studied to investigate for variation in performance of the subjects. After the analysis, we found evidence of order bias, but not for conjunction and exposure bias. We found that the subjects spent most of their time on earlier search engines and spent less time on subsequent search engines, (F (4, 92) =11.36, P<0.0001). The subjects had a significant difference in their click-through percentage by order of use of the engine, (F (4, 92) =2.89, P<0.05). In many statistical tests, there was a subject effect, but further analysis revealed this to be a scenario effect. In other words, motivation and size of the search target were found to influence the different steps of the search process. |
