Usability Evaluation Of Internet Search Engine For Obtaining Medical Knowledge Base On User Experience

Internet becomes one of the most important means for people to obtain health and medical information. It is often a first step for people to check basic information on diseases, and medicine and the searched results are often helpful to users. Among various Internet search engines, Google is the most widely used. More and more people use Internet search engines, especially Google, to learn diseases and possible treatments. Furthermore, more and more clinicians began to use search engine to help diagnosis. There are some studies suggests that non-physician may also occasionally obtain correct diagnoses through Internet-based search and this may help physician-patient interaction. All these studies indicate that search engines like Google could play an important role in obtaining and assessing medical information for professionals and lay people. In order to check the quality and effectiveness of search engine, some researchers have done a lot of study works. Some of their mechanism was too general and the systematic data didn't build up. Some researchers' studies were too systematic and didn't consider user's experience. Thus, it is more important for the search engines to meet public needs based on users' experience.The objective of this study is to conduct Internet search engines usability test in obtaining medical information. Usability testing as a software engineering technique and standard industry practice is critical for software validation. It often addresses the ease of use in terms of the human-computer interaction by collecting direct input on how real users use the system. Usability testing is different from usability inspection where experts apply different methods to evaluate a user interface without involving users. In this study, We conducted a hallway testing to evaluate the effectiveness of search engines such as Google etc. for obtaining medical information. We searched"Breast Cancer"using Google and the other three search engines Yahoo!, Bing and Ask.com. Professors from medical school and doctors from cancer clinic are invited to provided the standard of the most useful websites, which were well known to provide useful information about breast cancer to public. These websites have comprehensive information about breast cancer, such as understanding of breast cancer, symptoms, diagnosis, and so on. In each testing, volunteers scored their experience for each of the websites and the result was evaluated by a physician. We collected the data from volunteers and re-ranked those websites based on their scores. We used Java.net to obtain all the text content in these websites. SNOMED CT (Systematized Nomenclature of Medicine– Clinical Terms) was applied as a measure in the analysis. Index the data of websites contents to investigate the quantity of information shown in the websites, including their subpages. Collect the results of data index and re-rank the websites from high to low according to the quantity of information shown in the websites. We compared those rankings. Our study shows that expert-annotated websites that are highly useful can be generally found by these search engines. Many of the top 50 websites are informative, with extensive usage of the keywords. This does not seem to be affected significantly by the commercial nature of search engines, which allows websites to pay for higher ranking. Search engines can often identify some well-known sites although users' experience is diverse. Part of the reasons for search engines' success comes from the way that search engines re-rank sites based on hits of these popular websites. Hence, a useful website with high ranking is likely to stay with high ranking. On the other hand, this could be a double-edged sword. We found some helpful websites that users recognized were ranked beyond top 100. As users rarely visit websites ranking beyond top 100, these helpful websites have little chance to improve ranking. In conclusion, our study shows that search engine like Google is by and large an effective search engine for helping lay users in getting medical information, but there are some pitfalls in that some highly useful websites may be buried beyond highly ranked sites. In other words, the specificity is good while sensitivity may not be satisfactory when using Google in searching for medical information.It is important to point out the limitation of this study, i.e., the user experience in medical information or information obtained by patients does not necessarily match that from their doctors. In the era of patient-centered care, evidence-based practice, personalized medicine, and health care system reform, vast availability of health information on the Internet has a significant impact on modern medical practice. The patients should not be expected to fully understand the significance of the medical information that they searched. Given that patients deserve the best care with accurate diagnosis and the most effective treatment, it is the responsibility for medical professionals to explain the medical knowledge and terminologies in a lay language to their patients in the patient-centered decision-making process. Explanations of nature of disease, possible diagnostic procedures, therapeutic options, potential side effects and cost should be offered to their patients. Ultimately, patient's health care plan should reflect the best-available scientific evidence, the maximal benefit to each particular patient, high quality of life, and cost effective and feasible management, while Internet search results may provide a starting point for patient-doctor communication. To further foster this new trend of patient care model in the Internet age, proper evaluation of search engines for health information is critical. Our case study, although performed only in a small scale, provids the first-hand data from users in terms of their experience in using Internet to search for medical information, and is a useful attempt to evaluate the usability on the Internet search engine for obtaining medical information. To our knowledge, this is the first quantitative usability test from a software engineering perspective in applying Internet search for obtaining medical information. This study may provide some suggestions in improving the overall usability of health-related Internet search.