Contextual document models for searching the clinical literature

I have demonstrated that context-based methods can improve the precision of searches in the medical literature without significantly compromising the recall. Conventional retrieval methods are imprecise and inefficient. Traditional systems employ unmanageable indexing vocabularies or use representations that overwhelm searchers with irrelevant information. In this dissertation, I present a robust document representation that combines full-text terms with a contextual structure that facilitates precise searching.;The basis for this representation is the context model. Context models define the semantic themes that recur in clinical publications, such as the scELIGIBILITY/ scSELECTION criteria in clinical research articles. I have defined context models for clinical research articles, case reports, and review articles, and I have implemented a context-based retrieval system that represents clinical documents with full-text terms and the contexts in which those terms occur.;Context markup, a manual-indexing process, identifies contexts in clinical articles. Manual indexing is often inconsistent, even when performed by professionals. I conducted studies of intermarker consistency that measured the beyond-chance agreement of the context markup across clinician markers who received two hours of training. I observed substantial agreement in the markup of clinical research articles (kappa value = 0.77) and case reports (kappa value = 0.74), and moderate agreement in the markup for review articles (kappa value = 0.52). Thus, I have shown that context markup can be accurately reproduced by clinician markers with minimal training.;I compared full-text and context-based retrieval on a collection of 500 original research articles. At a fixed level of recall (which included only the most relevant documents), clinicians achieved a 5.6 percent increase in strict precision (p = 0.010). Comprehensive precision (which included both relevant and partially relevant documents) also increased by 6.3 percent (p = 0.022) with a 4.5 percent loss of comprehensive recall that was not statistically significant (p = 0.139). Thus, a contextual document representation can improve the precision of searches in the full-text medical literature without sacrificing the recall.