Developing and evaluating a document visualization system for information management

Documents have discourse structures that are common to documents of a particular type. We believe that document structure can be exploited in knowledge management so that information may be assessed quickly and displayed intelligently. This thesis describes a system that automatically extracts information from scientific articles and patent claims based on document structure. Based on the results of 101 documents, evaluation shows 72% accuracy for patents and 50% accuracy for scientific articles. To assess overall user satisfaction with the system an adapted usability method from human-computer interaction is used. This novel method is described and shows clear advantages over existing summarization evaluation methods. Based on the 4 most and least accurate documents from the accuracy evaluation, our system scored a user satisfaction level of 2.7/5 for patents and 3.9/5 for scientific articles. Discussion on extending the usability evaluation to other natural language systems is presented.