| Dynamic web sites commonly return information in the form of lists and tables. Although hand crafting an extraction program for a specific format is time-consuming but straightforward, it is desirable to automatically generate extraction programs from examples of lists and tables in html documents. Supervised approaches have been shown to achieve high accuracy, but they require manual labeling of training examples, which is also time consuming. Fully unsupervised approaches, which extract rows and columns by detecting regularities in the data, cannot provide sufficient accuracy for practical domains. We describe a novel technique, Post-supervised Learning, which exploits unsupervised learning to avoid the need for training examples, while minimally involves the user to achieve high accuracy. We have developed unsupervised algorithms to extract the number of rows and adopted a dynamic programming algorithm for extracting columns. Our method achieves high performance with minimal user input compared to fully supervised techniques. For 14 typical web sources tested by our method, all lists and tables are correctly found and the average accuracy of extracting data field is almost 100 percent. |
