Grammar Inference for Web Documents

Presentational XML documents, such as XHTML or Presentation MathML, use XML tags mainly for formating purposes, while descriptive XML applications, such as a wellstructured movie database, use tags to structure data items in a semantically meaningful way. There is little semantic connection between tags in a presentational XML document and its content, so the tagging is often complex and seemingly ambiguous. These differences make inference of the underlying structure more difficult for presentational XML. The problem of schema or grammar inference has been studied mostly for descriptive XML, and proposed solutions are often ineffective for presentational XML. On the other hand, there are many applications such as data extraction tools and special-purpose search engines that need to infer structure from presentational XML. Current proposals for such systems provide only partial solutions to this problem. Restrictions imposed by DTDs and XML Schemas make them insufficient to describe many presentational XML documents effectively. In this paper we use regular tree grammars to define a class of grammars that is able to model many published presentational XML documents. We also propose an algorithm to infer such grammars, and prove that we can infer an appropriate grammar with high probability from given samples. We also empirically evaluate our algorithm by applying it to various types of presentational XML and comparing it to other algorithms.