Treehouse Guest Lecture November 19, 2004: 3:30pm THO 101

Ambiguity Management in Deep Grammar Engineering

Tracy Holloway King

Natural language is extremely ambiguous, and the magnitude of this ambiguity becomes apparent when writing deep grammars for parsing and generating natural language. In this talk, I first outline some of the sources of ambiguity that affect deep grammars. I then show how the problem of ambiguity management has been approached in the XLE system. There are three main approaches in current use. The first is to use shallow preprocessing, such as POS tagging and named entity markup, to filter parses. The second is an implementation of (dis)preference marks for particular rules and lexical items. The third is to use stochastic disambiguation to determine the most probable parse. The three of these can be used in combination to create an efficient and accurate deep grammar for real text. Finally, I will talk briefly about the ambiguity management system, known as packing, used in XLE to efficiently manipulate the existing ambiguities.

(Slides)

Additional References:

• Maxwell, John T., III and Ronald M. Kaplan. 1993. The interface between phrasal and functional constraints. Computational Linguistics 19(4):571-590.
• Kaplan, Ronald M. et al. 2004. Speed and Accuracy in Shallow and Deep Stochastic Parsing. HLT-NAACL'04
• Riezler, Stefan et al. 2002. Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques. ACL'02
• Riezler, Stefan et al. 2003. Statistical sentence condensation using ambiguity packing and stochastic disambiguation methods for Lexical-Functional Grammar. HLT-NAACL'03
• Riezler, Stefan and Vasserman, Alexander. 2004. Incremental Feature Selection and L1 Regularization for Relaxed Maximum-Entropy Modeling. EMNLP'04

-- EmilyBender - 22 Nov 2004