CS 460NATURAL LANGUAGE PROCESSING
CONTENTS
Introduction Motivation Categorial Grammar Combinatory Categorial Grammar Three parts of formalism of CCG Subtypes Semantics CCG and Parsing Algorithm Conclusion
INTRODUCTION
Limitations of Context-free grammar “Peter is from England and Paul from Sweden” Knowledge about context needed due to missing
verb. Crossing dependencies cannot be resolved. “John kicks skillfully the ball”
Models based on lexical dependencies The dependencies are typically derived from a
context-free phrase structure tree Does not work well for the long-range
dependencies“Ram ka yeh baar baar Shyaam ke ghar jaana
mujhe pasand nahi” CCG
“mildly context-sensitive” formalism Provides the most linguistically satisfactory
account of the dependencies Is to facilitate recovery of unbounded
dependencies
Ref - nlp.korea.ac.kr/~hjchung/sprg/summary/021104.ppt
MOTIVATION (1/2)
MOTIVATION (2/2) Principle of Compositionality – Meaning of a
complex expression is determined by meaning of constituent expressions and rules used to combine them
CCG has close relation to (compositional) semantics - syntactic constituents combine as functions or according to a function-argument relationship.
Cross-linguistic generalizations can be made easily since the same set of rules always apply
Arguably psychologically plausible (since processing can proceed in a left-to-right fashion)
CATEGORIAL GRAMMAR (1/2)
Categorial Grammar (CG) involves syntactic objects with well defined syntactic types or categories and rules for combining them.
The rules of grammar are entirely conditioned on lexical categories.
There are lots of categories and only a small set of applicable rules.
CATEGORIAL GRAMMAR (2/2)
Categories Primitive Categories : N, NP, S etc.
Man – N The old man - NP
Functions : Combination of primitive categories, more specifically a function from one category to another.
S/NP NP/N (NP\S)/NP
FUNCTION TYPES
A simple categorial grammar may have just two function types -
B/A - type of a phrase that results in a phrase of type B when followed (on the right) by a phrase of type A.
A\B - type of a phrase that results in a phrase of type B when preceded (on the left) by a phrase of type A.
CATEGORIAL GRAMMAR
English grammar might have three basic types (N, NP and S). Other types can be derived -
Adjective – N/N Determiner – NP/N Intransitive verbs – NP\S Transitive verbs - (NP\S)/NP
The bad boy made that mess
NP/N N/N N (NP\S)/NP NP/N N
COMBINATORY CATEGORIAL GRAMMAR
Combinatory categorial grammar (CCG) is an efficiently parseable, yet linguistically expressive grammar formalism.
CCG is mildly context sensitive. Basic categorial grammar uses just forward and
backward application combinators. CCG also includes functional composition and type-
raising combinators. CCG provides incremental derivations (left to right)
to the language.
DEFINITION OF CCG
A CCG G = (VT , VN, f, S,R) is defined as follows:
– VT defines the finite set of all terminals. – VN defines the finite set of all nonterminals.
These nonterminals are also called “atomic categories” which can be combined into more complex functional categories by using the backward operator \ or the forward operator /.
– The function f maps terminals to sets of categories and corresponds to the first step in bottom-up parsing.
– The unique starting symbol is denoted by S – R describes a finite set of combinatory rules
FUNCTIONAL APPLICATION
The two basic rules used in Pure Categorial Grammar (AB Calculus)
Forward Application: (>)
X/Y Y => X Backward Application: (<)
Y X\Y => X
FUNCTIONAL APPLICATION (EXAMPLE)
Brazil defeated Germany np (s\np)/np np
------------------------------ >
s\np
----------------------------------------------- <
s The dog bit John
np/n n (s\np)/np np
------------------ > ---------------------- >
np s\np
------------------------------------- <
s
FUNCTIONAL COMPOSITION
Two functional types can compose if the domain of one type corresponds to the range of the other.
Forward Composition: (>B)
X/Y Y/Z =>B X/Z
Backward Composition: (<B)
Y\Z X\Y =>B X\Z
FUNCTIONAL COMPOSITION(EXAMPLE)
Ram likes football
s/(s\np) (s\np)/np np
----------------------------- >B
s/np
------------------------------- >
s
TYPE RAISING
Type raising combinators take elementary syntactic types (primitive types) to functor types.
Forward Type-Raising: (>T)
X =>T T/(T\X)
Backward Type-Raising: (<T)
X =>T T\(T/X)
TYPE RAISING (EXAMPLE)
Ram likes football np (s\np)/np np
--------- >T
s/(s\np)
----------------------------- >B
s/np
------------------------------- >
s
MODIFICATIONS
Above rules are order preserving In languages certain words can be permutes
without changing the meaning of sentence. E.g. Kahn blocked skillfully a powerful shot by
Rivaldo instead of skillfully blocked. Extra rules needed to parse such sentences.
CROSSED COMPOSTION
Forward Crossed Composition: (>Bx)
X/Y Y\Z =>B X\Z Forward crossed composition is generally
considered to be inactive in the grammar of English because it can induce some highly ungrammatical scrambled orders
Backward Crossed Composition: (<Bx)
Y/Z X\Y =>B X/Z
SUBSTITUTION
Allows a single resource to be utilized by two different functors.
Forward Substitution: (>S)
(X/Y)/Z Y/Z =>S X/Z
Backward Substitution: (<S)
Y\Z (X\Y)\Z =>S X\Z
EXAMPLE
team that I persuaded everyone to support n (n\n)/(s/np) np ((s\np)/(s\np))/np np/np (s\np)/np
------ >T ------------------------------------------- >B
s/(s\np) ((s\np)/(s\np))/np
------------------------------------------------------------ >S
(s\np)/np
----------------------------------------------------------- >B
s/np
--------------------------------------------- >
n\n
------------------------------ <
n
SUBTYPES (1/2)
In the simple terms described earlier John run or many coffee
are correct. Introduce type hierarchies For example , John is not only a NP it is also
singular. So we introduce NPsg as a subtype of NP. Anything that requires a NP will accept NPsg
as well. But there can be specific requirement of NPsg
where just NP will not fit.
http://www.wellnowwhat.net/blog/?p=294
SUBTYPES (2/2)
http://www.wellnowwhat.net/blog/?p=294
John run NPsg S\NPpl
––––––––––––– Cannot Apply
John runs NPsg S\NPsg
–––––––––––––<
many coffee NPpl/Npl Nmass
–––––––––––––––– Cannot Apply
much coffee NPmass/Nmass Nmass
––––––––––––––––––––> NPmass
SEMANTICS (1/2)
Most common way to represent semantics is through predicate calculus and lambdas
For example : Each word has a semantic content. The proper noun John has the content John’
(the ‘ distinguishing the semantic individual from the word represented with the same orthography).
Verb run would have the content λx.run’ x. Rules are :
http://www.wellnowwhat.net/blog/?p=294
X/Y Y λv.p(v) y ––––––––––––> X p(y)
Y X\Y y λv.p(v) –––––––––––––< X p(y)
SEMANTICS (1/2)
http://www.wellnowwhat.net/blog/?p=294
John runs NP S\NP John’ λx.run’(x) ––––––––––––––––––> S run’(John’)
John saw Frank NP (S\NP)/NP NP John’ λy.(λx.see’(x, y)) Frank’ –––––––––––––––––––––––––––> S\NP λx.see’(x, Frank’) ––––––––––––––––––––––––––––––< S see’(John’, Frank’)
CCG AND PARSING ALGORITHMS
Normal CYK Algorithm (discussed in class) It is exhaustive It explores all possible analyses of all possible
spans, irrespective of whether such analyses are likely to be part of the highest probability derivation.
Two methods Adaptive supertagging
A* parsing
ADAPTIVE SUPERTAGGING
Treats the assignment of lexical categories (or supertags) as a sequence tagging problem.
Lexical categories are pruned to contain only those with high posterior probability.
It is the extensive pruning of lexical categories that leads to substantially faster parsing times
Relaxing the pruning threshold for lexical categories whenever the parser fails to find an analysis.
The process either succeeds and returns a parse after some iteration or gives up after a predefined number of iterations
A* PARSING
A* search is an agenda-based best-first graph search algorithm
Finds the lowest cost parse exactly without necessarily traversing the entire search space
Items are processed from a priority queue, which orders them by the product of their inside probability and a heuristic estimate of their outside probability.
If heuristic is admissible then solution is guaranteed to be exact.
Klein and Manning, 2003
CONCLUSION (1/2)
Accurate, efficient wide-coverage parsing is possible with CCG
Mildly context sensitive.
Uses functors and function rules for parsing of sentences. The semantics is analyzed using lambda calculus/combinatory logic.
CONCLUSION (2/2)
REFERENCES (1/2) A Brief History of Grammar – Categorial Grammar (CG) and
Combinatory Categorial Grammar (CCG)
July 24th, 2009
(http://www.wellnowwhat.net/blog/?p=294) Wikipedia : Combinatory categorial grammar
(http://en.wikipedia.org/wiki/Combinatory_categorial_grammar )
Efficient CCG Parsing: A* versus Adaptive Supertagging – Michael Auli and Adam Lopez ACL 2011
Generative Models for Statistical Parsing with Combinatory Categorial Grammar
2002. 10. 23
Joon-Ho Lim NLP Lab., Korea University
REFERENCES (2/2)
Identifying Semantic Roles Using Combinatory Categorial Grammar - Daniel Gildea and Julia Hockenmaier, University of Pennsylvania
Building deep dependency structures with a wide-coverage CCG parser
Stephen Clark - ACL2002
Multi-Modal Combinatory Categorial Grammar
Jason Baldridge, Geert-Jan M. Kruijff