Computational Grammars

Azadeh Maghsoodi

History

Before 18001800-1900First 20s20sWorld War IILast 1950sNowadays

Before 1800

Traditional Grammar Correct Speech of a specific language Not scientific Rejected Useful issues: POS

1800-1900

Indian-European languages Language vs. Other languages Language vs. its history

Early 20s

Enough Philology! Language in a specific time

20s

America & Western Europe Intellectual Pattern Understanding Processes in human being

World War II

Math. Logic as a study tool Computer invention caused new App Abstract Mind model ends Behaviorism

Late 1950

Chomsky is coming! Formal Language Theory “Syntactic Structures” Language Categories

– Type 0: Natural (Irregular)– Type 1: Context sensitive– Type 2: Context free– Type 3: Regular

Late 1950 (continue)

Chomsky followers professes:– Generative grammar: Accurate and definite

enough for testing Generative Grammars

– Goal: Unaware knowledge of users– Biologic and inborn basis for linguistic abilities

Universal Grammar Shared structures

Nowadays

Motives – Discover human mind structure – Language process technology

Applications – Word processors– MT– Word predictors– Text predictors– UFIs / DB Queries– Information retrieval

Syntactic Model

Grammars

Parse Algorithms

Computational Grammars

Generative Grammars– Caused by Natural Language Theory– Introduced by Chomsky– Accurate and definite structures– Transformational grammar (TG)– Constraint-Based Lexicalist grammar (CBLG)

TG

Less computational efficiency Theoretical basis Complex rules Simple lexicons

TG (continue)

Chomsky hierarchy & First TG Standard Theory (1965) Extended Standard Theory Government & Binding Theory (1981-1988)

Standard Theory

Sentence – Deep structure– Surface structure

Generative TG– Basic part

Produce deep structure CFG

– Transformational part Transformational Rules

Transformational Rules

Convert deep structure to surface structure Transformational Rule ~ Transformation Example: (same deep structures)

– (i) The boys place the book on the table.– (ii) The boy has placed the book on the table.– (iii) Did the boy place the book on the table?

Transformational Rules (example)

A deep structure:

S

NP VP

N D

the boy

Aux V NP

will place The book

Transformational Rules (example)

To produce yes/no question:– Using a Move Transformation– S[NP VP [AUX V NP]] S[AUX NP VP[V NP]]

S

NP VP

Aux V NP

S

Aux NP VP

V NP

Government and Binding Theory (GB)

Universal grammar theory Learning a language = confirming a small set

of parameters + learning lexicons Move α: deep structure to surface structure ‘Move α’ moves anything to anywhere Some constraints correct ‘Move α’

GB (continue)

Lexicons

Deep Struct

Surface Struct

Logical FormPhonological Form

Move-α

LF Move-αStylistic &

Phonological Rules

GB (continue)

Minimalist Program (MP)– Choose the best candidate instead of direct

production– Under study

CBLG

Based on TGs Increase computational efficiency of

grammars Simple rules Complex lexicons Psychological Computational

CBLG (continue)

Constraint-Based architecture– Constraint satisfaction more important than

transformational derivation

Strict lexicalism– Lexicons: syntactic atoms of a language– Independent Internal structure from syntactic

constraints

CBLG (continue)

Surface structures are produced directly Most computational grammars are CBLG

Computational Grammars

Unification grammar (UG) Categorical grammar (CG) Dependency grammar (DG) Link grammar Lexical/Functional grammar (LFG) Tree Adjoining grammar (TAG) Generalized Phrase Structure grammar (GPSG) Head Driven Phrase Structure grammar (HPSG)

Unification Grammar (UG)

Lots of CBLs are UG Augmented CFG

– CFG can’t recognize long distance dependencies– A generalized form of CFG + A set of features– Augmented Transition Network (ATN)– Definite Clause Grammar (DCG)

Unification Grammars

UG (continue)

Unification Grammars– Feature structures are extended– No need to CFGs– Grammar ~ A set of constraints between feature

structures– Key concept: Subsumption relation

UG (continue)

CAT verb

ROOT cry CAT verb

ROOT cry

CAT verb VFORM present

VFORM present

(Unificator)

UG (example)

S NP VP Unification grammar:

X0 X1 X2 CAT 0 = 5

CAT 1 = NP

CAT 2 VP

AGR 0 = AGR 1 = AGR 2

VFORM 0 = VORM 2

UG (continue)

More grammar information are stored in lexicons

Less grammar rules Using DAGs

ATN Grammar

Transitive network ~ Expanded Finite-State machine

ATN Grammar ~ A set of transitive networks Features Constraints

Categorical Grammar (CG)

Lots of bases are omitted No difference between lexicons and none-

lexicons Part Of Speech is replaced by some complex

category NP/S : NP is on the right NP\S : NP is on the left

CG (example)

Peter : NP

Likes : (NP\S)/NP

Peanuts : NP

Passionately : (NP\S)\(NP\S)

Peter likes peanuts passionately.

CG (example)

S

NP NP\S

Peter NP\S (NP\S)\(NP\S)

(NP\S)/NP NP

Likes peanuts

passionately

Dependency Grammar (DG)

American linguists Based on TGs Dependencies between words Dependency tree

V

N play Adv

boys well

Link Grammar

Planarity phenomenon Legal sequence of words:

– Satisfy local necessities (satisfaction)– No crossed conjunctions (planarity)– One connected graph (connectivity)

CFG Lexical grammars

– Grammar is distributed between words Probability models Voice recognition Hand-written recognition

Link Grammar (example)

linking requirements:

Link Grammar (example)

linking requirements are satisfied

Link grammar (example)

Not part of a language

Lexical-Functional Grammar (LFG)

Unification grammar Not TG ATN research and its deficiencies introduced

LFG Group structures 4 structures

Tree Adjoining Grammar (TAG)

Between CFG and CSG Grammar rules are a set of initial trees Initial trees are anchored trees Two main operations:

– Substitution– Adjoin

High accuracy

TAG (example)

S VP S

NP VP + VP ADV NP VP

V NP VP ADV

V NP

TAG (continue)

High accuracy Apps in NLP

– MT– Information retrieval– …

Generalized Phrase Structure grammar (GPSG)

Only CFLs CFG Rules

– Immediate Dominance (ID)– Linear Precedence (LP)

Head Driven Phrase Structure grammar (HPSG)

Lexical grammar Based on unification Increase computational potency of GPSG Simple CFG Complex lexicons

Applications

Parse Algorithms

Top-Down parsing Bottom-Up parsing (*)

Parse Algorithms

Top-Down parsing Chart parser

– Dynamic Programming Recursive Transition Network (RTN)

– ATN grammar LR parser

– Shift-Reduce algorithms Cocke-Younger-Kasami parser (CYK)

– Dynamic Programming– CNF grammar

Efficient Algorithms

Chart parser CYK parser

Questions???