LING 388Language and Computers

Lecture Lecture 99

9/9/3030/03/03

Sandiway FONGSandiway FONG

Administrivia

ReminderReminder Homework 2 due this ThursdayHomework 2 due this Thursday No late submissions pleaseNo late submissions please Need help?Need help?

Talk to Charles or me after the class to set up Talk to Charles or me after the class to set up an appointmentan appointment

We’re here to help you understand the We’re here to help you understand the conceptsconcepts

Recap

Regular Grammars

FSA Regular Expressions

DCG

Recap

Extensions to regular grammarsExtensions to regular grammars

1.1. Use left and right recursive rulesUse left and right recursive rules

2.2. Use multiple terminal/non-terminals on the rightUse multiple terminal/non-terminals on the right

((Not a problem for DCGNot a problem for DCG))

=>=>

Last time, a DCG for the non-regular (context-free) Last time, a DCG for the non-regular (context-free) language Llanguage Lnn={a={annbbnn | n>=0} using 1. | n>=0} using 1.

Now, an alternative DCG for LNow, an alternative DCG for Lnn using 2. using 2.

Multiple Non-Terminals/Terminals on the Right

Free from the single non-terminal/terminal Free from the single non-terminal/terminal symbol restriction imposed by regular symbol restriction imposed by regular grammars…grammars…

we can encode Lwe can encode Lnn ={a ={annbbnn | n>=0 } | n>=0 } using using

just 2 context-free rules as follows:just 2 context-free rules as follows:S -> S -> s --> [].s --> [].S -> aSbS -> aSb s --> [a],s,[b].s --> [a],s,[b].

Multiple Non-Terminals/Terminals on the Right

ReminderReminder:: This rather simple-looking grammar cannot be encoded with a This rather simple-looking grammar cannot be encoded with a

FSAFSA Requires “working memory” unavailable to the FSARequires “working memory” unavailable to the FSA Intuitively, given …Intuitively, given …

• s --> [a],s,[b].s --> [a],s,[b].recall the regular grammar -> FSA construction idearecall the regular grammar -> FSA construction idea

S a

• How to keep track of the bs as we read the as?

Multiple Non-Terminals/Terminals on the Right

However, GHowever, Gnn can be simulated using a can be simulated using a

Push-Down Automata (PDA)Push-Down Automata (PDA) i.e. a FSA plus a stacki.e. a FSA plus a stack

bb

…b

S

a / push b

B

b / pop b

b / pop b

Stack

Multiple Non-Terminals/Terminals on the Right

DCG rule mapping to Prolog clauses:DCG rule mapping to Prolog clauses: s --> [].s --> [].

s(L,L).s(L,L). s --> [a],s,[b].s --> [a],s,[b].

s(L1,L4) :- s(L1,L4) :- ‘‘C’(L1,a,L2), s(L2,L3), ‘C’(L3,b,L4).C’(L1,a,L2), s(L2,L3), ‘C’(L3,b,L4).

L1

L2

L3

L4

•Prolog query ?- s(X,[]).

Multiple Non-Terminals/Terminals on the Right

Informally, we have the following Prolog sentential Informally, we have the following Prolog sentential forms…forms… ?- s([a,a,a,b,b,b],[]).?- s([a,a,a,b,b,b],[]).

aa s([a,a,b,b,b],X) b s([a,a,b,b,b],X) b• a aa a s([a,b,b,b],X’) b b s([a,b,b,b],X’) b b

– a a aa a a s([b,b,b],X”) b b b s([b,b,b],X”) b b b– a a a s([b,b,b],[b,b,b]) a a a s([b,b,b],[b,b,b]) b b b b b b – Note: [b,b,b] - [b,b,b] = []Note: [b,b,b] - [b,b,b] = []

• a a s([a,b,b,b],[b,b]) ba a s([a,b,b,b],[b,b]) b b b• Note: [a,b,b,b] – [b,b] = [a,b]Note: [a,b,b,b] – [b,b] = [a,b]

a s([a,a,b,b,b],[b]) ba s([a,a,b,b,b],[b]) b Note: [a,a,b,b,b] – [b] = [a,a,b,b]Note: [a,a,b,b,b] – [b] = [a,a,b,b]

So far…

We have encoded LWe have encoded Lnn = { a = { annbbnn | n >= 0 } using two | n >= 0 } using two different DCGs containing:different DCGs containing:

1.1. Left and right recursive rulesLeft and right recursive rules2.2. Three symbols on the rightThree symbols on the right

Now let’s examine one more technique:Now let’s examine one more technique:

Modifying regular grammar GModifying regular grammar Gabab where L(G where L(Gabab) = { a) = { a++bb++ } } without changing the rule format without changing the rule format

i.e. without using either 1. or 2.i.e. without using either 1. or 2.

More on the capabilities of DCG rules

To do this, we’ll need two more things …To do this, we’ll need two more things … The ability to call Prolog predicates from DCG The ability to call Prolog predicates from DCG

rules, rules,

andand The capability for non-terminals to be more The capability for non-terminals to be more

than just simple namesthan just simple namesgeneral structures like: general structures like:

• a(X) a(X) b(X,Y)b(X,Y)

Calling Prolog Predicates We can insert calls to any Prolog predicate, either built-in or user-We can insert calls to any Prolog predicate, either built-in or user-

defined, anywhere using the notation { … }defined, anywhere using the notation { … } Example:Example:

s --> [a], { playChess }, [b], { playGo }.s --> [a], { playChess }, [b], { playGo }. Perhaps a slightly extreme example …Perhaps a slightly extreme example …

• playChess/0 and playGo/0 are not SWI-Prolog built-insplayChess/0 and playGo/0 are not SWI-Prolog built-ins

but the point is that this capability gives the DCG formalism but the point is that this capability gives the DCG formalism the expressive power of Turing Machines (Turing, 1949) aka the expressive power of Turing Machines (Turing, 1949) aka “general computing power”“general computing power”

Somewhat more down-to-earth and relevant for LSomewhat more down-to-earth and relevant for Lnn … … we can use { … } to do arithmeticwe can use { … } to do arithmetic

Non-terminals as structures In general, non-terminals need not be limited to names In general, non-terminals need not be limited to names

onlyonly Some instances where we’ll find non-terminals as Some instances where we’ll find non-terminals as

structures useful:structures useful: Subject/Inflection agreementSubject/Inflection agreement

John likes MaryJohn likes Mary [[IPIP [ [NPNP John ][ John ][I’I’ [ [InflInfl ] …]] ] …]] I like MaryI like Mary [[IPIP [ [NPNP I ][ I ][I’I’ [ [InflInfl ] …]] ] …]] John is tiredJohn is tired We are tiredWe are tired IP -> NP(A) I’(A)IP -> NP(A) I’(A)

• AA a variable controlling agreement between the subject a variable controlling agreement between the subject and inflectionand inflection

Non-terminals as structures

In PrologIn Prolog IP -> NP(A) I’(A)IP -> NP(A) I’(A)=>=> ip --> np(A), ibar(A).ip --> np(A), ibar(A).

• AA a variable controlling agreement between the a variable controlling agreement between the subject and inflectionsubject and inflection

Simple implementation:Simple implementation:• np(sg) --> [john].np(sg) --> [john]. np(pl) --> [we]. np(pl) --> [we].• propagates subject number (sg/pl) into non-propagates subject number (sg/pl) into non-

terminal terminal ibar ibar

Non-terminals as structures Another use, to build parse trees as Prolog structures…Another use, to build parse trees as Prolog structures…

s(s(NP,VP)) --> np(NP), vp(VP).s(s(NP,VP)) --> np(NP), vp(VP). np(np(N)) --> pronoun(N).np(np(N)) --> pronoun(N). np(np(det(the),n(ball))) --> [the,ball].np(np(det(the),n(ball))) --> [the,ball]. pronoun(i) --> [i].pronoun(i) --> [i]. pronoun(we) --> [we].pronoun(we) --> [we]. vp(vp(V)) --> unergative(V).vp(vp(V)) --> unergative(V). vp(vp(V,NP)) --> transitive(V), np(NP).vp(vp(V,NP)) --> transitive(V), np(NP). unergative(v(ran)) --> [ran].unergative(v(ran)) --> [ran]. transitive(v(hit)) --> [hit].transitive(v(hit)) --> [hit].

Prolog queryProlog query ?- s(X,[i,hit,the,ball],[]).?- s(X,[i,hit,the,ball],[]).

NoteNote: extra parameter precedes difference list: extra parameter precedes difference list AnswerAnswer

X = s(np(i),vp(v(hit),np(det(the),n(ball))))X = s(np(i),vp(v(hit),np(det(the),n(ball))))

From a+b+ to anbn

Define Define GGabab such that L(Gsuch that L(Gabab) =) = a a++bb++ We have seen this grammar before as:We have seen this grammar before as:

s --> [a], b.s --> [a], b. b --> [a], b.b --> [a], b. b --> [b], c.b --> [b], c. b --> [b].b --> [b]. c --> [b], c.c --> [b], c. c --> [b].c --> [b].

From a+b+ to anbn

Idea: Idea: Add a parameter as a counter to the non-terminals to count the number of Add a parameter as a counter to the non-terminals to count the number of

aas and s and bbss Input stringInput string Counter ValueCounter Value aa 11 aaaa 22 aaaaaa 33 aaab aaab 22 aaabb aaabb 11 aaabbb aaabbb 00

So:So: every time we see an every time we see an aa, we add 1 to the counter, we add 1 to the counter every time we see a every time we see a bb, we subtract 1 from the counter, we subtract 1 from the counter

This way we can be sure the numbers of This way we can be sure the numbers of aas and s and bbs matchs match

From a+b+ to anbn

Idea: Idea: Add a parameter to the non-terminals to Add a parameter to the non-terminals to

count the number of count the number of aas and s and bbss s --> [a], bs --> [a], b(1)(1).. 1 means we have seen one 1 means we have seen one aa bb(N)(N) --> [a], b --> [a], b(M)(M).. where M is N+1where M is N+1 bb(N)(N) --> [b], c --> [b], c(M)(M).. where M is N-1where M is N-1 bb(N)(N) --> [b]. --> [b]. N must be 1N must be 1 cc(N)(N) --> [b], c --> [b], c(M)(M).. where M is N-1where M is N-1 cc(N)(N) --> [b]. --> [b]. N must be 1N must be 1

From a+b+ to anbn

DCG rules:DCG rules: s --> [a], b.s --> [a], b. s --> [a], b(1).s --> [a], b(1). b --> [a], b.b --> [a], b. b(N) --> [a], {M is N+1}, b(M). b(N) --> [a], {M is N+1}, b(M). b --> [b], c.b --> [b], c. b(N) --> [b], {M is N-1}, c(M).b(N) --> [b], {M is N-1}, c(M). b --> [b].b --> [b]. b(1) --> [b].b(1) --> [b]. c --> [b], c.c --> [b], c. c(N) --> [b], {M is N-1}, c(M).c(N) --> [b], {M is N-1}, c(M). c --> [b].c --> [b]. c(1) --> [b].c(1) --> [b].

Context-Free Grammars

Upgrading from regular to context-free Upgrading from regular to context-free grammars:grammars: By removing all restrictions from the By removing all restrictions from the

right side of the ruleright side of the ruleX -> X -> { V { VNN u V u VTT } }**

How powerful are context-free grammars?How powerful are context-free grammars?

Context-Free Grammars

The Expressive Power of Formal GrammarsThe Expressive Power of Formal Grammars Chomsky Hierarchy:Chomsky Hierarchy:

Type-0 General rewrite rulesType-0 General rewrite rules

Type-1 Context-sensitive rulesType-1 Context-sensitive rules aannbbnnccnn

• Type-2 Context-free rulesType-2 Context-free rules• aannbbnn

– Type-3 Regular grammar rulesType-3 Regular grammar rules– aa++bb++

DCG and Chomsky Hierarchy

Regular Grammars=

Type-3

FSA Regular Expressions

DCG = Type-0

Type-2Type-1

DCG is more powerful than Context-Free

Next time…Next time… We can write a DCG for the context-We can write a DCG for the context-

sensitive languagesensitive language

LLabcabc = { a = { annbbnnccnn | n >= 0 } | n >= 0 }

ButBut::• No context-free grammar can encode LNo context-free grammar can encode Labcabc