LING 438/538Computational Linguistics

Sandiway Fong

Lecture 6: 9/7

2

Administrivia

• Homework 1– acknowledgments sent out– reviewed in class today

3

Homework 1 Review

• [438/538] Homework Question 1 (3pts)

– (A) How many inference steps does it take to run the following query: • ?- app([1,2,3,4],

[5],L).

– (B) How many inference steps does it take to run the following query:• ?- app([1],

[2,3,4,5],L).

– (C) Explain why the number of steps differ despite the fact both queries return the same result.

• Definition– app([],L2,L2). (base case)– app([X|L1],L2,[X|L3]) :-

app(L1,L2,L3). (recursive case)

• Key thing to notice• app( [X|L1], L2, [X|L3]) :- • app( L1, L2, L3).

• each time we recurse– the first argument is reduced to its tail, i.e.

from matching [X|L1] to just L1

• also, each time we recurse– the second argument is passed along

unchanged, i.e. stays as L2

• therefore– the number of recursive calls here

depends entirely on the length of the 1st list

4

Homework 1 Review

• [438/538] Homework Question 1 (3pts)

– (A) How many inference steps does it take to run the following query: • ?- app([1,2,3,4],

[5],L).

– (B) How many inference steps does it take to run the following query:• ?- app([1],

[2,3,4,5],L).

– (C) Explain why the number of steps differ despite the fact both queries return the same result.

• Definitionapp([],L2,L2). (BC : base case)app([X|L1],L2,[X|L3]) :-

app(L1,L2,L3). (RC: recursive case)

• Tracing[trace] ?- app([1,2,3,4],[5],L).

Call: (7) app([1, 2, 3, 4], [5], _G329) ? (RC)

Call: (8) app([2, 3, 4], [5], _G395) ? (RC)

Call: (9) app([3, 4], [5], _G398) ? (RC)

Call: (10) app([4], [5], _G401) ? (RC)

Call: (11) app([], [5], _G404) ? (BC)

Exit: (11) app([], [5], [5]) ? creep

Exit: (10) app([4], [5], [4, 5]) ? creep

Exit: (9) app([3, 4], [5], [3, 4, 5]) ? creep

Exit: (8) app([2, 3, 4], [5], [2, 3, 4, 5]) ? creep

Exit: (7) app([1, 2, 3, 4], [5], [1, 2, 3, 4, 5]) ? creep

L = [1, 2, 3, 4, 5]

5

Question 2

• [438/538] Homework Question 2 (6 pts)

– give the answer and– draw the complete computation

tree (see lecture 2) for the following queries?- app(X,Y,[1,2,3]).?- app(_,[X],[1,2]).

• Computation tree?- app(X,Y,[1,2,3]).(BC) X=[], Y=L2, [1,2,3]=L2X=[] Y=[1,2,3](RC) X=[X’|L1’] Y=L2’ [1,2,3]=[X’|L3’]• ?- app(L1’,L2’,[2,3]).• (BC) L1’=[], L2’=L2”, [2,3]=L2”• X=[1] Y=[2,3]• (RC) L1’=[X”|L1”] L2’=L2” [2,3]=[X”|L3”]

– ?- app(L1”,L2”,[3]).– (BC) L1”=[], L2”=L2”’, [3]=L2”’– X=[1,2] Y=[3]– (RC) L1”=[X”’|L1”’] L2”=L2”’

[3]=[X”’|L3”’]• ?- app(L1”’,L2”’,[]).• (BC) L1”’=[], L2”’=L2””, []=L2””• X=[1,2,3] Y=[]• (RC) DOESN’T MATCH

Definitionapp([],L2,L2). (BC : base case)app([X|L1],L2,[X|L3]) :- app(L1,L2,L3). (RC: recursive case)

BC

RC

6

Question 2

• [438/538] Homework Question 2 (6 pts)

– give the answer and– draw the complete computation

tree (see lecture 2) for the following queries?- app(X,Y,[1,2,3]).?- app(_,[X],[1,2]).

• Computation tree?- app(_,[X],[1,2]).(BC) _=[], [X]=L2, [1,2]=L2 DOESN’T MATCH(RC) _=[X’|L1’] [X]=L2’ [1,2]=[X’|L3’]• ?- app(L1’,L2’,[2]).• (BC) L1’=[], L2’=L2”, [2]=L2”• X=2• (RC) L1’=[X”|L1”] L2’=L2” [2]=[X”|L3”]

– ?- app(L1”,L2”,[]).– (BC) L1”=[], L2”=L2”’, []=L2”’

DOESN’T MATCH– (RC) DOESN’T MATCH

Definitionapp([],L2,L2). (BC : base case)app([X|L1],L2,[X|L3]) :- app(L1,L2,L3). (RC: recursive case)

BC

RC

7

Question 2

• [438/538] Homework Question 2

• (2 pts)– what does the general form of the

2nd query compute for some arbitrary list L??- app(_,[X],L).

• Key thing to notice?- app(L1,L2,L3).concatenates L1 to L2 giving L3but can also be used to split L3 into L1

and L2e.g. ?- app([1,2],[3,4],[1,2,3,4]).

• in ?- app(_,[X],L). we are constraining the 2nd list to be of form [X], i.e. to be a list containing exactly one item X

• in ?- app(_,[X],[1,2,3,4]). this list can only be split as [1,2,3] and [4]

in general, ?- app(_,[X],L). computes X as the last element of the list L

?- app(_,[X],L). is the same as?- last(L,X). defined in lecture 3

8

Homework 1 Review

• [optional 438/mandatory 538]

• Homework Question 3 (4 pts)– explain what happens for

the following query?

?- app(X,Y,Z).

• tracing shows it matches the base case• and then proceeds to repeatedly match

the recursive case followed by the base case

• each recursion lengthens lists X and Z by one (variable) element

• but Z always includes list X followed by list Y (invariant)

• answers generated are of form:X = []Y = Z ;

X = [_G394]Z = [_G394|Y] ;

X = [_G394, _G400]Z = [_G394, _G400|Y] ;

X = [_G394, _G400, _G406]Z = [_G394, _G400, _G406|Y] ;...

9

Last Time

• began a new topic– grammar

• <N,T,P,S>• N = nonterminals, T = terminals• P = production rules, S = start symbol

– grammar hierarchy• Chomsky’s type-0 through type-3 classification

– type-1 context-sensitive– type-2 context-free– type-3 regular

• generative power/capacity of each class determined by restrictions on what can appear in a production rule

– LHS → RHS “LHS rewrites/expands to RHS”

10

Last Time

• Prolog’s definite clause grammar (DCG) system– rules have form:

• LHS --> RHS.• LHS = single nonterminal• RHS = conjunction (,) of terminals, nonterminals and

Prolog queries• both terminals and nonterminal symbols are represented

by Prolog symbols, distinguished notationally by:• [a] terminal symbol a• a nonterminal symbol a• {a} Prolog query for predicate a/0

11

Grammar as a computer program

• advantage of using Prolog– grammar is a Prolog program– uses the same computation rule

1. s --> [a],b.2. b --> [a],b.3. b --> [b],c.4. b --> [b].5. c --> [b],c.6. c --> [b].

• query– call the grammar using a

regular Prolog query– ?- s(List,[]).

• not a coincidencegrammar is translated into a

regular Prolog program

12

Grammar as a computer program

• SWI-Prolog automatically rewrites DCG rules – into regular Prolog rules

when loaded in from a file• so it can run the grammar

like a regular Prolog program

• using the same computation rule

– match the database facts and rules from the top for each (sub)-goal

– but not from the command line

– example?- assert((s --> [a],b)).

Yes

?- listing.

s-->[a], b.

Yes

?- s([a],[]).

ERROR: Undefined procedure: s/2

13

Grammar as a computer program

• let’s look at the translation ofs --> [a],b.

• translated version iss(L1, L3) :- 'C'(L1, a, L2), b(L2, L3).

• when using?- listing.?- trace.

all you will see is the translated version, so it’s important we learn how to read it

• Notes– it adds a pair of list

arguments to each nonterminal

• this is why we call the grammar with

• ?- s(List,[]).

L1=List, L3=[]

– it calls a builtin predicate 'C’/3 for each terminal

• including two list arguments around the terminal symbol

14

Grammar as a computer program

• piece-by-piece translation...

s --> [a] , b .

s(L1,L3) :- 'C'(L1, a, L2) , b(L2,L3) .

• list pairs (difference lists)– L1 = the list we supply for parsing– L3 = []

• (what’s left over from L1 after parsing nonterminal s)

– L2 = intermediate list • (what’s left over from L1 parsing terminal a)• this in turn serves as input to nonterminal b

• example

L1 = [a,b,b]

L3 = []

L2 = [b,b]

means– nonterminal s covers/spans [a,b,b]– terminal a spans the difference between [a,b,b] and [b,b] – nonterminal b spans [b,b]minus []

15

Grammar as a computer program

• difference list pairs...

s --> [a] , b .

s(L1,L3) :- 'C'(L1, a, L2) , b(L2,L3) .

• example– L1 = [a,b]

– L3 = []

– L1 = [a,b] – L2 = [b]– 'C’/3 removes a

from L1 leaving L2

– L2 = [b] – L3 = []– b/2 spans L2

minus L3

16

Difference Lists

• difference lists– L1 - L2– L2 is a suffix (sub-list) of L1

• examples (well-formed) – [a,a,b,b] – [] = [a,a,b,b]– [a,a,b,b] – [b] = [a,a,b]– [a,a,b,b] – [a,b,b] = [a]– [a,a,b,b] – [a,a,b,b] = []

• in L1 – L2, L2 must be a suffix of L1 to be well-formed• • examples (ill-formed)

– [a,a,b,b] – [a,a] (L2 prefix, not suffix)– [a,a,b,b] – [a,a,a,b,b,b] (not a suffix)

17

Difference Lists

• example query explained– ?- s([a,a,b,b,b],[]).

• query uses the start symbol s with two arguments: – (1) sequence (as a list) to be recognized and– (2) the empty list []

– s spans [a,a,b,b,b] - []• i.e. the whole sequence

• equivalent query– ?- s([a,a,b,b,b,d,e],[d,e]).– s is true if s covers the [a,a,b,b,b]prefix of [a,a,b,b,b,d,e]

• or– in showing s is true, – we consume part of [a,a,b,b,b,d,e]from left to right up to (but not including)

[d,e]

•Reason for using difference lists? Computational efficiency–Prolog can chop the input list down an element at a time (linear time) without taking the whole list apart and calling append to extract sub-sequences

18

Extra Arguments

• we’ve already seen – a pair of extra arguments

• (shown in blue) • are added through Prolog translation for the input lists

– s --> [a],b.– s(L1, L3) :- 'C'(L1, a, L2), b(L2, L3).

• we can also manually add (still) more arguments to our nonterminals– why? so we can implement useful things like agreement and parse

trees

–examples(S) --> [a],b(B).

s(S,L1, L3) :- 'C'(L1, a, L2), b(B,L2, L3).

added arguments appearbefore the input lists

19

Extra Arguments: Parse Tree

– want Prolog to return more than just Yes/No answers

– in case of Yes, we can compute a syntax tree representation of the parse

• example– sheeptalk language– ba! baa! baaa! ba...a!– Prolog grammar– s --> [b], a, [‘!’].– a --> [a]. (base case)– a --> [a], a. (right recursive case)

s

b ‘!’

a

a

a

a

s(b,a(a,a(a)),’!’)

20

Extra Arguments: Parse Tree

• Prolog grammar– s --> [b], a, [‘!’].– a --> [a]. (base case)– a --> [a], a. (right recursive case)

s

b ‘!’

a

a

a

a

s(b,a(a,a(a)),’!’)

• base case– a --> [a].– a(subtree) --> [a].– a(a(a)) --> [a].

• recursive case– a --> [a], a.– a(subtree) --> [a], a(subtree).– a(a(a,A)) --> [a], a(A).

21

Extra Arguments: Parse Tree

• Prolog grammar– s --> [b], a, [’!’].– a --> [a]. (base case)– a --> [a], a. (right recursive case)

s

b ‘!’

a

a

a

a

s(b,a(a,a(a)),’!’)

• base and recursive cases– a(a(a)) --> [a].– a(a(a,A)) --> [a], a(A).

• start symbol case– s --> [b], a, [’!’].– s(tree) --> [b], a(subtree), [’!’].– s(s(b,A,’!’) ) --> [b], a(A), [’!’].

22

Extra Arguments: Parse Tree

• Prolog grammar– s --> [b], a, [’!’].– a --> [a]. (base case)– a --> [a], a. (right recursive case)

s

b ‘!’

a

a

a

a

s(b,a(a,a(a)),’!’)

• Prolog grammar computing a parse– a(a(a)) --> [a].– a(a(a,A)) --> [a], a(A).– s(s(b,A,’!’)) --> [b], a(A), [’!’].

23

Next Time

• type-3: regular grammars