YANGYANG 1 Chap 5 LL(1) Parsing LL(1) left-to-right scanning leftmost derivation 1-token lookahead...

transcript

YANGYANG 1Chap 5 LL(1)

Parsing

Chap 5 LL(1) Parsing

• LL(1) left-to-right scanning leftmost derivation 1-token lookahead

• parser generator:

Parsing becomes the easiest!

Modifying parsers is also convenient.

Parsing

Given the productions

A 1 A 2 ..... A n

During a (leftmost) derivation,

... A ... ... 1 ... or ... 2 ... or ... n ...

Which route should we choose?

(Try-and-error is not a good idea.)

» Use the lookahead symbols.

Parsing

• Consider the situation:We are about to expand a nonterminalA and there are several productions whose LHS are A:

.....A n

We choose one of the productionsbased on the lookahead token.

Which one should we choose?

Consider First(1) First(2) ...... First(n)and

if i , then consider also Follow(A). *

Parsing

• Define

predict(A ) =First() (if First() then Follow(A))

• If the lookahead token a predict(A) then we use the production A to expand A.

• What if a predict(A 1) and a predict(A 2)?

• What if a predict(A) for all productions A whose LHS are A?

Parsing

Property of LL(1) grammars:

If a grammar is LL(1), then

for any two productions

First(Follow(A)) First(Follow(A)) =

Parsing

Given the FIRST and FOLLOW sets in Fig. 5-2 and 5-3, calculate the predictset for each production.

Parsing

§5.2 LL(1) Parse Table

• The predict() function may be represented as an LL(1) parse table.

T: Vn * Vt P {error}

a b ...... A 3 B error ....

T[A, a] = A if apredict(A)

= error otherwise

• A grammar is LL(1) iff all entries in the parse table contain a unique production or the error flag.

Parsing

Figure 5.5 The LL(1) table for Micro

Parsing

5.3 LL(1) parsers

• Similar to scanners, there are two kinds of parsers:

1. built-in: recursive descent

2. table-driven

Parsing

1. built-in

stmt() { token = next_token(); switch(token) { case ID: /*production 5:stmt-->ID:=<exp>;*/ match(ID); match(ASSIGN); exp(); match(SEMICOLON); break; case READ: /*production 6*/ ... case WRITE: /*production 7*/ ... default: syntax_error(....); } }

Parsing

•It is obvious that these recursive descentparsing procedures can be generatedautomatically from the grammar.

grammar LL(1) table

parser generator

recursive descent parser

• However, it is difficult for the parsergenerator to integrate the semantic routines into the (generated) recursive descent parser automatically.

Parsing

2. table-driven parser

(+) generic driver

Only the LL(1) table needs to bechanged when the grammar ismodified.

(+) non-recursive (faster)

Parser maintains a stack itself.No recursive calls.

Parsing

lldriver(){

push( START_SYMBOL );a := next_token;while stack is not empty do{

X := symbol on stack topif ( X is a nondeterminal &&

T[X, a] == XY1Ym )’)pop(1);push Ym, Ym-1, , Y1

else if ( x == a )pop(1);a := next_token();

else if ( x is an action symbol ) pop(1); call correspond routine

else sntax_error();}

YANGYANG 14Chap 5 LL(1) ParsingChap 5 LL(1) Parsing

Ex. begin A := B - 3 + A; end $

a = begin

X = <GOAL><GOAL> parsestack

Trace the action of the parser on this example.

YANGYANG 15Chap 5 LL(1) ParsingChap 5 LL(1) Parsing

5.5 Action symbols

• Action symbols may be processed by the parser in a similar way.

1. in recursive descent parsers

Ex. gen_action( “ID:=<exp>#assign” );”) will generate the following code:

match(ID); match(ASSIGN); exp(); assign(); match(semicolon);

• Parameters are transmitted through a semantic stack.

• Semantic stack is a stack of semantic records.• Parser stack is a stack of grammar (and action) symbols.

Parsing

2. in LL(1) driver

• Action symbols are pushed into the parse stack in the same way as grammar symbols.

• When action symbols are on stack top, the driver calls corresponding semantic routines.

• See previous slide for lldriver.

• Parameters are also transmitted through semantic stack.

Parsing

§5.6 Making grammars LL(1)

• Not all grammars are LL(1). However, some non-LL(1) grammars can be made LL(1) by simple modifications.

• When is a grammar not LL(1)?When there is an entry in the parsetable that contains more than oneproductions.

Ex. ...... ID ...... .... <stmt> 2,5 ....

This is called a conflict, which means we do not know which production to use when <stmt> is on stack top and ID is the next input token.

Parsing

• Conflicts are classfied into two categories: 1. common prefix 2. left recursion

• Common prefixEx.

Consider when <stmt> is on stacktop, ‘if’ is the next input token. Wecannot choose which production to use at this time.

In general, if we have two productions

and First() First() , then we have a conflict.

Parsing

• Solution: factor out the common prefix

Parsing

2. left recursion: productions of the form:

• grammar with left-recursive productions are not LL(1) because we may have

same lookahead

Parsing

• Problem: left recursion

AA A A

Intuition: all the strings derivable from A have the form: , , , , , ,

Solution: replace the productions So we may use the following productions instead:

AT AT T TT

Parsing

Ex. Given the left-recursive grammar:

EE + T ET TT * P TP PID

After eliminating left recursion, we get

ET A A A+ T A TP B B B* P B PID

Parsing

3. more general solution ex.

<stmt><label> <unlabeled stmt><label>ID :<label><unlabeled stmt>ID := <exp> ;

We cannot decide which production to use when <label> is on the stack top and ID is the next token:

lookahead lookahead ID ID

Parsing

• Solution: use the following productions(which essentially look ahead 2 tokens)

<stmt>ID <suffix> <suffix>: <unlabeled stmt> <suffix>:= <exp> ; <unlabeled stmt>ID := <exp> ;

Try two examples:

A: B := C ;

B := C ;

Parsing

4. For more difficult cases, we use semantic routines to help parsing.

Ex. In Ada, we may declare arrays as

A: array(I .. J, BOOLEAN)

A straightforward grammar is (for array bound)

<bound><exp> .. <exp> <bound>ID <exp>ID <exp>… and ID First(<exp>)

This grammar is not LL(1) because we cannot make a decision when <bound> is on stack top and ID is the next token.

Parsing

• Solution: <bound> <exp> <tail> <tail> <tail> .. <exp>

• All grammars can be transformed into Greibach Normal Form, in which a production has the form:

terminal

So given a grammar G, we can do

GGNFno common prefix no left recursion but still NOT LL(1)!Ex. Sa A a Sb A b a Ab A consider A is on stacktop; b is next token.

Parsing

§5.7 The dangling-else problem

• Consider if a then if b then x := 1 else x := 2

Two possibilities: a a T T F b b T F T x := 2x := 1 x := 2 x := 1

The problem is which ‘if’ the ‘else’belong to.• In essence, we are trying to find an LL(1) grammar for the set { [i ]j | ij0}

But is it possible?

Parsing

• 1st attempt: G1

S[ S C S C] C

This grammar is ambiguous. Consider [ [ ]

[ S C [ S C

[ S C [ S C ]

Parsing

• 2nd attempt: we can make ] be associated with the nearest unpaired [ as follows:

S[ SSTT [ T ]T

This grammar is not ambiguous. Consider [ [ ] S

[ T ]However, this grammar is not LL(1), either. Consider the case when S is on stack top and [ is the next input token. [First( [ S ) [First( T )This grammar can be parsed with a bottom-up parser, but not a top-downparser.

Parsing

• Solution: conflicts + special rules

1. GS ;2. Sif S E3. Sother4. Eelse S5. E

The parse table if else other ; G 1 1 S 2 3 E 4,5 5 conflicts

We can enforce that T[E, else] = 4th rule.

This essentially forces ‘else’ to be matched with the nearest unpaired ‘if’.

Parsing

• Alternative solution: change the language.

• Add ‘end if’ at the end of every ‘if’.

Sif S E Sother Eelse S end if Eend if

Parsing

§5.9 Properties of LL(1) parsers:

• A correct leftmost parse is guaranteed.

• All LL(1) grammars are un-ambiguous.

• linear time and linear space

Parsing

§ llgen

of the book

output from llgen

*definedecrtn 1ifprocess 2

Parsing

§ LL(k) parsing

• Recall a grammar is LL(1) only if

for any two productions A and A,

First(Follow(A))First(Follow(A)) =

• To generalize, we write

for any two productions A and A

Firstk(Followk(A)) Firstk(Followk(A)) =

if G is strong LL(k).

• The word ‘strong’ means G imposes too strong a condition.

Parsing

• Consider GS $ Sa A a Sb A b a Ab A

– This grammar is not LL(1)When A is on stack top and b is next token, we cannot choose between

Ab and A.

stack input b ..... A ......

-- Does it help if we can look ahead two tokens? NO! if the next two tokens are bb then we should choose Ab. if the next two tokens are ba then we cannot make a choice.

Parsing

case 1. input is aba a A A S a a G $ $ $

lookahead match lookahead ab a ba at this point, we should choose Ab

case 2. input is bba b A A b b S a a G $ $ $

lookahead match lookahead bb b ba at this point, we should choose A

Parsing

So the problem is not the limited numberof lookahead tokens.

The problem is in the ‘context’.

Parsing

• Therefore, the grammar is not strong LL(1).

• Actually, we can verify that the grammar is not strong LL(k) for all k1 by verify that

Firstk( ba$ ) Firstk( bFollowk(A) ) Firstk( Followk(A) )

for all k1

Parsing

• However, it is possible to parse the language of the grammar under the following conditions:

1. look ahead two tokens 2. from left to right 3. using the left context

We call such grammars LL(2), rather than strong LL(2).

• Note that LL(2) strong LL(2)

LL(1) = strong LL(1)

YANGYANG 1 Chap 5 LL(1) Parsing LL(1) left-to-right scanning leftmost derivation 1-token lookahead...

Documents