– 1 – CSCE 531 Spring 2006

Lecture 8 Bottom Up Parsing

Lecture 8 Bottom Up Parsing

Topics Topics Overview Bottom-Up Parsing Handles Shift-reduce parsing Operator precedence parsing

Readings: 4.6Readings: 4.6

Homework: Test 1 – Feb 15Homework: Test 1 – Feb 15

February 6, 2006

CSCE 531 Compiler Construction

– 2 – CSCE 531 Spring 2006

OverviewOverviewLast TimeLast Time

First and Follow LL(1) property

Today’s Lecture Today’s Lecture Panic mode error recovery in Predictive parsing Overview Bottom-Up Parsing Handles Shift-reduce parsing Operator precedence parsing

Homework: Homework: LL(1) table for core (pdf email handout) grammar Test 1 Feb 15

– 3 – CSCE 531 Spring 2006

Panic Mode Error Recovery for LL(1)Panic Mode Error Recovery for LL(1)

– 4 – CSCE 531 Spring 2006

Recall Bottom-up Parsing Recall Bottom-up Parsing

Bottom-up parsers Bottom-up parsers

Start at the leaves and grow tree toward rootStart at the leaves and grow tree toward root

As input is consumed, encode possibilities on a As input is consumed, encode possibilities on a stackstack

– 5 – CSCE 531 Spring 2006

Bottom-up Parsing: TerminologyBottom-up Parsing: Terminology

A derivation consists of a series of rewrite stepsA derivation consists of a series of rewrite steps

SS 00 11 22 … … nn––11 nn

Each Each ii is a sentential form and if is a sentential form and if nn T* ({ tokens}* ) then it T* ({ tokens}* ) then it is a sentenceis a sentence

To get To get ii from from i–1i–1, expand some A, expand some A i–1i–1 by using A by using A i.e., Replace the occurrence of A i–1 with to get i

In a rightmost derivation, A would be the last nonterminal In a rightmost derivation, A would be the last nonterminal

inin i–1i–1

– 6 – CSCE 531 Spring 2006

Bottom-up ParsingBottom-up Parsing

In finding a rightmost derivation In finding a rightmost derivation

SS 00 11 … … ii––11 ii … … nn––11 nn

At each step we need to find a place where the right-hand At each step we need to find a place where the right-hand side of a production occurs and replace it with the non-side of a production occurs and replace it with the non-terminal on the LHS.terminal on the LHS.

Again we are building the parse tree from leaves to rootAgain we are building the parse tree from leaves to root

As we are constructing it is not always connectedAs we are constructing it is not always connected

Nodes with no parent in the partially constructed tree Nodes with no parent in the partially constructed tree form its upper fringe form its upper fringe

Since each replacement of Since each replacement of with with AA shrinks the upper shrinks the upper fringe, we call it a reduction.fringe, we call it a reduction.

– 7 – CSCE 531 Spring 2006

Example of Indentifying ReductionsExample of Indentifying Reductions

EE E+E | E * E | ( E ) | id | num E+E | E * E | ( E ) | id | num

Rightmost derivation of x * (z + w) with Rightmost derivation of x * (z + w) with token token sequence id * ( id + id )sequence id * ( id + id )

EEE * EE * E E * E * ( E )( E ) E * ( E * ( E + EE + E ) ) E * ( E + E * ( E + idid ) ) E * ( E * (idid+id) +id) idid * ( id + id ) * ( id + id )

FrontierFrontier id id * * ( ( id id + + id id ))

– 8 – CSCE 531 Spring 2006

Finding HandlesFinding Handles

The parser must find a substring The parser must find a substring in the tree’s frontier in the tree’s frontier that matches some production Athat matches some production A that occurs that occurs as as one step in the rightmost derivation one step in the rightmost derivation

We call this substring We call this substring a handle a handle

SS rmrm 11 … … rmrm ii––11==ααAw Aw rmrm ααw = w = ii rmrm … … rmrm nn

Because Because ii is a right-sentential form, the substring to the is a right-sentential form, the substring to the right of a handle (w) contains only tokensright of a handle (w) contains only tokens

– 9 – CSCE 531 Spring 2006

Figure 4.20 – a Tree with Handle AβFigure 4.20 – a Tree with Handle Aβ

A

β

α = α1 α2 … αk

S

w = w1 w2 … ws

…

– 10 – CSCE 531 Spring 2006

Grammar Unambiguous Unique HandlesGrammar Unambiguous Unique Handles

Sketch of Proof:Sketch of Proof:

1.1. GG is unambiguous implies is unambiguous implies

2.2. rightmost derivation is unique, which impliesrightmost derivation is unique, which implies

3.3. there is a unique production there is a unique production AA applied to derive applied to derive ii from from i–i–11

4.4. and a unique position and a unique position kk at which at which AA is applied is applied

5.5. thus the handle is uniquethus the handle is unique

This all follows from the definitionsThis all follows from the definitions

– 11 – CSCE 531 Spring 2006

Shift-reduce ParsingShift-reduce Parsing

A parser based on recognizing handles is called a Shift A parser based on recognizing handles is called a Shift Reduce parsersReduce parsers

A shift-reduce parser has just four actionsA shift-reduce parser has just four actions Shift: the next token is pushed (shifted) onto the stackShift: the next token is pushed (shifted) onto the stack Reduce: the right end of a handle is on the top of stackReduce: the right end of a handle is on the top of stack

Locate the left end of the handle within the stack Pop the handle off stack & push appropriate lhs

Accept — stop parsing & report successAccept — stop parsing & report success Error — call an error handling/recovery routineError — call an error handling/recovery routine

– 12 – CSCE 531 Spring 2006

Shift-reduce ParsingShift-reduce Parsing

Accept - finish upAccept - finish up

Error - error recovery, panic mode againError - error recovery, panic mode again

Shift Shift push the current token call to the scanner to get the next token

Reducing by A Reducing by A ββ requires requires Popping β from the stack Push A

– 13 – CSCE 531 Spring 2006

Stack Implementation of Shift-Reduce ParsersStack Implementation of Shift-Reduce ParsersStackStack InputInput ActionAction

$$ id * ( id + id ) $id * ( id + id ) $ Shift idShift id

$ id $ id * ( id + id ) $* ( id + id ) $ Reduce EReduce Eidid

$ E$ E * ( id + id ) $* ( id + id ) $ Shift *Shift *

$ E *$ E * ( id + id ) $( id + id ) $ Shift (Shift (

$ E * ($ E * ( id + id ) $id + id ) $ Shift idShift id

$ E * ( id$ E * ( id + id ) $+ id ) $ Reduce EReduce Eidid

$ E * ( E$ E * ( E + id ) $+ id ) $ Shift +Shift +

$ E * ( E +$ E * ( E + id ) $id ) $ Shift idShift id

$ E * ( E + id$ E * ( E + id ) $) $ Reduce EReduce Eidid

$ E * ( E + E$ E * ( E + E ) $) $ Reduce EReduce EE+EE+E

$ E * ( E$ E * ( E ) $) $ Shift )Shift )

$ E * ( E )$ E * ( E ) $$ Reduce EReduce E ( E ) ( E )

$ E * E$ E * E $$ Reduce EReduce E ( E ) ( E )

– 14 – CSCE 531 Spring 2006

More ExamplesMore Examples

Now what aboutNow what about

id * ( id + id ) + idid * ( id + id ) + id

See fig 4.22 for another exampleSee fig 4.22 for another example

– 15 – CSCE 531 Spring 2006

Justification of the Stack ImplementationJustification of the Stack Implementation

Consider two derivationsConsider two derivations

(1)(1) S S … … ααAz Az αβαβByz Byz … … αβμαβμyzyz Rewriting by A βBy

(2)(2) S S … … ααBxAz BxAz ααBxyz Bxyz … … αμαμxyz xyz Rewriting by A y

– 16 – CSCE 531 Spring 2006

Justification of the Stack ImplementationJustification of the Stack ImplementationConsider the first derivationConsider the first derivation

(1)(1) S S … … ααAz Az αβαβByz Byz αβμαβμyzyz Rewriting by A βBy, and then B μ

StackStack InputInput ActionsActions

$ $ αα ββ μμ yz$yz$ Reduce by BReduce by B μμ

$ $ αα ββ B B yz$yz$

Since B is the rightmost nonterminal in Since B is the rightmost nonterminal in αβαβByz, the handle cannot be all Byz, the handle cannot be all in the stackin the stack

So the parser can shift the string of tokens y onto the stackSo the parser can shift the string of tokens y onto the stack

StackStack InputInput ActionsActions

$ $ αα ββ B y B y z$z$ Reduce by AReduce by A ββByBy

$ $ αα A A z$z$

– 17 – CSCE 531 Spring 2006

Justification of the Stack ImplementationJustification of the Stack ImplementationConsider the second derivationConsider the second derivation

(2) S (2) S … … ααBxAz BxAz ααBxyz Bxyz αμαμxyz xyz Rewriting by A y and then B μ

StackStack InputInput ActionsActions

$ $ αα μμ xyz$xyz$ Reduce by BReduce by B μμ

$ $ αα B B xyz$xyz$

Now the parser can shift the string of tokens xy onto the stackNow the parser can shift the string of tokens xy onto the stack

StackStack InputInput ActionsActions

$ $ αα B x y B x y z$z$ Reduce by AReduce by A y y

$ $ αα B x A B x A z$z$

– 18 – CSCE 531 Spring 2006

An Important Lesson about HandlesAn Important Lesson about Handles

To be a handle, a substring of a sentential form To be a handle, a substring of a sentential form must must have two properties:have two properties: It must match the right hand side of some rule A There must be some rightmost derivation from the goal symbol

that produces the sentential form with A as the last production applied

Simply looking for right hand sides that match strings is Simply looking for right hand sides that match strings is not good enoughnot good enough

Critical Question: Critical Question: How can we know when we have found a How can we know when we have found a handle without generating lots of different derivations?handle without generating lots of different derivations? Answer: we use look ahead in the grammar along with tables

produced as the result of analyzing the grammar. LR(1) parsers build a DFA that runs over the stack & finds them

– 19 – CSCE 531 Spring 2006

An Important Lesson about HandlesAn Important Lesson about Handles

To be a handle, a substring of a sentential form To be a handle, a substring of a sentential form must must have two properties:have two properties: It must match the right hand side of some rule A There must be some rightmost derivation from the goal

symbol that produces the sentential form with A as the last production applied

We have seen that simply looking for right hand sides We have seen that simply looking for right hand sides that match strings is not good enoughthat match strings is not good enough

Critical Question: Critical Question: How can we know when we have How can we know when we have found a handle without generating lots of different found a handle without generating lots of different derivations?derivations? Answer: we use look ahead in the grammar along with tables

produced as the result of analyzing the grammar. o There are a number of different ways to do this.o We will look at two: operator precedence and LR parsing

– 20 – CSCE 531 Spring 2006

Model of an LR ParserModel of an LR Parser

aa11 …… aaii …… aann$$

ssmm

XXmm

ssm-1m-1

XXm-1m-1

……

ss00

Stack

input

outputLR ParsingProgram

Parsing Table

Action goto

– 21 – CSCE 531 Spring 2006

LR Parsing AlgorithmLR Parsing Algorithm

– 22 – CSCE 531 Spring 2006

Expression LR-Parsing Table fig 4.31Expression LR-Parsing Table fig 4.31

StateState ActionAction gotogoto

IdId ++ ** (( )) $$ EE TT FF

00 S5S5 S4S4 11 22 33

11 S6S6 acceptaccept

22 R2R2 S7S7 R2R2

33 R4R4 R4R4 R4R4

44 S5S5 S4S4 88 22 33

55 R6R6 R6R6 R6R6

66 S5S5 S4S4 99 33

77 S5S5 S4S4 1010

88 S6S6 S11S11

99 R1R1 S7S7 R1R1 R1R1

1010 R3R3 R3R3 R3R3 R3R3

1111 R5R5 R5R5 R5R5 R5R5

– 23 – CSCE 531 Spring 2006

LR Parse of id * ( id + id ) + id LR Parse of id * ( id + id ) + id StackStack InputInput ActionAction

0 0 id * ( id + id ) + id id * ( id + id ) + id $$ Shift idShift id