Top-Down Parsing

1

Relationship between parser types

2

Recursive descent

• Recursive descent parsers simply try to build a top-down parse tree.

• It would be better if we always knew the correct action to take.

• It would be better if we could avoid recursive procedure calls during parsing.

3

Predictive parsers• A predictive parser always knows which

production to use, ( to avoid backtracking ) • Example: for the productions•

stmt -> if ( expr ) stmt else stmt | while ( expr ) stmt | for ( stmt expr stmt ) stmt

• a recursive descent parser would always know which production to use, depending on the input token.

4

Transition diagrams• Transition diagrams can describe recursive parsers,

just like they can describe lexical analyzers, • (but the diagrams are slightly different.)

• Construction:1. Eliminate left recursion from G2. Left factor G3. For each non-terminal A, do

1. Create an initial and final (return) state2. For each production A -> X1 X2 … Xn, create a path from

the initial to the final state with edges X1 X2 … Xn.

5

Example transition diagrams• An expression

grammar with left recursion

• With ambiguity • E -> E+T | T• T -> T*F | F• F -> (E) | id

6

Corresponding transition diagrams:

Eliminating the ambiguity E -> T E’E’ -> + T E’ | εT -> F T’T’ -> * F T’ | εF -> ( E ) | id

The parsing table and parsing program

• The table is a 2D array M[A,a] where A is a nonterminal symbol and a is a terminal or $.

• At each step, the parser considers the top-of-stack symbol X and input symbol a: If both are $, accept If they are the same (nonterminals), pop X, advance

input If X is a nonterminal, consult M[X,a]. – If M[X,a] is “ERROR” call an error recovery routine.

Otherwise, if M[X,a] is a production of the grammar X -> UVW, replace X on the stack with WVU (U on top)

7

Predictive parsing without recursion

• To get rid of the recursive procedure calls, we maintain our own stack.

8

Example• Use the table-driven predictive parser to parse

id + id * id• Assuming parsing table

9

Initial stack is $EInitial input is id + id * id $

E -> T E’E’ -> + T E’ | εT -> F T’T’ -> * F T’ | εF -> ( E ) | id

Building a predictive parse table

• The construction requires two functions: • 1. FIRST • 2. FOLLOW

10

For First

• For a string of grammar symbols α, FIRST(α) is the set of terminals that begin all possible strings derived from α. If α =*> ε, then ε is also in FIRST(α).

• E -> T E’• E’ -> + T E’ | ε• T -> F T’• T’ -> * F T’ | ε• F -> ( E ) | id

11

FIRST(E) = FIRST (T) = FIRST (F) = {( , id }FIRST(E’) = {+ , }FIRST(T) = {( , id} FIRST(T’) = { *, }FIRST(F) = {( , id }

For Follow

• FOLLOW(A) for non terminal A is the set of terminals that can appear immediately to the right of A in some sentential form. If A can be the last symbol in a sentential form, then $ is also in FOLLOW(A).

• E -> T E’• E’ -> + T E’ | ε• T -> F T’• T’ -> * F T’ | ε• F -> ( E ) | id

12

Follow (E) = { ) , $ }Follow (E’) = Follow (E)= { ) ,$ }Follow (T) = { +, Follow (E)}= {+ , ) , $}Follow (T’) = {+, ) ,$}Follow ( F) = {*, +, ), $ }

How to compute FIRST(α)

1. If X is a terminal, FIRST(X) = X.2. Otherwise (X is a nonterminal),

1. 1. If X -> ε is a production, add ε to FIRST(X)2. 2. If X -> Y1 … Yk is a production, then place a in

FIRST(X) if for some i, a is in FIRST(Yi) and Y1…Yi-1 =*> ε.

• Given FIRST(X) for all single symbols X,• Let FIRST(X1…Xn) = FIRST(X1)• If ε FIRST(X∈ 1), then add FIRST(X2), and so on…

13

How to compute FOLLOW(A)

• Place $ in FOLLOW(S) (for S the start symbol)• If A -> α B β, then FIRST(β)-ε is placed in

FOLLOW(B)• If there is a production A -> α B or a

production A -> α B β where β =*> ε, then everything in FOLLOW(A) is in FOLLOW(B).

• Repeatedly apply these rules until no FOLLOW set changes.

14

Example FIRST and FOLLOW

• For our favorite grammar:E -> TE’E’ -> +TE | εT -> FT’T’ -> *FT’ | εF -> (E) | id

• What is FIRST() and FOLLOW() for all nonterminals?

15

Parse table construction withFIRST/FOLLOW

• Basic idea: if A -> α and a is in FIRST(α), then we expand A to α any time the current input is a and the top of stack is A.

• Algorithm:• For each production A -> α in G, do:• For each terminal a in FIRST(α) add A -> α to M[A,a]• If ε FIRST(α), for each terminal b in FOLLOW(A), do:∈• add A -> α to M[A,b]• If ε FIRST(α) and $ is in FOLLOW(A), add A -> α to M[A,$]∈• Make each undefined entry in M[ ] an ERROR

16

Example predictive parse table construction

• For our favorite grammar:E -> TE’E’ -> +TE | εT -> FT’T’ -> *FT’ | εF -> (E) | id

• What the predictive parsing table?

17

LL(1) grammars• The predictive parser algorithm can be applied to

ANY grammar.• But sometimes, M[ ] might have multiply defined

entries.• Example: for if-else statements and left factoring:

stmt -> if ( expr ) stmt optelseoptelse -> else stmt | ε

• When we have “optelse” on the stack and “else” in the input, we have a choice of how to expand optelse (“else” is in FOLLOW(optelse) so either rule is possible)

18

LL(1) grammars

• If the predictive parsing construction for G leads to a parse table M[ ] WITHOUT multiply defined entries,we say “G is LL(1)”

19

1 symbol of lookahead

Leftmost derivation

Left-to-right scan of the input

LL(1) grammars

• Necessary and sufficient conditions for G to be LL(1):

• If A -> α | β1. There does not exist a terminal a such that

a FIRST(α) and a FIRST(∈ ∈ β)2. At most one of α and β derive ε3. If β =*> ε, then FIRST(α) does not intersect with

FOLLOW(β).

20

This is the same as saying thepredictive parser alwaysknows what to do!

Model of a non recursive predictive parser.

21

a + b $ a + b $

XXYYZZ$$

Input bufferInput buffer

stackstack

Predictive parsingprogram/driver

Predictive parsingprogram/driver

Parsing Table M Parsing Table M

Moves made by predictive parser on input id + id * id

STACKSTACK INPUTINPUT OUTPUTOUTPUT

$$EE$$EE' T' T$$EE' T' F' T' F$$EE' T' ' T' idid$$EE' T'' T'$$EE' ' $$EE' T ' T ++$$EE' T' T$$EE' T' F' T' F$$EE' T' ' T' idid$$EE' T' ' T' $$EE' T' F' T' F * *$$EE' T' F' T' F$$EE' T' ' T' idid$$EE' T'' T'$$EE' ' $$

id id + + idid * * idid$$id id + + idid * * idid$$id id + + idid * * idid$$id id + + idid * * idid$$

+ + idid * * idid$$+ + idid * * idid$$+ + idid * * idid$$

idid * * idid$$idid * * idid$$idid * * idid$$

* * idid$$* * idid$$

idid$$idid$$

$$$$$$

EE TT EE''TT F TF T''FF idid

TT'' EE'' + + TT EE''

TT F TF T''FF idid

TT'' * * F TF T''

FF idid

TT'' EE''

22

Nonrecursive Predictive Parsing• 1. If X = a = $, the parser halts and announces successful completion of parsing.• 2. If X = a $, the parser pops X off the stack and advances the input pointer to the

next input symbol.• 3. If X is a nonterminal, the program consults entry M[X, a] of the parsing table M.

This entry will be either an X-production of the grammar or an error entry. If, for example, M[X, a] = {X UVW}, the parser replaces X on top of the stack by WVU (with U on top). As output, we shall assume that the parser just prints the production used; any other code could be executed here. If M[X, a] = error, the parser calls an error recovery routine.

23

Parsing table M for grammar

NONTER-NONTER-MINALMINAL

INPUT SYMBOLINPUT SYMBOL

IdId ++ ** (( )) $$

EE

EE''

TT

TT''

FF

EE TE'TE'

TT FT'FT'

F F id id

E'E' + +TE'TE'

T' T' T'T' * *FT'FT'

EE TE'TE'

TT FT'FT'

FF ((EE))

E'E'

T'T'

E'E'

T'T'

24

Top-down parsing recap

• RECURSIVE DESCENT parsers are easy to build, but inefficient, and might require backtracking.

• TRANSITION DIAGRAMS help us build recursive descent parsers.

• For LL(1) grammars, it is possible to build PREDICTIVE PARSERS with no recursion automatically. Compute FIRST() and FOLLOW() for all nonterminals Fill in the predictive parsing table Use the table-driven predictive parsing algorithm

25