Parsing Top-Down. 2 Top-Down parsers The top-down parser must start at the root of the tree and...

Parsing Top-DownParsing Top-Down

Parsing Top-DownParsing Top-Down 22

Top-Down parsersTop-Down parsers

The top-down parser must start at the root The top-down parser must start at the root of the tree and determine, from the token of the tree and determine, from the token stream, how to grow the parse tree that stream, how to grow the parse tree that results in the observed tokens.results in the observed tokens.

This approach runs into several problems, This approach runs into several problems, which we will now deal with.which we will now deal with.


Left RecursionLeft Recursion

Productions of the form (A->AProductions of the form (A->A) are left ) are left recursive. recursive.

No Top-Down parser can handle left No Top-Down parser can handle left recursive grammars. recursive grammars.

Therefore we must re-write the grammar Therefore we must re-write the grammar and eliminate both direct and indirect left and eliminate both direct and indirect left recursion. recursion.


How to eliminate Left Recursion (direct) How to eliminate Left Recursion (direct) – Given: Given:

A -> AA -> A11 | A | A22 | A | A33 | … | …

A -> A -> 11 | | 22 | | 33 | ... | ...

– Introduce A' Introduce A' A -> A -> 11 A' | A' | 22 A' | A' | 33 A' | ... A' | ...

A' -> A' -> | | 11 A' | A' | 22 A' | A' | 33 A' | ... A' | ...

– Example: Example: S -> Sa | b S -> Sa | b Becomes Becomes S -> bS' S -> bS' S' -> S' -> | a S' | a S'


How to remove ALL Left Recursion. How to remove ALL Left Recursion. – 1.Sort the nonterminals 1.Sort the nonterminals

– 2.for each nonterminal2.for each nonterminalif B -> Aif B -> A and A -> and A -> 11 | | 22 | | 33 | ... | ...

then B -> then B -> 11| | 22 | | 33 | ... | ...

– 3.After all done, remove immediate left 3.After all done, remove immediate left recursion. recursion.


Example: Example: – S -> aA | b | cSS -> aA | b | cS– A -> Sd | e A -> Sd | e

becomesbecomes

– S -> aA | b | cSS -> aA | b | cS– A -> aAd | bd | cSd | eA -> aAd | bd | cSd | e

note: the S in A(3) -> but it is NOT left note: the S in A(3) -> but it is NOT left recursion recursion


BacktrackingBacktracking

One way to carry out a top-down parse is One way to carry out a top-down parse is simply to have the parser try all applicable simply to have the parser try all applicable productions exhaustively until it finds a tree. productions exhaustively until it finds a tree.

This is sometimes called the brute force This is sometimes called the brute force method. method.

It is similar to depth-first search of a graph It is similar to depth-first search of a graph

Tokens may have to be put back on the Tokens may have to be put back on the input stream input stream


Given a grammar: Given a grammar: – S -> ee | bAc | bAeS -> ee | bAc | bAe– A -> d | cAA -> d | cA

A Backtracking algorithm will not work A Backtracking algorithm will not work properly with this grammar. properly with this grammar.

Example: input string is bcdeExample: input string is bcde– When you see a b you select S -> bAc When you see a b you select S -> bAc – This is wrong since the last letter is e not cThis is wrong since the last letter is e not c


The solution is Left Factorization. The solution is Left Factorization. – Def:Def: Left FactorizationLeft Factorization: -- create a new : -- create a new

non-terminal for a unique right part of a left non-terminal for a unique right part of a left factorable production. factorable production.

Left Factor the grammar given Left Factor the grammar given previously. previously. – S -> ee | bAQS -> ee | bAQ– Q -> c | eQ -> c | e– A -> d | cAA -> d | cA


Recursive-Descent ParsingRecursive-Descent Parsing

There is one function for each non-There is one function for each non-terminal these functions try each terminal these functions try each production and call other functions for non-production and call other functions for non-terminals. terminals.

The stack is invisible for CFG's The stack is invisible for CFG's

The problem is -- a new grammar requires The problem is -- a new grammar requires new code. new code.


Example: Example: – S -> bA | cS -> bA | c– A -> dSd | eA -> dSd | e

Code: Code: function S: boolean;function S: boolean; beginbegin S := true;S := true; if token_is ('b') thenif token_is ('b') then if A thenif A then writeln('S --> writeln('S --> bA')bA') elseelse S := false;S := false;

elseelse

if token_is ('c') if token_is ('c') thenthen

writeln ('S --> c')writeln ('S --> c')

elseelse

beginbegin

error ('S');error ('S');

S := falseS := false

endend

end; { S }end; { S }


function A: booleanfunction A: boolean

beginbegin

A := true;A := true;

if token_is ('d') thenif token_is ('d') then

beginbegin

if S thenif S then

if token_is ('d') thenif token_is ('d') then

writeln('A --> dSd');writeln('A --> dSd');

elseelse

beginbegin

error ('A'); error ('A');

A := falseA := false

endend

elseelse A := falseA := false endend elseelse if token_is ('e') if token_is ('e') thenthen writeln ('A --> e') writeln ('A --> e') elseelse beginbegin error ('A');error ('A'); A := falseA := false endendend; { A }end; { A }


Input String: bdcd Input String: bdcd



Predictive ParsersPredictive Parsers

The goal of a predictive parser is to know The goal of a predictive parser is to know which characters on the input string trigger which characters on the input string trigger which productions in building the parse tree.which productions in building the parse tree.

Backtracking can be avoided if the parser Backtracking can be avoided if the parser had the ability to look ahead in the grammar had the ability to look ahead in the grammar so as to anticipate what terminals are so as to anticipate what terminals are derivable (by leftmost derivations) from each derivable (by leftmost derivations) from each of the various nonterminals on the RHS.of the various nonterminals on the RHS.


First (First () ) (you construct first() of (you construct first() of

RHS’s)RHS’s)– 1.if 1.if begins with a terminal x, begins with a terminal x,

then first(then first() = x. ) = x.

– 2.if 2.if = =**=> => , , then first(then first( includes includes . .

– 3.First(3.First() = {) = {}. }. – 4.if 4.if begins with a nonterminal A, begins with a nonterminal A,

then first(then first() includes first(A) - {) includes first(A) - {}}


Follow(Follow() ) – 1.if A is the start symbol, 1.if A is the start symbol,

then put the end marker $ into follow(A). then put the end marker $ into follow(A).

– 2.for each production with A on the right 2.for each production with A on the right hand side hand side

Q -> xAy Q -> xAy

A) if y begins with a terminal q, A) if y begins with a terminal q, – q is in follow(A). q is in follow(A).

else follow(A) includes first(y)-else follow(A) includes first(y)-. .

B) if y = B) if y = , or y is nullable (y =, or y is nullable (y =**=> => ) ) – then add follow(Q) to follow(A). then add follow(Q) to follow(A).


Grammar: Grammar: – E -> T QE -> T Q– Q -> + T Q | - T Q | epsilonQ -> + T Q | - T Q | epsilon– T -> F RT -> F R– R -> * F R | / F R | epsilonR -> * F R | / F R | epsilon– F -> ( E ) | IF -> ( E ) | I

Construction of First and Follow Sets: Construction of First and Follow Sets:

First(E->TQ) = First(E->TQ) = – First(T) = First(T) = – First(F) = {i,(} First(F) = {i,(}

First(Q->+TQ) = {+} First(Q->+TQ) = {+} First(Q->-TQ) = {-} First(Q->-TQ) = {-} First(Q->First(Q->) = {) = {} } First(R-> *FR) = {*}First(R-> *FR) = {*}

First(R->/FR) = {/} First(R->/FR) = {/} First(R->First(R->) = {) = {} } First(F-> (E) ) = {(}First(F-> (E) ) = {(}First(F-> I) = {I}First(F-> I) = {I}

Follow(E) = {$,)} Follow(E) = {$,)} Follow(Q) = Follow(Q) = Follow(E) = {$, )} Follow(E) = {$, )} Follow(T) = Follow(T) = – First(Q) - First(Q) - + +

Follow(E) = Follow(E) = – {+,-} + {),$} {+,-} + {),$}

Follow(R) = Follow(R) = Follow(T) Follow(T) Follow(F) = Follow(F) = – First(R) - First(R) - + +

Follow(T) = Follow(T) = – {*,/} + {+,-,),$} {*,/} + {+,-,),$}



First(E) = First(T) = First(F) = {i,(} First(E) = First(T) = First(F) = {i,(}

First(Q) = {+,-,First(Q) = {+,-,} }

First(R) = {*,/,First(R) = {*,/,} }

Follow(E) = {$,)} Follow(E) = {$,)}

Follow(Q) = Follow(E) = {$, )} Follow(Q) = Follow(E) = {$, )}

Follow(T) = First(Q) - Follow(T) = First(Q) - + Follow(E) + Follow(E) {+,-} + {),$} {+,-} + {),$}

Follow(R) = Follow(T) Follow(R) = Follow(T)

Follow(F) = First(R) - Follow(F) = First(R) - + Follow(T) + Follow(T) {*,/} + {+,-,),$} {*,/} + {+,-,),$}


LL(1) Grammars LL(1) Grammars – In a predictive parser, Follow tells us when In a predictive parser, Follow tells us when

to use the epsilon productions. to use the epsilon productions.

– Def:Def: LL(1)LL(1) -- -- LLeft to Right Scan of the eft to Right Scan of the tokens, tokens, LLeftmost derivation, eftmost derivation, 11 token token lookahead. lookahead.


For a grammar to be LL(1), we require For a grammar to be LL(1), we require that for every pair of productions A -> that for every pair of productions A -> || – 1.First(1.First()-)- and First( and First()-)- must be disjoint. must be disjoint. – 2.if 2.if is nullable, then First( is nullable, then First() and ) and

Follow(A) must be disjoint. Follow(A) must be disjoint.

– if rule 1 is violated, we may not know which if rule 1 is violated, we may not know which right hand side to chooseright hand side to choose

– if rule 2 is violated, we may not know when if rule 2 is violated, we may not know when to choose to choose or or ..



Table-Driven Predictive ParsersTable-Driven Predictive Parsers

Grammar Grammar – E -> E + T | E - T | TE -> E + T | E - T | T– T -> T * F | T / F | FT -> T * F | T / F | F– F -> ( E ) | IF -> ( E ) | I

Step 1: Eliminate Left Recursion.Step 1: Eliminate Left Recursion.


Grammar without left recursion Grammar without left recursion – E -> T QE -> T Q– Q -> + T Q | - T Q | epsilonQ -> + T Q | - T Q | epsilon– T -> F RT -> F R– R -> * F R | / F R | epsilonR -> * F R | / F R | epsilon– F -> ( E ) | IF -> ( E ) | I

It is easier to show you the table, and It is easier to show you the table, and how it is used first, and to show how the how it is used first, and to show how the table is constructed afterward.table is constructed afterward.


TableTable


Driver Algorithm:Driver Algorithm:– Push $ onto the stack Push $ onto the stack – Put a similar end marker on the end of the Put a similar end marker on the end of the

string. string. – Push the start symbol onto the stack. Push the start symbol onto the stack.


– While (stack not empty do) While (stack not empty do) Let x = top of stack and a = incoming token. Let x = top of stack and a = incoming token. If x is in T (a terminal)If x is in T (a terminal) if x == a then pop x and goto next input token if x == a then pop x and goto next input token else error else error else (nonterminal)else (nonterminal) if Table[x,a] if Table[x,a] pop x pop x push Table[x,a] onto stack in reverse order push Table[x,a] onto stack in reverse order else errorelse error

It is a successful parse if the stack is It is a successful parse if the stack is empty and the input is used up. empty and the input is used up.


Example 1: (i+i)*iExample 1: (i+i)*i




Example 2: (i*)Example 2: (i*)



Constructing the Predictive Constructing the Predictive Parser TableParser Table

Go through all the productions. Go through all the productions. X -> X -> is your typical production. is your typical production.

– 1.For all terminals a in First(1.For all terminals a in First(), except ), except , , Table[X,a] = Table[X,a] = . .

– 2.If 2.If = = , or if , or if is in first( is in first() then For ALL a in ) then For ALL a in Follow(X), Table[X,a] = Follow(X), Table[X,a] = ..

So, Construct First and Follow for all Left So, Construct First and Follow for all Left and right hand sides. and right hand sides.


ConflictsConflicts

A conflict occurs if there is more than 1 A conflict occurs if there is more than 1 entry in a table slot. This can sometimes entry in a table slot. This can sometimes be fixed by Left Factoring, ... be fixed by Left Factoring, ...

If a grammar is LL(1) there will not be If a grammar is LL(1) there will not be multiple entries. multiple entries.


Summary of Top DownSummary of Top Down

Left Recursion Left Recursion

Left Factorization Left Factorization

First (A) First (A)

Follow (A) Follow (A)

Predictive Parsers (table driven) Predictive Parsers (table driven)

Date post:	13-Dec-2015
Category:	Documents
Upload:	ashlee-melton
View:	231 times
Download:	0 times

Parsing Top-Down. 2 Top-Down parsers The top-down parser must start at the root of the tree and...

Documents