3. Top-Down Syntax AnalysisContextSyntax DeﬁnitionsParsersContext-Free GrammarsTop-Down...

Context Syntax Definitions Parsers Context-Free Grammars Top-Down Parsing HACS as a Parser Generator

3. Top-Down Syntax Analysis

Eva Rose Kristoffer Rose

NYU Courant InstituteCompiler Construction (CSCI-GA.2130-001)

http://cs.nyu.edu/courses/fall14/CSCI-GA.2130-001/lecture-3.pdf

September 18, 2014

Eva Rose, Kristoffer Rose Compiler Construction (CSCI-GA.2130-001) 3. Top-Down Syntax Analysis September 18, 2014 1 / 80

http://cs.nyu.edu/courses/fall14/CSCI-GA.2130-001/lecture-3.pdf


1 Context

2 Syntax Definitions

3 Parsers

4 Context-Free Grammars

5 Top-Down ParsingFIRST and FOLLOWPredictive Parsing

6 HACS as a Parser Generator



Second compilation phasesource program

Lexical Analysis--

Syntax AnalysisTokens

--

Semantic AnalysisTree

--

SymbolTable

Intermediate Representation GeneratorTree

--

OptimizerIR--

Code GeneratorIR--

Machine-Dependent Code OptimizerIR--

target machine code��



Example

position = initial + rate * 60

scanned into list of tokens:

〈id,1〉〈=〉〈id,2〉〈+〉〈id,3〉〈∗〉〈num,60〉1 position2 initial3 rate



Example

parsed into syntax tree:

=

〈id,1〉ww

+''

〈id,2〉ww

∗''

〈id,3〉ww

〈num,60〉''

according to precedence rules...



Example

Some tree-structuring syntax rules:

I block indentation rules (Python)I block delimiters like {...} (Java, C)I grouping rules like ( ... ) (most languages)I built-in algebraic precedence rules (most languages)I statements vs expressions ...



1 Context


3 Parsers






How do we specify language syntax?

I Context-free grammarI special notation (BNF)I Set of rules (productions)

Example:

if (x=2) print("yep"); else print("nope");

Corresponds to a rule:

stm → if (expr) stm else stm



How do we specify language syntax?

I Context-free grammarI special notation (BNF)I Set of rules (productions)

Example:

if (x=2) print("yep"); else print("nope");

Corresponds to a rule:




Production rules

stm

production head

77→

“can have the form”

OOif (expr) stm else stm︸︷︷︸

production body

__



Production rules


Nonterminals need more rules to define them.Terminals are well defined, no more rules define them.



Components of context-free grammar.

1 Set of terminal symbols.2 Set of nonterminal symbols.3 Set of productions.

I The head is non-terminal.I The body is a sequence of terminals and non-terminals.

4 Designation of one nonterminal as the starting symbol.



Example

list→ list + digitlist→ list− digitlist→ digit

digit→ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

1 What are the terminals here?2 What are the nonterminals?3 What does the grammar generate?



Derivations

Derivation algorithm:

I given the grammar (production rules),I begin with the start symbols,I repeatedly replace nonterminals (head) with their bodies,I the generated set of terminals defines the language of that

grammar.

Example: How do we derive the string 9− 5 + 7?



Derivations

list→ list + digit | list− digit | digitdigit→ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

Deriving the string 9− 5 + 7:

list⇒ list + digit⇒ list− digit + digit⇒ digit− digit + digit⇒ 9− digit + digit⇒ 9− 5 + digit⇒ 9− 5 + 7



Parsing

Given a string of terminals, figure out how to picture a tree from the start symbol of thegrammar where all the terminals are at the leaves.

A→ XYZ A

Root is labeled by the start symbolqq

X Y

Interior nodes are nonterminals

Z

Each leaf is a terminal or ε

ll

The process of finding such a “grammar” tree is called parsing. The “grammar” tree iscalled a parse tree.



Exercise

list→ list + digit | list− digit | digitdigit→ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

Derive a parse tree for the string 9− 5 + 7.



Example

string→ string + string | string− string | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

What about a parse tree for the string 9− 5 + 7 with this grammar?



Ambiguity

A grammar is ambiguous if it has more than one parse tree generating thesame string of terminals.

string

string

string

9

− string

5

+ string

2

string

string

9

− string

string

5

+ string

2

Two parse trees for 9− 5 + 2. Which is right?



Examples

Are any of the following grammars ambiguous?

1 S→ + S S | − S S | a2 S→ S ( S ) S | ε



Associativity of operators

How will you evaluate 9−5−2?

I If 5 goes with the ‘−’ on the left: (9−5)−2 we say theoperator is left associative.

I If 5 goes with the ‘−’ on the right: 9−(5−2) we say theoperator is right associative.



Associativity of operators

How do we express associativity in production rules?

I Left associative (9−5)−2:

term→ term− digit | digitdigit→ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

I Right associative 9−(5−2):

term→ digit− term | digitdigit→ 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9



Precedence of operators

What if operators are different, e.g., 9−5 ∗ 2?

I If ‘∗’ takes operands before ‘−’, it is said to have higherprecedence.

I Another example: logical operators. . .



Precedence of operators

How to present precedence in productions?

expr→ expr + term | expr− term | term

term→ term ∗ factor | term/factor | factor

factor→ digit | (expr)

The example shows both operator precedence (‘∗’, ‘/’ over‘+’, ‘−’) and left associativity.



1 Context


3 Parsers






LexicalAnalysis

SyntaxAnalysis

SemanticAnalysis

TokenStream

Token-- SyntaxanalysergetNextToken

mm

Parse Tree-- Semanticanalyser

getNextTreemm

Symboltable

$$

dd

��

OO

zz

::



Parsing

The parsing task:

I We have: set of grammar productions.I We have: string of terminals for the grammar.I We need: find a parse tree that generates the string.



Example: parsing

Given these grammar productions:

stmt→ expr| if (expr) stmt| for (optexpr ; optexpr ; optexpr) stmt| other

optexpr→ expr | ε

and this string:

for (ε ; expr ; expr)other



Example: parsing

How do we generate this parse tree in an operational manner?

stmt

for ( optexpr

ε

; optexpr

expr

; optexpr

expr

) stmt

other



Construction of a parse tree

Three general kinds of parsers:

Universal from any grammar, but too inefficient.Bottom-up identifying the string symbols as terminals,

constructing the tree from leaves to root.Top-down beginning with the start symbol as the root,

constructing the tree from root to leaves.



Definition

Given an input string top-down parsing is defined as follows:I Start naming the root with the starting (nonterminal)

symbol.I Repeat whilst scanning the input sting, one symbol at a

time:1 At node N labeled with nonterminal A: select a production for A

and construct children at N with the symbols in the productionbody.

2 Find the next unexpanded node/nonterminal, typically theleftmost unexpanded nonterminal in the tree

“select a production for A” is guided by the current input symbol.



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Example

stmtOO

forOO (OO

optexpr

ε

;OO

optexpr

exprOO

;OO

optexpr

exprOO

)OO

stmt

otherOO

forOO (OO

;OO

exprOO

;OO

exprOO

)OO

otherOO



Parsing

Sometimes choosing the right production involves trial anderror, whence backtracking!



Parsing

Predictive parsing: NO backtracking!

I Part of a category called recursive-descent parsing aretop-down methods, based on recursive procedures.

I The lookahead symbol unambiguously determines the flowof control.



Parsing

Designing a predictive parser.

I A procedure is designed for every nonterminal/production.I Examination of the lookahead symbol chooses a

production.I No conflict between two bodies with the same head may

occur.I The procedure mimics the body of the chosen production:

I nonterminals are procedure calls,I terminals are matched and lookahead is advanced.



Left recursion: a problem!

Left recursive grammar:

expr→ expr + term | term

Eliminating left recursion:

expr→ term factorfactor→ + term factor | ε

Both generating: term, term+term, term+term+term, . . .



1 Context


3 Parsers






Context Free Grammars, revisited

Start symbol, Nonterminals, Terminals, Productions

expression→ expression+ termexpression→ expression− termexpression→ term

term→ term ∗ factorterm→ term / factorterm→ factor

factor→ (expression)factor→ id



Derivations, revisited

I Start with start symbol.I Each step: a nonterminal is replaced with the body of a

production.

Example:

E→ E + E | E ∗ E | − E | (E) | id

Deriving –(id + id)

E⇒ −E⇒ −(E + E)⇒ −(id + E)⇒ −(id + id)




⇒ means derive in one step.∗⇒ means derive in zero or more steps.+⇒ means derive in one or more steps.

1 α∗⇒ α for any string α (reflective).

2 α∗⇒ β, and β ∗⇒ γ, then α ∗⇒ γ (transitive).

3 Same properties for +⇒.




I Leftmost derivations: the leftmost nonterminal in eachsentential is always chosen. α⇒lm β

I Rightmost derivations: the rightmost nonterminal in eachsentential is always chosen. α⇒rm β



Example

S→ SS + | SS ∗ | a

and the string aa+a*1 Give leftmost derivation for the string.2 Give rightmost derivation for the string.3 Give parse tree for the string.



Parse trees and derivations

Relationship:

I parse trees are graphical representations of derivations,I parse trees filter out the order of nonterminal

replacements,I many-to-one relationship between derivations and parse

trees.



Example

E→ E + E | E ∗ E | − E | (E) | id

The string −(id + id) has two derivations:I E⇒ −E⇒ −(E)⇒ −(E + E)⇒ −(id + E)⇒ −(id + id)I E⇒ −E⇒ −(E)⇒ −(E + E)⇒ −(E + id)⇒ −(id + id)

...but one parse tree!



Context-free grammars vs regular expressions

I Grammars are more powerful notations than regularexpressions.

Every construct that can be described by a regular expressioncan be described by a grammar, but not vice versa.

Example:{anbn | n ≥ 1}



From RE to NFA

A regular expression:(a|b)∗abb

as NFA:

0start //

a��

b

GG 1a // 2b // 3b //



From NFA to context-free grammar

Now apply the conversion algorithm:1 For each state i of NFA, create nonterminal Ai.

2 If state i has transition to state j on input a, add the productionAi → aAj. If transition happens on ε, add the production Ai → Aj.

3 If i is an accepting state, add Ai → ε.

4 If i is start state, make Ai the start symbol of the grammar.

which renders:

A0 → aA0 | bA0 | aA1

A1 → bA2

A2 → bA3

A3 → εEva Rose, Kristoffer Rose Compiler Construction (CSCI-GA.2130-001) 3. Top-Down Syntax Analysis September 18, 2014 46 / 80


Relevant question

If grammars are so much more powerful than regularexpressions, why not use them during lexical analysis?

I Lexical (token) descriptions are quite simple patterns.I REs easier to understand for simple patterns.I Easier to generate an efficient lexical analyser from a

simple REs.



How can we enhance our grammar?

1 Eliminate ambiguity.2 Eliminate left-recursion.3 Left factoring.



Eliminating ambiguity

Sometimes we can re-write the grammar to eliminateambiguity.

stmt→ if expr then stmt| if expr then stmt else stmt (4.14)| other

consider the parse trees for:

if expr then if expr then stmt else stmt

How can we fix it?Eva Rose, Kristoffer Rose Compiler Construction (CSCI-GA.2130-001) 3. Top-Down Syntax Analysis September 18, 2014 49 / 80


Eliminating ambiguity

stmt→ matched_stmt| open_stmt

matched_stmt→ if expr then matched_stmt else matched_stmt| other

open_stmt→ if expr then stmt| if expr then matched_stmt else open_stmt

Now re-consider: if expr then if expr then stmt else stmtEva Rose, Kristoffer Rose Compiler Construction (CSCI-GA.2130-001) 3. Top-Down Syntax Analysis September 18, 2014 50 / 80


Eliminating Left-Recursion

A→ Aα | β can be rewritten:A→ βA′

A′ → αA′ | ε

A→ Aα1 | . . . | Aαm | β1 | . . . | βn can be rewritten:

A→ β1A′ | . . . | βnA′

A′ → α1A′ | . . . | αmA′ | ε



Example: Eliminating Left-Recursion

How about something like:

S→ A a | bA→ A c | S d | ε (4.18)

Second line can be rewritten: A→ A c | A a d | b d | εResulting in:

S→ A a | bA→ b d A′ | A′

A′ → c A′ | a d A′ | ε



Left Factoring

A way to delay the decision of what production rule to expand by.

General rule: A→ α β1 | α β2 rewritten to:A→ α A′

A′ → β1 | β2

Example:

stmt→ if expr then stmt else stmt| if expr then stmt

rewritten to:

stmt→ EXP else stmt | EXPEXP→ if expr then stmt



1 Context


3 Parsers






Top-Down Parsing

Once grammar is on its optimal form:1 Constructing a parse tree, for an input string, starting at

the root:I parse tree build in preorder (depth-first).

2 Finding a left-most derivation.3 At each step of the top-down parse:

I determine which production to be applied,I matching terminal symbols in the production body with the input

string symbols.



Example

E→ E + T | TT → T ∗ F | F (4.1)F → ( E ) | id

Left-recursion elimination:

E→ T E′

E′ → + T E′ | εT → F T′ (4.2)T′ → ∗ F T′ | εF → ( E ) | id



Example: Top-Down Parsing

Consider the string: id + id ∗ id

E ⇒lm

ET E′ ⇒lm

ET

F T′E′ ⇒lm

ET

Fid

T′E′ ⇒lm

ET

Fid

T′

ε

E′

⇒lm

ET

Fid

T′

ε

E′

+ T E′⇒lm

ET

Fid

T′

ε

E′

+ TF T′

E′




⇒lm

E

T

F

id

T′

ε

E′

+ T

F

id

T′

E′

⇒lm

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F T′

E′

⇒lm

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F

id

T′

E′

⇒lm

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F

id

T′

ε

E′




⇒lm

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F

id

T′

ε

E′

ε



Top-Down parser categories

Recursive-Descent Parser general form of top-down parsers.I May require backtracking.

Predictive Parser special case of recursive-descent parsers.I No backtracking necessary.I Constructed from LL(k) grammars, k ≥ 1.

An LL(k) grammar implies scanning input from Left, retrievingLeftmost derivation, using k lookahead symbols to reach aresolution.



Choosing the right production

Key parsing problem: determining the production to be appliedfor some nonterminal A.

FIRST and FOLLOW helps choosing the production based on thenext input symbol.



Definition: FIRST and FOLLOW

DefinitionDefine FIRST(α), where α is any string of grammar symbols, to be the set ofterminals that begin strings derived from α. If α ∗⇒ ε, then ε is also inFIRST(α).

DefinitionDefine FOLLOW(A), for non-terminal A, to be the set of terminals a that canappear immediately to the right of A in some sentential form; that is, the setof terminals a such that there exists a derivation of the form S ∗⇒ αAaβ forsome α and β.



Algorithm: FIRST Set for a Grammar

Repeat for each production until a fixed point (with stable sets) is reached:

1 If X is a terminal, then FIRST(X) = {X}

2 If X is a nonterminal and X → Y1 . . .Yk, k ≥ 1, a ∈ FIRST(X) ifa ∈ FIRST(Yi), where ε is in all of FIRST(Y1) . . . FIRST(Yi−1); that is,Y1 . . .Yi−1

∗⇒ ε. If ε ∈ FIRST(Yj), for all j = 1, . . . , k, k ≥ 1, then add ε toFIRST(X).

3 If X → ε is a production, then add ε to FIRST(X).



Algorithm: FOLLOW Set for a Grammar

Repeat for each production until a fixed point (with stable sets) is reached:

1 Place $ in FOLLOW(S), where S is the start symbol, and $ is the inputright endmarker.

2 If there is a production A→ αBβ, then everything in FIRST(β) except εis in FOLLOW(B).

3 If there is a production A→ αB, or a production A→ αBβ whereFIRST(β) contains ε, then everything in FOLLOW(A) is in FOLLOW(B).



Example

FIRST FOLLOW

E→ T E′ ( id ) $

E′ → + T E′ | ε + ε ) $

T → F T′ ( id + ) $

T′ → ∗ F T′ | ε ∗ ε + ) $

F → ( E ) | id ( id ∗ + ) $



LL(1)—Left (to right) Leftmost (derivation) with 1 (input symbol)

Three requirements for productions A→ α | β:1 FIRST(α) and FIRST(β) are disjoint (including for ε).2 If ε ∈ FIRST(α) then FIRST(β) and FOLLOW(A) are disjoint.3 If ε ∈ FIRST(β) then FIRST(α) and FOLLOW(A) are disjoint.



Example (why is left recursion a problem?)

FIRST FOLLOW

E→ E + T | T ( id + ) $

T → T ∗ F | F ( id + ∗ ) $

F → ( E ) | id ( id + ∗ ) $

We cannot decide how to parse from the first symbol!



Example (why is left recursion a problem?)

FIRST FOLLOW

E→ E + T | T ( id + ) $

T → T ∗ F | F ( id + ∗ ) $

F → ( E ) | id ( id + ∗ ) $

We cannot decide how to parse from the first symbol!



Predictive Parsing Table

Input: LL(1) grammar G.Output: Parsing table M.Method: For each production A→ α in the grammar:

1 For each a ∈ FIRST(α) add A→ α to M[A, a].2 If ε ∈ FIRST(α) then for each b ∈ FOLLOW(A) add

A→ α to M[A, b]. (If ε ∈ FIRST(α) and$ ∈ FOLLOW(A) then add A→ α to M[A, $].)

Any entries M[A, a] that have no content are set toerror.



Parser Table

NON-TERMINALINPUT SYMBOL

id + ∗ ( ) $E E→ TE′ E→ TE′

E′ E′ → +TE′ E′ → ε E′ → εT T → FT′ T → FT′

T′ T′ → ε T′ → ∗FT′ T′ → ε T′ → εF F → id F → (E)

INPUT STACK OUTPUT



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id + id ∗ id$ E $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id + id ∗ id$ E $id + id ∗ id$ T E′ $ E→ TE′



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT


id + id ∗ id$ F T′ E′ $ T → FT′



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id + id ∗ id$ T E′ $ E→ TE′

id + id ∗ id$ F T′ E′ $ T → FT′

id + id ∗ id$ id T′ E′ $ F → id



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id + id ∗ id$ F T′ E′ $ T → FT′

id + id ∗ id$ id T′ E′ $ F → id+ id ∗ id$ T′ E′ $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id + id ∗ id$ id T′ E′ $ F → id+ id ∗ id$ T′ E′ $+ id ∗ id$ T′ E′ $ T′ → ε



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

+ id ∗ id$ T′ E′ $+ id ∗ id$ T′ E′ $ T′ → ε+ id ∗ id$ E′ $ E′ → +TE′



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

+ id ∗ id$ T′ E′ $ T′ → ε+ id ∗ id$ E′ $ E′ → +TE′

+ id ∗ id$ +T E′ $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

+ id ∗ id$ E′ $ E′ → +TE′

+ id ∗ id$ +T E′ $id ∗ id$ T E′ $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

+ id ∗ id$ +T E′ $id ∗ id$ T E′ $id ∗ id$ F T′ E′ $ T → FT′



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id ∗ id$ T E′ $id ∗ id$ F T′ E′ $ T → FT′

id ∗ id$ id T′ E′ $ F → id



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id ∗ id$ F T′ E′ $ T → FT′

id ∗ id$ id T′ E′ $ F → id∗ id$ T′ E′ $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id ∗ id$ id T′ E′ $ F → id∗ id$ T′ E′ $∗ id$ ∗ F T′ E′ $ T′ → ∗FT′



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

∗ id$ T′ E′ $∗ id$ ∗ F T′ E′ $ T′ → ∗FT′

id$ F T′ E′ $



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

∗ id$ ∗ F T′ E′ $ T′ → ∗FT′

id$ F T′ E′ $id$ id T′ E′ $ F → id



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id$ F T′ E′ $id$ id T′ E′ $ F → id$ T′ E′ $ T′ → ε



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

id$ id T′ E′ $ F → id$ T′ E′ $ T′ → ε$ E′ $ E′ → ε



Parser Table


id + ∗ ( ) $E E→ TE′ E→ TE′



INPUT STACK OUTPUT

$ T′ E′ $ T′ → ε$ E′ $ E′ → ε$ $



Example: Top-Down Predictive Parsing

INPUT STACK OUTPUT


id + id ∗ id$ F T′ E′ $ T → FT′

id + id ∗ id$ id T′ E′ $ F → id+ id ∗ id$ T′ E′ $+ id ∗ id$ T′ E′ $ T′ → ε+ id ∗ id$ E′ $ E′ → +TE′

+ id ∗ id$ + T E′ $id ∗ id$ T E′ $id ∗ id$ F T′ E′ $ T → FT′


id$ F T′ E′ $id$ id T′ E′ $ F → id$ T′ E′ $ T′ → ε$ E′ $ E′ → ε$ $

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F

id

T′

ε

E′

ε



Example: Top-Down Predictive Parsing

INPUT STACK OUTPUT


id + id ∗ id$ F T′ E′ $ T → FT′

id + id ∗ id$ id T′ E′ $ F → id+ id ∗ id$ T′ E′ $+ id ∗ id$ T′ E′ $ T′ → ε+ id ∗ id$ E′ $ E′ → +TE′

+ id ∗ id$ + T E′ $id ∗ id$ T E′ $id ∗ id$ F T′ E′ $ T → FT′


id$ F T′ E′ $id$ id T′ E′ $ F → id$ T′ E′ $ T′ → ε$ E′ $ E′ → ε$ $

E

T

F

id

T′

ε

E′

+ T

F

id

T′

∗ F

id

T′

ε

E′

ε



Errors

I Lexical.I Syntactic.I Semantic.I Logical.

Detect early? Recover? Avoid repetition?I Panic! (for a while, “skip to ;”)I Phrase-level.I Error Productions.I Global Correction.



Errors





Errors





Errors





Errors





1 Context


3 Parsers






HACS Encoding

E→ E + T | TT → T ∗ F | F (4.1)F → ( E ) | id

sort E | J〈E〉 + 〈T〉K | J〈T〉K ;sort T | J〈T〉 ∗ 〈F〉K | J〈F〉K ;sort F | J ( 〈E〉 ) K | J〈Id〉K ;



HACS Encoding

E→ E + T | TT → T ∗ F | F (4.1)F → ( E ) | id




HACS Encoding Details


I Every choice (= production) introduced by “ |”I Explicit “JK” for concrete syntaxI Explicit “〈〉” for nonterminal and terminal referencesI sort means kind of syntax tree nodeI Direct left recursion is permitted



HACS Encoding Details


I Every choice (= production) introduced by “ |”I Explicit “JK” for concrete syntaxI Explicit “〈〉” for nonterminal and terminal referencesI sort means kind of syntax tree nodeI Direct left recursion is permitted



HACS Encoding with Precedence & Associativity


sort E | J〈E@1〉 + 〈E@2〉K@1| J〈E@2〉 ∗ 〈E@3〉K@2| sugar J(〈E#1@1〉)K@3 →E#1 | J〈Id〉K@3 ;



HACS Encoding with Precedence & Associativity





HACS Encoding with Precedence & Associativity Details


I Single sort captures abstract syntax tree (AST)I Precedence marker "@"n for precedence and associativity.I sugar specifies concrete-only syntax.



HACS Encoding with Precedence & Associativity Details


I Single sort captures abstract syntax tree (AST)I Precedence marker "@"n for precedence and associativity.I sugar specifies concrete-only syntax.



HACS Expression Parser

1 sort Stat | J 〈Name〉 := 〈Exp〉 ; K| J{ 〈Stat∗〉 } K;23 sort Exp | J 〈Exp@1〉 + 〈Exp@2〉 K@14 | J 〈Exp@2〉 ∗ 〈Exp@3〉 K@25 | J 〈Int〉 K@36 | J 〈Float〉 K@37 | J 〈Name〉 K@38 | sugar J(〈Exp#〉) K@3 →Exp# ;9

10 sort Name | symbol J〈Id〉 K;



HACS Summary

I Unusual brackets JK〈〉. . .I Automatically does immediate left recursion elimination.I sort covers multiple non-terminals.I Precedence & associativity handled automatically (the @

markers).I sugar.I (symbol for names – that’s next week.)

Oh, and it is just a front-end for JavaCC and the CRSX rewriteengine.



HACS Summary

I Unusual brackets JK〈〉. . .I Automatically does immediate left recursion elimination.I sort covers multiple non-terminals.I Precedence & associativity handled automatically (the @

markers).I sugar.I (symbol for names – that’s next week.)

Oh, and it is just a front-end for JavaCC and the CRSX rewriteengine.



Project Milestone 1

Project Milestone 1 Released!Due 10/6 (Monday) 8am



Questions?

[email protected] [email protected]


Date post:	15-Jul-2020
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

3. Top-Down Syntax AnalysisContextSyntax DeﬁnitionsParsersContext-Free GrammarsTop-Down...

Documents