Lecture 05 syntax analysis 2
Download as PPT, PDF5 likes1,688 views
The document discusses various topics related to syntax analysis and parsing, including ambiguous grammars, elimination of ambiguity, resolving problems with ambiguous and left recursive grammars, top-down parsing using recursive descent, predictive parsing, recursive predictive parsing, the FIRST and FOLLOW functions, and an example of computing FIRST and FOLLOW sets.
Lecture 05 syntax analysis 2
- 3. AMBIGUOUS GRAMMAR More than one Parse Tree for some sentence. The grammar for a programming language may be ambiguous Need to modify it for parsing. Also: Grammar may be left recursive. Need to modify it for parsing. 3
- 4. ELIMINATION OF AMBIGUITY Ambiguous A Grammar is ambiguous if there are multiple parse trees for the same sentence. Disambiguation Express Preference for one parse tree over others Add disambiguating rule into the grammar 4
- 5. RESOLVING PROBLEMS: AMBIGUOUS GRAMMARS Consider the following grammar segment: stmt → if expr then stmt | if expr then stmt else stmt | other (any other statement) If E1 then S1 else if E2 then S2 else S3 simple parse tree: stmt stmt stmtexpr exprE1 E2 S3 S1 S2 then then else else if if stmt stmt 5
- 6. EXAMPLE : WHAT HAPPENS WITH THIS STRING? If E1 then if E2 then S1 else S2 How is this parsed ? if E1 then if E2 then S1 else S2 if E1 then if E2 then S1 else S2 vs. 6
- 7. PARSE TREES: IF E1 THEN IF E2 THEN S1 ELSE S2 Form 1: stmt stmt stmtexpr E1 S2 then elseif expr E2 S1 thenif stmt stmt expr E1 thenif stmt expr E2 S2S1 then else if stmt stmt Form 2: 7
- 8. REMOVING AMBIGUITY Take Original Grammar: stmt → if expr then stmt | if expr then stmt else stmt | other (any other statement) Revise to remove ambiguity: stmt → matched_stmt | unmatched_stmt matched_stmt → if expr then matched_stmt else matched_stmt | other unmatched_stmt → if expr then stmt | if expr then matched_stmt else unmatched_stmt Rule: Match each else with the closest previous unmatched then. 8
- 9. RESOLVING DIFFICULTIES : LEFT RECURSION A left recursive grammar has rules that support the derivation : A ⇒ Aα, for some α.+ Top-Down parsing can’t reconcile this type of grammar, since it could consistently make choice which wouldn’t allow termination. A ⇒ Aα ⇒ Aαα ⇒ Aααα … etc. A→ Aα | β Take left recursive grammar: A → Aα | β To the following: A → βA’ A’ → αA’ | ∈ 9
- 10. WHY IS LEFT RECURSION A PROBLEM ? Consider: E → E + T | T T → T * F | F F → ( E ) | id Derive : id + id + id E ⇒ E + T ⇒ How can left recursion be removed ? E → E + T | T What does this generate? E ⇒ E + T ⇒ T + T E ⇒ E + T ⇒ E + T + T ⇒ T + T + T … How does this build strings ? What does each string have to start with ? 10
- 11. RESOLVING DIFFICULTIES : LEFT RECURSION (2) Informal Discussion: Take all productions for A and order as: A → Aα1 | Aα2 | … | Aαm | β1 | β2 | … | βn Where no βi begins with A. Now apply concepts of previous slide: A → β1A’ | β2A’ | … | βnA’ A’ → α1A’ | α2A’| … | αm A’ | ∈ For our example: E → E + T | T T → T * F | F F → ( E ) | id E → TE’ E’ → + TE’ | ∈ T → FT’ T’ → * FT’ | ∈ F → ( E ) | id 11
- 12. RESOLVING DIFFICULTIES : LEFT RECURSION (3) Problem: If left recursion is two-or-more levels deep, this isn’t enough S → Aa | b A → Ac | Sd | ∈ S ⇒ Aa ⇒ Sda Algorithm: Input: Grammar G with ordered Non-Terminals A1, ..., An Output: An equivalent grammar with no left recursion 1. Arrange the non-terminals in some order A1=start NT,A2,…An 2. for i := 1 to n do begin for j := 1 to i – 1 do begin replace each production of the form Ai → Ajγ by the productions Ai → δ1γ | δ2γ | … | δkγ where Aj → δ1|δ2|…|δk are all current Aj productions; end eliminate the immediate left recursion among Ai productions 12
- 13. USING THE ALGORITHM Apply the algorithm to: A1 → A2a | b| ∈ A2 → A2c | A1d i = 1 For A1 there is no left recursion i = 2 for j=1 to 1 do Take productions: A2 → A1γ and replace with A2 → δ1 γ | δ2 γ | … | δk γ| where A1→ δ1 | δ2 | … | δk are A1 productions in our case A2 → A1d becomes A2 → A2ad | bd | dWhat’s left: A1→ A2a | b | ∈ A2 → A2 c | A2 ad | bd | d Are we done ? 13
- 14. USING THE ALGORITHM (2) No ! We must still remove A2 left recursion ! A1→ A2a | b | ∈ A2 → A2 c | A2 ad | bd | d Recall: A → Aα1 | Aα2 | … | Aαm | β1 | β2 | … | βn A → β1A’ | β2A’ | … | βnA’ A’ → α1A’ | α2A’| … | αm A’ | ∈ Apply to above case. What do you get ? A1→ A2a | b | ∈ A2 → bdA2’ | dA2’ A2’ → c A2’ | adA2’ | ∈ 14
- 15. REMOVING DIFFICULTIES : ∈- MOVES Transformation: In order to remove A→ ∈ find all rules of the form B→ uAv and add the rule B→ uv to the grammar G. Why does this work ? E → TE’ E’ → + TE’ | ∈T → FT’ T’ → * FT’ | ∈ F → ( E ) | id Examples: A1 → A2 a | b A2 → bd A2’ | A2’ A2’ → c A2’ | bd A2’ | ∈ A is Grammar ∈-free if: 1. It has no ∈-production or 2. There is exactly one ∈-production S → ∈ and then the start symbol S does not appear on the right side of any production. 15
- 16. REMOVING DIFFICULTIES : CYCLES How would cycles be removed ? Make sure every production is adding some terminal(s) (except a single ∈ -production in the start NT)… e.g. S → SS | ( S ) | ∈ Has a cycle: S ⇒ SS ⇒ S S → ∈ Transform to: S → S ( S ) | ( S ) | ∈ 16
- 17. REMOVING DIFFICULTIES : LEFT FACTORING Problem : Uncertain which of 2 rules to choose: stmt → if expr then stmt else stmt | if expr then stmt When do you know which one is valid ? What’s the general form of stmt ? A → αβ1 | αβ2 α : if expr then stmt β1: else stmt β2 : ∈ Transform to: A → α A’ A’ → β1 | β2 EXAMPLE: stmt → if expr then stmt rest rest → else stmt | ∈ 17
- 18. TOP DOWN PARSING Find a left-most derivation Find (build) a parse tree Start building from the root and work down… As we search for a derivation Must make choices: Which rule to use Where to use it May run into problems!! 18
- 19. TOP-DOWN PARSING Recursive-Descent Parsing Backtracking is needed (If a choice of a production rule does not work, we backtrack to try other alternatives.) It is a general parsing technique, but not widely used. Not efficient Predictive Parsing no backtracking efficient needs a special form of grammars (LL(1) grammars). Recursive Predictive Parsing is a special form of Recursive Descent parsing without backtracking. Non-Recursive (Table Driven) Predictive Parser is also known as LL(1) parser. 19
- 36. RECURSIVE DESCENT PARSING (BACKTRACKING) Successfully parsed!! 36
- 37. RECURSIVE-DESCENT PARSING ALGORITHM A recursive-descent parsing program consists of a set of procedures – one for each non-terminal Execution begins with the procedure for the start symbol Announces success if the procedure body scans the entire input void A(){ for (j=1 to t){ /* assume there is t number of A-productions */ Choose a A-production, AjX1X2…Xk; for (i=1 to k){ if (Xi is a non-terminal) call procedure Xi(); else if (Xi equals the current input symbol a) advance the input to the next symbol; else backtrack in input and reset the pointer } } } 37
- 38. PREDICTIVE PARSER When re-writing a non-terminal in a derivation step, a predictive parser can uniquely choose a production rule by just looking the current symbol in the input string. A → α1 | ... | αn input: ... a ....... current token 38
- 39. PREDICTIVE PARSER (EXAMPLE) stmt → if ...... | while ...... | begin ...... | for ..... When we are trying to write the non-terminal stmt, if the current token is if we have to choose first production rule. When we are trying to write the non-terminal stmt, we can uniquely choose the production rule by just looking the current token. We eliminate the left recursion in the grammar, and left factor it. But it may not be suitable for predictive parsing (not LL(1) grammar). 39
- 40. RECURSIVE PREDICTIVE PARSING Each non-terminal corresponds to a procedure. Ex: A → aBb (This is only the production rule for A) proc A { - match the current token with a, and move to the next token; - call ‘B’; - match the current token with b, and move to the next token; } 40
- 41. RECURSIVE PREDICTIVE PARSING (CONT.) A → aBb | bAB proc A { case of the current token { ‘a’: - match the current token with a, and move to the next token; - call ‘B’; - match the current token with b, and move to the next token; ‘b’: - match the current token with b, and move to the next token; - call ‘A’; - call ‘B’; } } 41
- 42. RECURSIVE PREDICTIVE PARSING (CONT.) When to apply ε-productions. A → aA | bB | ε If all other productions fail, we should apply an ε- production. For example, if the current token is not a or b, we may apply the ε-production. Most correct choice: We should apply an ε- production for a non-terminal A when the current token is in the follow set of A (which terminals can follow A in the sentential forms). 42
- 43. RECURSIVE PREDICTIVE PARSING (EXAMPLE) A → aBe | cBd | C B → bB | ε C → f proc C { match the current token with f, proc A { and move to the next token; } case of the current token { a: - match the current token with a, and move to the next token; proc B { - call B; case of the current token { - match the current token with e, b: - match the current token with b, and move to the next token; and move to the next token; c: - match the current token with c, - call B and move to the next token; e,d: do nothing - call B; } - match the current token with d, } and move to the next token; f: - call C } } follow set of B first set of C 43
- 44. FIRST FUNCTION 44 • FIRST (E) = • FIRST (E’) = • FIRST (T) = • FIRST (T’) = • FIRST (F) =
- 47. FIRST - EXAMPLE P i | c | n T S Q P | a S | b S c S T R b | ε S c | R n | ε T R S q FIRST(P) = FIRST(Q) = FIRST(R) = FIRST(S) = FIRST(T) = 47
- 48. FIRST - EXAMPLE S a S e | S T S T R S e | Q R r S r | ε Q S T | ε FIRST(S) = FIRST(R) = FIRST(T) = FIRST(Q) = 48
- 49. FOLLOW SETS FOLLOW(A) is the set of terminals (including end marker of input - $) that may follow non- terminal A in some sentential form. FOLLOW(A) = {c | S ⇒+ …Ac…} ∪ {$} if S ⇒+ … A For example, consider L ⇒+ (())(L)L Both ‘)’ and end of file can follow L NOTE: ε is never in FOLLOW sets 49
- 50. COMPUTING FOLLOW(A) 1. If A is start symbol, put $ in FOLLOW(A) 2. Productions of the form B α A β, Add FIRST(β) – {ε} to FOLLOW(A) 3. Productions of the form B α A or B α A β where β ⇒* ε Add FOLLOW(B) to FOLLOW(A) 50
- 51. EXAMPLE E T E′ E′ + T E′ | ε T F T′ T′ * F T′ | ε F ( E ) | id FIRST(E) = {(, id} FIRST(E′) = {+, ε} FIRST(T) = {(, id} FIRST(T′) = {*, ε} FIRST(F) = {(, id}} FOLLOW(E) = {$} FOLLOW(E′) = FOLLOW(T) = FOLLOW(T′) = FOLLOW(F) = Assume the first non-terminal is the start symbol Using rule #1 1. If A is start symbol, put $ in FOLLOW(A) 51
- 52. EXAMPLE E T E′ E′ + T E′ | ε T F T′ T′ * F T′ | ε F ( E ) | id FIRST(E) = {(, id} FIRST(E′) = {+, ε} FIRST(T) = {(, id} FIRST(T′) = {*, ε} FIRST(F) = {(, id}} FOLLOW(E) = {$, )} FOLLOW(E′) = FOLLOW(T) = {+} FOLLOW(T′) = FOLLOW(F) = {*} Using rule #2 2. Productions of the form B α A β, Add FIRST(β) – {ε} to FOLLOW(A) 52
- 53. EXAMPLE E T E′ E′ + T E′ | ε T F T′ T′ * F T′ | ε F ( E ) | id FIRST(E) = {(, id} FIRST(E′) = {+, ε} FIRST(T) = {(, id} FIRST(T′) = {*, ε} FIRST(F) = {(, id}} FOLLOW(E) = {$, )} FOLLOW(E′) = FOLLOW(E) = {$, )} FOLLOW(T) = {+} ∪ FOLLOW(E′) = {+, $, )} FOLLOW(T′) = FOLLOW(T) = {+, $, )} FOLLOW(F) = {*} ∪ FOLLOW(T′) = {*, +, $, )} Using rule #3 3. Productions of the form B α A or B α A β where β ⇒* ε Add FOLLOW(B) to FOLLOW(A) 53
- 54. EXAMPLE S ( A) | ε A T E E & T E | ε T ( A ) | a | b | c FIRST(T) = FIRST(E) = FIRST(A) = FIRST(S) = FOLLOW(S) = FOLLOW(A) = FOLLOW(E) = FOLLOW(T) = 54
- 55. EXAMPLE S ( A) | ε A T E E & T E | ε T ( A ) | a | b | c FIRST(T) = {(,a,b,c} FIRST(E) = {&, ε } FIRST(A) = {(,a,b,c} FIRST(S) = {(, ε} FOLLOW(S) = {$} FOLLOW(A) = { ) } FOLLOW(E) = FOLLOW(A) = { ) } FOLLOW(T) = FIRST(E) ∪ FOLLOW(E) = {&, )} 55
- 56. EXAMPLE S a S e | B B b B C f | C C c C g | d | ε FIRST(C) = FIRST(B) = FIRST(S) = FOLLOW(C) = FOLLOW(B) = FOLLOW(S) = {$} Assume the first non-terminal is the start symbol 1. If A is start symbol, put $ in FOLLOW(A) 56 2. Productions of the form B α A β, Add FIRST(β) – {ε} to FOLLOW(A) 3. Productions of the form B α A or B α A β where β ⇒* ε Add FOLLOW(B) to FOLLOW(A)
- 57. EXAMPLE S a S e | B B b B C f | C C c C g | d | ε FIRST(C) = {c,d,ε} FIRST(B) = {b,c,d,ε} FIRST(S) = {a,b,c,d,ε} FOLLOW(C) = FOLLOW(B) = FOLLOW(S) = { }$, e {c,d} ∪ FOLLOW(S) = {c,d,e,$} {f,g} ∪ FOLLOW(B) = {c,d,e,f,g,$} 57
- 58. QUESTIONS ? 58