SS UI Lecture 4
Download as PPTX, PDF•0 likes•250 views
This document discusses the specification and recognition of tokens in compiling. It begins by defining tokens and regular expressions, which are used to formally specify tokens. Regular expressions define patterns for strings. The document then explains how to recognize tokens by defining patterns for language elements using regular expressions. Finally, it provides examples of transition diagrams that can be used to recognize tokens like relational operators, identifiers, numbers, and whitespace.
![Extensions
One or more instances: (r)+
Zero of one instances: r?
Character classes: [abc]
Example:
letter_ -> [A-Za-z_]
digit -> [0-9]
id -> letter_(letter|digit)*](https://image.slidesharecdn.com/f4cb320f-ea6e-4096-955e-4a70ce551544-150915075912-lva1-app6892/85/SS-UI-Lecture-4-8-320.jpg)
![Recognition of tokens (cont.)
The next step is to formalize the patterns:
digit -> [0-9]
Digits -> digit+
number -> digit(.digits)? (E[+-]? Digit)?
letter -> [A-Za-z_]
id -> letter (letter|digit)*
If -> if
Then -> then
Else -> else
Relop -> < | > | <= | >= | = | <>
We also need to handle whitespaces:
ws -> (blank | tab | newline)+](https://image.slidesharecdn.com/f4cb320f-ea6e-4096-955e-4a70ce551544-150915075912-lva1-app6892/85/SS-UI-Lecture-4-10-320.jpg)
SS UI Lecture 4
- 1. System Software (5KS03) Unit 1 : Introduction to Compiling Lecture : 4 Specification of Tokens A S Kapse, Assistant Professor, Department Of Computer Sci. & Engineering Anuradha Engineering College, Chikhli
- 2. Contents… Specification of tokens Recognition of tokens
- 3. Objectives… Upon completion of this lecture, you will be able To understand the use of tokens To understand Role of lexical analyses to recognition of tokens To understand regular expression To use of transition diagram
- 4. Review…./ Concepts What do you mean by tokens? What do you mean by parser and scanner? What do you mean by regular expression?
- 5. Specification of tokens In theory of compilation regular expressions are used to formalize the specification of tokens Regular expressions are means for specifying regular languages Example: Letter_(letter_ | digit)* Each regular expression is a pattern specifying the form of strings
- 6. Regular expressions Ɛ is a regular expression, L(Ɛ) = {Ɛ} If a is a symbol in ∑then a is a regular expression, L(a) = {a} (r) | (s) is a regular expression denoting the language L(r) ∪ L(s) (r)(s) is a regular expression denoting the language L(r)L(s) (r)* is a regular expression denoting (L9r))* (r) is a regular expression denting L(r)
- 7. Regular definitions d1 -> r1 d2 -> r2 … dn -> rn Example: letter_ -> A | B | … | Z | a | b | … | Z | _ digit -> 0 | 1 | … | 9 id -> letter_ (letter_ | digit)*
- 8. Extensions One or more instances: (r)+ Zero of one instances: r? Character classes: [abc] Example: letter_ -> [A-Za-z_] digit -> [0-9] id -> letter_(letter|digit)*
- 9. Recognition of tokens Starting point is the language grammar to understand the tokens: stmt -> if expr then stmt | if expr then stmt else stmt | Ɛ expr -> term relop term | term term -> id | number
- 10. Recognition of tokens (cont.) The next step is to formalize the patterns: digit -> [0-9] Digits -> digit+ number -> digit(.digits)? (E[+-]? Digit)? letter -> [A-Za-z_] id -> letter (letter|digit)* If -> if Then -> then Else -> else Relop -> < | > | <= | >= | = | <> We also need to handle whitespaces: ws -> (blank | tab | newline)+
- 11. Transition diagrams Transition diagram for relop
- 12. Transition diagrams (cont.) Transition diagram for reserved words and identifiers
- 13. Transition diagrams (cont.) Transition diagram for unsigned numbers
- 14. Transition diagrams (cont.) Transition diagram for whitespace
- 15. Video on Compilers 1. Lexical Analysis : The role of lexical analyzer 2. Input buffering
- 16. Questions.. 1. Define tokens? 2. What do you mean by regular expression 3. Explain the process of regular expression. 4. What is mean by transition diagram?
- 17. Homework.. 1. What is parser? 2What is mean by analysis and synthesis. 3. Describe the following example. area=pi * r * r + 45