An Introduction to ANTLR
Download as PPTX, PDF4 likes2,314 views
This document provides an introduction to ANTLR, including: - What ANTLR is and how it works by taking a grammar specification and generating source code for a parser - A brief history of ANTLR and examples of its common uses - An overview of new features in ANTLR v4 like supporting all grammars and generating listener and visitor classes - Steps for getting started with ANTLR and examples of simple grammars and language applications built with ANTLR
An Introduction to ANTLR
- 1. Introduction to By: Morteza Zakeri Iran University of Science and Technology Fall 2016
- 2. Contents at a Glance I • What is ANTLR? • LL Grammars • History • Motivation • What is New in ANTLR v4? • ANTLR Components: How it Works? • Getting Start with ANTLR v4 9 February 2017 Introduction to ANTLR - M.Zakeri Page 2 of 31
- 3. Contents at a Glance II • ANTLR Hello World! • Integrating a Generated Parser into a Java Program • A Starter ANTLR Project: ArrayInit • Building a Language Application • Customizing BaseListener Methods • Congratulation! • References 9 February 2017 Introduction to ANTLR - M.Zakeri Page 3 of 31
- 4. What is ANTLR? • ANTLR (pronounced Antler), or Another Tool For Language Recognition, is a parser generator that uses LL(*) for parsing. • ANTLR takes as input a grammar that specifies a language and generates as output source code for a recognizer for that language. • supported generating code in Java, C#, JavaScript, Python2 and Python3. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 4 of 31
- 5. LL(K) Grammars • An LL parser is a top-down parser for a subset of context-free languages. • It parses the input from Left to right, performing Leftmost derivation of the sentence. • An LL parser is called an LL(k) parser if it uses k tokens of look-ahead when parsing a sentence. • The LL(K) parser is a deterministic pushdown automaton with the ability to peek on the next k input symbols without reading. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 5 of 31
- 6. LL(*) Grammars • An LL parser is called an LL(*) parser (an LL-regular parser) if it is not restricted to a finite k tokens of look- ahead, but can make parsing decisions by recognizing whether the following tokens belong to a regular language. • LL (LL(1), LL(k), LL(*)) grammars can be parsed by recursive descent parsers. • In fact ANTLR is recursive descent parser Generator! 9 February 2017 Introduction to ANTLR - M.Zakeri Page 6 of 31
- 7. History • Initial release: • February 1992; 24 years ago. • Stable release: • 4.5.1 / July 15, 2015; 14 months ago • Its maintainer is: • Professor Terence Parr • University of San Francisco. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 7 of 31
- 8. Motivation • It’s widely used in academia and industry to build all sorts of languages, tools, and frameworks. • Twitter search uses ANTLR for query parsing, with more than 2 billion queries a day. • Oracle uses ANTLR within the SQL Developer IDE and its migration tools. • The NetBeans IDE parses C++ with ANTLR. • The HQL language in the Hibernate object-relational mapping framework is built with ANTLR. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 8 of 31
- 9. What is New in ANTLR v4? I • The most important new feature is: • ANTLR v4 gladly accepts every grammar you give it! • with one exception regarding indirect left recursion, i.e. grammars rules x which refer to y which refer to x. • ANTLR v4 automatically rewrites left-recursive rules such as expr into non left-recursive equivalents. • The only constraint is that the left recursion must be direct, where rules immediately reference themselves. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 9 of 31
- 10. What is New in ANTLR v4? II • ANTLR v4 dramatically simplifies the grammar rules used to match syntactic structures. • like programming language arithmetic expressions. • ANTLR v4 also automatically generates parse-tree walkers in the form of listener and visitor pattern implementations. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 10 of 31
- 11. What is New in ANTLR v4? III • ANTLR v4 de-emphasizes embedding actions (code) in the grammar, favoring listeners and visitors instead. • Listeners and visitors are the familiar design patterns. • ANTLR parsers use a new parsing technology called Adaptive LL(*) or ALL(*) (“all star”). • ANTLR v3’s LL(*) parsing strategy is weaker than v4’s ALL(*). 9 February 2017 Introduction to ANTLR - M.Zakeri Page 11 of 31
- 12. ANTLR Components: How it Works? 9 February 2017 Introduction to ANTLR - M.Zakeri ANTLR Grammar (*.g4) ANTLR jar file ANTLR Tool ANTLR runtime (parse-time) API *.class Input Text (.txt) Java Compiler The Result Our Compiler Generated Code *.java & *.token Page 12 of 31
- 13. Getting Start with ANTLR v4: Linux 9 February 2017 Introduction to ANTLR - M.Zakeri Page 13 of 31
- 14. Getting Start with ANTLR v4: Windows 9 February 2017 Introduction to ANTLR - M.Zakeri Page 14 of 31
- 15. ANTLR Hello World! 9 February 2017 Introduction to ANTLR - M.Zakeri Page 15 of 31
- 16. ANTLR Hello World! 9 February 2017 Introduction to ANTLR - M.Zakeri Page 16 of 31
- 17. Do all with my own bat file! 9 February 2017 Introduction to ANTLR - M.Zakeri Page 17 of 31
- 18. Integrating a Generated Parser into a Java Program • We can integrate the ANTLR generated code into a larger application. • We’ll see simple example in next slides for recognition structures like {1,{2,3},4} in C or JAVA. • Then we’ll look at a simple Java main() that invokes our initializer parser and prints out the parse tree like TestRig’s -tree option. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 18 of 31
- 19. A Starter ANTLR Project: ArrayInit 9 February 2017 Introduction to ANTLR - M.Zakeri Page 19 of 31
- 20. A Starter ANTLR Project: ArrayInit 9 February 2017 Introduction to ANTLR - M.Zakeri Page 20 of 31
- 21. A Starter ANTLR Project: ArrayInit 9 February 2017 Introduction to ANTLR - M.Zakeri Page 21 of 31
- 22. A Starter ANTLR Project: ArrayInit • Here’s how to compile everything and run Test: 9 February 2017 Introduction to ANTLR - M.Zakeri Page 22 of 31
- 23. A Starter ANTLR Project: ArrayInit • ANTLR parsers also automatically report and recover from syntax errors. • For example, here’s what happens if we enter an initializer that’s missing the final curly brace: 9 February 2017 Introduction to ANTLR - M.Zakeri Page 23 of 31
- 24. Building a Language Application I • An application that merely checks syntax is not that impressive! • Continuing with our array initializer example, our next goal is to translate not just recognize initializers. • For example, let’s translate Java short arrays like {99,3,451} to "u0063u0003u01c3" where 63 is the hexadecimal representation of the 99 decimal. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 24 of 31
- 25. Building a Language Application II • To move beyond recognition, an application has to extract data from the parse tree. • ANTLR automatically generates a listener infrastructure for us. • These listeners are like the callbacks on GUI widgets (for example, a button would notify us upon a button press) or like SAX events in an XML parser. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 25 of 31
- 26. Building a Language Application III • To write a program that reacts to the input, all we have to do is implement a few methods in a subclass of ArrayInitBaseListener. • The basic strategy is to have each listener method print out a translated piece of the input when called to do so by the tree walker. • All we know is that our listener gets notified at the beginning and end of phrases associated with rules in the grammar and we don’t even have to know that the runtime is walking a tree to call our methods. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 26 of 31
- 27. Customizing BaseListener Methods 9 February 2017 Introduction to ANTLR - M.Zakeri Page 27 of 31
- 28. Language Application Main Class 9 February 2017 Introduction to ANTLR - M.Zakeri Page 28 of 31
- 29. Run and Test! • Let’s build the translator and try it on our sample input: 9 February 2017 Introduction to ANTLR - M.Zakeri Page 29 of 31
- 30. Congratulation! • It works! We’ve just built our first translator, without even touching the grammar! • All we had to do was implement a few methods that printed the appropriate phrase translations. • Listeners effectively isolate the language application from the grammar, making the grammar reusable for other applications. 9 February 2017 Introduction to ANTLR - M.Zakeri Page 30 of 31
- 31. References 1. The Definitive ANTLR 4 Reference • Terence Parr, The Pragmatic Programmers, LLC; 2012. 2. ANTLR 4 Official Website: • http://www.antlr.org/ 3. ANTLR page on Wikipedia • https://en.wikipedia.org/wiki/ANTLR 9 February 2017 Introduction to ANTLR - M.Zakeri Page 31 of 31