Compiler Construction | Lecture 5 | Transformation by Term Rewriting

Eelco Visser
CS4200 Compiler Construction
TU Delft
September 2018
Lecture 5: Transformation by Term Rewriting

No deadlines
- There are no deadlines for homework

- I have removed the ‘advisory’ deadlines

- Make at your own pace; but important to prepare for exam

Reading material on parsing
- Read sections 4.1, 4.2, 4.3, 4.5, 4.6 of ‘Dragon’ book

- In particular: FIRST and FOLLOW set, constructing SLR table

New assignments
- Week 1.2: templates, completion

- Week 1.3: LR parsing

Slides on parsing extended with more examples
!2
Notes on Homework

Would you be interested in
peer review of homework
assignments?
3

Channel on research group Slack
Where many Spoofax users hang out
Ask questions about Spoofax
(Not answers to assignments)
No guarantee to quick response, or answer at all
Send me an email if you want an invitation
!4
#spoofax-users

This Lecture
!5
Source
Code
Editor
Parse
Abstract
Syntax
Tree
Transform
Abstract
Syntax
Tree
Deﬁne transformations on abstract syntax trees (terms) using rewrite rules

Reading Material
6
The following papers add background, conceptual exposition,
and examples to the material from the slides. Some notation and
technical details have been changed; check the documentation.

!7
Term rewrite rules deﬁne transformations on (abstract syntax)
trees. Traditional rewrite systems apply rules exhaustively. This
paper introduces programmable rewriting strategies to control
the application of rules, the core of the design of the Stratego
transformation language.

Note that the notation for contextual rules is no longer
supported by Stratego. However, the technique to implement
contextual rules still applies.
https://doi.org/10.1145/291251.289425
ICFP 1998

!8
Stratego/XT combines SDF2 and Stratego into toolset for
program transformation.

This paper gives a high-level overview of the concepts.

The StrategoXT.jar is still part of the Spoofax distribution.
https://doi.org/10.1007/978-3-540-25935-0_13
Lecture Notes in Computer Science 2003

!9
Spoofax combines SDF2 and Stratego into a language
workbench, i.e. an IDE for creating language definition from
which IDEs for the defined languages can be generated.

A distinctive feature of Spoofax is live language development,
which supports developing a language definition and programs
in the defined language in the same IDE instance.

Spoofax was developed for Eclipse, which is still the main
development platform. However, Spoofax Core is now
independent of any IDE.
https://doi.org/10.1145/1932682.1869497
OOPSLA 2010
Note that the since the publication of this paper, we have
introduced more declarative approaches to name and type
analysis, which will be the topic of the next lectures.

!10
Documentation for Stratego at
the metaborg.org website.
http://www.metaborg.org/en/latest/source/langdev/meta/lang/stratego/index.html

!11
This paper deﬁnes the formal semantics of the full Stratego
language, including its scoped dynamic rules feature.
If you want to dig deeper
These slides document most features of the base language
without dynamic rules and without giving the formal
semantics.

Transformation Architecture
12
This presentation uses the Stratego Shell for explaining the behavior of
Stratego programs. The Stratego Shell is currently not supported by Spoofax

Parsing: From Text to Terms
!23
Let(
[ FunDec(
"fact"
, [FArg("n", Tp(Tid("int")))]
, Tp(Tid("int"))
, If(
Lt(Var("n"), Int("1"))
, Int("1")
, Seq(
[ Times(
Var("n")
, Call(
Var("fact")
, [Minus(Var("n"), Int("1"))]
)
)
]
)
)
)
]
, [Call(Var("fact"), [Int("10")])]
)
let function fact(n : int) : int =
if n < 1 then 1 else (n * fact(n - 1))
in fact(10)
end

Syntax of Terms
!24
module Terms
sorts Cons Term
lexical syntax
Cons = [a-zA-Z][a-zA-Z0-9]*
context-free syntax
Term.App = <<Cons>(<{Term ","}*>)>
Zero()
Succ(Zero())
Cons(A(), Cons(B(), Nil()))

Syntax of Terms
!25
module Terms
sorts Cons Term
lexical syntax
context-free syntax
Term.List = <[<{Term ","}*>]>
Term.Tuple = <(<{Term ","}*>)>
Zero()
Succ(Zero())
[A(), B()]

module ATerms
sorts Cons Term
lexical syntax
Cons = String
Int = [0-9]+
String = """ StringChar* """
StringChar = ~["n]
StringChar = "" ["]
context-free syntax
Term.Str = <<String>>
Term.Int = <<Int>>
Term.List = <[<{Term ","}*>]>
Term.Tuple = <(<{Term ","}*>)>
Syntax of Terms
!26
0
1
[A(), B()]
Var(“x”)
Let(
[ Decl(“x”, IntT()),
Decl(“y”, BoolT())
]
, Eq(Var(“x”), Int(0))
)

Syntax of A(nnotated) Terms
!27
module ATerms
sorts Cons Term
lexical syntax
// more lexical syntax omitted
context-free syntax
Term.Anno = <<PreTerm>{<{Term “,"}*>}>
Term = <<PreTerm>>
PreTerm.Str = <<String>>
PreTerm.Int = <<Int>>
PreTerm.App = <<Cons>(<{Term ","}*>)>
PreTerm.List = <[<{Term ","}*>]>
PreTerm.Tuple = <(<{Term ","}*>)>
Var(“x”){Type(IntT())}

Signatures
!29
context-free syntax
Exp.Uminus = [- [Exp]]
Exp.Power = [[Exp] ** [Exp]]
Exp.Times = [[Exp] * [Exp]]
Exp.Divide = [[Exp] / [Exp]]
Exp.Plus = [[Exp] + [Exp]]
Exp.Minus = [[Exp] - [Exp]]
Exp.CPlus = [[Exp] +i [Exp]]
Exp.CMinus = [[Exp] -i [Exp]]
Exp.Eq = [[Exp] = [Exp]]
Exp.Neq = [[Exp] <> [Exp]]
Exp.Gt = [[Exp] > [Exp]]
Exp.Lt = [[Exp] < [Exp]]
Exp.Geq = [[Exp] >= [Exp]]
Exp.Leq = [[Exp] <= [Exp]]
Exp.True = <true>
Exp.False = <false>
Exp.And = [[Exp] & [Exp]]
Exp.Or = [[Exp] | [Exp]]
Signature declares argument and
return types of term constructors
Signature is automatically generated
from SDF3 productions
Stratego compiler only checks arity
of constructor applications
signature
constructors
Uminus : Exp -> Exp
Power : Exp * Exp -> Exp
Times : Exp * Exp -> Exp
Divide : Exp * Exp -> Exp
Plus : Exp * Exp -> Exp
Minus : Exp * Exp -> Exp
CPlus : Exp * Exp -> Exp
CMinus : Exp * Exp -> Exp
Eq : Exp * Exp -> Exp
Neq : Exp * Exp -> Exp
Gt : Exp * Exp -> Exp
Lt : Exp * Exp -> Exp
Geq : Exp * Exp -> Exp
Leq : Exp * Exp -> Exp
True : Exp
False : Exp
And : Exp * Exp -> Exp
Or : Exp * Exp -> Exp

Desugaring
!31
desugar: IfThen(e1, e2) -> IfThenElse(e1, e2, NoVal())
if x then
printint(x)
else
()
IfThen(
Var("x")
, Call(
"printint"
, [Var("x")]
)
)
IfThenElse(
Var("x")
, Call(
"printint"
, [Var("x")]
)
, NoVal()
)
pattern matching pattern instantiation
e1
e2
if x then
printint(x)

More Desugaring
!32
desugar: Uminus(e) -> Bop(MINUS(), Int("0"), e)
desugar: Plus(e1, e2) -> Bop(PLUS(), e1, e2)
desugar: Minus(e1, e2) -> Bop(MINUS(), e1, e2)
desugar: Times(e1, e2) -> Bop(MUL(), e1, e2)
desugar: Divide(e1, e2) -> Bop(DIV(), e1, e2)
desugar: Eq(e1, e2) -> Rop(EQ(), e1, e2)
desugar: Neq(e1, e2) -> Rop(NE(), e1, e2)
desugar: Leq(e1, e2) -> Rop(LE(), e1, e2)
desugar: Lt(e1, e2) -> Rop(LT(), e1, e2)
desugar: Geq(e1, e2) -> Rop(LE(), e2, e1)
desugar: Gt(e1, e2) -> Rop(LT(), e2, e1)
desugar: And(e1, e2) -> IfThenElse(e1, e2, Int("0"))
signature
constructors
PLUS: BinOp
MINUS: BinOp
MUL: BinOp
DIV: BinOp
EQ: RelOp
NE: RelOp
LE: RelOp
LT: RelOp
Bop: BinOp * Expr * Expr -> Expr
Rop: RelOp * Expr * Expr -> Expr

Constant Folding
!33
x := 21 + 21 + x
eval: Plus(Int(i1), Int(i2)) -> Int(i3)
where <addS> (i1, i2) => i3
x := 42 + x
Assign(
Var("x")
, Plus(
Plus(
Int("21")
, Int("21")
)
, Var("x")
)
)
Assign(
Var("x")
, Plus(
Int("42")
, Var("x")
)
)

More Constant Folding
!34
eval: Bop(PLUS(), Int(i1), Int(i2)) -> Int(<addS> (i1, i2))
eval: Bop(MINUS(), Int(i1), Int(i2)) -> Int(<subtS> (i1, i2))
eval: Bop(MUL(), Int(i1), Int(i2)) -> Int(<mulS> (i1, i2))
eval: Bop(DIV(), Int(i1), Int(i2)) -> Int(<divS> (i1, i2))
eval: Rop(EQ(), Int(i), Int(i)) -> Int("1")
eval: Rop(EQ(), Int(i1), Int(i2)) -> Int("0") where not( <eq> (i1, i2) )
eval: Rop(NE(), Int(i), Int(i)) -> Int("0")
eval: Rop(NE(), Int(i1), Int(i2)) -> Int("1") where not( <eq> (i1, i2) )
eval: Rop(LT(), Int(i1), Int(i2)) -> Int("1") where <ltS> (i1, i2)
eval: Rop(LT(), Int(i1), Int(i2)) -> Int("0") where not( <ltS> (i1, i2) )
eval: Rop(LE(), Int(i1), Int(i2)) -> Int("1") where <leqS> (i1, i2)
eval: Rop(LE(), Int(i1), Int(i2)) -> Int("0") where not( <leqS> (i1, i2))

Application: Spoofax
Builders
35

Spoofax Builders
!36
rules
build :
(node, _, ast, path, project-path) -> result
with
<some-transformation> ast => result
Builder gets AST as parameter

Tiger: AST Builder
!37
rules // Debugging
debug-show-aterm:
(node, _, _, path, project-path) -> (filename, result)
with
filename := <guarantee-extension(|"aterm")> path
; result := node
trans/tiger.str
module Syntax
menus
menu: "Syntax" (openeditor)
action: "Format" = editor-format (source)
action: "Show parsed AST" = debug-show-aterm (source)
editor/syntax.esv
Builder gets AST as parameter

Tiger: Formatting Builder
!39
module Syntax
menus
menu: "Syntax" (openeditor)
action: "Format" = editor-format (source)
action: "Show parsed AST" = debug-show-aterm (source)
rules
editor-format:
(node, _, ast, path, project-path) -> (filename, result)
with
ext := <get-extension> path
; filename := <guarantee-extension(|$[pp.[ext]])> path
; result := <pp-debug> node
editor/syntax.esv
trans/pp.str
rules
pp-Tiger-string =
parenthesize-Tiger
; prettyprint-Tiger-start-symbols
; !V([], <id>)
; box2text-string(|120)
pp-debug :
ast -> result
with
result := <pp-Tiger-string> ast
<+ …
; result := “"
trans/pp.str

Tiger: Parenthesize
!40
module pp/Tiger-parenthesize
imports
libstratego-lib
signatures/-
strategies
parenthesize-Tiger =
innermost(TigerParenthesize)
rules
TigerParenthesize :
Or(t_0, t_1) -> Or(t_0, Parenthetical(t_1))
where <(?For(_, _, _, _)
+ ?While(_, _)
+ ?IfThen(_, _)
+ ?If(_, _, _)
+ ?Assign(_, _)
+ ?Array(_, _, _)
+ ?Or(_, _)
+ fail)> t_1
TigerParenthesize :
And(t_0, t_1) -> And(Parenthetical(t_0), t_1)
where <(?For(_, _, _, _)
+ ?While(_, _)
+ ?IfThen(_, _)
+ ?If(_, _, _)
+ ?Assign(_, _)
+ ?Array(_, _, _)
+ ?Or(_, _)
+ fail)> t_0
pp/Tiger-parenthesize.str
context-free priorities
Exp.And
> Exp.Or
> Exp.Array
> Exp.Assign
> …

Tiger: Pretty-Print Rules
!41
rules
prettyprint-Tiger-Exp :
If(t1__, t2__, t3__) -> [ H(
[SOpt(HS(), "0")]
, [ S("if ")
, t1__'
, S(" then")
]
)
, t2__'
, H(
[SOpt(HS(), "0")]
, [S("else")]
)
, t3__'
]
with t1__' := <pp-one-Z(prettyprint-Tiger-Exp)
<+ pp-one-Z(prettyprint-completion-aux)> t1__
with t2__' := <pp-indent(|"2")> [
<pp-one-Z(prettyprint-Tiger-Exp)
<+ pp-one-Z(prettyprint-completion-aux)> t2__ ]
with t3__' := <pp-indent(|"2")> [
<pp-one-Z(prettyprint-Tiger-Exp)
<+ pp-one-Z(prettyprint-completion-aux)> t3__ ]
context-free syntax
Exp.If = <
if <Exp> then
<Exp>
else
<Exp>
>
syntax/Control-Flow.sdf3
pp/Control-Flow-pp.str

Tiger: Desugaring
!43
function printboard() = (
for i := 0 to N-1 do (
for j := 0 to N-1 do
print(if col[i]=j then " O" else " .");
print("n")
);
print("n")
)
function printboard() = (
let
var i : int := 0
in
while i < N - 1 do ( (
let
var j : int := 0
in
while j < N - 1 do (
print(if col[i] = j then
" O"
else
" .");
j := j + 1
)
end;
print("n")
);
i := i + 1
)
end;
print("n")
)
Expressing for in terms of while++

Tiger: Desugaring Builder
!44
rules // Desugaring
editor-desugar :
with
filename := <guarantee-extension(|"des.tig")> path
; result := <desugar-all; pp-Tiger-string>node
editor-desugar-ast :
with
filename := <guarantee-extension(|"aterm")> path
; result := <desugar-all>node
trans/tiger.str
module Transformation
menus
menu: "Transform" (openeditor) (realtime)
action: "Desugar" = editor-desugar (source)
action: "Desugar AST" = editor-desugar-ast (source)
editor/Transformation.esv

Tiger: Desugaring Transformation
!45
module desugar
imports signatures/Tiger-sig
imports …
strategies
desugar-all = topdown(try(desugar))
rules
desugar :
For(
Var(i)
, e1
, e2
, e_body
) ->
Let(
[VarDec(i, Tid("int"), e1)]
, [ While(
Lt(Var(i), e2)
, Seq(
[ e_body
, Assign(Var(i), Plus(Var(i), Int("1")))
]
)
)
]
)

Tiger: Outline View Builder
!48
module Syntax
views
outline view: editor-outline (source)
expand to level: 3
editor/syntax.esv
trans/outline.str
module outline
imports
signatures/Tiger-sig
signatures/Types-sig
signatures/Records-sig
signatures/Variables-sig
signatures/Functions-sig
signatures/Bindings-sig
libspoofax/editor/outline
pp
rules
editor-outline:
(_, _, ast, path, project-path) -> outline
where
outline := <simple-label-outline(to-outline-label)> ast

Tiger: Outline View Rules
!49
rules
to-outline-label :
Let(_, _) -> $[let]
to-outline-label :
TypeDec(name, _) -> $[type [name]]
to-outline-label :
FunDec(name, _, t, _) -> $[fun [name] : [<pp-tiger-type> t]]
to-outline-label :
ProcDec(name, _, _) -> $[fun [name]]
to-outline-label :
VarDecNoType(name, _) -> $[var [name]]
to-outline-label :
VarDec(name, _, _) -> $[var [name]]
trans/outline.str
rules
to-outline-label :
Call(f, _) -> $[[f]()]
to-outline-label :
If(_, _, _) -> $[if]
to-outline-label :
For(Var(name), _, _, _) -> $[for [name]]
// etc.

Outline View: Generic Strategy
!50
module libspoofax/editor/outline
imports …
signature
constructors
Node : Label * Children -> Node
rules
simple-label-outline(s1) =
collect-om(to-outline-node(s1, fail), conc)
custom-label-outline(s1, s2) =
collect-om(origin-track-forced(s2) <+ to-outline-node(s1, s2), conc)
to-outline-node(s1, s2):
term -> Node(label, children)
where
random := <next-random>;
label := <origin-track-forced(s1; term-to-outline-label)> term;
children := <get-arguments; custom-label-outline(s1, s2)> term
term-to-outline-label =
is-string
<+ ?term{a}; origin-text; ?label; !label{a}
<+ write-to-string // fallback
Language-independent
strategy for collecting
outline nodes
Term structure for outline
nodes

Tiger: Completion Rules
!52
rules
suggest-completions(|completions):
Exp-Plhdr() -> <add-completions(
| ( "If"
, If(
<try(inline-completions(|Exp-Plhdr()))> Exp-Plhdr()
, <try(inline-completions(|Exp-Plhdr()))> Exp-Plhdr()
, <try(inline-completions(|Exp-Plhdr()))> Exp-Plhdr()
)
)
)
; fail> completions
context-free syntax
Exp.If = <
if <Exp> then
<Exp>
else
<Exp>
>
syntax/Control-Flow.sdf3
completion/Control-Flow-cp.str

Rewriting =
Matching & Building
53

Combining Rewrite Rules
with Strategies
71

Parameterized
Rewrite Rules
84

f(x,y|a,b): lhs -> rhs
- strategy or rule parameters x,y

- term parameters a,b

- no matching

f(|a,b): lhs -> rhs
- optional strategy parameters

f(x,y): lhs -> rhs
- optional term parameters

f: lhs -> rhs
!85
Parameterized Rewrite Rules

Parameterized Rewrite Rules: Map
!86
[ 1
, 2
, 3
]
[ 2
, 3
, 4
]
map(inc)
inc
1
2
3
2
3
4inc
inc
map(s): [] -> []
map(s): [x|xs] -> [<s> x | <map(s)> xs]

Parameterized Rewrite Rules: Zip
!87
[ 1
, 2
, 3
]
[ (1,4)
, (2,5)
, (3,6)
]
[ 4
, 5
, 6
]
zip
[ 1
, 2
, 3
]
[ 5
, 7
, 9
]
[ 4
, 5
, 6
]
zip(add)
map(add)
zip(s): ([],[]) -> []
zip(s): ([x|xs],[y|ys]) -> [<s> (x,y) | <zip(s)> (xs,ys)]
zip = zip(id)

Parameterized Rewrite Rules: Fold
!88
[1,2,3] foldr(!0,add) 6
[]
0
[3]
3
[2,3]
56
[1,2,3]
foldr(s1,s2): [] -> <s1>
foldr(s1,s2): [x|xs] -> <s2> (x,<foldr(s1,s2)> xs)

Traversal: Topdown
!113
Const
Mul
Const
3 4 5
Const
Add1
switch: Add(e1, e2) -> Add(e2, e1)
switch: Mul(e1, e2) -> Mul(e2, e1)
topdown(s) = s ; all(topdown(s))
topdown(switch)

Traversal: Topdown
!114
Mul
Const
3
Const
4 5
Const
Add1
2
topdown(switch)

Traversal: Topdown
!115
Mul
Const
3
Const
5 4
Const
Add1
2
3
topdown(switch)

Traversal: Topdown/Try
!116
Const
Mul
Const
3 4 5
Const
Add1
topdown(try(switch))

!117
Mul
Const
3
Const
4 5
Const
Add1
2

!118
Mul
Const
3
Const
5 4
Const
Add1
2
3
4
5
6
7
8

Traversal: Alltd
!119
Const
Mul
Const
3 4 5
Const
Add1
alltd(s) = s <+ all(alltd(s))
alltd(switch)

Traversal: Alltd
!120
Mul
Const
3
Const
4 5
Const
Add1
alltd(s) = s <+ all(alltd(s))
alltd(switch)

Traversal: bottomup
!121
Const
Mul
Const
3 4 5
Const
Add1
2
3
4
bottomup(s) = all(bottomup(s)) ; s
bottomup(switch)

Traversal: Bottomup
!122
Const
Mul
Const
3 4 5
Const
Add1
2
3
bottomup(try(switch))

Traversal: Bottomup
!123
Const
Mul
Const
3 4 5
Const
Add1
2
3
4
5
6
7
8

Traversal: Bottomup
!124
Const
Mul
Const
3 4 5
Const
Add1
2
3
4

Traversal: Bottomup
!125
Const
Mul
Const
3 5 4
Const
Add1
2
3
4

Traversal: Bottomup
!126
Const
Mul
Const
3 5 4
Const
Add1

Traversal: Bottomup
!127
Mul
Const
3
Const
5 4
Const
Add1

Traversal: Innermost
!130
Const
Mul
Const
3 4 5
Const
Add1
2
3
4
5
6
7
8
innermost(s) = bottomup(try(s ; innermost(s)))
innermost(switch)

!131
Const
Mul
Const
3 4 5
Const
Add1
2
3
4
innermost(switch)

!132
Const
Mul
Const
3 5 4
Const
Add1
2
3
4
9
10
11
12
innermost(switch)

!133
Const
Mul
Const
3 5 4
Const
Add1
2
3
4
innermost(switch)

!134
Const
Mul
Const
3 4 5
Const
Add1
2
3
4
innermost(switch)

Type-Unifying
Transformations
141

!175
Except where otherwise noted, this work is licensed under

Compiler Construction | Lecture 5 | Transformation by Term Rewriting

More Related Content

What's hot (20)

Similar to Compiler Construction | Lecture 5 | Transformation by Term Rewriting (20)

More from Eelco Visser (13)

Recently uploaded (20)

Compiler Construction | Lecture 5 | Transformation by Term Rewriting