Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation

Applicative Logic Meta-Programming
as the foundation for
Template-based Program Transformation
Coen De Roover 
cderoove@vub.ac.beSoftware Engineering Laboratory Software Languages Lab
Osaka Brussels

Introducing Belgium: location and languages
Dutch 
6.23 million
French 
3.32 million
German 
0.07 million
Germany
Luxemburg
France
Netherlands

Introducing Belgium: Brussels
Jeanneke pis
Zinneke pisAtomium
Grote Markt - Grand Place
Manneken pis

Introducing Belgium: painters
Magritte (1964)
Bruegel (1563)
Rubens (1615) Ensor (1889)
van Eyck (1434) van der Weyden (1445)
Permeke (1929)

Introducing Belgium: food
Waffles (from Brussels) Waffles (from Liège)
Chocolates French Fries

Introducing Belgium: drinks
For 25 little-known facts: http://cheeseweb.eu/2009/08/25-belgium/

Motivation: repeating changes within a program
annotation for properties of which
evolution is to be tracked
more typesafe version
before
after
many
public class BreakStatement extends Statement {
@EntityProperty(value = SimpleName.class)
private EntityIdentifier label;
!
public EntityIdentifier getLabel() {
return label;
}
!
public void setLabel(EntityIdentifier label) {
this.label = label;
}
}
private EntityIdentifier<SimpleName> label;
!
public EntityIdentifier<SimpleName> getLabel() {
return label;
}
!
public void setLabel(EntityIdentifier<SimpleName> label) {
this.label = label;
}
}

Table II
STUDY SUBJECTS
Eclipse JDT core Eclipse SWT Mozilla Project
Type IDE IDE Several projects associated with Internet
Period of development 2001/06 ⇠ 2009/02 2001/05 ⇠ 2010/05 1998/03 ⇠ 2008/05
Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07
Total revisions 17000 revisions 21530 revisions 200000 revisions
# of bugs 1812 1256 11254
# of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%)
# of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%)
2 3 4 5 6 7
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
2 3 4 5 6 7 8 9 10 11 12
Percentage
Eclipse SWT
0%
10%
20%
30%
40%
50%
60%
70%
2 4 6 8 10 12 14 16 18 21 25 29
Percentage
Mozilla
Figure 1. The number of times that the same bug is fixed
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
70%
Percentage
Eclipse SWT
0%
5%
10%
15%
20%
25%
30%
35% 0
1
3
4
7
1
6
1
8
6
8
4
1
5
9
3
4
7
4
Percentage
Mozilla
Table II
STUDY SUBJECTS
Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07
# of bugs 1812 1256 11254
# of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%)
# of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%)
0%
10%
20%
30%
40%
50%
60%
70%
80%
2 3 4 5 6 7
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
2 3 4 5 6 7 8 9 10 11 12
Percentage
Eclipse SWT
0%
10%
20%
30%
40%
50%
60%
70%
2 4 6 8 10 12 14 16 18 21 25 29
Percentage
Mozilla
0%
10%
20%
30%
40%
50%
60%
70%
0
3
6
9
12
16
19
25
29
38
44
52
60
80
97
103
130
166
495
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
70%
0
4
8
16
22
27
35
47
58
73
84
119
172
265
319
428
957
Percentage
Eclipse SWT
0%
5%
10%
15%
20%
25%
30%
35%
0
31
63
94
127
161
196
231
278
336
398
464
531
585
659
733
804
887
1014
Percentage
Mozilla
Figure 2. The number of days taken for the supplementary fix to appear since an initial fix
Table II
STUDY SUBJECTS
Study period 2004/07 ⇠ 2006/07 2004/07 ⇠ 2006/07 2003/04 ⇠ 2005/07
# of bugs 1812 1256 11254
# of Type I bugs 1405 (77.54%) 954 (75.96%) 7562 (67.19%)
# of Type II bugs 407 (22.46%) 302 (24.04%) 3692 (32.81%)
0%
10%
20%
30%
40%
50%
60%
70%
80%
2 3 4 5 6 7
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
2 3 4 5 6 7 8 9 10 11 12
Percentage
Eclipse SWT
0%
10%
20%
30%
40%
50%
60%
70%
2 4 6 8 10 12 14 16 18 21 25 29
Percentage
Mozilla
0%
10%
20%
30%
40%
50%
60%
70%
0
3
6
9
12
16
19
25
29
38
44
52
60
80
97
103
130
166
495
Percentage
Eclipse JDT core
0%
10%
20%
30%
40%
50%
60%
70%
0
4
8
16
22
27
35
47
58
73
84
119
172
265
319
428
957
Percentage
Eclipse SWT
0%
5%
10%
15%
20%
25%
30%
35%
0
31
63
94
127
161
196
231
278
336
398
464
531
585
659
733
804
887
1014
Percentage
Mozilla
Figure 2. The number of days taken for the supplementary fix to appear since an initial fix
Motivation: repeating changes within a program
correctly
bugs requiring
supplementary patches
amount of supplements
(~ missed occurrences)
days before first
supplement
[“An empirical study of supplementary bug fixes” Park et al. MSR 2012]

Repeating Changes: using existing tool support
requires specifying
subjects of the transformation (LHS)
their state afterwards or change actions (RHS)
carefully ensuring
no unwarranted changes are applied
no required changes are missed
where to apply a change
what change to apply

Repeating Changes: using existing tool support
Brussels’
logic meta-programming
Ekeko
Eclipse plugin
applications
program and corpus analysis
program transformation
meta-programming library for Clojure
causally connected
applicative meta-programming
script queries over workspace
specify code characteristics declaratively, leave search to logic engine
manipulate workspace
tool building
Building Development Tools Interactively
using the EKEKO Meta-Programming Library
Coen De Roover
Software Languages Lab
Vrije Universiteit Brussel, Belgium
Email: cderoove@vub.ac.be
Reinout Stevens
Software Languages Lab
Vrije Universiteit Brussel, Belgium
Email: resteven@vub.ac.be
Abstract—EKEKO is a Clojure library for applicative logic
meta-programming against an Eclipse workspace. EKEKO has
been applied successfully to answering program queries (e.g.,
“does this bug pattern occur in my code?”), to analyzing project
corpora (e.g., “how often does this API usage pattern occur
in this corpus?”), and to transforming programs (e.g., “change
occurrences of this pattern as follows”) in a declarative manner.
These applications rely on a seamless embedding of logic queries
in applicative expressions. While the former identify source code
of interest, the latter associate error markers with, compute
statistics about, or rewrite the identified source code snippets.
In this paper, we detail the logic and applicative aspects of the
EKEKO library. We also highlight key choices in their implemen-
tation. In particular, we demonstrate how a causal connection
with the Eclipse infrastructure enables building development
tools interactively on the Clojure read-eval-print loop.
I. INTRODUCTION
EKEKO is a Clojure library that enables querying and
manipulating an Eclipse workspace using logic queries that
are seamlessly embedded in functional expressions. Recent
applications of EKEKO include the GASR tool for detect-
ing suspicious aspect-oriented code [1] and the QWALKEKO
tool for reasoning about fine-grained evolutions of versioned
code [2]. In this paper, we describe the meta-programming
facilities offered by EKEKO and highlight key choices in
their implementation1
. We also draw attention to the highly
interactive manner of tool building these facilities enable.
II. RUNNING EXAMPLE: AN ECLIPSE PLUGIN
More concretely, we will demonstrate how to build a
lightweight Eclipse plugin entirely on the Clojure read-eval-
print loop. We will use this plugin as a running example
throughout the rest of this paper. Our Eclipse plugin is to
support developers in repeating similar changes throughout an
entire class hierarchy. It is to associate problem markers with
fields that have not yet been changed. In addition, it is to
present developers a visualization of these problems. Finally,
it is to provide a “quick fix” that applies the required changes
correctly.
Figure 1 illustrates the particular changes that need to be
repeated. The raw EntityIdentifier type of those fields
within a subclass of be.ac.chaq.model.ast.java.ASTNode
1The EKEKO library, its implementation, and all documentation is freely
available from https://github.com/cderoove/damp.ekeko/.
public c l a s s BreakStatement extends Statement {
/ / Before changes :
@EntityProperty ( value = SimpleName . c l a s s )
private E n t i t y I d e n t i f i e r l a b e l ;
/ / A f t e r changes :
@EntityProperty ( value = SimpleName . c l a s s )
private E n t i t y I d e n t i f i e r <SimpleName> l a b e l ;
/ / . . . ( a . o . , accessor methods change a c c o r d i n g l y )
}
Fig. 1: Example changes to be repeated.
that carry an @EntityProperty annotation, is to receive a type
parameter that corresponds to the annotation’s value key.
III. ARCHITECTURAL OVERVIEW
The EKEKO library operates upon a central repository of
project models. These models contain structural and behavioral
information that is not readily available from the projects
themselves. The models for Java projects include abstract
syntax trees provided by the Eclipse JDT parser, but also
control flow and data flow information computed by the SOOT
program analysis framework [3].
An accompanying Eclipse plugin automatically maintains
the EKEKO model repository. To this end, it subscribes to
each workspace change and triggers incremental updates or
complete rebuilds of project models. As a result, the infor-
mation operated upon by the EKEKO library is always up-to-
date. In addition, this plugin provides an extension point that
enables registering additional kinds of project models. The
KEKO extension, for instance, builds its project model from
the results of a partial program analysis [4] —enabling queries
over compilation units that do not build correctly.
IV. LOGIC PROGRAM QUERYING
The EKEKO library enables querying and manipulating pro-
grams using logic queries and applicative expressions respec-
tively. We detail the former first. Section V discusses the latter.
The program querying facilities relieve tool builders from
implementing an imperative search for source code that ex-
hibits particular characteristics. Instead, developers can specify
these characteristics declaratively through a logic query. The
[WCRE-CSMR14]

Think of it asquerying a databaseof programinformation!
Logic meta-programming (LMP)

Logic relations: ast/2 and has/3 “tables”
relation between a ?node 
of an Abstract Syntax Tree (AST) 
and its ?type
(ast
?type
?node)
(has
?property
?node
?value)
LM
P
relation of AST nodes and the
values of their properties

(ekeko
[?statement
?expression]

(ast
:ReturnStatement
?statement)

(has
:expression
?statement
?expression))

Logic querying: AST relations
SELECT
ast.node
as
?statement,

has.value
as
?expression

FROM

ast,
has

WHERE

ast.type
=

:ReturnStatement
AND

has.property
=
:expression
AND

has.node
=
ast.node;

SQL query
relation of all AST nodes of type :ReturnStatement
and the value of their :expression property
equivalent logic query
LM
P

(defn

expression|returned

[?expression]

(fresh
[?statement]

(ast
:ReturnStatement
?statement)

(has
:expression
?statement
?expression)))
Logic programming: defining relations LM
P
local variable
(ekeko*
[?returned
?type]

(expression|returned
?returned)

(ast|expression-‐type
?returned
?type)

(type|binary
?type)
returned expressions of a type defined in compiled rather than source code
defining relation:
using newly defined relation:

control and
data flow 
structural
syntactic for(Object i : collection)
Ekeko Library: relations
class Ouch {
int hashCode() {
return ...;
}
}
scanner = new Scanner();
...
x.close();
...
scanner.next();
(ast ?kind ?ast)
(has ?property ?ast ?value)
(ast-encompassing|method+ ?ast ?m)
(ast-encompassing|type+ ?ast ?t)
(classfile-type ?binaryfile ?type)
(type-type|sub+ ?type ?subtype)
(type-name|qualified ?type ?qname)
(advice-shadow ?advice ?shadow)
for logic meta-programming
(method|soot-cfg ?m ?cfg)
(unit|soot-usebox ?u ?ub)
(local|soot-pointstoset ?l ?p)
(soot|may|alias ?l1 ?l2)

Ekeko Library: functions
(remove-node node)
(replace-node node newnode)
(change-property node property value)
(apply-and-reset-rewrites!)
for applicative meta-programming
(visualize nodes edges
:layout layout
:node|label labelfn
:edge|label labelfn
. . .)
(add-problem-marker marker node)
(register-quickfix marker rewritefn)
(reduce—workspace fn initval)
(wait-for-builds-to-finish)
rewriting
visualizing
tooling

before
after
private EntityIdentifier label;
!
public EntityIdentifier getLabel() {
return label;
}
!
public void setLabel(EntityIdentifier label) {
this.label = label;
}
}
private EntityIdentifier<SimpleName> label;
!
public EntityIdentifier<SimpleName> getLabel() {
return label;
}
!
public void setLabel(EntityIdentifier<SimpleName> label) {
this.label = label;
}
}
Repeating Changes: using Ekeko
Live
Dem
o!

Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation

Repeating Changes: using Ekeko
specifying changes requires significant expertise
and most state of the art tools
often requires multiple iterations
Imperative Program Transformation by Rewriting 53
says that an assignment x := v (where v is a variable) can be replaced by
= c if the “last assignment” to v was v := c (where c is a constant). The side
ndition formalises the notion of “last assignment”, and will be explain later in
e paper:
n : (x := v) =⇒ x := c
if
n ⊢ A△
(¬def(v) U def(v) ∧ stmt(v := c))
conlit(c)
The rewrite language has several important properties:
The specification is in the form of a rewrite system with the advantages of
succinctness and intuitiveness mentioned above.
The rewrite system works over a control flow graph representation of the
program. It does this by identifying and manipulating graph blocks which
are based on the idea of basic blocks but with finer granularity.
The rewrites are executable. An implementation exists to automatically de-
termine when the rewrite applies and to perform the transformation just
from the specification.
no unwarranted changes are applied
no required changes are missed generalization
refinement

Repeating Changes : introducing Ekeko/X
more intuitively
1/ template-based transformation specifications
2/ matching and rewriting directives in templates
3/ template mutation operators
Towards

(ekeko/x

!

?exp.setAge(?arg)

!

=>

!

new
Integer(?arg))

(ekeko/x <LHS1>…<LHSn>
=>
<RHS1>…<RHSn>)
template on LHS:
matches identify subjects
template on RHS:
generates code for every LHS match

(ekeko/x

?exp.setAge(?arg)

=>

new
Integer(?arg))

will not match setAge(5)
no destination for generated code
(ekeko/x

[?exp]@[relax-‐receiver].setAge(?arg)

=>

[new
Integer(?arg)]@[(replace
?arg)])

[<component>]@[<directive>]
matching directive
rewriting directive

[]@[orlarger] class ?name extends [Component]@[orsubtype] {
[[public]@[member]
void acceptVisitor(?type ?param) {[
System.out.println(?string);
[?x]@[(must-alias ?param)].?visitMethod(this);
]@[orflow]
}]@[member]
} public class OnlyLoggingLeaf extends Component {
public void acceptVisitor(ComponentVisitor v) {
System.out.println("Only logging.");
}
}
public class SillyLeaf extends OnlyLoggingLeaf {
super.acceptVisitor(v);
ComponentVisitor temp = v;
temp.visitSuperLogLeaf(this);
}
}
rich inter-proceduralsemantics!

[]@[orlarger] class ?name extends [Component]@[orsubtype] {
[[public]@[member]
void acceptVisitor(?type ?param) {[
System.out.println(?string);
[?receiver]@[(must-alias ?param)].?visitMethod(this);
]@[orflow]
}]@[member]
} public class OnlyLoggingLeaf extends Component {
System.out.println("Only logging.");
}
}
public class SillyLeaf extends OnlyLoggingLeaf {
super.acceptVisitor(v);
ComponentVisitor temp = v;
temp.visitSuperLogLeaf(this);
}
}
rich inter-proceduralsemantics!

return
age;
return
?exp;introduce-variable
generalize-aliases
atomic
generalization
compound
generalization
public
boolean
hasChildren(Object
element)
{

if
(element
==
null)

return
false;

return
getChildren(element).length
>
0;

}
public
boolean
hasChildren(Object
?param)
{

if
([?ref1]@[(must-‐alias
?param)]
==
null)

return
false;

return
getChildren([?ref2]@[(must-‐alias
?param)]).length
>
0;

}

Applications: beyond repeating changes
1 List<User> getRoleUser () {
2 List<User> listUsers = new ArrayList<User>();
3 List<User> users = this.userDao.getUsers();
4 List<Role> roles = this.roleDao.getRoles();
5 for (User u : users) {
6 for (Roles r : roles) {
7 if (u.roleId().equals(r.roleId())) {
8 User userok = u;
9 listUsers.add(userok);
10 }}}
11 return listUsers;
12 }
Figure 1: Sample code that implements join operation in applica-
tion code, abridged from actual source for clarity
1 List listUsers := [ ]; int i, j = 0;
2 List users := Query(SELECT ⇤ FROM users);
3 List roles = Query(SELECT ⇤ FROM roles);
4 while (i < users.size()) {
5 while (j < roles.size()) {
6 if (users[i].roleId = roles[j].roleId)
7 listUsers := append(listUsers, users[i]);
8 ++j;
9 }
10 ++i;}
Figure 2: Sample code expressed in kernel language
c 2 constant ::= True | False | number literal | string litera
e 2 expression ::= c | [ ] | var | e.f | {fi = ei } | e1 op e2 | ¬
| Query(...) | size(e) | geter
(es )
| append(er , es ) | unique(e)
c 2 command ::= skip | var := e | if(e) then c1 else c2
| while(e) do c | c1 ; c2 | assert e
op 2 binary op ::= ^ | _ | > | =
Figure 4: Abstract syntax of the kernel language
Entry Point Identifier
Java
Inferred
SQL
Java
Code Fragment Identifier
Java
Code
fragment
Java
Application source +
config files
Code Inliner
Inlined persistent
data methods
VC Computation
Invariant + Postcondition
Synthesizer
Formal Verification
Transform
method b
Java
Java
XML
Kernel Language Compiler
Figure 5: QBS architecture
9 listUsers.add(userok);
10 }}}
11 return listUsers;
12 }
Figure 1: Sample code that implements join operation in applica-
tion code, abridged from actual source for clarity
1 List listUsers := [ ]; int i, j = 0;
2 List users := Query(SELECT ⇤ FROM users);
3 List roles = Query(SELECT ⇤ FROM roles);
4 while (i < users.size()) {
5 while (j < roles.size()) {
6 if (users[i].roleId = roles[j].roleId)
7 listUsers := append(listUsers, users[i]);
8 ++j;
9 }
10 ++i;}
Figure 2: Sample code expressed in kernel language
Postcondition
listUsers = ⇡`(./' (users, roles))
where
'(eusers , eroles ) := eusers .roleId = eroles .roleId
` contains all the ﬁelds from the User class
Translated code
1 List<User> getRoleUser () {
2 List<User> listUsers = db.executeQuery(
3 "SELECT u
4 FROM users u, roles r
5 WHERE u.roleId == r.roleId
6 ORDER BY u.roleId, r.roleId");
7 return listUsers; }
Figure 3: Postcondition as inferred from Fig. 1 and code after query
transformation
domain-specific optimizations
API migration
program renovation
domain-specific refactorings
refactorings to eliminate or maintain clones
convert anonymous classes to Java 8 lambda expressions
migrate an application from using SWING to using SWT
[Cheung et al., PLDI13]

Ongoing work involving Ekeko/X
and possible collaborations
summarizing similar code fragments into template
action: template mutations
goal: template that covers all fragments
start state: code fragment
state space search
efficacy of atomic vs compound mutation operators

Ongoing work involving Ekeko/X
and possible collaborations
buggy
fixed
evolved
clone B
clone B
evolved
clone B
fixed
system variant A system variant B system variant C
evolved
clone C
clone C
evolved
clone C
fixed
PATCH
PATCH'
PATCH''??
supporting clone-and-own reuse
repeat changes across variants
impact analysis of divergences on transformation
by mutating transformation

Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation

More Related Content

What's hot (19)

Viewers also liked (9)

Similar to Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation (20)

Recently uploaded (20)

Applicative Logic Meta-Programming as the foundation for Template-based Program Transformation