Roulette

Summary

The goal of the project is to reimplement part of Jackpot functionality as an Eclipse plugin. It wouldn't be a straight port by copy-pasting its source and adapting into Eclipse environment. The implementation will reuse Eclipse JDT Plugin: especially its JDT AST/DOM manipulation classes and ASTRewrite mechanism.

Jackpot funcionality now

Jackpot is an experimental tool for automated code transformations. It introduces its own DSL that is quite straightforward to learn and use. The DSL reuses

Java syntax to represent match-patterns and transformations.

The simplest case of a rule expressing a simple transform is :

	<code_before> => <code_after>

Jackpot searches the code that matches the pattern on the left hand of the "=>" operator. Every matched code structure is then replaced by corresponding code located at the right hand of the "=>" operator.

Example:

	i++ => i--

The rule above will match any postfix incrementation expression of any variable named "i" and replace it with a postfix decrementation expression.

	{ i++; j++; k++; } => { k--; j--; i-- }

This one will match any consecutive postfix incrementations of i, j and k and replace them with code on the right.

To make the rules more flexible, Jackpot introduces concept of meta-variables (wildcards). These act quite like groups in regular expressions. The metavariables are simple Java identifiers starting with "$".

Example:

	$variable++ => $variable--

The rule above will match ANY postfix incrementation expression of ANY variable and replace it with a postfix decrementation expression.

NOTE: Java syntax does not forbid using names starting with "$" but it is generally percieved as bad practice against naming conventions and no one uses it. That's why Jackpot creators found it quite safe to denote metavars like that.

Meta-variales can also express a sequence of statements. Such meta-vars start and end with "$". Example:

	{ $p$; $t $v = $e; return $v; } => { $p$; return $e; }

The above rule eliminates unnecessary temporary variable denoted here as $v. $p$ denotes an arbitrary sequence of statements preceding the one that matches $t $v.

To make the rules more specific, guard-expressions can be used :

	<code_before> => <code_after> :: <guard_expression>

If guard-expression is present (after "::" operator), it is evaluated for every matched code. The transformation is applied only when the guard-expression

evaluates to true. The guard-expressions can be constructed of following elements :

• operators: instanceof, &&, ||
• predicates: assignedIn(A,B), declaredIn(A,B), referencedIn(A,B), isInstance(Object,Type), statementcontext(), hasComment, isConstant, isLiteral, isClassIdentifier, isEmpty, sideEffectFree, hasVariableDeclarations, isStatement, isTrue, isFalse, isNull, isNullTree, isStatic, couldThrow, referenced, assigned, parameter, local

Example:

	$s.equals("") => $s.isEmpty() :: $s instanceof java.lang.String;

The rule will be applied only if $s is a String.

Jackpot allows performing "actions" on matched code :

	<code> => <action>

Example:

	$s.countTokens() => transformationFailure("I dont know what to do with countTokens()")

Available actions: note(Text), comment(Text), transformationFailure(Message)

Syntactic sugar for easing API migration :

	mapclass <class_A> => <class_B>;

Example:

	mapclass com.packageA.ClassA => com.packageB.ClassB;

It will replace each use of ClassA with appropriate use of ClassB, where applicable.

How Jackpot works

The heart of Jackpot is org.netbeans.modules.jackpot.rules.parser.TransformParser class. It extends ScriptParser and uses classes from com.sun.tools.javac.*. It parses the rule and visits input code and construct a class (GeneratedMatcher) that will write new code replacing the input code. The generated on the fly class extends GeneratedMatcher. It is then compiled on the fly with PluginCompiler and loaded into VM and executed (rewrite() method).

TransformParser is also responsible for matching meta-variables.

This class is quite big and cries for better design, it's one of the main reasons of reimlpementing it rather than reusing it.

GeneratedMatcher is the base class for the on-the-fly-compiled rewriter. It uses com.sun.source.tree.*, com.sun.tools.javac.* and javax.lang.model.* packages to manipulate Java DOM.

Scope of reimplementation

what will be reimplemented

• parsing of a rule in the form :

	<code_before> => <code_after> :: <guard_expression>

• matching the pattern, including meta-variable ($expr, $statemebnts$) matching
• presenting found mathes to the user in a dialog box
• rewriting matched code with corresponding replacement code
• mechanism to execute appropriate guard-expressions
• one guard-expression implementation: instanceof
• a set of unit tests extensively excercising provided functionality

what will be skipped

• actions - as this is not the core functionality
• mapclass - as this is just a syntactic shortcut, not the core functionality
• guard-expressions other than 'instanceof'

Re-Implementation Concept

Engine

• input: Rule, CompilationUnit, IJavaProject, collection of Library objects
• output: a collection of TextEdit
• matches given rule to one java file at a time
• applies a transformation
• invokes guards implemented by Library objects

RuleParser

• input: rule as text
• output: a bean representing the rule compounds
• Parses

	<match-pattern> => <replacement> :: <guard_expr>

into AST representation of <match-pattern> and AST for <replacement>. Ignores <guard_expr>. Ideally, it should extend the JDT parser/compiler just like Jackpot extends sun's compiler classes to process both Java code and additional "=>" and "::" operators. But it seems quite hard to extends JDT compilar because of its comlpexity and some internal packages being hidden behind plugin boundaries. Parsing of a rule might be done with JavaCC-generated or ANTLR-generated parser based on Java grammar extended with these two operators. It will be easier than trying to extend JDT.

Matcher (extends ASTMatcher) class that compares two AST subtrees

• input: 2x ASTNode
• output: answer if the two given subtrees match or not, mapping metavariables to particular ASTNodes (if the subtrees match)
• The matching algorithm is aware of meta-variables.

PatternScanner (extends ASTVisitor) class

• input: match-pattern-tree, CompilationUnit
• output: a collection of ASTNodes matching given pattern
• It visits a CompilationUnit top-down. On every ASTNode it visits, it uses ASTMatcher to check if the subtree rooted at currently processed ASTNode matches the pattern-tree (kinda brute-force but feasible). If subtree rooted under the given ASTNode matches the pattern-tree, the class records the match and continues on its sibling nodes.

Rewriter

• input: IJavaProject, CompilationUnit, match-pattern root-ASTNode, mapping metavars into ASTNodes, replacement-tree,
• output: rewritten code of CompilationUnit
• It retrieves the source code of CompilationUnit from IJavaProject and creates a ASTRewrite object for it. Then it clones replacement-tree within CompilationUnit's AST object and replaces metavars with ASTNodes from appropriate mapping. Eventually, it replaces match-pattern root-ASTNode with the root of the cloned replacement-tree. Finally, it commits the rewrite and saves the changes into file, notifying other plugins about changed resources/model.

Library

• registers own methods as guards
• every method is an implementation of a guard
• every parameter of a method is an ASTNode object
• every method returns boolean

Problems

• JDT parser is hard to extend with "=>" and "::" operators
• JDT parser won't accept $statements$ wildcard

Links

• Last year idea proposal "Support for user-defined refactorings in JDT"
• see discussion on soc-dev@eclipse.org
• http://jackpot.netbeans.org