Revision as of 13:59, 24 November 2015

This page gives an overview of how to proceed when you want to generify a Java project (use Java 5 generics). It describes the steps we took to generify the org.eclipse.jdt.ui project. Most of these steps should also be applicable for other projects.

Basics

• Make sure you understand what you're doing:
  • Read Evolving Java-based APIs, especially Evolving Java-based APIs 2#Turning_non-generic_types_and_methods_into_generic_ones:

    So if you plan to generify an API, remember that you only get one chance (release), to get it right. In particular, if you change a type in an API signature from the raw type "List" to "List<?>" or "List<Object>", you will be locked into that decision. The moral is that generifying an existing API is something that should be considered from the perspective of the API as a whole rather than piecemeal on a method-by-method or class-by-class basis.

  • Read about the relevant Java Programming Language Enhancements
  • Read the Java Language Specification
  • Understand the concept of erasure, especially with respect to API compatibility (in short: For binary compatibility, only the erasure matters).

• Generifying APIs requires special care:
  • Communicate to API clients that you're going to generify APIs
  • Perform the work in a branch and only merge back once you've validated the changes by updating several clients of your API.
  • If in doubt, don't add type parameters to types. If really necessary, you can still parameterize types later. But you cannot add/remove/modify type parameters after the initial release! If you add type parameters, all your clients will have to add type arguments all over their code to resolve raw type problems.

Setup

• Run with enough memory (e.g. -Xmx600M)
• Make sure you're in sync with HEAD and that other committers are informed about your plans (you don't want to merge non-trivial changes, since you're going to touch a big percentage of your LOC).
• In MANIFEST.MF, set Bundle-RequiredExecutionEnvironment: JavaSE-1.7
• On the build path, change the "JRE System Library" to "Execution Environment: JavaSE-1.7"
  • this will also set the right compliance options on the "Project Properties > Java Compiler" page
• Make sure the minor segment of the bundle version has been incremented.
• Make a pass over "Project Properties > Java Compiler > Errors/Warnings" and turn interesting problems for 1.5 language constructs to Warning or Error:
  • Set all options under Annotations to Warning/Enabled, except for the last one (disable "Suppress optional errors...")
  • Our general rule (JDT UI team) is that we set options to Error iff:
    • there are no false positives (i.e. no "potential" problems)
    • we don't have hundreds of instances of the problem and fixing all of them would take too much time
    • it's not a local problem that often occurs during editing and where an error would be distracting (e.g. unused stuff)
  • Before we commit a file, we always try to get rid of all warnings (if possible with reasonable effort)
  • If you can't generify all dependencies, the Ignore unavoidable generic type problems compiler option can help you get rid of uninteresting raw type warnings
    • To disable these warnings in the PDE build and Tycho, add javacWarnings..=-unavoidableGenericProblems to your build.properties file
• Adjust code formatter settings for generics and annotations if necessary

Clean Up

• Run Clean Ups:
  • Missing Code > Add missing Annotations > @Override
  • Missing Code > Add missing Annotations > @Deprecated
  • Unnecessary Code > Remove unnecessary casts (you can also enable the others in Unnecessary Code)

The @Override annotation will make some comments unnecessary. Here are two regular expressions to find boilerplate comments (replace matches with nothing):

^[ \t]*/\*\*[\s\*]*\{@inheritDoc\}[\s\*]*\*/\s*\R
^[ \t]*/\*[\s\*]*\(non-Javadoc\)[\s\*]*@see [^@/]+(?:@since \d+\.\d+)?\s*\*/\s*\R

Or both in one little monster:

^[ \t]*/\*(?:\*[\s\*]*\{@inheritDoc\}[\s\*]*|[\s\*]*\(non-Javadoc\)[\s\*]*@see [^@/]+(?:@since \d+\.\d+)?\s*)\*/\s*\R

I also found a few matches with this pattern:

^[ \t]*/\*[\s\*]*\(non-Javadoc\)[\s\*]*Method declared [oi]n \w+\.?[\s\*]*\*/\s*\R

As always, make sure you look at the changes before committing them! Some comments started as (non-Javadoc), but accumulated interesting explanations later on.

Required Projects

• Generify APIs of required projects. If you have non-generified dependencies, the "Infer Generic Type Arguments" refactoring will wrongly infer <Object> in too many cases (since that's the best guess when APIs use raw types).

E.g to generify the org.eclipse.jdt.core.dom APIs, I used this regex search on the whole project:

\(element ?[Tt]ype: \{@link (\w+)\}\)[^/]*\R\t \*/\R\tpublic List

... and then I just replaced the match with:

$0<$1>

This worked quite well because the Javadocs of these APIs are very clean, consistent, and they contain the element type information.

Caveat: After this, client code that calls these APIs and then adds elements to the parameterized list must make sure the elements are of the right type. In some cases, this means that you have to add type parameters to other methods as well.

E.g. in ASTRewrite, the right generic version of:

ASTNode createCopyTarget(ASTNode node)

is:

<T extends ASTNode> T createCopyTarget(T node)

If you can't release these changes, you'll run into trouble when compiling certain client code, e.g.:

	SingleVariableDeclaration p1= xxx;
	SingleVariableDeclaration newParam = (SingleVariableDeclaration) rewrite.createCopyTarget(p1);

The original version of createCopyTarget requires a cast on the second line, but the generified version doesn't. I don't have a good solution for this problem.

Here's another regex I used to generify property descriptors in the DOM AST:

Search: (public static final Child(?:List)?PropertyDescriptor)( [A-Z_]+\s*=\s*new Child(?:List)?PropertyDescriptor)(\(\w+\.class, "\w+", (\w+)\.class)
Replace: \1<\4>\2<\4>\3

• If you can't generify all dependencies, the Ignore unavoidable generic type problems compiler option can help you get rid of uninteresting raw type warnings.

Generify your project

• Manually add type arguments to your subtypes of generic types:
  • Search for implementations of Comparable, Comparator, Iterator, and of the Collection types (Collection, List, Set, Map)
• Set the Problems view to "Group By > Java Problem Type", open the "Type Safety and Raw Types" group, and look at the "Class is a raw type" problems
  • Class types are a bit special, and it usually pays out to generify them manually (IAdaptable#getAdapter(Class<T>), I'm looking at you).
• Generify your own container types (e.g. MultiMaps, adapters for UI list or tree widgets).
• Run "Refactor > Infer Generic Type Arguments" on the whole project (both options checked)
• Organize Imports
• Fix remaining compile errors manually.
• Check that the changes make sense and don't affect APIs:
  • Be aware that the Infer Generic Type Arguments refactoring does not create wildcard type bounds, but in APIs, that would often be the right thing to do.
• Note that some constructs cannot be automatically generified, e.g. custom map implementations, or code that uses the same slots for E and Collection<E>.
• Search for commented type arguments that have sometimes been used (e.g. Map/*<String, Integer>*/ fStringToInteger;):
  • If equal to the actual arguments, remove the comments:

Search: \s*/\*\s*(<[^>]+>)\s*\*/\s*\1
Replace: \1
(Caveat: Search expression does not work for nested types like List<List<String>>)

• If different from the actual arguments, take a closer look:

Search:  \s*/\*\s*(<[^>]+>)\s*\*/\s*(?!\1)<[^>]+>
Search:  \w+\s*/\*\s*[^/]+)\s*\*/\s*<[^>]+>

• Query for Collection/*String*/<String> (note the missing < > in the comment):

Search: \s*/\*\s*<?\s*([^*]+)\s*>?\s*\*/\s*<\1>
Replace: <\1>

• Make a last manual pass over types with Object as type argument (could be correct, but is often not):

Search: <Object|Object>

• Check all APIs to make sure their erasure didn't change and the type arguments have the necessary wildcards (roughly "<? super X>" for containers on which you need to call setters, "<? extends X>" for containers on which you need to call getters, and "<X>" for cases that should be completely restricted). Note that adding type parameters and arguments to APIs is like adding new API: You have exactly one chance to get it right. Changing generics later will be a breaking change most of the time.
• Run "Clean Up > Enhanced for loop" if you want

Closing remarks

• Set the Problems view to "Group By > Java Problem Type" and try to get get the "Type Safety and Raw Types" group empty.
• Collection#toArray(T[]) is not typesafe, see Bug 7023484. Be careful when writing new code, since the compiler will not detect bugs in this case.