Difference between revisions of "JDT Core/Null Analysis"

Revision as of 15:20, 30 January 2011

This page discusses a proposed improvement in the static null analysis of the JDT compiler.

See also bug 186342.

Introduction

The static analysis of the JDT compiler detects many potential programming problems related to the null-ness of variables: dereferencing a null value (-> NPE), redundant null checks etc.

However, the current analysis is restricted to flow analysis within one method. No assumptions can be made about

• arguments flowing into a method
• return values from method calls and
• field reads.

In order to include these elements in the analysis one could either

• use whole program analysis (very expensive - not feasible for a (incremental) compiler)
• explicit contracts via an extended type system or annotations

The second option is well explored in research and some existing tools (like the Checker Framework, JML, FindBugs) already introduce specific annotations to this end.

One could argue that advanced analysis should be left to specialized tools but having something like this in the JDT compiler should show two benefits:

• feedback is more immediate and it is available for all JDT users without installing more software
• analysis might be more precise than existing tools, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).

A preparatory discussion of the design space can be found here: /Brainstorming.

Actual Strategy in the JDT

Disclaimer: this is work in progress. No promise is made that this particular feature will be part of any particular release of the JDT.

By default the JDT does not support inter-procedural null analysis, however, a prototype exists allowing the JDT to be configured to use annotations for null contracts across method boundaries. The prototypical implementation is currently based on the OT/Equinox technology for better maintainability at this stage. This particular prototype is known to have negative impact on the compiler performance, which is however no indication about how the final in-place implementation will perform.

Installing the prototype

• Get an Eclipse SDK ≥ 3.7M5
• Enter this update URL:
  • http://download.eclipse.org/objectteams/updates/contrib
• Select and install these features:
  • JDT Null Annotation Checker (Incubation)
  • Object Teams Equinox Integration (Incubation)

The code is hosted at svn://dev.eclipse.org/svnroot/tools/org.eclipse.objectteams/trunk/plugins/org.eclipse.objectteams.jdt.nullity

Example usage

In order to try the prototype against any existing Java project the following steps should help (I tried it using the JDT/Core as an example):

• Prepare the following compiler configuration
  • enable compliance 1.5 or higher
  • enable all null-related warnings
  • disable warnings regarding generics if the project doesn't use generics (don't spam the problems view)

• Since there isn't yet any UI for the new compiler preferences the following lines should be added manually into .settings/org.eclipse.jdt.core.prefs

org.eclipse.jdt.core.compiler.annotation.nullable=annotations.Nullable
org.eclipse.jdt.core.compiler.annotation.nonnull=annotations.NonNull
org.eclipse.jdt.core.compiler.annotation.nulldefault=nonnull

• the first two options specify the fully qualified names of those annotation types to be used for null contracts.
• the third option tells the compiler that any types that have no explicit null contract should be assumed as non-null (applies to method parameters and method return type).

• Create the annotation types specified above like this (in their correct package):

@Retention(RetentionPolicy.CLASS) public @interface Nullable { }
@Retention(RetentionPolicy.CLASS) public @interface NonNull { }

• Build (Project > Clean)
  • At this point you should see plenty of new errors and warnings

Cleaning up

The sheer number of new problems may look intimidating but that's where quickfixes will come to the rescue. Currently the following problems offer a quickfix:

• Null contract violation: returning null from a method declared as @NonNull

  Note that the mentioned @NonNull declaration is implicit via the global default

• Null contract violation: return value can be null but method is declared as @NonNull

  Use with care: the compiler has no clear indication if @Nullable was actually intended or not

• Redundant null check: The variable foo cannot be null at this location

  Only those occurrences that concern a method parameter

• Null comparison always yields false: The variable bar cannot be null at this location

  Only those occurrences that concern a method parameter

These quickfixes can be applied...

• individually (Ctrl-1)
• all occurrences per file (via the hover)
• all occurrences (via context menu in the Problems view)

Compiler configuration explained

By default the JDT does not recognize any null annotations but it can be configured to do so. For this purpose three independent options are proposed:

1. Specify the names of annotation types to be used for marking nullable vs. nonnull types.

   use this if you want to achieve compatibility with annotations defined by some other tool (or a future standard, should one be defined eventually).

   Implemented

2. Emulate null annotation types that do not exist on the build path

   use this if you don't have any actual null annotation types or if you just want to avoid having to configure your build path for null annotations.

   Partially implemented in the prototype: the Java Model and DOM do not support emulated annotation types and thus the UI will cause some exceptions in this mode.

3. Implicitly import null annotation types in every Java file

   use this if you want to use null annotations by their simple name even without a corresponding import statement in your sources.

   NOT implemented

If at least one of these options is defined / enabled, the compiler will start analysing null contracts. If "emulate" and "implicitly import" are enabled, no further preparations are required for using null annotations in your code.

Conversely, if null contracts are enabled ..

1. ... without specifying annotation type names

   → the following built-in defaults will be used:

   • nullable = org.eclipse.jdt.annotation.Nullable
   • nonnull = org.eclipse.jdt.annotation.NonNull

2. ... without enabling annotation type emulation

   → the specified annotation types have to be provided on the build path (either as source or binary files).

3. ... without enabling implicit annotation type imports

   → annotation types have to be imported or referenced by their fully qualified name.

Null Contracts

Once properly configured the JDT compiler supports specification and checking of null contracts. Each part of a null contract implies an obligation for one side and a guarantee for the other side.

Method Parameters

When a method parameter is specified as nullable this defines the obligation for the method implementation to cope with null values without throwing NPE. Clients of such a method enjoy the guarantee that it is safe to call the method with a null value for the given parameter.

When a method parameter is specified as nonnull all clients are obliged to ensure that null will never be passed as the value for this parameter. Thus the method implementation may rely on the guarantee that null will never occur as the value for this parameter.

Method Returns

The situation is reversed for method returns. All four cases are summarized by the following table:

Local Variables

Null contracts can also be defined for local variables although this doesn't improve the analysis by the compiler, because local variables can be fully analyzed without annotations, too. Here the main advantage of null annotations is in documenting intentions.

The following is an example of a program where all sides adhere to their respective part of the contract:

public class Supplier {
    // this method requires much but delivers little
    @Nullable String weakService (@NonNull String input, boolean selector) {
        if (selector)
            return input;
        else
            return null;
    }
    // this method requires nothing and delivers value
    @NonNull String strongService (@Nullable String input) {
        if (input == null)
           return "";
        else
           return input.toUpperCase();
    }
}
public class Client {
    void main(boolean selector) {
        Supplier supplier = new Supplier();
        @Nullable String value = supplier.weakService("OK", selector);
        @NonNull String result = supplier.strongService(value);
        System.out.println(result.toLowerCase());
    }
}

Notes:

• Althoug we know that toUpperCase() will never return null, the compiler does not know as long as java.lang.String does not specify null contracts. Therefor the compiler has to raise a warning that it has "insufficient nullness information" to guarantee contract adherence.
• Null annotations for the local variables are redundant here, as the nullness information can be fully derived from the variable's initialization (and no further assignments exist).

Null Contract Inheritance

A method that overrides or implements a corresponding method of a super type (class or interface) inherits the full null contract. Its implementation will thus be checked against this inherited contract. For the sake of safe polymorphic calls, null contracts must be inherited without modification, or be redefined in the following ways:

• A nonnull method parameter may be relaxed to a nullable parameter. The additional checks have to be performed in the body of the overriding method. Callers of the super type must still pass nonnull, while callers which are aware of the sub type may pass null.
• A nullable method return (or a return with no null annotation) may be tightened to a nonnull return. The additional checks must again be performed in the body of the overriding methods. Callers of the super type still have to check for null, whereas callers which are aware of the sub type may safely assume nonnull return values.

Any overrides that attempt to change a null contract in the opposite directions will raise a compile time error.

This explicitly implies that callers only need to inspect the null contract of the statically declared type of a call target to safely assume that all runtime call targets will adhere (at least) to the contract as specified in the statically declared type, even if the runtime type of the call target is any sub type of the declared type.

Future

The following bugzillas address future improvements of the above strategy (order roughly by priority):

• bug 334455 - UI for new preferences regarding null annotations
• bug 334457 - [compiler][null] check compatibility of inherited null contracts
• bug 331647 - [compiler][null] support flexible default mechanism for null-annotations
• bug 331651 - [compiler][null] Support nullity profiles for libraries
• bug 331649 - [compiler][null] consider null annotations for fields