Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "JDT Core/Null Analysis"

(→‎Installing the prototype)
(extracted section on contracts and adjusted to a new scheme not using the word "contracts")
Line 37: Line 37:
 
but having something like this in the JDT compiler should show two benefits:
 
but having something like this in the JDT compiler should show two benefits:
 
* feedback is more immediate and it is available for all JDT users without installing more software
 
* feedback is more immediate and it is available for all JDT users without installing more software
−
* analysis might be more precise than existing tools, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).
+
* analysis might be more precise than some existing tools provide, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).
  
 
A preparatory discussion of the design space can be found here: [[/Brainstorming]].
 
A preparatory discussion of the design space can be found here: [[/Brainstorming]].
Line 46: Line 46:
 
'''Disclaimer''': this is work in progress. No promise is made that this particular feature will be part of any particular release of the JDT.
 
'''Disclaimer''': this is work in progress. No promise is made that this particular feature will be part of any particular release of the JDT.
  
−
By default the JDT does not support inter-procedural null analysis, however, a prototype exists allowing the JDT to be configured to use annotations for null contracts across method boundaries. The prototypical implementation is currently  
+
By default the JDT does not support inter-procedural null analysis, however, a prototype exists allowing the JDT to be configured to use annotations for extended null checking. The prototypical implementation is currently  
 
based on the [[:Category:OTEquinox|OT/Equinox]] technology for better maintainability at this stage. This particular prototype is known to have negative impact on the compiler ''performance'', which is however no indication about how the final in-place implementation will perform.
 
based on the [[:Category:OTEquinox|OT/Equinox]] technology for better maintainability at this stage. This particular prototype is known to have negative impact on the compiler ''performance'', which is however no indication about how the final in-place implementation will perform.
 +
 +
===Specifying nullness===
 +
 +
Null annotations in method signatures can be interpreted as [[/Null Contracts|null contracts]], however, a more general approach considers null annotations as an extension of the type system. Eventually - that is once JSR 308 can be used - all type references should either include or exclude null, which allows for complete checking of any possible dereferencing of null. In other words, a fully annotated program which passes the type checker will never raise an NPE at runtime.
 +
 +
To achieve this guarantee two annotations are used. The specific annotations can be selected as a preference, but the following defaults are provided (see [[#Compiler configuration explained]]):
 +
* org.eclipse.jdt.annotation.NonNull
 +
* org.eclipse.jdt.annotation.Nullable
 +
 +
For any variable who's type is annotated with @NonNull (or the configured equivalent) the following rules apply:
 +
* It is illegal to bind null or a value that can be null to the variable. (For fields and local variables this applies to initialization and assignments, for method argument binding a value means to pass an actual argument in a method call).
 +
* It is legal and safe to dereference such a variable for accessing a field or a method of the bound object.
 +
 +
For any variable who's type is annotated with @Nullable (or the configured equivalent) the following rules apply:
 +
* It is legal to bind null or a value that can be null to the variable (see details above).
 +
* It is illegal to dereference such a variable for either field or method access.
 +
 +
The above rules imply that the value from a @NonNull variable can be bound to a variable annotated with @Nullable, but the opposite direction is generally illegal.
 +
Only immediately after an explicit null check can a @Nullable variable be treated as being @NonNull for the sake of binding to another @NonNull variable or for dereferencing.
  
 
===Installing the prototype===
 
===Installing the prototype===
  
 
* Get an [http://download.eclipse.org/eclipse/downloads/drops/R-3.7-201106131736/index.php Eclipse SDK 3.7] or use your favorite [http://www.eclipse.org/downloads/ Indigo package].
 
* Get an [http://download.eclipse.org/eclipse/downloads/drops/R-3.7-201106131736/index.php Eclipse SDK 3.7] or use your favorite [http://www.eclipse.org/downloads/ Indigo package].
−
** ''(instructions for helios will follow shortly)''
+
** ''(no backport to helios, sorry)''
 
** Some of the operations below (multi quickfixes) require lots of memory, you may want to add s.t. like <code>-Xmx800m</code> already now
 
** Some of the operations below (multi quickfixes) require lots of memory, you may want to add s.t. like <code>-Xmx800m</code> already now
 
* Enter this update URL:
 
* Enter this update URL:
Line 84: Line 103:
 
* Build (Project > Clean)
 
* Build (Project > Clean)
 
** At this point you should see plenty of new errors and warnings
 
** At this point you should see plenty of new errors and warnings
−
** There are known issues with incremental building, so a full build may be required at times.
+
** If problems are not reported immediately, please try a full build (if this makes a difference please drop a note in {{bug|186342}}).
  
 
====Cleaning up====
 
====Cleaning up====
Line 141: Line 160:
  
 
It is the user's responsibility to make the required annotation types available on the build path.
 
It is the user's responsibility to make the required annotation types available on the build path.
−
 
−
===Null Contracts===
 
−
Once properly configured the JDT compiler supports specification and checking of null contracts. Each part of a null contract implies an obligation for one side and a guarantee for the other side.
 
−
 
−
====Method Parameters====
 
−
 
−
When a method parameter is specified as '''nullable''' this defines the <font color="red">obligation</font> for the '''method implementation''' to cope with null values without throwing NPE. '''Clients''' of such a method enjoy the <font color="green">guarantee</font> that it is safe to call the method with a null value for the given parameter.
 
−
 
−
When a method parameter is specified as '''nonnull''' all '''clients''' are <font color="red">obliged</font> to ensure that null will never be passed as the value for this parameter. Thus the '''method implementation''' may rely on the <font color="green">guarantee</font> that null will never occur as the value for this parameter.
 
−
 
−
====Method Returns====
 
−
The situation is reversed for method returns. All four cases are summarized by the following table:
 
−
 
−
{|border=1
 
−
! !! caller !! method implementation
 
−
|-
 
−
| nullabel parameter
 
−
| <font color="green">may safely pass null without checking</font>
 
−
| <font color="red">must check before dereference</font>
 
−
|-
 
−
| nonnull parameter 
 
−
| <font color="red">must not pass null  </font>
 
−
| <font color="green">may use without checks</font>
 
−
|-
 
−
| nullable return   
 
−
| <font color="red">must check before dereference</font>
 
−
| <font color="green">can safely pass null</font>
 
−
|-
 
−
| nonnull return   
 
−
| <font color="green">may use without check</font>
 
−
| <font color="red">must not return null</font>
 
−
|-
 
−
|}
 
−
 
−
====Local Variables====
 
−
Null contracts can also be defined for local variables although this doesn't improve the
 
−
analysis by the compiler, because local variables can be fully analyzed without annotations, too. Here the main advantage of null annotations is in documenting intentions.
 
−
 
−
The following is an example of a program where all sides adhere to their respective part of the contract:
 
−
<source lang="java">
 
−
public class Supplier {
 
−
    // this method requires much but delivers little
 
−
    @Nullable String weakService (@NonNull String input, boolean selector) {
 
−
        if (selector)
 
−
            return input;
 
−
        else
 
−
            return null;
 
−
    }
 
−
    // this method requires nothing and delivers value
 
−
    @NonNull String strongService (@Nullable String input) {
 
−
        if (input == null)
 
−
          return "";
 
−
        else
 
−
          return input.toUpperCase();
 
−
    }
 
−
}
 
−
public class Client {
 
−
    void main(boolean selector) {
 
−
        Supplier supplier = new Supplier();
 
−
        @Nullable String value = supplier.weakService("OK", selector);
 
−
        @NonNull String result = supplier.strongService(value);
 
−
        System.out.println(result.toLowerCase());
 
−
    }
 
−
}
 
−
</source>
 
−
Notes:
 
−
* Althoug we know that toUpperCase() will never return null, the compiler does not know as long as <code>java.lang.String</code> does not specify null contracts. Therefor the compiler has to raise a warning that it has "insufficient nullness information" to guarantee contract adherence.
 
−
* Null annotations for the local variables are redundant here, as the nullness information can be fully derived from the variable's initialization (and no further assignments exist).
 
−
 
−
===Null Contract Inheritance===
 
−
A method that overrides or implements a corresponding method of a super type (class or interface) inherits the full null contract. Its implementation will thus be checked against
 
−
this inherited contract. For the sake of safe polymorphic calls,
 
−
null contracts must be inherited without modification, ''or'' be redefined in the following ways:
 
−
* A nonnull method parameter may be '''relaxed''' to a nullable parameter. The additional checks have to be performed in the body of the overriding method. Callers of the super type must still pass nonnull, while callers which are aware of the sub type may pass null.
 
−
* A nullable method return (or a return with no null annotation) may be '''tightened''' to a nonnull return. The additional checks must again be performed in the body of the overriding methods. Callers of the super type still have to check for null, whereas callers which are aware of the sub type may safely assume nonnull return values.
 
−
 
−
Any overrides that attempt to change a null contract in the opposite directions will raise a compile time error.
 
−
 
−
This explicitly implies that callers only need to inspect the null contract of the statically declared type of a call target to safely assume that all runtime call targets will adhere (at least) to the contract as specified in the statically declared type, even if the runtime type of the call target is any sub type of the declared type.
 
  
 
===Defaults at different levels===
 
===Defaults at different levels===
Line 228: Line 168:
  
 
The above mentioned preference (<code>org.eclipse.jdt.core.compiler.annotation.nulldefault</code>) allows to globally change this so that any declaration
 
The above mentioned preference (<code>org.eclipse.jdt.core.compiler.annotation.nulldefault</code>) allows to globally change this so that any declaration
−
to which no null annotation applies (directly or via inheritance) will be considered
+
to which no null annotation applies will be considered as either nullable or nonnull, depending on that specific setting.
−
as either nullable or nonnull, depending on that specific setting.
+
  
 
For more fine-grained control two additional annotations can be used. The qualified type
 
For more fine-grained control two additional annotations can be used. The qualified type
Line 239: Line 178:
  
 
These annotations can be applied to any Java type or package and affect all  
 
These annotations can be applied to any Java type or package and affect all  
−
method returns and parameters with undefined null status within their scope.
+
method returns and parameters with undefined null status within their scope. (More locations will be supported in the future).
  
 
A default declared at an outer scope can be overridden by a different default at an
 
A default declared at an outer scope can be overridden by a different default at an

Revision as of 13:59, 9 August 2011

This page discusses a proposed improvement in the static null analysis of the JDT compiler.

See also bug 186342.


Introduction

The static analysis of the JDT compiler detects many potential programming problems related to the null-ness of variables: dereferencing a null value (-> NPE), redundant null checks etc.

However, the current analysis is restricted to flow analysis within one method. No assumptions can be made about

  • arguments flowing into a method
  • return values from method calls and
  • field reads.

In order to include these elements in the analysis one could either

  • use whole program analysis (very expensive - not feasible for a (incremental) compiler)
  • explicit contracts via an extended type system or annotations

The second option is well explored in research and some existing tools (like the Checker Framework, JML, FindBugs) already introduce specific annotations to this end.

One could argue that advanced analysis should be left to specialized tools but having something like this in the JDT compiler should show two benefits:

  • feedback is more immediate and it is available for all JDT users without installing more software
  • analysis might be more precise than some existing tools provide, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).

A preparatory discussion of the design space can be found here: /Brainstorming.


Actual Strategy in the JDT

Disclaimer: this is work in progress. No promise is made that this particular feature will be part of any particular release of the JDT.

By default the JDT does not support inter-procedural null analysis, however, a prototype exists allowing the JDT to be configured to use annotations for extended null checking. The prototypical implementation is currently based on the OT/Equinox technology for better maintainability at this stage. This particular prototype is known to have negative impact on the compiler performance, which is however no indication about how the final in-place implementation will perform.

Specifying nullness

Null annotations in method signatures can be interpreted as null contracts, however, a more general approach considers null annotations as an extension of the type system. Eventually - that is once JSR 308 can be used - all type references should either include or exclude null, which allows for complete checking of any possible dereferencing of null. In other words, a fully annotated program which passes the type checker will never raise an NPE at runtime.

To achieve this guarantee two annotations are used. The specific annotations can be selected as a preference, but the following defaults are provided (see #Compiler configuration explained):

  • org.eclipse.jdt.annotation.NonNull
  • org.eclipse.jdt.annotation.Nullable

For any variable who's type is annotated with @NonNull (or the configured equivalent) the following rules apply:

  • It is illegal to bind null or a value that can be null to the variable. (For fields and local variables this applies to initialization and assignments, for method argument binding a value means to pass an actual argument in a method call).
  • It is legal and safe to dereference such a variable for accessing a field or a method of the bound object.

For any variable who's type is annotated with @Nullable (or the configured equivalent) the following rules apply:

  • It is legal to bind null or a value that can be null to the variable (see details above).
  • It is illegal to dereference such a variable for either field or method access.

The above rules imply that the value from a @NonNull variable can be bound to a variable annotated with @Nullable, but the opposite direction is generally illegal. Only immediately after an explicit null check can a @Nullable variable be treated as being @NonNull for the sake of binding to another @NonNull variable or for dereferencing.

Installing the prototype

The code is hosted at

svn://dev.eclipse.org/svnroot/tools/org.eclipse.objectteams/trunk/plugins/org.eclipse.objectteams.jdt.nullity

Browse it at

http://dev.eclipse.org/viewcvs/viewvc.cgi/trunk/plugins/org.eclipse.objectteams.jdt.nullity/?root=TOOLS_OBJECTTEAMS

Example usage

In order to try the prototype against any existing Java project the following steps should help (I tried it using the JDT/Core as an example):

  • Prepare the following compiler configuration
    • enable compliance 1.5 or higher
    • enable all null-related warnings
    • (disable warnings regarding generics if the project doesn't use generics — don't spam the problems view)
  • Import the example annotation types (source) from http://download.eclipse.org/objectteams/contrib/org.eclipse.jdt.annotations.zip
  • Since there isn't yet any UI for the new compiler preferences the following line should be added manually into .settings/org.eclipse.jdt.core.prefs
org.eclipse.jdt.core.compiler.annotation.nullanalysis=enabled
  • Define @NonNull as the default at the granularity of your choice (project/package/type):
    • project: add one more line to .settings/org.eclipse.jdt.core.prefs:
      org.eclipse.jdt.core.compiler.annotation.nulldefault=nonnull
    • package: add a file package-info.java with contents like this:
      @org.eclipse.jdt.annotation.NonNullByDefault package org.my.pack.age;
    • type: add @org.eclipse.jdt.annotation.NonNullByDefault to the type declaration.
  • Build (Project > Clean)
    • At this point you should see plenty of new errors and warnings
    • If problems are not reported immediately, please try a full build (if this makes a difference please drop a note in bug 186342).

Cleaning up

The sheer number of new problems may look intimidating but that's where quickfixes will come to the rescue. Currently the following problems offer a quickfix:

  • Quickfix error obj.gif Null contract violation: returning null from a method declared as @NonNull
  • Quickfix error obj.gif Null contract violation: return value can be null but method is declared as @NonNull
    Note that the mentioned @NonNull declaration may be implicit via an applicable default
    In the second case use only with care: the compiler has no clear indication if @Nullable was actually intended or not
    The fix is:
    Correction change.gif Declare method return as @Nullable
  • Quickfix error obj.gif Redundant null check: The variable foo cannot be null at this location
  • Quickfix error obj.gif Null comparison always yields false: The variable bar cannot be null at this location
    Quickfix only applies to those occurrences that concern a method parameter
    The fix is:
    Correction change.gif Declare method parameter as @Nullable
  • Quickfix error obj.gif Cannot relax null contract for method return, inherited method from T is declared as @NonNull
    Note again that the mentioned @NonNull declaration may be due to a default.
    The fix is:
    Correction change.gif Adjust overridden method from T, mark as returning @Nullable
  • Quickfix error obj.gif Cannot tighten null contract for parameter p, inherited method from T declares this parameter as @Nullable
  • Quickfix error obj.gif Cannot tighten null contract for parameter p, inherited method from T does not constrain this parameter.
    The second form occurs when no null default applies at the scope of the super method.
    The fix is:
    Correction change.gif Adjust overridden method from T, mark parameter as @NonNull

These quickfixes can be applied...

  • individually (Ctrl-1)
  • all occurrences per file (via the hover)
  • all occurrences (via context menu in the Problems view)

Note, that some quickfixes require to modify another compilation unit (file) than the one where the problem was observed. For these quickfixes the current implementation doesn't support fixing several equal issues in bulk (for the technical background see bug 337977).

Compiler configuration explained

By default the JDT does not recognize any null annotations but it can be configured to do so. For this purpose these options are proposed:

Previous versions of this proposal contained emulation and default import of annotation types, which may, however, not be supported by the final solution. See older versions of this page for details (≤ 20110226).
  1. Enable annotation based null analysis (enabled/disabled per project or workspace global).
    This is the master switch for everything described in this wiki page.
    The built-in default for this option is:
    org.eclipse.jdt.core.compiler.annotation.nullanalysis=disabled
  2. Specify the names of annotation types to be used for marking nullable vs. nonnull types as well as for defining the default per type or per package.
    Use this if you want to achieve compatibility with annotations defined by some other tool (or a future standard, should one be defined eventually).
    The built-in default values for these options are:
    org.eclipse.jdt.core.compiler.annotation.nonnull=org.eclipse.jdt.annotation.NonNull
    org.eclipse.jdt.core.compiler.annotation.nullable=org.eclipse.jdt.annotation.Nullable
    org.eclipse.jdt.core.compiler.annotation.nonnullbydefault=org.eclipse.jdt.annotation.NonNullByDefault
    org.eclipse.jdt.core.compiler.annotation.nullablebydefault=org.eclipse.jdt.annotation.NullableByDefault
  3. Define a global default for un-annotated entities (nonnull/nullable/unspecified)
    Use this if your code has a bias to either @NonNull or @Nullable. By defining a global default you save the effort of adding annotations to the majority of locations.
    There is no built-in default for this option. The option name is
    org.eclipse.jdt.core.compiler.annotation.nulldefault

It is the user's responsibility to make the required annotation types available on the build path.

Defaults at different levels

If no null annotations are used, the compiler uses the original Java semantics, where the following is legal for all variables of reference types:

  • assign null, and
  • dereference without check.

The above mentioned preference (org.eclipse.jdt.core.compiler.annotation.nulldefault) allows to globally change this so that any declaration to which no null annotation applies will be considered as either nullable or nonnull, depending on that specific setting.

For more fine-grained control two additional annotations can be used. The qualified type names of these annotations can be configured using these preferences: 

org.eclipse.jdt.core.compiler.annotation.nullablebydefault
org.eclipse.jdt.core.compiler.annotation.nonnullbydefault

The built-in values for these preferences are org.eclipse.jdt.annotation.NullableByDefault and org.eclipse.jdt.annotation.NonNullByDefault.

These annotations can be applied to any Java type or package and affect all method returns and parameters with undefined null status within their scope. (More locations will be supported in the future).

A default declared at an outer scope can be overridden by a different default at an inner scope. However, no means are provided for canceling a default (as to re-establish the original Java semantics).

Future

The following bugzillas address future improvements of the above strategy (order roughly by priority):

  • bug 334455 - UI for new preferences regarding null annotations
  • bug 337977 - Progress.gif [quick fix] Add quickfixes for null annotations
  • bug 334457 - [compiler][null] check compatibility of inherited null contracts
  • bug 331647 - Progress.gif [compiler][null] support flexible default mechanism for null-annotations
  • bug 331651 - [compiler][null] Support nullity profiles for libraries
  • bug 331649 - [compiler][null] consider null annotations for fields
Retrieved from "https://wiki.eclipse.org/index.php?title=JDT_Core/Null_Analysis&oldid=264020"

Copyright © Eclipse Foundation, Inc. All Rights Reserved.