Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "JDT Core/Null Analysis"

(Cleaning up)
 
(35 intermediate revisions by 2 users not shown)
Line 6: Line 6:
 
div.hover {
 
div.hover {
 
     background-color:#ffffde;
 
     background-color:#ffffde;
 +
}
 +
div.hover a.image {
 +
    float: left;
 +
    margin-right: 5px;
 
}
 
}
 
div.fix {
 
div.fix {
 
     background-color:#ffffff;
 
     background-color:#ffffff;
 +
}
 +
div.fix a.image {
 +
    float: left;
 +
    margin-right: 5px;
 
}
 
}
 
</css>
 
</css>
This page discusses a proposed improvement in the static null analysis of the JDT compiler.
+
This page describes continuing work on improving the static null analysis of the JDT compiler.
 +
 
 +
The initial master bug for this work was  {{FixedBug|186342}}, this part has been released for Eclipse Juno (JDT 3.8).
 +
 
 +
Later, "type annotations" (JSR 308, part of Java 8), have been adopted for null analysis via {{FixedBug|392099}} &mdash; released with JDT's support for Java 8 and then Luna (JDT 3.10).
 +
 
 +
Support for [[/External_Annotations|external annotations]] has been added via {{FixedBug|331651}}, released with Eclipse Mars (JDT 3.11).
 +
 
 +
'''User documentation''' can meanwhile be found in the online help:
 +
* [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_annotations.htm Using null annotations] (Java &le; 1.7)
 +
* [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_type_annotations.htm Using null type annotations] (Java &ge; 1.8)
 +
* [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_external_null_annotations.htm Using external null annotations]
  
See also {{bug|186342}}.
+
When '''migrating''' from SE5-style null annotations to null type annotations in Java 8, a few [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_type_annotations.htm%23compatibility compatibility considerations] are relevant.
  
  
Line 21: Line 40:
 
null checks etc.
 
null checks etc.
  
However, the current analysis is restricted to flow analysis '''within one method'''.
+
However, the analysis in JDT &le; 3.7 is restricted to flow analysis '''within one method'''.
 
No assumptions can be made about
 
No assumptions can be made about
 
* arguments flowing into a method
 
* arguments flowing into a method
Line 39: Line 58:
 
* analysis might be more precise than some existing tools provide, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).
 
* analysis might be more precise than some existing tools provide, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).
  
A preparatory discussion of the design space can be found here: [[/Brainstorming]].
 
  
 +
<table><tr><td>
 +
[[Image:Video.png]]</td><td>See also these presentations:
 +
* recording of the [http://eclipsecon.org/europe2011/ ECE 2011] [http://eclipsecon.org/europe2011/sessions/bye-bye-npe session]: '''[http://www.fosslc.org/drupal/content/bye-bye-npe Bye, bye, NPE]'''
 +
* [https://www.eclipsecon.org/na2014 ECNA 2014] session '''[https://www.eclipsecon.org/na2014/session/jdt-embraces-type-annotations JDT Embraces Type Annotations]'''
 +
* [https://www.eclipsecon.org/europe2014/ ECE 2014] session [https://www.eclipsecon.org/europe2014/session/deep-dive-void-advanced-null-type-annotations-35-minute-standard-talk A Deep Dive into the Void - Advanced Null Type Annotations]
 +
* [https://www.eclipsecon.org/europe2016/ ECE 2016] session [https://www.eclipsecon.org/europe2016/session/end-world-we-know-it-aka-your-last-nullpointerexception-1b-bugs The End of the world as we know it - AKA your last NullPointerException $1B bugs!] by Michael Vorburger
 +
* [https://www.eclipsecon.org/europe2017/ ECE 2017] session [https://www.eclipsecon.org/europe2017/session/null-type-annotations-practice Null type annotations in practice] by Till Brychcy
 +
</td></tr></table>
  
 
==Actual Strategy in the JDT==
 
==Actual Strategy in the JDT==
  
'''Disclaimer''': this is work in progress. No promise is made that this particular feature will be part of any particular release of the JDT.
+
By default the JDT does not support inter-procedural null analysis, however, starting with 3.8 the JDT can be configured to use annotations for extended null checking.
  
By default the JDT does not support inter-procedural null analysis, however, a prototype exists allowing the JDT to be configured to use annotations for extended null checking. The prototypical implementation is currently
+
Up-to-date documentation for the annotation-based null analysis and its new configuration options can be found in the Eclipse help (Eclipse 3.8 and greater):
based on the [[:Category:OTEquinox|OT/Equinox]] technology for better maintainability at this stage. This particular prototype is known to have negative impact on the compiler ''performance'', which is however no indication about how the final in-place implementation will perform.
+
* '''Java development user guide'''
 +
** '''Reference > Preferences > Java > Compiler > Errors/Warnings'''
 +
**: scroll down to '''Null analysis''' -- [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/reference/preferences/java/compiler/ref-preferences-errors-warnings.htm read online]
 +
** '''Tasks > Improving Java code quality'''
 +
*: '''> Using null annotations'''  [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_annotations.htm?cp=1_3_9_0 read online]
 +
*: '''> Using null type annotations'''  [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_type_annotations.htm&cp=1_3_9_1 read online] (''since Luna'')
  
 
===Specifying nullness===
 
===Specifying nullness===
  
Null annotations in method signatures can be interpreted as [[/Null Contracts|null contracts]], however, a more general approach considers null annotations as an extension of the type system. Eventually - that is once JSR 308 can be used - all type references should either include or exclude null, which allows for complete checking of any possible dereferencing of null. In other words, a fully annotated program which passes the type checker will never raise an NPE at runtime.
+
Null annotations in method signatures can be interpreted as [[/Null Contracts|null contracts]], however, a more general approach considers null annotations as an extension of the type system. Since the availability of JSR 308 all type references should either include or exclude null, which allows for complete checking of any possible dereferencing of null. In other words, a fully annotated program which passes the type checker will never raise an NPE at runtime.
  
To achieve this guarantee two annotations are used. The specific annotations can be selected as a preference, but the following defaults are provided (see [[#Compiler configuration explained]]):
+
To achieve this guarantee two annotations are used. The specific annotations types can be selected as a preference, but the following defaults are provided:
* [http://dev.eclipse.org/viewcvs/viewvc.cgi/trunk/contrib/org.eclipse.objectteams.jdt.nullity.tests/annotations/org/eclipse/jdt/annotation/NonNull.java?view=markup&root=TOOLS_OBJECTTEAMS org.eclipse.jdt.annotation.NonNull]
+
* [http://help.eclipse.org/topic/org.eclipse.jdt.doc.isv/reference/api/org/eclipse/jdt/annotation/NonNull.html org.eclipse.jdt.annotation.NonNull]
* [http://dev.eclipse.org/viewcvs/viewvc.cgi/trunk/contrib/org.eclipse.objectteams.jdt.nullity.tests/annotations/org/eclipse/jdt/annotation/Nullable.java?view=markup&root=TOOLS_OBJECTTEAMS org.eclipse.jdt.annotation.Nullable]
+
* [http://help.eclipse.org/topic/org.eclipse.jdt.doc.isv/reference/api/org/eclipse/jdt/annotation/Nullable.html org.eclipse.jdt.annotation.Nullable]
  
 
For any variable who's type is annotated with @NonNull (or the configured equivalent) the following rules apply:
 
For any variable who's type is annotated with @NonNull (or the configured equivalent) the following rules apply:
Line 66: Line 97:
  
 
The above rules imply that the value from a @NonNull variable can be bound to a variable annotated with @Nullable, but the opposite direction is generally illegal.
 
The above rules imply that the value from a @NonNull variable can be bound to a variable annotated with @Nullable, but the opposite direction is generally illegal.
Only immediately after an explicit null check can a @Nullable variable be treated as being @NonNull for the sake of binding to another @NonNull variable or for dereferencing.
+
Only after an explicit null check can a @Nullable variable be treated as being @NonNull for the sake of binding to another @NonNull variable or for dereferencing. For fields the situation is actually more complex &mdash; please read [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_annotations.htm?cp=1_3_9_0_4#fields "The case of fields"].
  
 
For interaction with inheritance see [[/Null Contracts#Null Contract Inheritance|Null Contract Inheritance]].
 
For interaction with inheritance see [[/Null Contracts#Null Contract Inheritance|Null Contract Inheritance]].
  
===Installing the prototype===
+
===Usage===
 +
In order to try the new analysis against any existing Java project the following steps should help:
  
* Get an [http://download.eclipse.org/eclipse/downloads/drops/R-3.7-201106131736/index.php Eclipse SDK 3.7] or use your favorite [http://www.eclipse.org/downloads/ Indigo package].
+
* Open the compiler preferences for your project:
** ''(no backport to helios, sorry)''
+
:* Ensure compliance is 1.5 or higher
** Some of the operations below (multi quickfixes) require lots of memory, you may want to add s.t. like <code>-Xmx800m</code> already now
+
:* Find the section '''Null analysis''' and select '''Enable annotation-based null analysis'''
* Enter this update URL:
+
::[[Image:annotation-based-null-analysis.png]]
** http://download.eclipse.org/objectteams/updates/contrib
+
:* You will be prompted to update the severity of some null-related problems, this is recommended.
* Select and install these features:
+
* Apply any of the annotations <code>@NonNull</code>, <code>@Nullable</code> or <code>@NonNullByDefault</code> in your code.
** JDT Null Annotation Checker (Early Access)
+
** The annotation will be unresolvable at first, but a quick fix is offered to update the project setup:
** Object Teams Equinox Integration
+
*: (see also: [http://help.eclipse.org/topic/org.eclipse.jdt.doc.user/tasks/task-using_null_annotations.htm&cp=1_3_9_0_2&anchor=buildpath_setup Help: Setup of the build path])
 +
*:* '''Copy library with default annotations to build path''' (plain Java projects)
 +
*: For Plug-in projects it is recommended to add an optional, versioned dependency to the bundle <code>org.eclipse.jdt.annotation</code>
 +
* Define <code>@NonNull</code> as the default at the granularity of your choice (package/type):
 +
** '''package''': add a file <code>package-info.java</code> with contents like this:
 +
**: <code>@NonNullByDefault package org.my.pack.age;</code>
 +
** '''type''': add <code>@NonNullByDefault</code> to the type declaration.
 +
* At this point you should see plenty of new errors and warnings
  
The code is hosted at
+
{{tip|Hints:|Setting the default to <code>@NonNull</code> is the recommended option for new projects, but for existing projects this could require a major clean-up in terms of inserting explicit <code>@Nullable</code> annotations in many locations. Experience shows, that this is a non-trivial task since in existing code the original intention, which parameter/return value should be allowed to be null, is usually blurred. Here starting with no default but adding individual annotations where the intention is clear will cause less disruption. This incremental approach should be seen as a long-term yet low effort task, not only for getting rid of NPEs but also for sorting the responsibilities in the code, which may have eroded over time.<br>For more hints on adpoting these annotations see [[JDT Core/Null Analysis/Adopting Null Annotations|Adoption Null Annotations]].}}
: svn://dev.eclipse.org/svnroot/tools/org.eclipse.objectteams/trunk/plugins/org.eclipse.objectteams.jdt.nullity
+
 
+
Browse it at
+
: http://dev.eclipse.org/viewcvs/viewvc.cgi/trunk/plugins/org.eclipse.objectteams.jdt.nullity/?root=TOOLS_OBJECTTEAMS
+
 
+
===Example usage===
+
In order to try the prototype against any existing Java project the following steps should help (I tried it using the JDT/Core as an example):
+
 
+
* Prepare the following compiler configuration
+
** enable compliance 1.5 or higher
+
** enable all null-related warnings
+
** (disable warnings regarding generics if the project doesn't use generics &mdash; don't spam the problems view)
+
* Import the example annotation types (source) from http://download.eclipse.org/objectteams/contrib/org.eclipse.jdt.annotations.zip
+
* Since there isn't yet any UI for the new compiler preferences the following line should be added manually into <code>.settings/org.eclipse.jdt.core.prefs</code>
+
org.eclipse.jdt.core.compiler.annotation.nullanalysis=enabled
+
* Define <code>@NonNull</code> as the default at the granularity of your choice (project/package/type):
+
** '''project''': add one more line to <code>.settings/org.eclipse.jdt.core.prefs</code>:
+
**: <code>org.eclipse.jdt.core.compiler.annotation.nulldefault=nonnull</code>
+
** '''package''': add a file <code>package-info.java</code> with contents like this:
+
**: <code>@org.eclipse.jdt.annotation.NonNullByDefault package org.my.pack.age;</code>
+
** '''type''': add <code>@org.eclipse.jdt.annotation.NonNullByDefault</code> to the type declaration.
+
* Build (Project > Clean)
+
** At this point you should see plenty of new errors and warnings
+
** If problems are not reported immediately, please try a full build (if this makes a difference please drop a note in {{bug|186342}}).
+
  
 
====Cleaning up====
 
====Cleaning up====
When applying the new analysis to a big existing project, the sheer number of new problems may look intimidating but that's where quickfixes will come to the rescue.
+
When applying the new analysis to a big existing project, the sheer number of new problems may look intimidating but that's where quick fixes will come to the rescue.
 
Currently the following problems offer a quickfix:
 
Currently the following problems offer a quickfix:
  
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Type mismatch: required '@NonNull Foo' but the provided value is null</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Null type mismatch: required '@NonNull Foo' but the provided value is null</div></div>
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Type mismatch: required '@NonNull Foo' but the provided value can be null</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Null type mismatch: required '@NonNull Foo' but the provided value is specified as @Nullable</div></div> ''not in JDT 3.8.0 - see {{bug|337977#c19}}
* <div class="b"><div class="hover">[[Image:Quickfix_warning_obj.gif]] Potential type mismatch: required '@NonNull Foo' but nullness of the provided value is unknown</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Null type mismatch: required '@NonNull Foo' but the provided value is inferred as @Nullable</div></div>
 +
* <div class="b"><div class="hover">[[Image:Quickfix_warning_obj.gif]] Null type safety: The expression of type Foo needs unchecked conversion to conform to '@NonNull Foo'</div></div>
 
*: '''Fixable for these locations: return statements''':
 
*: '''Fixable for these locations: return statements''':
 
*: Note that the mentioned @NonNull declaration may be implicit via an applicable default
 
*: Note that the mentioned @NonNull declaration may be implicit via an applicable default
*: In cases 2) and 3) use only with care: the compiler has no clear indication if @Nullable was actually intended or not
+
*: In cases 3) and 4) use only with care: the compiler has no clear indication if @Nullable was actually intended or not
 
*: The fix is:
 
*: The fix is:
 
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method return as @Nullable</div></div>
 
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method return as @Nullable</div></div>
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Redundant null check: The variable foo cannot be null at this location</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Null comparison always yields false: The variable x is specified as @NonNull</div></div>
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Null comparison always yields false: The variable bar cannot be null at this location</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Redundant null check: The variable x is specified as @NonNull</div></div>
*: '''Fixable for these locations: read access to a method parameter'''
+
*: '''Fixable for these locations: null check for a method parameter'''
 
*: The fix is:
 
*: The fix is:
 
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method parameter as @Nullable</div></div>
 
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method parameter as @Nullable</div></div>
Line 129: Line 145:
 
*: Note again that the mentioned @NonNull declaration may be due to a default.
 
*: Note again that the mentioned @NonNull declaration may be due to a default.
 
*: Possible fixes are:
 
*: Possible fixes are:
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method return as @NonNull</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change return type of foo(..) to '@NonNull'</div></div>
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Adjust overridden method from SuperFoo, mark as returning @Nullable</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change return type of overridden foo(..) to '@Nullable'</div></div>
 
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Illegal redefinition of parameter a, inherited method from SuperFoo declares this parameter as @Nullable</div></div>
 
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Illegal redefinition of parameter a, inherited method from SuperFoo declares this parameter as @Nullable</div></div>
 
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Illegal redefinition of parameter a, inherited method from SuperFoo does not constrain this parameter.</div></div>
 
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Illegal redefinition of parameter a, inherited method from SuperFoo does not constrain this parameter.</div></div>
Line 136: Line 152:
 
*: The second form occurs when no null default applies at the scope of the super method.
 
*: The second form occurs when no null default applies at the scope of the super method.
 
*: Possible fixes are:
 
*: Possible fixes are:
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method parameter as @Nullable</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change parameter type to '@Nullable'</div></div>
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Adjust overridden method from SuperFoo, mark parameter as @NonNull</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change parameter type in overridden 'foo(..)' to '@NonNull'</div></div>
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Missing null annotation: inherited method from Testing declares this parameter as @NonNull</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Missing non-null annotation: inherited method from SuperClass declares this parameter as @NonNull</div></div>
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Missing null annotation: inherited method from Testing declares this parameter as @Nullable</div></div>
+
* <div class="b"><div class="hover">[[Image:Quickfix_error_obj.gif]] Missing nullable annotation: inherited method from SuperClass declares this parameter as @Nullable</div></div>
 
*: '''Location: Parameter declaration of an overriding method'''
 
*: '''Location: Parameter declaration of an overriding method'''
*: Quickfix is either of:
+
*: Quick fix is either of:
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method parameter as @NonNull</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change parameter type to @NonNull</div></div>
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Declare method parameter as @Nullable</div></div>
+
*: <div class="b"><div class="fix">[[Image:Correction_change.gif]] Change parameter type to @Nullable</div></div>
  
These quickfixes can be applied...
+
These quick fixes can be applied...
 
* individually (Ctrl-1)
 
* individually (Ctrl-1)
 
* all occurrences per file (via the hover)
 
* all occurrences per file (via the hover)
 
* all occurrences (via context menu in the Problems view)
 
* all occurrences (via context menu in the Problems view)
Note, that some quickfixes require to modify another compilation unit (file) than the one
+
Note, that some quick fixes require to modify another compilation unit (file) than the one
 
where the problem was observed. For these quickfixes the current implementation doesn't
 
where the problem was observed. For these quickfixes the current implementation doesn't
 
support fixing several equal issues in bulk (for the technical background see
 
support fixing several equal issues in bulk (for the technical background see
 
{{bug|337977}}).
 
{{bug|337977}}).
 
===Compiler configuration explained===
 
By default the JDT does not recognize any null annotations but it can be configured to do so.
 
For this purpose these options are proposed:
 
::''Previous versions of this proposal contained ''emulation'' and ''default import'' of annotation types, which may, however, not be supported by the final solution. See older versions of this page for details (&le; 20110226).''
 
# '''Enable annotation based null analysis''' (enabled/disabled per project or workspace global).
 
#: ''This is the master switch for everything described in this wiki page''.
 
#: The built-in default for this option is:
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nullanalysis=disabled</code>
 
# Specify the '''names of annotation types''' to be used for marking nullable vs. nonnull types as well as for defining the default per type or per package.
 
#: ''Use this if you want to achieve compatibility with annotations defined by some other tool (or a future standard, should one be defined eventually).''
 
#: The built-in default values for these options are:
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nonnull=org.eclipse.jdt.annotation.NonNull</code>
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nullable=org.eclipse.jdt.annotation.Nullable</code>
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nonnullbydefault=org.eclipse.jdt.annotation.NonNullByDefault</code>
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nullablebydefault=org.eclipse.jdt.annotation.NullableByDefault</code>
 
# Define a '''global default''' for un-annotated entities (nonnull/nullable/''unspecified'')
 
#: ''Use this if your code has a bias to either @NonNull or @Nullable. By defining a global default you save the effort of adding annotations to the majority of locations.''
 
#: There is no built-in default for this option. The option name is
 
#:: <code>org.eclipse.jdt.core.compiler.annotation.nulldefault</code>
 
 
It is the user's responsibility to make the required annotation types available on the build path.
 
  
 
===Defaults at different levels===
 
===Defaults at different levels===
Line 182: Line 176:
 
* dereference without check.
 
* dereference without check.
  
The above mentioned preference (<code>org.eclipse.jdt.core.compiler.annotation.nulldefault</code>) allows to globally change this so that any declaration
+
To generally avoid these weak semantics you may want to declare that by default all types should be considered as nonnull.
to which no null annotation applies will be considered as either nullable or nonnull, depending on that specific setting.
+
  
For more fine-grained control two additional annotations can be used. The qualified type
+
This is done using the annotation 'NonNullByDefault'. The qualified type
names of these annotations can be configured using these preferences:
+
name of this annotation can be configured using the preference "'NonNullByDefault' annotation".
org.eclipse.jdt.core.compiler.annotation.nullablebydefault
+
The built-in value for these preference is <code>org.eclipse.jdt.annotation.NonNullByDefault</code>.
org.eclipse.jdt.core.compiler.annotation.nonnullbydefault
+
  
The built-in values for these preferences are <code>org.eclipse.jdt.annotation.NullableByDefault</code> and <code>org.eclipse.jdt.annotation.NonNullByDefault</code>.
+
* This annotation takes an optional parameter that can be used to ''cancel'' a default that may possible apply at the current location. This is useful when, e.g., sub-classing a legacy class without null annotation, where the sub-class sits in a place that would otherwise apply non-null as the default, which would make all overrides incompatible with inherited methods.
 +
** When using version 1.x of <code>org.eclipse.jdt.annotation</code> specify '''<code>false</code>''' as the annotation argument.
 +
** When using version 2.x of <code>org.eclipse.jdt.annotation</code> specify '''<code>{}</code>''' as the annotation argument.
  
These annotations can be applied to any Java type or package and affect all
+
This annotation can be applied to any package, Java type or method, and has the following effect:
method returns and parameters with undefined null status within their scope. (More locations will be supported in the future).
+
;Java &le; 1.7: The annotation affects all  method returns and parameters with undefined null status within their scope.
 +
:Starting with Eclipse Kepler, also fields are affected (see {{FixedBug|331649}}).
 +
;Java &ge; 1.8: Many more [http://help.eclipse.org/topic/org.eclipse.jdt.doc.isv/reference/api/org/eclipse/jdt/annotation/DefaultLocation.html locations] are affected, but local variables are intentionally unaffected by any default.
  
A default declared at an outer scope can be overridden by a different default at an
+
===External Annotations===
inner scope. However, no means are provided for canceling a default (as to re-establish
+
To close the gap of 3rd party libraries without formally defined null contracts, starting with Eclipse Mars JDT supports the concept of [[/External_Annotations|external annotations]].
the original Java semantics).
+
 
+
===Future===
+
The following bugzillas address future improvements of the above strategy (order roughly by priority):
+
* {{bug|334455}} - UI for new preferences regarding null annotations
+
* {{bug|337977}} - [[Image:Progress.gif]] [quick fix] Add quickfixes for null annotations
+
* {{bug|334457}} - [compiler][null] check compatibility of inherited null contracts
+
* {{bug|331647}} - [[Image:Progress.gif]] [compiler][null] support flexible default mechanism for null-annotations
+
* {{bug|331651}} - [compiler][null] Support nullity profiles for libraries
+
* {{bug|331649}} - [compiler][null] consider null annotations for fields
+
  
 +
==Status==
 +
 +
===Done===
 +
At the current point the following bugs are resolved:
 +
 +
'''Since 3.8 (Juno):'''
 +
* {{FixedBug|186342}}.- [[Image:Ok_green.gif]] [compiler][null] Using annotations for null checking
 +
* {{FixedBug|334455}} - [[Image:Ok_green.gif]] UI for new preferences regarding null annotations (plus a dup: {{FixedBug|364815}}).
 +
* {{FixedBug|334457}} - [[Image:Ok_green.gif]] [compiler][null] check compatibility of inherited null contracts
 +
* {{FixedBug|331647}} - [[Image:Ok_green.gif]] [compiler][null] support flexible default mechanism for null-annotations
 +
* {{FixedBug|365208}} - [[Image:Ok_green.gif]] [compiler][batch] command line options for annotation based null analysis
 +
'''Since 4.3 (Kepler):'''
 +
* {{FixedBug|331649}} - [[Image:Ok_green.gif]] [compiler][null] consider null annotations for fields
 +
* {{FixedBug|383368}} - [[Image:Ok_green.gif]] [compiler][null] syntactic null analysis for field references
 +
* {{FixedBug|337977}} - [[Image:Ok_green.gif]] [quick fix] Add quickfixes for null annotations
 +
* {{FixedBug|388281}} - [[Image:Ok_green.gif]] [compiler][null] inheritance of null annotations as an option
 +
'''Since 4.4 (Luna):'''
 +
* {{FixedBug|392099}} - [[Image:Ok_green.gif]] [1.8][compiler][null] Apply null annotation on types for null analysis - ''requires Java 8''
 +
'''Since 4.5 (Mars):'''
 +
* {{FixedBug|331651}} - [[Image:Ok_green.gif]] [compiler][null] Support external null annotations for libraries - see [[/External_Annotations]]
 +
'''Since 4.8 (Photon):'''
 +
* {{bug|507109}} - [[Image:Ok_green.gif]] [9] Consider @NonNullByDefault in module-info
 +
* {{bug|518839}} - [[Image:Ok_green.gif]] [null] Null-checks should honor @TypeQualifierDefault to reduce the scope of what's checked
 +
 +
===Future===
 +
The following bugzillas address future improvements of the above strategy:
 +
* {{bug|414237}} - [[Image:Glass.gif]] [compiler][null] Support a @LazyNonNull annotation for fields
 
[[Category:JDT]]
 
[[Category:JDT]]

Latest revision as of 12:37, 10 March 2018

This page describes continuing work on improving the static null analysis of the JDT compiler.

The initial master bug for this work was bug 186342, this part has been released for Eclipse Juno (JDT 3.8).

Later, "type annotations" (JSR 308, part of Java 8), have been adopted for null analysis via bug 392099 — released with JDT's support for Java 8 and then Luna (JDT 3.10).

Support for external annotations has been added via bug 331651, released with Eclipse Mars (JDT 3.11).

User documentation can meanwhile be found in the online help:

When migrating from SE5-style null annotations to null type annotations in Java 8, a few compatibility considerations are relevant.


Introduction

The static analysis of the JDT compiler detects many potential programming problems related to the null-ness of variables: dereferencing a null value (-> NPE), redundant null checks etc.

However, the analysis in JDT ≤ 3.7 is restricted to flow analysis within one method. No assumptions can be made about

  • arguments flowing into a method
  • return values from method calls and
  • field reads.

In order to include these elements in the analysis one could either

  • use whole program analysis (very expensive - not feasible for a (incremental) compiler)
  • explicit contracts via an extended type system or annotations

The second option is well explored in research and some existing tools (like the Checker Framework, JML, FindBugs) already introduce specific annotations to this end.

One could argue that advanced analysis should be left to specialized tools but having something like this in the JDT compiler should show two benefits:

  • feedback is more immediate and it is available for all JDT users without installing more software
  • analysis might be more precise than some existing tools provide, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).


Video.pngSee also these presentations:

Actual Strategy in the JDT

By default the JDT does not support inter-procedural null analysis, however, starting with 3.8 the JDT can be configured to use annotations for extended null checking.

Up-to-date documentation for the annotation-based null analysis and its new configuration options can be found in the Eclipse help (Eclipse 3.8 and greater):

  • Java development user guide
    • Reference > Preferences > Java > Compiler > Errors/Warnings
      scroll down to Null analysis -- read online
    • Tasks > Improving Java code quality
    > Using null annotations read online
    > Using null type annotations read online (since Luna)

Specifying nullness

Null annotations in method signatures can be interpreted as null contracts, however, a more general approach considers null annotations as an extension of the type system. Since the availability of JSR 308 all type references should either include or exclude null, which allows for complete checking of any possible dereferencing of null. In other words, a fully annotated program which passes the type checker will never raise an NPE at runtime.

To achieve this guarantee two annotations are used. The specific annotations types can be selected as a preference, but the following defaults are provided:

For any variable who's type is annotated with @NonNull (or the configured equivalent) the following rules apply:

  • It is illegal to bind null or a value that can be null to the variable. (For fields and local variables this applies to initialization and assignments, for method argument binding a value means to pass an actual argument in a method call).
  • It is legal and safe to dereference such a variable for accessing a field or a method of the bound object.

For any variable who's type is annotated with @Nullable (or the configured equivalent) the following rules apply:

  • It is legal to bind null or a value that can be null to the variable (see details above).
  • It is illegal to dereference such a variable for either field or method access.

The above rules imply that the value from a @NonNull variable can be bound to a variable annotated with @Nullable, but the opposite direction is generally illegal. Only after an explicit null check can a @Nullable variable be treated as being @NonNull for the sake of binding to another @NonNull variable or for dereferencing. For fields the situation is actually more complex — please read "The case of fields".

For interaction with inheritance see Null Contract Inheritance.

Usage

In order to try the new analysis against any existing Java project the following steps should help:

  • Open the compiler preferences for your project:
  • Ensure compliance is 1.5 or higher
  • Find the section Null analysis and select Enable annotation-based null analysis
Annotation-based-null-analysis.png
  • You will be prompted to update the severity of some null-related problems, this is recommended.
  • Apply any of the annotations @NonNull, @Nullable or @NonNullByDefault in your code.
    • The annotation will be unresolvable at first, but a quick fix is offered to update the project setup:
    (see also: Help: Setup of the build path)
    • Copy library with default annotations to build path (plain Java projects)
    For Plug-in projects it is recommended to add an optional, versioned dependency to the bundle org.eclipse.jdt.annotation
  • Define @NonNull as the default at the granularity of your choice (package/type):
    • package: add a file package-info.java with contents like this:
      @NonNullByDefault package org.my.pack.age;
    • type: add @NonNullByDefault to the type declaration.
  • At this point you should see plenty of new errors and warnings
Idea.png
Hints:
Setting the default to @NonNull is the recommended option for new projects, but for existing projects this could require a major clean-up in terms of inserting explicit @Nullable annotations in many locations. Experience shows, that this is a non-trivial task since in existing code the original intention, which parameter/return value should be allowed to be null, is usually blurred. Here starting with no default but adding individual annotations where the intention is clear will cause less disruption. This incremental approach should be seen as a long-term yet low effort task, not only for getting rid of NPEs but also for sorting the responsibilities in the code, which may have eroded over time.
For more hints on adpoting these annotations see Adoption Null Annotations.


Cleaning up

When applying the new analysis to a big existing project, the sheer number of new problems may look intimidating but that's where quick fixes will come to the rescue. Currently the following problems offer a quickfix:

  • Quickfix error obj.gif Null type mismatch: required '@NonNull Foo' but the provided value is null
  • Quickfix error obj.gif Null type mismatch: required '@NonNull Foo' but the provided value is specified as @Nullable
    not in JDT 3.8.0 - see bug 337977#c19
  • Quickfix error obj.gif Null type mismatch: required '@NonNull Foo' but the provided value is inferred as @Nullable
  • Quickfix warning obj.gif Null type safety: The expression of type Foo needs unchecked conversion to conform to '@NonNull Foo'
    Fixable for these locations: return statements:
    Note that the mentioned @NonNull declaration may be implicit via an applicable default
    In cases 3) and 4) use only with care: the compiler has no clear indication if @Nullable was actually intended or not
    The fix is:
    Correction change.gif Declare method return as @Nullable
  • Quickfix error obj.gif Null comparison always yields false: The variable x is specified as @NonNull
  • Quickfix error obj.gif Redundant null check: The variable x is specified as @NonNull
    Fixable for these locations: null check for a method parameter
    The fix is:
    Correction change.gif Declare method parameter as @Nullable
    Otherwise a null check may indeed be unnecessary and should be deleted.
  • Quickfix error obj.gif The return type is incompatible with the @NonNull return from SuperFoo.foo()
    Location: declaration of an overriding method
    Note again that the mentioned @NonNull declaration may be due to a default.
    Possible fixes are:
    Correction change.gif Change return type of foo(..) to '@NonNull'
    Correction change.gif Change return type of overridden foo(..) to '@Nullable'
  • Quickfix error obj.gif Illegal redefinition of parameter a, inherited method from SuperFoo declares this parameter as @Nullable
  • Quickfix error obj.gif Illegal redefinition of parameter a, inherited method from SuperFoo does not constrain this parameter.
    Location: Parameter declaration of an overriding method
    The second form occurs when no null default applies at the scope of the super method.
    Possible fixes are:
    Correction change.gif Change parameter type to '@Nullable'
    Correction change.gif Change parameter type in overridden 'foo(..)' to '@NonNull'
  • Quickfix error obj.gif Missing non-null annotation: inherited method from SuperClass declares this parameter as @NonNull
  • Quickfix error obj.gif Missing nullable annotation: inherited method from SuperClass declares this parameter as @Nullable
    Location: Parameter declaration of an overriding method
    Quick fix is either of:
    Correction change.gif Change parameter type to @NonNull
    Correction change.gif Change parameter type to @Nullable

These quick fixes can be applied...

  • individually (Ctrl-1)
  • all occurrences per file (via the hover)
  • all occurrences (via context menu in the Problems view)

Note, that some quick fixes require to modify another compilation unit (file) than the one where the problem was observed. For these quickfixes the current implementation doesn't support fixing several equal issues in bulk (for the technical background see bug 337977).

Defaults at different levels

If no null annotations are used, the compiler uses the original Java semantics, where the following is legal for all variables of reference types:

  • assign null, and
  • dereference without check.

To generally avoid these weak semantics you may want to declare that by default all types should be considered as nonnull.

This is done using the annotation 'NonNullByDefault'. The qualified type name of this annotation can be configured using the preference "'NonNullByDefault' annotation". The built-in value for these preference is org.eclipse.jdt.annotation.NonNullByDefault.

  • This annotation takes an optional parameter that can be used to cancel a default that may possible apply at the current location. This is useful when, e.g., sub-classing a legacy class without null annotation, where the sub-class sits in a place that would otherwise apply non-null as the default, which would make all overrides incompatible with inherited methods.
    • When using version 1.x of org.eclipse.jdt.annotation specify false as the annotation argument.
    • When using version 2.x of org.eclipse.jdt.annotation specify {} as the annotation argument.

This annotation can be applied to any package, Java type or method, and has the following effect:

Java ≤ 1.7
The annotation affects all method returns and parameters with undefined null status within their scope.
Starting with Eclipse Kepler, also fields are affected (see bug 331649).
Java ≥ 1.8
Many more locations are affected, but local variables are intentionally unaffected by any default.

External Annotations

To close the gap of 3rd party libraries without formally defined null contracts, starting with Eclipse Mars JDT supports the concept of external annotations.

Status

Done

At the current point the following bugs are resolved:

Since 3.8 (Juno):

  • bug 186342.- Ok green.gif [compiler][null] Using annotations for null checking
  • bug 334455 - Ok green.gif UI for new preferences regarding null annotations (plus a dup: bug 364815).
  • bug 334457 - Ok green.gif [compiler][null] check compatibility of inherited null contracts
  • bug 331647 - Ok green.gif [compiler][null] support flexible default mechanism for null-annotations
  • bug 365208 - Ok green.gif [compiler][batch] command line options for annotation based null analysis

Since 4.3 (Kepler):

  • bug 331649 - Ok green.gif [compiler][null] consider null annotations for fields
  • bug 383368 - Ok green.gif [compiler][null] syntactic null analysis for field references
  • bug 337977 - Ok green.gif [quick fix] Add quickfixes for null annotations
  • bug 388281 - Ok green.gif [compiler][null] inheritance of null annotations as an option

Since 4.4 (Luna):

  • bug 392099 - Ok green.gif [1.8][compiler][null] Apply null annotation on types for null analysis - requires Java 8

Since 4.5 (Mars):

Since 4.8 (Photon):

  • bug 507109 - Ok green.gif [9] Consider @NonNullByDefault in module-info
  • bug 518839 - Ok green.gif [null] Null-checks should honor @TypeQualifierDefault to reduce the scope of what's checked

Future

The following bugzillas address future improvements of the above strategy:

  • bug 414237 - Glass.gif [compiler][null] Support a @LazyNonNull annotation for fields

Back to the top