JDT Core/Null Analysis/Beta

Obsolete content
This pages captures work that was in progress during 2012-2013, which is obsolete by now

After basic support for annotation-based analysis by the JDT compiler has been released for Eclipse Juno / JDT 3.8, more advanced functionality is currently in an experimental state.

On this page we provide early access to a variant of the JDT (JDT/Core and JDT/UI plus annotations) that includes:

• Support for null annotations for fields
• A new quick fix for resolving newly reported problems
• Small fixes for late detected issues in existing quick fixes for null-related problems

We publish this variant with the intention to gather community feedback regarding a specific solution -- please see the questions below.

Installation

• You need Eclipse Juno or newer using any package that includes the JDT.
• Next add the following software site for installation:

  Juno GA

  http://www.eclipse.org/jdt/core/beta-null-annotations-for-fields

  Juno SR1

  http://www.eclipse.org/jdt/core/beta-null-annotations-for-fields/R4.2.1

  Kepler Milestones

  http://www.eclipse.org/jdt/core/beta-null-annotations-for-fields/4.3milestones

• Select and install the feature JDT patch for null annotations for fields (early access)

Compatibility

This feature updates the JDT to a variant that is compatibly with the JDT release 3.8, adding the functionality explained below.

Note, that from an installation point of view, this feature will be incompatible with future builds in the Kepler development stream. I.e., before upgrading to a future build you may have to uninstall this experimental feature, first.

We will, however, provide updates for this feature to match the Juno service releases SR1 and SR2.

Code written with this experimental version will also compile OK with the JDT 3.8 and any future versions with the following caveats:

• as soon as null annotations are used for fields a corresponding version of 'org.eclipse.jdt.annotation' must be used, to ensure that these annotations are indeed applicable to fields.
• the option to tolerate direct nullcheck-dereference sequences (see below: #Syntactic analysis) may perhaps be withdrawn from future releases.

Other than that any code that passes compilation with the experimental version should also pass compilation with any official release of the JDT (≥ 3.8).

Null Annotations for Fields

Background

During the Juno development cycle, work was started to include fields into the annotation-based null analysis. However, discussions showed that different people are expecting different behavior from the null analysis. At JDT_Core/Null_Analysis/Options an attempt was made to summarize the core difficulties.

In a nutshell this discussion boils down to the following conflicting options:

1. Do we want null analysis to be strict so that in a fully annotated program no potential null pointer access can go undetected?
2. Do we want null analysis to be more lenient to silently accept programs that are safe under specific assumptions that, however, cannot be verified by the null analysis?

Option 1 is attractive as a long term goal since this will enable us to detect not only some more, but all potential NPEs already during compilation.

Option 2 is attractive because it results in fewer errors/warnings to be acted upon, hoping that the remaining errors/warnings correspond to the most likely causes for null pointer access at runtime.

Option 1 represents a "pessimistic" viewpoint, considering any field dereference as dangerous unless proven to be safe.

Option 2 represents a more "optimistic" viewpoint, suppressing some errors/warnings that are considered less likely causes for null pointer access at runtime.

Experimental Solution

The implementation published here extends the existing null analysis for the "pessimistic" viewpoint mentioned above:

• Annotations @NonNull and @Nullable are supported for fields, too.
• No flow analysis is supported for fields, i.e., each access to a field is analysed based only on the field's annotation, regardless of any previous assignments or null checks.

More speficically the following checks are performed:

@NonNull fields

• Each non-null field must be properly initialized to a non-null value:
  • member fields must be definitely initialized in one of
    • field initialization
    • constructors, or
    • an initializer block
  • static fields must be definitely initialized in one of
    • field initialization
    • a static initializer block
• Each assignment to a non-null field must assign a value that is known to be non-null, either based on a null annotation of the source variable or based on flow analysis.

@Nullable fields

• Each dereference of a nullable field is flagged as an error or warning.

Obviously, this makes it difficult to work with a nullable field, but the experimental implementation supports two options:

• the analysis can be configured for tolerating the dereference of a nullable field if it directly follows a null-check for the same field, i.e., no code what so ever is tolerated between the check and the dereference. See "syntactic analysis" below.
• a quick fix is offered for resolving the problem using a new local variable:
  • the field's value is assinged to a new local variable
  • the local variable is checked for null
  • the dereference is performed using the local variable, guarded by the null-check.

The quick fix transforms this code:

class C {
    @Nullable Exception e;
    void printEx() {
        e.printStackTrace();
    }
}

into:

class C {
    @Nullable Exception e;
    void printEx() {
        final Exception localE = e;
        if (localE != null) {
            localE.printStackTrace();
        } else {
            // TODO handle null value
        }
    }
}

The advantage of this solution lies in the fact that it is fully safe against any of the dangers discussed: neither side effects, nor aliasing, nor concurrency can cause a null pointer access, even with arbitrary code between the null check and the dereference.

Effectively non-null fields

Any final field that is directly initialized to a value that is statically known to be non-null is considered as effectively non-null, even if no @NonNull annotation is attached.

This feature is suspended both in the beta featurea as well as in recent builds towards the Kepler release. See bug 237236 and linked bugs for background discussion.

Syntactic analysis

While the compiler performs no flow analysis for fields, it can be configured to consider some constellations as safe, which are matched just by their syntactic structure.

• This feature can be enabled using the compiler preference

[ ] Enable syntactic null analysis for fields (sub-option of "Enable annotation-based null analysis")

With this option enabled, the compiler will record any event where a field is either checked for non-null or assigned with a non-null value. The code immediatly following such protection will be allowed to dereference the field without triggering an error/warning, although theoretically concurrency could still invalidate the protection. The protection ends after the next statement, any method call or one of a few other events. In this analysis two field references are considered to be equivalent if their (qualified) names match.

class SyntacticAnalysis {
 
   @Nullable Exception e;
 
   void printException1() {
      if (e != null)
         e.printStackTrace();   // tolerated with no error/warning if this option is enabled
   }
   void printException2() {
      e = new Exception();
      e.printStackTrace();      // tolerated with no error/warning if this option is enabled
   }
   @NonNull Exception getException() {
      if (e != null)
         return e;              // tolerated with no error/warning if this option is enabled
      else
         return new Exception("new exception");
   }
   void printException3() {
      if (e != null) {
           cleanUp();           // could possibly set e to null
           e.printStackTrace(); // flagged as potential null pointer access
      }
   }
}

It is important to note that this analysis can only detect the most obvious cases, no algebra is applied to the nullness information, i.e., if a pattern isn't obviously safe (modulo concurrency) it is not considered by the analysis and the analysis falls back to relying solely on the declared annotation.

The solution using local variables is stronger than syntactic analysis in several regards:

• it is safe even under concurrency.
• it doesn't suffer from the tight constraints of the syntactic analysis, since local variables can be well analysed, essentially independent of code style.

Questions

We solicit feedback from early adopters regarding the following questions:

• Does initialization for non-null fields incur undue problems?
  • E.g.: does your code require to initialize such a field in a non-constructor method invoked during object initialization?
• Does the introduction of local variables incur overheads that seem unnecessary?
• Do you use any specific coding patterns that you feel to be safe, but the compiler doesn't accept?
• Would it help if the quick fix could also be used to re-use the same local variable for multiple statements dereferencing the same field value?

For any answers to the above, we'd be interested also to hear about how frequently a particular situation occurs in your code.

Please submit any feed back as comments in bug 383371.

Based on this feedback we will discuss to what extent we will add more analysis and heuristics to suppress errors/warnings that users feel to be distracting. Yet, the analysis implemented at this stage is intended to remain as part of any future solution (no guarantee implied, though).

Observations collected so far

Initialization of @NonNull fields

• Observation 1: some frameworks require a no-arg constructor for reflective instantiation and assign required fields immediately after and before passing the object into client code.
  • Proposal 1.a: it has been proposed to introduce @SuppressWarnings("initialization") to silence errors/warnings relating to the above.
  • Proposal 1.b: the analysis could be told about some annotations like @Inject which cause the framework to fill in valid values.
  • Both proposals are cheap implementation-wise.
• Observation 2: in some situations we are uncertain about initialization but want to enforce that after initialization a field may never again become null.
  • Proposal 2: support an annotation @LazyNonNull (aka. @EventuallyNonNull, @MonoNonNull (for monotonous)). This annotation would prohibit assigning null or potentially null values to the field (which is an interesting assertion to be checked). Thus it would allow flow analysis for these fields with the same strength as for local variables.
  • The essence of this proposal is backed by several scientific publications, but it is not cheap implementation-wise and needs further clarification regarding supported kinds of field references (cf. what kinds of field references to include?).