Skip to main content
Jump to: navigation, search

JDT Core Programmer Guide/ECJ/Testing

< JDT Core Programmer Guide‎ | ECJ
Revision as of 18:57, 2 July 2020 by (Talk | contribs) (run.javac)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

For testing the compiler it is not necessary to ever run a maven build.

Running org.eclipse.jdt.core.tests.RunCompilerTests (or any of its constituents) as a JUnit (3) Plug-in Test suffices. For faster launching, open the "Main" page of the run dialog and choose "Headless Mode" as the application.

For selectively running only a single test method, or a group of test methods, see that each test class has a static block with a commented assignment to TEST_NAMES and similar. Uncomment and insert the name of the desired test and run the class. Note, that the given name will be used for prefix-matching, so tests testMethod1, ... testMethod13 can all be run by specifying "testMethod". Please try not to commit this change of TEST_NAMES.

Parameters of the compiler test suite


Specify at which compliance levels a test should be run by adding a VM argument like this (using a comma separated list of values):


Some compiler tests not only compile but also execute the compiled class files. For those ensure that the test is executed on a JRE that is the same or newer as the newest compliance level specified.

Not specifying a compliance level causes the tests to be executed for each compliance level compatible with the current running JRE, which can be very time consuming.


This special test mode is used to compare the behaviors of ecj and javac.

Typical VM arguments for this mode look like this:

-Drun.javac=enabled -Djdk.root=/home/java/jdk1.8.0_212 -Dcompliance=1.8

Dedicated Jenkins jobs exist to test in this mode for different compliance levels:

These jobs are triggered once per week (on weekends).

Each of these jobs demonstrates a number of differences (as failures) as summarized most recently in bug 404648#c121.

The general umbrella bug for differences between ecj and javac is bug 564617.

Known differences are documented using constants of type <</code> during test invocation. For conveniently passing the correct "Excuse" in each affected test, it is recommended to use class "Runner" as shown below, and its field javacTestOptions.


Flow analysis has separate implementations for the first 64 variables and subsequent ("extra") variables. Since most tests only exercise the first implementation, a mode has been added to explicitly test the second implementation (by faking 64 additional variables).


For better test coverage, this mode is enabled for one of the time-scheduled jobs: eclipse.jdt.core-run.javac-10

Since that job is known to fail, watching for regressions of this job is particularly relevant.

Authoring / editing tests

different styles of test invocation

Inside each test method of the compiler test suite, an invocation like runConformTest, runNegativeTest passes the source file and additional parameters to the actual testing infra structure.

Traditionally, a myriad of overloads of these methods has been created to accommodate the needs of different tests, which makes picking the appropriate method tedious and still uninteresting parameters have to be provided in many cases.

More recently, class org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.Runner has been created to enable providing exactly the relevant test parameters before invoking one of the run*Test() methods.

Back to the top