Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

JDT Core Programmer Guide/ECJ/Testing

< JDT Core Programmer Guide‎ | ECJ
Revision as of 14:29, 2 July 2020 by Stephan.herrmann.berlin.de (Talk | contribs) (Created page with "For testing the compiler it is '''not necessary''' to ever run a '''maven''' build. Running <code>org.eclipse.jdt.core.tests.RunCompilerTests</code> (or any of its constituen...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

For testing the compiler it is not necessary to ever run a maven build.

Running org.eclipse.jdt.core.tests.RunCompilerTests (or any of its constituents) as a JUnit (3) Plug-in Test suffices. For faster launching, open the "Main" page of the run dialog and choose "Headless Mode" as the application.


For selectively running only a single test method, or a group of test methods, see that each test class has a static block with a commented assignment to TEST_NAMES and similar. Uncomment and insert the name of the desired test and run the class. Note, that the given name will be used for prefix-matching, so tests testMethod1, ... testMethod13 can all be run by specifying "testMethod". Please try not to commit this change of TEST_NAMES.


Parameters of the compiler test suite

compliance

Specify at which compliance levels a test should be run by adding a VM argument like this (using a comma separated list of values):

-Dcompliance=1.8,14

Some compiler tests not only compile but also execute the compiled class files. For those ensure that the test is executed on a JRE that is the same or newer as the newest compliance level specified.

Not specifying a compliance level causes the tests to be executed for each compliance level compatible with the current running JRE, which can be very time consuming.

run.javac

This special test mode is used to compare the behaviors of ecj and javac.

Typical VM arguments for this mode look like this:

-Drun.javac=enabled -Djdk.root=/home/java/jdk1.8.0_212 -Dcompliance=1.8


Dedicated Jenkins jobs exist to test in this mode for different compliance levels:

These jobs are triggered once per week (on weekends).


Each of these jobs demonstrates a number of differences (as failures) as summarized most recently in bug 404648#c134.

The general umbrella bug for differences between ecj and javac is bug 564617.

Known differences are documented using constants of type <code.org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.JavacTestOptions.Excuse</code> during test invocation. For conveniently passing the correct "Excuse" in each affected test, it is recommended to use class "Runner" as shown below, and its field javacTestOptions.

jdt.flow.test.extra

Flow analysis has separate implementations for the first 64 variables and subsequent ("extra") variables. Since most tests only exercise the first implementation, a mode has been added to explicitly test the second implementation (by faking 64 additional variables).

-Djdt.flow.test.extra=true


For better test coverage, this mode is enabled for one of the time-scheduled jobs: eclipse.jdt.core-run.javac-10

Since that job is known to fail, watching for regressions of this job is particularly relevant.

Authoring / editing tests

different styles of test invocation

Inside each test method of the compiler test suite, an invocation like runConformTest, runNegativeTest passes the source file and additional parameters to the actual testing infra structure.

Traditionally, a myriad of overloads of these methods has been created to accommodate the needs of different tests, which makes picking the appropriate method tedious and still uninteresting parameters have to be provided in many cases.

More recently, class org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.Runner has been created to enable providing exactly the relevant test parameters before invoking one of the run*Test() methods.

Copyright © Eclipse Foundation, Inc. All Rights Reserved.