Revision as of 11:28, 10 August 2010

Eclipse SDK Test Results as part of eclipse-build

Using the Linux Tools project's eclipse-build output, it is possible to run the SDK JUnit tests with ant runTests. You can pass in -DdebugTests=true or -DverboseTests=true (or both) for help with debugging test failures.

Requirements for running the tests

• Xvnc (it will default to your main X display if this isn't found at /usr/bin/Xvnc or /usr/X11/bin/Xvnc)
• a functioning eclipse-build
• bash, sed, etc.

Tracking down the root cause of the failure

There are many, many ways that a test could be failing. The first step is to check the test plugin out from CVS (:pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse is a good place to look. /cvsroot/rt is also a good place to look.). Make sure you have the same version checked out as the one that failed (ex. R3_5_1). You'll probably also need to check out org.eclipse.test and org.eclipse.test.performance so get the properly-tagged version of them as well.

You may need to set up an Execution Environment (EE) or two. You can press Ctrl-3 and type execution env and you should see the option to select the Execution Environments preference. Select it and ensure you have proper EEs set up for the bundles you're trying to use. A common trick is to make, say, JavaSE-1.6 prentend to be J2SE-1.4. This isn't the best move from a preventing-coding-errors standpoint, but it will probably do for debugging tests.

Once everything is compiled (you have no errors in your Problems view), try running the tests by right-clicking on the test plugin and selecting Run -> Run As -> JUnit Plug-In Test. If the tests bring up a workbench window, don't touch it in case your doing so affects the outcome of the test run. You should see the JUnit view come up and the results should start filling themselves in. If the error happens here, you can debug it directly. Good luck :) If the tests do not fail when run from your workbench, it will be more difficult to track down the cause of the failure. You can try inspecting the code around the failure location to see if you can spot something that may be sensitive to the location from which it is run. You can set breakpoints and step through the code just like any other plugin.

Running tests from eclipse-build

./runtests.sh is the script that runs the tests. You can modify it to only run one or more sets. It will output the VNC display number it is using. You can connect to this display to watch what's going on with vncviewer :${display number given in output of script} and enter the password VNCpassword1 (secure, I know).

Modifying Test Sources And Re-running Tests

First let's suppose that you've already run the build script and have run all test cases. Now you've found some tests that are failing. You've fixed them, from a checked out version but you'd like to modify the source, build them, and run all the Eclipse SDK tests again to confirm that your changes do not break anything else. Here's a quick way to do this.

If the changes you made were from a checked out version, then you'll need to replicate your changes in the actual eclipse-build tests source at :

${basedir}/testsBuild/eclipse-sdktests-${buildTag}-fetched-src/plugins/ThePluginYouWantToModify/

Now run : 'ant clean' in the directory (there should be a build.xml file there)

Now run : 'eclipse -application org.eclipse.ant.core.antRunner'

This should have generated a new jar file in the directory.

Copy this file and make it overwrite the older jar found in :

${basedir}/build/eclipse-sdktests-${buildTag}-fetched-src/testsInstallation/eclipse/ThePluginYouWantToModify/

Now you can go to the ${basedir} and run the tests.

Test runs where there are failures or errors

See [eclipse-build\%20\[tests\];bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;short_desc_type=allwordssubstr;component=PackagingTools;classification=Technology;product=Linux%20Tools this Bugzilla query] for bugs tracking failures and/or errors.