WTP/Build/WTP Build Mechanics and details

WTP Build Mechanics

How to release for a build

The platform's releng tool should be used to "release" projects to the map files. Be sure to have the appropriate branch of releng project (which has the map files) loaded in your workspace. R1_0_maintenance for 1.0.x builds, R1_5_maintenance for 1.5.x builds, R2_0_maintenance for 2.0.x builds, R3_0_maintenance for 3.0.x builds and HEAD for 3.1 builds.

As projects are versioned, please use the "standard" format, in UTC time, following vYYYYMMDDHHMM. This is important as these cvs tags become the qualifier field of the plugin's version.

Note: do NOT use underscores in the CVS version, as there are some issues with Eclispe tooling when it finds an underscore in a plugin or version qualifier, see [89428].

The build files are in a CVS project called

    /releng.control

The idea is that anything that is "machine specific" or "cruise control" specific is in releng.control. The CVS version should be in synch with the eclipse.org PPC machine, but other machines can be kept in "parallel" files, e.g. see

cruise-lomboz.properties 

and rename them when deployed.

Please note, one implication is that "releng.builder" isn't downloaded and checked automatically by CruiseControl, in other words, changes in it won't trigger a build. So tools such as API checkers will have to be downloaded or triggered by some other means.

The "whatisfixed" tools are off by default because it takes almost as long as the build itself.

How to watch the builds

Do you enjoy watching screens of console logs fly by as much as I? Waiting for that joyous final result? Well then you too can do that with the following information.

We are currently doing our builds on build.eclipse.org. Any committer can logon to that server using their committer ID and password. I use something like

  ssh david_williams@build.eclipse.org

Once there the main work we do is in /shared/webtools/. If you want to "watch" the console log while the compiiler and ant files are running, you can use

   tail -F /shared/webtools/releng.control/out.txt -n200

to watch the last part of the main log (during compile, etc.). This is the fastest way to find out if the build failed. Once it does fail, there's some diagnostic messages that might be informative in the last 20 or 50 lines. (Do be careful, though, the build always says "Build Successful" as its very very last message, and that message just means all the ant scripts were any to completely execute (not that they did so successfully).

During tests, you can watch test results console at,

   tail -F /shared/webtools/test-wtp-S/results/consolelogs/wtptestlog.txt -n200

In fact, I do this so much, I define these aliases in my .bash_proflie

   alias tailout='tail -F /shared/webtools/releng.control/out.txt -n200'
   alias tailtest-i='tail -F /shared/webtools/test-wtp-I/results/consolelogs/wtptestlog.txt -n200'
   alias tailtest-m='tail -F /shared/webtools/test-wtp-M/results/consolelogs/wtptestlog.txt -n200'
   alias tailtest-r='tail -F /shared/webtools/test-wtp-R/results/consolelogs/wtptestlog.txt -n200'
   alias tailtest-s='tail -F /shared/webtools/test-wtp-S/results/consolelogs/wtptestlog.txt -n200'

Here the handy thing to watch for is if a test doesn't seem to make any progress at all, after 10-20 minutes. That's a sure sign of a hang. And, here's how to get some diagnotic information on Linux. First, "break out" of the tail program with ctrl-c. Then get a list of processes that are running by using

    ps -ef | grep java

Usually, torwards the end, you'll see a task that mentions, in the long (wrapped) command line, a recognizable JUnit suite. For that process, looking toward the left of the long (wrapped) line, get its process ID ... usually 4 or 5 digits. Then (almost there!) type "kill -3 <process-id>". I forget exactly what the '-3' means to Unix, but to Java, it means to take a dump of current data and thread states, You'll see a few messages to the console that a java dump wsa greated. That's the file that's handy to attach to a bugzills. (use sftp copy, if you do not have handier tools). That file is usually located in the /eclipse directory in use during the test.

Feel free to edit this document with further tips (or, improve the directions above is I left out some steps, or, you've noticed things are slightly different than written here.

Use of the Eclipse archives Site

For reference:

Out of date milestones, releases, and translations can be found at http://archive.eclipse.org/webtools/downloads/

Or, if you are navigating there directly, see /home/data/httpd/archive.eclipse.org/webtools/downloads/drops

There are also out of date non-released pre-req drivers at /home/data/httpd/archive.eclipse.org/webtools/downloads/drivers

Please note, there is a bug with the download pages on the archive site. The links still point to the "downloads" area instead of the "archived" area. Manually edit the URL to go to the archive area to download the zip file.

Managing builds for multiple WTP build streams

Following the 1.5 release, we started building WTP for 1.0, 1.5 and 2.0 streams concurrently. This required some modififications:

First, we added a property named "buildBranch" to the ant scripts in releng.wtpbuilder. The value of the buildBranch is provided by the parent script that triggers the build. Automated builds are triggered by cruisecontrol, so we also organized cruise projects by build type (i.e. R, S, I, N) and build branch (e.g. R1.0, R1.5, R2.0).

Second, the download areas for declared builds, committers and archives have been re-organized. Previosly, all builds were kept under the same folder (e.g. webtools/downloads/drops). Now there is a seperate folder for each build branch under drops. It goes like

 drops/
   + R1.5
   |  ...
   + R1.0/
   |   + R-1.0.2...
   |   + I-I200607081130...
   + R1.5
   |  ...
   + R2.0
   |  ...
   

The translations are still left in the same place in the archives area.

We also updated the declare script in committers area. Previosly the script required only one parameter, the full build stamp (e.g. ./declare.sh I-I200607130237-200607130237 ). Now, we provide two parameters to this build script, the build branch and the build name as usual (e.g. ./declare.sh R2.0 I-I200607130237-200607130237)