Automated ECF Build

Introduction

Here's Part 3 of my notes on setting up our automated build system. There are a number of questions (open for discussion) interspersed throughout this document.

The ECF automated build system automates two builds. We called one Auto Build and the other Daily Build. Current docs spec out another automated build called Nightly which is similar but not identical to the Daily Build I'm describing. We'll merge the operation of Daily and Nightly into one type of build and whether we call it Daily or Nightly at this point does not matter.

This system will be deployed on ecf2. The user interface will do more than automated builds. It will also manage user-initiated builds. These user-initiated builds may include Integration, Stable, Release, and Maintenance builds.

User-Initiated Builds

The downloads page on http://www.eclipse.org/ecf defines these user-initiated (non-automated) build types in detail. Here's a summary.

• Integration Builds are builds that have been judged stable enough to be used for integration with other eclipse projects.

• Stable Builds are like Integration Builds but they have been judged to be stable enough for most people to use ,where most people means a greater user set than those integrating with other Eclipse projects. Do we still need this build type? We should consider merging Integration and Stable builds into the same build.

• Release Builds are major releases, like R1.0.0. ECF currently now has a Release Candidate 2 (RC2), but not yet an official release.

• Maintenance Builds are fixes to a Release Build. They are cumulative as bugs are fixed. They are merged into the next release.

• A release is tagged in the CVS repository as a release. What is the exact string used?

CVS Tags

How do these user-initiated builds relate to CVS tags? Here's a screenshot of current CVS tags for ECF. Note that the topmost tag is v20070606-1500, meaning that the files associated with that tag define a build made on 6/6/2007 at 15:00 hours.

This latest tag is older than the HEAD, which defines the very latest ECF files available. The HEAD should always successfully build. This latest tag corresponds to the files presented for download. The CVS tag is embedded in the name of the zip. The text on the download page identifies this download as RC2.

If you download one of those files (org.eclipse.ecf.sdk-20070606-1500.zip) and unzip it you see the following. Note that the jars have a version string (for example v20070606-1500) that indicates when they were made. Also, the feature directories contain a Feature Version Suffix (randomly generated) in their names.

When should these Feature Version Suffixes be used? You can determine whether to generate them or not by setting the property genFVSuffix to true or false.

The zips provided on the download page are also available through the update within Eclipse. (Help --> Software Updates.) Note that the features have the Feature Version Sufffix.

Files Available for Download

It appears that what we offer for download is the top of the CVS tag list. Should the ECF download page also offer Integration/Stable Builds, Maintenance Builds, and Daily Builds? And if so, how should they be designated?

The Two Automated Builds

These are the Auto Build and the Daily Build. Both are controlled by CruiseControl. Each is a separate project.

Other than scheduling, the differences are that the Daily Build after a successful build ftp's the built files and that it uses different criteria for sending notification email.

I don't know whether either of these builds should set the Feature Version Suffix for the feature directories.

Also, I noted that the Nightly Build does not include the Update Site. The Update Site (Help --> Software Updates) is what is used to update from within Eclipse. It makes more sense for this to be the latest release rather than the Daily Build. However, having two Update Sites is a possibility.

Auto Build

CruiseControl attempts an Auto Build every 30 minutes with a quietperiod of 5 minutes. I gave it a buildType of A for Auto. The only place this buildType is used is to name the directory that contains the built files. I also chose to set genFVSuffix.

<modificationset quietperiod="300">

<cvs localworkingcopy="/home/ted/workanon/org.eclipse.ecf/plugins" />

</modificationset>

<schedule interval="1800">

<ant buildfile="cc-build.xml" target="ecf.build" >

<property name="buildType" value="A" />

<property name="genFVSuffix" value="true" />

</ant>

</schedule>

Note the directory A-20070610-1724.

The zip contains jars that have HEAD in their names. So does jars under updateSite.

The Auto Build should sends no mail when it fails; When it succeeds, it should send mail to ecf-build@eclipse.org. Right now it appears to be sending both success and failure mail. I'm not sure why. I thnk CruiseControl wants to send mail wo whoever modified the CVS repository, and I've aliased contributors to ecf-build.

Daily Build

CruiseControl attempts a Daily build once a day at (for now) 15:00 hours. I gave it a buildType of D for Daily. I chose for the sake of variety not to include a Feature Version Suffix.

<modificationset quietperiod="300">

<cvs localworkingcopy="/home/ted/workanon/org.eclipse.ecf/plugins" />

</modificationset>

<schedule>

<ant buildfile="cc-build.xml"

time="1500"

target="ecf.copy" >

<property name="buildType" value="D" />

<property name="genFVSuffix" value="false" />

</ant>

</schedule>

Note the directory D-20070610-1500.

If the build is successful, the zip files and update site are ftp'ed to a download site. Currently,this is just another directory on halibut called /opt/ecf_deploy.

Note that the <schedule/> for the Daily Build calls the ant target ecf.copy which does the ftp and depends on the target ecf. The target ecf is the same one called by the Auto Build, except is is given a different buildType (D) in the config.xml file. The target ecf checks for this buildType and sets forceContextQualifier accordingly.

<condition property="forceContextQualifier" value="${timestamp}">

<equals arg1="${buildType}" arg2="D" />

</condition>

<java classname="org.eclipse.core.launcher.Main" classpath="${eclipse.home}/startup.jar"

In config.xml, here is how the ftp is done.

<onsuccess>

<antpublisher

antscript="/opt/apache-ant-1.7.0/bin/ant"

antWorkingDir="/opt/build.ecf"

buildfile="uploadZIP.xml"

target="deploy" />

<antpublisher

antscript="/opt/apache-ant-1.7.0/bin/ant"

antWorkingDir="/opt/build.ecf"

buildfile="uploadUPDATE.xml"

target="deploy" />

</onsuccess>

The ftp is just calling ant twice with to different ant files that do the ftp. <ftp/> is an optional ant task, and for it to work, you have to download and install two libraries: commons-net-1.4.1.jar and jakarta-oro-2.0.8.jar. Put them in /opt/apache-ant-1.7.0/lib. Get them from the Apache Jakarta project.

The resulting files in /opt/ecf_deploy do not show HEAD but rather the datetime.

The updateSite is there, but should be named update, not updateSite. Here is a screenshot of files on dev.eclipse.org.

The Daily Build sends both success and failure mail to ecf-build@eclipse.org. It also sends mail to ecf-dev@eclipse.org. During this test period, I'm only sending mail to ecf-build.

The User Interface

Remember we have two builds going on: Auto Build (the project is called called ecf) and the Daily Build (the project is called called ecfDaily). Here's how it looks using CruiseControl's web reporting tool. I've used this tool to force either of the builds.

To use the web reporting tool, you have to create a war file and install it in the webapps folder for TomCat and run TomCat. If you click on the project name you go into a tabbed window that also displays the JMX console. To run the JMX console, you must have started CruiseControl with -port 8000.

I've also used the CruiseControl GUI. To use this GUI you must start CruiseControl with -rmiport 1099.

/opt/cruisecontrol-2.6.2/main/bin/cruisecontrol.sh -configfile config.xml -port 8000 -rmiport 1099

Then, download cruisecontrol-gui-2.jnlp and open it up with FireFox. Configure a server to monitor.

You can force a build by selecting a project, dropping down the Projects list and choosing ForceBuild.