How To Create Automated Build Scripts

Instructions for producing automated builds of Higgins components on the Eclipse build server

Entry conditions

Before creating automated builds scripts for Eclipse server you should have completed build scripts for your component(s) (usually build.xml and buildaux.xml). These scripts should work correctly from command line and have all necessary targets for producing jars, wars, Eclipse plugins etc. Also, no absolute build paths should be used inside build scripts. You should have an Eclipse commiter's account.

Requirements to server's side build scripts

In addition to internal component's build scripts you should create server's side scripts. These scripts should carry out the next operations: checkout from CVS repository necessary component(s) and get all required projects and libraries. execute required Ant tasks from internal component's build scripts in order to compile project(s) and produce required jars, wars, plugins, Javadocs etc.

• copy all created files to the separated folder (folder in work directory with ${buildLabel} name).
• copy all build logs, HTML files and any other required files to ${buildLabel} directory.
• modify prepared index.html file using Ant 'replace' task in order to make complete HTML download page.
• make any other required changes. E.g. add build manifest file with information about current build.
• copy all files from ${buildLabel} directory into web-directory of this Higgins component. Usually project's web directory has the next structure:

/lastNightlyBuild
/lastStableBuild
/builds
   /N-N20070401-200704011001
   /N-N20070403-200704030509
   /S-S20070401-200704011012
   ...
   /N-N20070404-200704040508

sub folders in 'builds' directory contain results of respective build. 'lastNightlyBuild' and 'lastStableBuild' directories accordingly contain results of the last nightly build an results of the last Stable build

Except ant build scripts you should create an additional files:

• build.properties. This file required for setting necessary variables which will be used in Ant build scripts. Example of this file below:

workDirectory=/shared/technology/higgins/work-dir
buildDirectory=/shared/technology/higgins/work-dir/builds
failsDirectory=/shared/technology/higgins/work-dir/builds/fails
testBase=/shared/technology/higgins/work-dir/tests
eclipse.home=/shared/technology/higgins/work-dir/eclipse
workspace=/shared/technology/higgins/work-dir/workspace
projectTitle=Token Service
webHome=/home/www/technology/higgins

• maillogger.properties. This one required for sending e-mails in the event that build of current component failed. Example below:

MailLogger.from=higgins_automated_builds@eclipse.org
MailLogger.failure.to=paul@parityinc.net
MailLogger.failure.subject=IdAS build failed 
MailLogger.success.notify=off

• build.sh. This file required for triggering Ant build files. Because of using 'crontab' utility,Ant scripts should be executed using java. Example of this file below:

# !/bin/sh
JAVA_HOME=/opt/public/common/ibm-java2-ppc64-50

workDirectory=/shared/technology/higgins/work-dir

log_base=/shared/technology/higgins/work-dir/builds/logs

_CLASSPATH=$JAVA_HOME/lib/tools.jar:/shared/common/apache-ant-1.6.5/lib/ant.jar:/shared/common/apache-ant-1.6.5/lib/ant-launcher.jar

build_base=/shared/technology/higgins/static-resources/scripts/sts

usage="usage: build [-mapVersionTag HEAD|<branch name>] [-vm <url to java executable to run build>] [-bc <bootclasspath>] [-target <buildall target to execute>] [-buildID <buildID, e.g. 2.1.2>]  [-ftp <userid> <password>] I|M"

rm -rf /shared/technology/higgins/work-dir/builds
rm -rf /shared/technology/higgins/work-dir/tests
rm -rf /shared/technology/higgins/work-dir/eclipse
rm -rf /shared/technology/higgins/work-dir/workspace

mkdir /shared/technology/higgins/work-dir/logs
mkdir /shared/technology/higgins/work-dir/eclipse
mkdir /shared/technology/higgins/work-dir/builds
mkdir /shared/technology/higgins/work-dir/builds/logs
mkdir /shared/technology/higgins/work-dir/builds/fails
mkdir /shared/technology/higgins/work-dir/tests
mkdir /shared/technology/higgins/work-dir/workspace

while [ "$#" -gt 0 ]
do
    case "$1" in
	-mapVersionTag) mapVersionTag="$2"; shift;;
	-vm) vm="$2"; shift;;
	-bc) bootclasspath="$2"; shift;;
	-target) target="$2"; shift;;
	-buildID) buildID="$2"; shift;;
	-ftp) ftpUser="$2"; ftpPassword="$3"; shift;;
	*) buildType="$1"; shift;;
    esac;
    shift
done

if [ "x$buildType" = "x" ]
then
    echo "$usage"
    exit 1
else

if [ "x$buildID" = "x" ]
then
$JAVA_HOME/bin/java -Dant.home=/shared/common/apache-ant-1.6.5 -classpath $_CLASSPATH org.apache.tools.ant.Main -f $build_base/build.xml -logger org.apache.tools.ant.listener.MailLogger -propertyfile maillogger.properties -DbuildType=$buildType > $log_base/build.log
else
$JAVA_HOME/bin/java -Dant.home=/shared/common/apache-ant-1.6.5 -classpath $_CLASSPATH org.apache.tools.ant.Main -f $build_base/build.xml -logger org.apache.tools.ant.listener.MailLogger -propertyfile maillogger.properties -DbuildType=$buildType -DbuildId=$buildID > $log_base/build.log
fi
fi

• additional prepared files for web UI.

Setting up build scripts

To force created build scripts to work you should commit created build scripts into Eclipse CVS repository. Use the following parameters (you should have Eclipse committer's account):

• Connection type: :ext:
• Host: dev.eclipse.org
• Repository Path: /cvsroot/technology/
• Project: org.eclipse.higgins/builds/org.eclipse.higgins.releng.auto

After that, you should login to build.eclipse.org server via ssh. Use your commiter's login and password. Change current directory to '/shared/technology/higgins/static-resources/scripts/'. There you can find 'beginBuild.xml' file. Add into targer 'N' of this file the following lines:

<exec executable="/bin/sh" dir="${basedir}/{-Your-project-path-}">
     <arg value="${basedir}/{-Your-project-name-}/build.sh"/>
     <arg value="N"/>
</exec>

where {-Your-project-path-} is the path to the project which you are going to build automatically. This path is the same to path inside the org.eclipse.higgins.releng.auto package in CVS repository.

If everything is correct, your build scripts will be triggered every night automatically.

Directory structure on build.eclipse.org server

The directory we use for builds is '/shared/technology/higgins/'. This directory has next structure:

• /static-resources - inside this directory we have all necessary archives, such as Eclipse platform and other required plugins/libraries. Also there located org.eclipse.releng.basebuilder project. You may use it to build Eclipse plugins.
  • /static-resources/stripts - In this folder are located initial (and checkouted) build scripts.
• /work-dir - In this folder are located the temporary files required for builds. Build scripts should clean this folder before each build.
  • /work-dir/eclipse - If Eclipse runtime is required, Eclipse platform should be unpackaged into this folder.
  • /work-dir/builds - directory for temporary files of the current build
  • /work-dir/tests - directory for temporary files of the tests

Links

• Higgins Home
• Nightly Component Builds