Graphical Modeling Framework/Build

Introduction

This document describes how to build GMF components from the command line using Ant scripts, including the running of unit tests and operation under CruiseControl.

Note that while there is no mandated standard for builds among Eclipse projects, they generally follow a similar process to what is described here; that is, they leverage the org.eclipse.releng.basebuilder framework. This framework enables headless operation of the Eclipse PDE build functionality. The GMF build process was developed using examples found in the Platform, GEF, EMF, VE, and WTP projects.

A release engineering tool is available for use with this framework. Information on its use can be found below.

Prerequisites

A number of prerequisites need to be installed on the build machine, as follows:

1. CVS version 1.10 or higher
2. J2SE 5.0
3. Info-Zip zip and unzip
4. Ant
5. CruiseControl (optional)
6. Tomcat or similar webserver (optional)

For more information on PDE Build, refer to the plug-in itself.

For examples, refer to the org.eclipse.releng.eclipsebuilder plug-in and its readme.

Setup

With the prerequisites installed, follow these steps to setup your environment and build the GMF components:

1. Add the zip, unzip, and cvs executables to your PATH environment variable.
   
2. Add the jre/bin directory of the installed JDK to your PATH environment variable.
   
3. Add the ant/bin directory of the installed Ant to your PATH environment variable.
   
4. Create a directory for the build, e.g.

• mkdir /gmfbuild
• Note that you will want to place this directory close to the root of the drive on Windows, due to path length limitations. Optionally, you may choose to use the -DbuildRoot property to target a location near the root.

1. Install the v3.1 Eclipse release engineering basebuilder (this installation will be used to run the build tools, it is not itself a GMF installation): Export org.eclipse.releng.basebuilder from the CVS repository /home/eclipse into the build directory, e.g.

• cd /gmfbuild
• cvs -d :pserver:anonymous@dev.eclipse.org:/home/eclipse export -r HEAD org.eclipse.releng.basebuilder

1. Export org.eclipse.gmf.releng.builder into the build directory. cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/technology export -r HEAD -d org.eclipse.gmf.releng.builder org.eclipse.gmf/releng/org.eclipse.gmf.releng.builder
2. Before running the build, you may wish to adjust some of the build properties (see [#antbuildproperties|below]). In particular, you may want to comment-out the 'useMirror' property found in the main build.properties file (in the 'scripts' directory). By default, the GMF build machine uses a local Tomcat installation for an update site and CruiseControl reporting. Technically, the build should function without a local mirror and default to the main update site for dependency installation, which is obviously not desired for frequent builds.

By default, the build will create the following structure, assuming a 'gmfbuild' root was created as indicated above. This structure also shows the default expected Tomcat and CruiseControl locations, which are configurable:

/gmfbuild						Root of build (named as in instructions above)
	/build					Root of the build output (configurable with -DbuildRoot property)
		/artifacts				Root of component build results, with a subdirectory for each
		/checkout				CVS structure used by CruiseControl
		/doc				Build directory of the 'doc' component
		/eclipse				Base installation to compile against and execute tests
		/examples				Build directory of the 'examples' component
		/logs				Location of CruiseControl logs
		/runtime				Build directory of the 'runtime' component
		/sdk				Build directory of the 'sdk' component
		/tests				Build directory of the 'tests' component
		/updateSite				Target for generated update jars, replicated on local mirror if used
	/org.eclipse.gmf.releng.builder					GMF releng builder
		/doc				Script to build the org.eclipse.gmf.doc feature
		/examples				Script to build the org.eclipse.gmf.examples feature
		/runtime				Script to build the org.eclipse.gmf feature
		/scripts				Directory containing build, tests, update, cruise, etc. scripts. Execute from this directory.
		/sdk				Script to build the org.eclipse.gmf.sdk feature
		/tests				Script to bulid the org.eclipse.gmf.tests feature
	/org.eclipse.releng.basebuilder					Eclipse releng builder
/java						Root of Tomcat and CruiseControl
	/cruisecontrol-2.3.1					CruiseControl installation
	/Tomcat 5.5					Tomcat installation

Run the build

cd to the 'scripts' directory under org.eclipse.gmf.releng.builder and execute the build command passing the desired component to target (on Windows) :

• cd org.eclipse.gmf.releng.builder\scripts
• build.bat -Dcomponent=sdk | runtime | examples | tests | doc

Optionally, you can use the following command:

• java -jar ..\org.eclipse.releng.basebuilder\startup.jar -application org.eclipse.ant.core.antRunner -Dcomponent=sdk | runtime | examples | tests | doc

Note that the download of the Eclipse SDK and/or test framework may fail using the Ant 'get' task, so you may have to download these elements and place in the 'build' folder manually. These zips are maintained for future use in the 'build' directory to make -DcleanBase builds quicker. Notice that unless a local mirror is installed, each dependent feature (EMF, GEF, etc.) will be downloaded from its respective update site with each -DcleanBase build. Therefore, it is recommended that a local mirror be used to speed up the process and to reduce stress on the update sites.

The component property value sets the name of the directory in org.eclipse.gmf.releng.builder which contains the Ant scripts that build a given GMF component, as descibed below:

Component (directory name in org.eclipse.gmf.releng.builder)	Description
sdk	This directory contains the scripts used to build the GMF SDK feature (org.eclipse.gmf.sdk). A GMF SDK includes the runtime feature and is comprised of binaries and sources.
runtime	This directory contains the scripts used to build the GMF runtime feature (org.eclipse.gmf). The GMF runtime component contains only those binaries required to run a GMF-based application.
examples	This directory contains the scripts used to build the GMF example feature (org.eclipse.gmf.examples) with sources. Note that the SDK build is a prerequisite.
tests	This directory contains the scripts used to build and execute the GMF unit tests feature (org.eclispe.gmf.tests). Note that the SDK and examples builds are prerequisites.
doc	This directory contains the scripts used to build the GMF documentation feature (org.eclipse.gmf.doc).

A number of properties described below can be passed in via the build command to override the default settings. Of course, direct modification of the properties is an alternative. Note that if you choose to set the 'buildDirectory' property, it will need to end in the name of the component. It is recommended that 'buildRoot' be used instead, as it will have the component name appended to form the 'buildDirectory' property by default. By default, 'buildRoot' will be a sibling of the org.eclipse.gmf.releng.builder directory.

Examples: build -Dcomponent=sdk -javacFailOnError=false -DjavacVerbose=false -DbuildRoot=d:\mybuild

This builds a GMF SDK in the directory d:\mybuild\sdk. "-DjavaVerbose=false" indicates that no compile logs will be generated and "-javacFailOnError=false" will cause the build to not fail if there are any compile errors.

build -Dcomponent=runtime -DbuildRoot=d:\gmfbuilds -DbaseLocation=d:\eclipse\eclipse3.1

This builds a GMF runtime in the directory "d:\gmfbuilds\runtime" yet uses an installation of eclipse located at 'd:\eclipse\eclipse3.1' as the base to build against.

build -Dcomponent=tests -DbuildRoot=d:\mybuild -DcleanBase=true -Dclean=true

This builds the GMF test plug-ins from scratch, as 'clean' is used. The build will also delete the base installation and unzip a fresh copy of Eclipse and install GMF's dependencies. Note that an 'sdk' build is a necessary dependency of the 'tests' build.

Using a local mirror

The GMF build process utilizes the Eclipse update manager to install its dependencies. If you plan to run a clean build frequently, it is recommended that you configure a local mirror to provide the features for installation. Simply uncomment the 'useMirror' property set in the build.properties file(s) and install a local HTTP server to enable this functionality. It is recommended that Tomcat is used, as it can also serve to provide the CruiseControl reporting described below.

GMF builds utilize the update manager via the command line for the installation of build dependencies (e.g. EMF & GEF), and also to install GMF itself for unit tests runs. This has the added benefit of testing update site packaging and leverages the ability to update and maintain a local mirror site. The alternative is to download archives from the Eclipse website, which may turn out to be a better approach ;-)

To use a local mirror, you will need to install Tomcat and perform the following steps to setup the local update site:

• Adjust the 'localUpdateSite' and 'localUpdateSitePath' properties in the main build.properties to reflect yourconfiguration.
• cd org.eclipse.gmf.releng.builder\scripts
• run 'build initLocalMirror'
• run 'build' command with -Dcomponent argument as described above

Run the build and then navigate to http://localhost/updateSite (or similar, depending on your configuration). If you had run the build previously, you may want to rerun with -DcleanBase=true option in order to force the mirror and install of GMF dependencies locally.

Without the mirror running, the build will resort to using the official update sites for dependencies, while using a simple unzip of GMF components (when running tests). When running, each build will produce update site jars which are then copied to the update site directory.

Using CruiseControl

The GMF project utilizes CruiseControl to manage its automated build process. The following basic steps are required to leverage CruiseControl:

• Install CruiseControl (version 2.3.1 is used currently)
• Check the settings of build.properties and config.xml files, modifying as necessary. In particular, the schedule interval, email server and addresses.
• If using Tomcat for the reporting component, follow the CC instructions to install the cruisecontrol.war and modify the web.xml file to point at the build's log and artifact locations. These will be specified in the CC config.xml file and are by default /build/logs and /build/ccpublish, respectively. The installation is as simple as copying the cc/reporting/jsp/dist/cruisecontrol.war file into the tomcat/webapps directory.
• You may want to adjust the maximum memory setting for Tomcat, and copy the xalan.jar provided with CruiseControl into the WEB-INF/lib directory of the exploded cruisecontrol webapp to avoid problems rendering larger reports.
• To run CruiseControl as a service on Windows, follow the instructions provided here.

To invoke CruiseControl, follow the instructions below:

• cd org.eclipse.gmf.releng.builder\scripts
• cruise

Note that the checkout command is run initially and produces a full checkout structure of GMF in a 'checkout' subfolder of 'build'. This is necessary for CC to check for changes and kick off a build. It seems necessary for this duplication to exist, as the PDE build uses the 'export' CVS command and not 'checkout' as is required when checking for changes.

Also, note that it should be possible for much of the scripts here and in the build itself to be generated from plug-ins and features. Perhaps future versions of the build will take advantage of these possibilities.

Ant build.properties

There are a number of Ant properties declared in the build.properties file, most of which are provided by the org.eclipse.releng.basebuilder utility and are described in this document. Below are a list of additional properties found in GMF's build process and their description.

Each property can be overridden by setting them at the command line at build time or by changing them directly in the component's build.properties file, or a map definition's build.cfg file.

Component build.properties
buildRoot	This directory is the root of the build folders for each component (e.g. sdk, tests, etc.) and the CruiseControl folders (when used). By default, it is a sibling to the releng folders. In GMF builds, it is recommended that you stick to setting buildRoot and avoid the buildDirectory property found within the scripts.
cleanBase	Normally commented out, but if present will cause the base to be deleted prior to the build. This will remove the base Eclipse installation and force an updated install of all dependencies.
clean	This will normally cause the cleaning of all output directories prior to the build, but in GMF will also cause a cleaning of all checkout areas to force an update from CVS.
skipTests	This is only found in the tests component and will prevent the execution of unit tests following the build and packaging steps.
Map build.cfg
TODO:	The properties found here are presently self-explanatory and largely to be unnecessary as most information can be gotten from feature.xml files (as is planned for future versions of the build process).

Project Structure

The CVS structure for GMF organizes the project by plug-ins, features, tests, etc., as seen below:

/cvsroot/technology						CVS ROOT (dev.eclipse.org)
	/org.eclipse.gmf					Graphical Modeling Framework subproject
		/contributions				GMF code contributions
		/doc				Project documentation
		/examples				GMF examples
		/features				GMF features
		/plugins				GMF plugins
		/releng				Build procedure scripts, update site, etc.
		/tests				GMF unit tests

Executing the tests

This section describes the process for executing the unit tests packaged in the GMF test component. The tests are normally installed and executed as part of the build process, while these instructions are intended for those who wish to invoke the tests after installed in their local environment.

• Install the prerequisites: The GMF unit tests require a few dependencies to be installed prior to execution. These are all available or listed on the GMF download page. Presently, you will need to install the GMF SDK and Examples, along with the UML2 project runtime component, in addition to their dependencies.
• Setting the test properties: Modify the properties found at the top of the main test.xml file to match your configuration. This file is found in the org.eclipse.gmf.tests.source_X.X.X folder under the plugins directory of your Eclipse installation.
• Running the tests: From the org.eclipse.gmf.tests.source_X.X.X folder in the plugins directory of your Eclipse installation, you will need to invoke Ant and pass in the test.xml as the build script file (-f option). When complete, a /testresults/html folder will contain the test output.

Developer information

This section outlines some of the procedures and practices for GMF developers to maintain a smooth build/test environment. In particular, information on using the releng tool and how to add a plug-in to the build is covered.

• Adding or removing a plug-in: As the build depends on features remaining accurate, be sure to update the feature.xml for listed plug-ins and also check the 'Dependencies' list, using the Compute feature of the PDE if desired. You will also need to update the appropriate map file(s) to indicate the changes to plug-ins and their location in the repository. Alternatively, email the developer mailing list with changes requested and let the maintainer of the build effect the change.
• Adding a test plug-in: Each test plug-in needs to have a corresponding test.xml file in order to be run as part of the build. See existing test.xml files for an example. Be sure to appropriately choose either 'ui-test' or 'core-test' for the target within this file. Information on what these mean is found in the readme for the testing framework. Finally, each test plug-in is listed for execution in the test.xml file found in the sourceTemplatePlugin directory of org.eclipse.gmf.tests-feature. You will need to add your new test plug-in to this list.

Gotchas

The build process is fairly complex, and will hopefully improve with more being generated/automated. A few things to watch out for are listed below:

• If the build stops during the fetch with an error message telling you to check your map files, open the generated fetch_*.xml file in your component build directory. If it is cutoff (malformed), check your CVS connection, as it is likely the cause.
• If you are running a build and get the message that 'plug-in org.*_X_X_X' cannot be found, check that you have declared/installed the necessary dependencies for the plug-in.
• If you are building a feature that contains a generate.* option in its build.properties file, and you are getting '.../build.xml file does not exist' messages, check that your plug-ins do not contain any cyclic dependencies. Specifically, look at any fragments you may have to see if they have their host plug-in listed as a dependency.