Difference between revisions of "Corona/Releng/Build"

Revision as of 06:55, 27 October 2006

Installation and Configuration of Automated Build Process for Corona

The document describes how to configure and run automated builds of Corona project. The description is intended for Linux platform.

Build prerequisities

For headless build the infrastructure provided by the PDE and the RelEng (Release Engineering) plugins is needed. To set up the build environment it is required to have:

• Java5 visible in PATH
• cvs, zip and unzip visible in PATH
• standalone package org.eclipse.releng.basebuilder (the newest version), enable ftp client in org.apache.ant:
  • copy jar file commons-net-1.4.1.jar to plugins/org.apache.ant.XXX/lib
  • modify MANIFEST.MF of org.apache.ant: add commons-net-1.4.1.jar to Bundle-ClassPath
• Web Tools Project all-in-one Eclipse with the following non-standard plugins:
  • org.eclipse.ecf-0.9.1
  • org.eclipse.equinox.log
  • org.eclipse.equinox.event
  • org.eclipse.equinox.http

Steps Required for Setting Headless Build Platform

Let the directory, in which the build is run, be "build". org.eclipse.releng.basebuilder should be installed in "build/..". The following configuration should be made:

1. Change build.properties file
   • baseLocation - the directory in which eclipse-SDK is installed
   • base - the directory baseLocation/..
   • ftpServer - IP of ftp server to copy build results
   • ftpUser, ftpPassword, remoteDirectory - connection parameters on the remote server
2. Change internalBuildTools/testConfigs/linux-local/testing.properties file:
   • testDir - directory where automated testing framework will be installed
3. Prepare build/../download directory. It should contain all 3rd party libraries in the following configuration: for each plugin requiring the libraries, there should be a directory named: "pluginname"/lib containing the jar files.
4. Set JAVA_HOME to point at the JDK installation
5. Change update.eclipse.properties for feed generation
   • branch - the number of branch that is build
   • buildAlias - branch number and name

Setting Configuration for Automated Tests

1. In order to run automated tests, copy zips of the following plugins into build directory:
   • ecf, eclipse-SDK, eclipse-test-framework, emf-sdo-xsd-SDK
   • additionally, it is required to have equinox.log and equinox.event jars and apache.xerces packed into a zip containing eclipse/plugins sudirectory.

Execution of Build Scripts

The main build script is:

• buildAll.sh X - X denotes the type of build, that is X = M, I, N, S, runs the build, tests, generates html data and published build results
  1. for example, to execute a maintenace build, call ./buildAll.sh M
• buildFeed.sh - generates and publishes RSS Feed

Operations Done During Build Phases

---

Execution of buildCorona script comprises:

• cleaning previous build output
• fetching build scripts and properties required by the build process such as:

1. run.bat/run.sh - scripts starting an Eclipse instance as AntRunner
2. buildCorona.xml - build script file controling the build sequence
3. customTargets.xml - build script file enabling customization of the build process, defines the map files to be fetched and the features to be built
4. build.properties - contains the properties needed for fetching and building the sources

• executing run.bat/run.sh script

---

Build Phases

Build Plug-ins (buildAll task)

buildAll task is responsible for building Corona's plug-ins and creating Corona's features.
The most important tasks from this build steps will be described below.

• getMapFiles task (customTarget.xml).

This task is responsible for retrieving corona's map files into local build directory. Map files describe the location of corona's plug-in's and features. Location of map files is specified in build.properties file in following section:

 #skipMaps=true
 mapsRepo=:pserver:anonymous@206.191.52.50:/cvsroot/technology
 mapsRoot=org.eclipse.corona/maps
 mapsCheckoutTag=HEAD

• allElementsDelegator task(allElements.xml).

  This task consists following subTasks:

  • allElements.org.eclipse.corona.client

    Responsible for retrieving and building org.eclipse.corona.client feature.

  • allElements.org.eclipse.corona.server

    Responsible for retrieving and building org.eclipse.corona.server feature.

  • allElements.org.eclipse.corona.doc-feature

    Responsible for retrieving and creating Corona's documentation feature.

  • allElements.org.eclipse.corona.test.client

    Responsible for retrieving and creating Corona's client test feature.

  • allElements.org.eclipse.corona.test.server

    Responsible for retrieving and creating Corona's server test feature.

  If you'd like to add new feature please have in mind following things

  • Create/update proper map file located in org.eclipse.corona/maps

    feature@org.eclipse.corona.<feature name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify feature location>

    plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>

    plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>

  • Add new section within allElements.xml file:
    • in target allElementsDelegator

      <antcall target="allElements.org.eclipse.corona.<feature name>"/>

    • create new target allElements.org.eclipse.corona.<feature name>

      <target name="allElements.org.eclipse.corona.<feature name>">

        <ant antfile="${genericTargets}" target="${target}" >

          <property name="type" value="feature" />

          <property name="id" value="org.eclipse.corona.<feature name>" />

        </ant>

      </target>

    • create new target assemble.org.eclipse.corona.<feature name>

        <target name="assemble.org.eclipse.corona.client">

          <ant antfile="${assembleScriptName}" dir="${buildDirectory}"/>

        </target>

Test Plug-ins (test task)

Publish Plug-ins (publish task)

Important tasks/files for publishing corona to the web site:

index.html generation

• <build directory>/testManifest.xml

  The file contain two types of top level elements:

  • <zipType>

    Example

      <zipType name="zipTypeName">

        <platform id="ID" name="applicable platforms" fileName='<a href="file.zip">file.zip</a>'></platform>

      </zipType>

    • zipType is a category of binary distributable element.

      Zip type name should be an unique identifier. It will be used in index.html.template file as a placeholder where platform information will be put.

    • platform is a description of a platform-specific or non-platform specific binary distributable.

      id - unique identifier for the platform relevant to the zip file referenced in fileName attribute. Used to map compile logs and test logs listed in the manifest to a specific platform of a zipType.

      name - text that should appear on download page to describe where the binary can be installed, ie. "ALL" or "Windows 98/ME/2000/XP".

      fileName - html source inserted on download index page to produce link to the distributable.

  • <logFile>

    Example

      <logFile name="pluginname/myjar.bin.log">

        <effectedFile id="ID"></effectedFile>

      </logFile>

    • logFile - maps compile logs and JUnit test results to a given download.

      effectedFile - references the value of the id attribute for a platform element. Used to mark a <platform> with a status depending on the log content.

• <build directory>/templateFiles/index.html.template

  The template for creating index.html page, this file should contain placeHolders for rows of links to platform-specific and/or platform-independant runtimes for a given download type. Place holders are defined in testManifext.xml file as a zipType name attribute and should be also defined in buildAll.xml file, in publish task as a dropTokenList property.

  <property name="dropTokenList" value="%sdk%,%runtime%,%examples%,%tests%" />

• <build directory>/templateFiles/index.php.template
• <build directory>/templateFiles/testResults.php.template

Push Plug-ins (push task)

Build Corona Documentation Plugin

Corona's build process is able to build corona's documentation feature, following prerequisites are required:

• Linux machine.
• Installed ANT build tool.
• JAVA_HOME environment variable pointing to Java home directory e.g. 'export JAVA_HOME=/etc/jdk1.5.0_09'.
• ANT_HOME environment variable pointing to ANT home directory e.g. 'export ANT_HOME=/etc/apache-ant-1.6.5'.
• javaDoc property set in build.properties file, as follows

 # Default for java doc generation.
 javaDoc=true

See in HINTS section how to generate JavaDoc without a documentation plugin

Results of successful build process

Stages of build process

• preBuild - fetching and concatenating map files (in corona.map dynamic setting of cvs tag name is made on the basis of fetchTag property value defined inside build.properties)
• fetch - retrieving project resources on the basis of the map files definition
• generate - generating build scripts for fetched resources
• process - compiling sources, coping results to build output directory or creating jar files containing the build output
• assemble - creating zip archive containg all build results

Useful links concerning eclipse build process

• Build and Test Automation for plug-ins and features
• Automating Builds with PDE BUILD
• RelEng FAQ

Hints

JavaDoc generation

Corona's JavaDoc always will be generated and integrated within documentation plugin. There is a possibility to generate javadoc without documentation plugin. Following prerequisites has to be set.

• LINUX
  • Installed ANT.
  • JAVA_HOME environment variable pointing to Java home directory.
  • ANT_HOME envitonment variable pointing to ANT home directory.
  • Documentation plugin should be on the same level as other corona's plugins

To start generation run script antJavadoc.sh located in org.eclipse.corona.doc\build.

• WINDOWS
  • Installed cygwin.
  • JAVA_HOME environment variable pointing to Java home directory.
  • ANT_HOME envitonment variable pointing to ANT home directory.
  • Documentation plugin should be on the same level as other corona's plugins

To start generation run under cygwin script antJavadoc.sh located in org.eclipse.corona.doc\build.