Corona/Releng/Build

From Eclipsepedia

Jump to: navigation, search
Eclipse Home Wiki Home Development

Contents

Installation and Configuration of Automated Build Process for Corona

The document describes how to configure and run automated builds of Corona project. The description is intended for Linux platform.

Build prerequisities

To set up the build environment it is required to have:

  • Java5 visible in PATH
  • Set JAVA_HOME to point at the JDK installation
  • cvs, zip and unzip visible in PATH

Steps Required for Setting Headless Build Platform

Setting Up Build Platform Using a Script

A script that sets up build environment is available under CVS at /cvsroot/technology/org.eclipse.corona/build/setupBuild.sh. The script downloads all required packages and configures build environment. Tasks performed by setupBuild.sh correspond those described in Manual Setting Up Build Platform.

To set up build platform:

  • get setupBuild.sh from CVS
  • change setupBuild.sh file properties to executable:
    chmod u+x setupBuild.sh
  • run setupBuild.sh with parameters:
    • build_host - folder that will be set up as Eclipse instance used to build Corona"
    • cvs_connection - connection type (typically "ext" or "pserver"), some CVS may support only one type
    • ext connection type excludes anonymous user, default "anonymous"
    • cvs_user_name - user name used to connect to CVS server, "anonymous" is accepted for pserver
    • connection only, default "pserver"

For a detailed discussion on firewall and security restrictions see Troubleshooting

Adjusting Build Configuration To Your Machine

Despite automated configuration of many issues, it is necessary to edit property files to adjust to machine specific settings. The following configuration changes should be made:

  1. Edit build/build.properties file and change the following settings:
    • ftpServer - IP of ftp server to copy build results
    • ftpUser, ftpPassword, remoteDirectory - connection parameters on the remote server
  2. Edit and change internalBuildTools/testConfigs/linux-local/testing.properties file:
    • testDir - directory where automated testing framework will be installed
  3. Edit and change update.eclipse.properties for feed generation
    • branch - the number of branch that is build
    • buildAlias - branch number and name

Manual Setting Up Build Platform

For headless build the infrastructure provided by the PDE and the RelEng (Release Engineering) plugins is needed.

  • Import from CVS /cvsroot/technology/org.eclipse.corona/build/
    Let the directory, in which the build is run, be "build"
  • Import from CVS standalone package org.eclipse.releng.basebuilder (the newest version)
org.eclipse.releng.basebuilder should be installed in "build/../org.eclipse.releng.basebuilder".
  • Enable ftp client in org.apache.ant plugin of org.eclipse.releng.basebuilder:
    • copy jar file commons-net-1.4.1.jar to plugins/org.apache.ant.XXX/lib
    • modify MANIFEST.MF of org.apache.ant: add commons-net-1.4.1.jar to Bundle-ClassPath
  • Download and install Web Tools Project all-in-one Eclipse and additional plugins:
    • org.eclipse.ecf-0.9.1
    • org.eclipse.equinox.log
    • org.eclipse.equinox.event
    • org.eclipse.equinox.http
  • Prepare build/../download directory. It should contain a file named build.zip with 3rd party libraries that have not been IP approved yet. The file can be downloaded from

http://people.apache.org/~jhawkins/

  • Edit build/build.properties and modify:
    • baseLocation - the directory in which eclipse-wtp-all-in-one is installed
    • base - the directory baseLocation/..
  • Configure property files of the build as described in section Adjusting Build Configuration To Your Machine earlier on this page.

Manually Setting Up Configuration for Automated Tests

To enable automated tests during build:

  • Supply the following zip/tar files in "build" folder:
    • org.eclipse.ecf as ecf.zip
    • eclipse-SDK-3.2.1-linux-gtk.tar.gz
    • eclipse-test-framework-3.2.1 zip file
    • wtp-all-in-one-sdk-R-1.5.1 tar.gz file
  • Pack eclipse.equinox.log.jar, eclipse.equinox.htt.jar and eclipse.equinox.event.jar to the file named equinox.additions.zip. The jar files must be placed under eclipse/plugins folder in the zip file. Place the zip file in "build".

Setting Download Side Configuration

Results of each build are propagated to an http server under DocumentRoot/corona/drops/buildId subdirectory. The folder DocumentRoot/corona must contain the files: build_typese.html, dlconfig.txt and index.php. The files are available in plugin org.eclipse.releng.basebuilder under subfolder builderTemplate\publishingFiles\dropSiteRootFiles\

Execution of Build Scripts

The main build script is:

  • buildAll.sh X - X denotes the type of build, that is X = M, I, N, S, runs the build, tests, generates html data and published build results
    1. for example, to execute a maintenace build, call ./buildAll.sh M
  • buildFeed.sh - generates and publishes RSS Feed

Enabling New Plugins in Build

In order to include a new plugin in build, it is necessary to:

  • add the plugin to feature (either org.eclipse.corona.client or org.eclipse.corona.server or both), check for plugin version and unpack option
  • add the path to plugin to maps (corona.client.map or corona.server.map or both)
  • if required, copy 3rd party jars to the download directory on the build server

Features and maps are located in Corona CVS repository.

Operations Done During Build Phases


Execution of buildCorona script comprises:

  • cleaning previous build output
  • fetching build scripts and properties required by the build process such as:
  1. run.bat/run.sh - scripts starting an Eclipse instance as AntRunner
  2. buildCorona.xml - build script file controling the build sequence
  3. customTargets.xml - build script file enabling customization of the build process, defines the map files to be fetched and the features to be built
  4. build.properties - contains the properties needed for fetching and building the sources
  • executing run.bat/run.sh script

Build Phases

Build Plug-ins (buildAll task)

buildAll task is responsible for building Corona's plug-ins and creating Corona's features.
The most important tasks from this build steps will be described below.

  • getMapFiles task (customTarget.xml).

This task is responsible for retrieving corona's map files into local build directory. Map files describe the location of corona's plug-in's and features. Location of map files is specified in build.properties file in following section:

 #skipMaps=true
 mapsRepo=:pserver:anonymous@206.191.52.50:/cvsroot/technology
 mapsRoot=org.eclipse.corona/maps
 mapsCheckoutTag=HEAD
  • allElementsDelegator task(allElements.xml).
    This task consists following subTasks:
    • allElements.org.eclipse.corona.client
      Responsible for retrieving and building org.eclipse.corona.client feature.
    • allElements.org.eclipse.corona.server
      Responsible for retrieving and building org.eclipse.corona.server feature.
    • allElements.org.eclipse.corona.doc-feature
      Responsible for retrieving and creating Corona's documentation feature.
    • allElements.org.eclipse.corona.test.client
      Responsible for retrieving and creating Corona's client test feature.
    • allElements.org.eclipse.corona.test.server
      Responsible for retrieving and creating Corona's server test feature.
    If you'd like to add new feature please have in mind following things
    • Create/update proper map file located in org.eclipse.corona/maps
      feature@org.eclipse.corona.<feature name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify feature location>
      plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>
      plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>
    • Add new section within allElements.xml file:
      • in target allElementsDelegator
        <antcall target="allElements.org.eclipse.corona.<feature name>"/>
      • create new target allElements.org.eclipse.corona.<feature name>
        <target name="allElements.org.eclipse.corona.<feature name>">
          <ant antfile="${genericTargets}" target="${target}" >
            <property name="type" value="feature" />
            <property name="id" value="org.eclipse.corona.<feature name>" />
          </ant>
        </target>
      • create new target assemble.org.eclipse.corona.<feature name>
          <target name="assemble.org.eclipse.corona.client">
            <ant antfile="${assembleScriptName}" dir="${buildDirectory}"/>
          </target>


Test Plug-ins (test task)

Publish Plug-ins (publish task)

Important tasks/files for publishing corona to the web site:

index.html generation

  • <build directory>/testManifest.xml
    The file contain two types of top level elements:
    • <zipType>
      Example
        <zipType name="zipTypeName">
          <platform id="ID" name="applicable platforms" fileName='<a href="file.zip">file.zip</a>'></platform>
        </zipType>
      • zipType is a category of binary distributable element.
        Zip type name should be an unique identifier. It will be used in index.html.template file as a placeholder where platform information will be put.
      • platform is a description of a platform-specific or non-platform specific binary distributable.
        id - unique identifier for the platform relevant to the zip file referenced in fileName attribute. Used to map compile logs and test logs listed in the manifest to a specific platform of a zipType.
        name - text that should appear on download page to describe where the binary can be installed, ie. "ALL" or "Windows 98/ME/2000/XP".
        fileName - html source inserted on download index page to produce link to the distributable.
    • <logFile>
      Example
        <logFile name="pluginname/myjar.bin.log">
          <effectedFile id="ID"></effectedFile>
        </logFile>
      • logFile - maps compile logs and JUnit test results to a given download.
        effectedFile - references the value of the id attribute for a platform element. Used to mark a <platform> with a status depending on the log content.
  • <build directory>/templateFiles/index.html.template
    The template for creating index.html page, this file should contain placeHolders for rows of links to platform-specific and/or platform-independant runtimes for a given download type. Place holders are defined in testManifext.xml file as a zipType name attribute and should be also defined in buildAll.xml file, in publish task as a dropTokenList property.
    <property name="dropTokenList" value="%sdk%,%runtime%,%examples%,%tests%" />
  • <build directory>/templateFiles/index.php.template
  • <build directory>/templateFiles/testResults.php.template

Push Plug-ins (push task)

Build Corona Documentation Plugin

Corona's build process is able to build corona's documentation feature, following prerequisites are required:

  • Linux machine.
  • Installed ANT build tool.
  • JAVA_HOME environment variable pointing to Java home directory e.g. 'export JAVA_HOME=/etc/jdk1.5.0_09'.
  • ANT_HOME environment variable pointing to ANT home directory e.g. 'export ANT_HOME=/etc/apache-ant-1.6.5'.
  • javaDoc property set in build.properties file, as follows
 # Default for java doc generation.
 javaDoc=true

See in HINTS section how to generate JavaDoc without a documentation plugin

Results of successful build process

Stages of build process

  • preBuild - fetching and concatenating map files (in corona.map dynamic setting of cvs tag name is made on the basis of fetchTag property value defined inside build.properties)
  • fetch - retrieving project resources on the basis of the map files definition
  • generate - generating build scripts for fetched resources
  • process - compiling sources, coping results to build output directory or creating jar files containing the build output
  • assemble - creating zip archive containg all build results

Useful links concerning eclipse build process

Hints

JavaDoc generation

Corona's JavaDoc always will be generated and integrated within documentation plugin. There is a possibility to generate javadoc without documentation plugin. Following prerequisites has to be set.

  • LINUX
    • Installed ANT.
    • JAVA_HOME environment variable pointing to Java home directory.
    • ANT_HOME envitonment variable pointing to ANT home directory.
    • Documentation plugin should be on the same level as other corona's plugins

To start generation run script antJavadoc.sh located in org.eclipse.corona.doc\build.

  • WINDOWS
    • Installed cygwin.
    • JAVA_HOME environment variable pointing to Java home directory.
    • ANT_HOME envitonment variable pointing to ANT home directory.
    • Documentation plugin should be on the same level as other corona's plugins

To start generation run under cygwin script antJavadoc.sh located in org.eclipse.corona.doc\build.