Difference between revisions of "Corona/Releng/Build"

From Eclipsepedia

< Corona‎ | Releng
Jump to: navigation, search
(Manually Setting Up Configuration for Automated Tests)
Line 57: Line 57:
 
=== Manually Setting Up Configuration for Automated Tests ===
 
=== Manually Setting Up Configuration for Automated Tests ===
 
To enable automated tests during build:
 
To enable automated tests during build:
* Supply the following zip/tar files in "build" folder:
+
* Supply the following zip/tar build/../download3.3 - those are versions of projects to test with. You must rename files manually.
** org.eclipse.ecf as ecf.zip
+
** ecf-sdk-latest.zip
** eclipse-SDK-3.2.1-linux-gtk.tar.gz
+
** eclipse-equinox-latest.zip
** eclipse-test-framework-3.2.1 zip file
+
** eclipse-test-framework-latest.zip
** wtp-all-in-one-sdk-R-1.5.1 tar.gz file
+
** eclipse-sdk-latest-linux-gtk.tar.gz
* Pack eclipse.equinox.log.jar, eclipse.equinox.http.jar and eclipse.equinox.event.jar to the file named equinox.additions.zip. The jar files must be placed under eclipse/plugins folder in the zip file. Place the zip file in "build".
+
** emf-sdo-xsd-sdk-latest.zip
  
 
== Adjusting Build Configuration To Your Machine ==
 
== Adjusting Build Configuration To Your Machine ==

Revision as of 05:58, 15 May 2007

Eclipse Home Wiki Home Development

Contents

Installation and Configuration of Automated Build Process for Corona

The document describes how to configure and run automated builds of Corona project. The description is intended for Linux platform.

Build prerequisities

To set up the build environment it is required to have:

  • Java5 visible in PATH
  • Set JAVA_HOME to point at the JDK installation
  • cvs, zip and unzip visible in PATH

Steps Required for Setting Headless Build Platform

Automatic Set Up by Script

Script needs update - use Manual setup for now.

A script that sets up build environment is available under CVS at /cvsroot/technology/org.eclipse.corona/build/setupBuild.sh. The script downloads all required packages and configures build environment. Tasks performed by setupBuild.sh correspond those described in Manually Setting Up Build Platform.

To set up build platform:

  • get setupBuild.sh from CVS
  • change setupBuild.sh file properties to executable:
    chmod u+x setupBuild.sh
  • run setupBuild.sh with parameters:
    • -h build_host - folder that will be set up as Eclipse instance used to build Corona"
    • [ -c cvs_connection ] - connection type (typically "ext" or "pserver")
    ext connection type excludes anonymous user, default "anonymous"
    • [ -u cvs_user_name ] - user name used to connect to CVS server, "anonymous" is accepted for pserver
    connection only, default "pserver"

For a detailed discussion on firewall and security restrictions see Troubleshooting Network Configuration

Manual Setup

Manually Setting Up Build Platform

For headless build the infrastructure provided by the PDE and the RelEng (Release Engineering) plugins is needed.

  • Import from CVS /cvsroot/technology/org.eclipse.corona/build/
    Let the directory, in which the build is run, be "build"
  • Import from CVS standalone package org.eclipse.releng.basebuilder (the latest version) org.eclipse.releng.basebuilder should be installed in "build/../org.eclipse.releng.basebuilder".
  • Download and install:
    • Eclipse Platform
    • ECF 1.0
    • Equinox Bundles: org.eclipse.equinox.log, org.eclipse.equinox.event, org.eclipse.equinox.http
    • EMF XSD SDO
  • Edit build/build.properties and modify:
    • baseLocation - the directory in which Eclipse platform is installed
    • base - the directory baseLocation/..
  • If you want to automatically publish build results:
    • Enable ftp client in org.apache.ant plugin of org.eclipse.releng.basebuilder:
      • copy jar file commons-net-1.4.1.jar to plugins/org.apache.ant.XXX/lib
      • modify MANIFEST.MF of org.apache.ant: add commons-net-1.4.1.jar to Bundle-ClassPath
    • Configure property files of the build as described in section Adjusting Build Configuration To Your Machine earlier on this page.

Manually Setting Up Configuration for Automated Tests

To enable automated tests during build:

  • Supply the following zip/tar build/../download3.3 - those are versions of projects to test with. You must rename files manually.
    • ecf-sdk-latest.zip
    • eclipse-equinox-latest.zip
    • eclipse-test-framework-latest.zip
    • eclipse-sdk-latest-linux-gtk.tar.gz
    • emf-sdo-xsd-sdk-latest.zip

Adjusting Build Configuration To Your Machine

Despite automated configuration of most issues, it is necessary to edit property files to adjust the build to machine specific settings. The following configuration changes should be made:

  1. Edit and change build/build.properties file:
    • ftpServer - IP of ftp server to copy build results
    • ftpUser, ftpPassword, remoteDirectory - connection parameters on the remote server
  2. Edit and change the file internalBuildTools/testConfigs/linux-local/testing.properties:
    • testDir - directory where automated testing framework will be installed
  3. Edit and change update.eclipse.properties file
    • branch - the number of branch that is build
    • buildAlias - branch number and name

Configuring Download Side

Results of each build should be propagated to an http server under DocumentRoot/corona/drops/buildId subdirectory. The folder DocumentRoot/corona must contain the files: build_typese.html, dlconfig.txt and index.php. The files are available in plugin org.eclipse.releng.basebuilder under subfolder builderTemplate\publishingFiles\dropSiteRootFiles\

Troubleshooting Network Configuration

Network access problems.

The setupBuild script uses two types of transport to fetch required resources: cvs and http. Depending on security settings enabled by your system administrator, one or both of there connections can be restricted. Typical solutions are:

  • for www access: define http_proxy system variable
  • for cvs access: check which access method is available. cvs can connect to a server using either ext (i.e. ssh protocol) or pserver. pserver access can be used if and only if the user is anonymous. ssh access requires a valid user and a password.

Execution of Build Scripts

Now the build is configured on our internal Linux machine. It will be ported to the vserver before Europa release. To run the build:

  • log in to the machine
  • cd to build.cvs folder
  • delete the src folder in build.cvs (the folder contains old build sources and results)
  • if you want to build on a specified tag, update the myFetchTag property to the desired tag / branch in the build.properties; for trunk, set HEAD
  • execute the main build script, the script is:
    • buildAll.sh X - X denotes the type of build, that is X = M, I, N, S, runs the build, tests, generates html data and published build results
for example, to execute a maintenace build, call ./buildAll.sh M
  • buildFeed.sh - generates and publishes RSS Feed

Operations Done During Build Phases


Execution of buildCorona script comprises:

  • cleaning previous build output
  • fetching build scripts and properties required by the build process such as:
  1. run.bat/run.sh - scripts starting an Eclipse instance as AntRunner
  2. buildCorona.xml - build script file controling the build sequence
  3. customTargets.xml - build script file enabling customization of the build process, defines the map files to be fetched and the features to be built
  4. build.properties - contains the properties needed for fetching and building the sources
  • executing run.bat/run.sh script

Build Phases

Build Plug-ins (buildAll task)

buildAll task is responsible for building Corona's plug-ins and creating Corona's features.
The most important tasks from this build steps will be described below.

  • getMapFiles task (customTarget.xml).

This task is responsible for retrieving corona's map files into local build directory. Map files describe the location of corona's plug-in's and features. Location of map files is specified in build.properties file in following section:

 #skipMaps=true
 mapsRepo=:pserver:anonymous@206.191.52.50:/cvsroot/technology
 mapsRoot=org.eclipse.corona/maps
 mapsCheckoutTag=HEAD
  • allElementsDelegator task(allElements.xml).
    This task consists following subTasks:
    • allElements.org.eclipse.corona.client
      Responsible for retrieving and building org.eclipse.corona.client feature.
    • allElements.org.eclipse.corona.server
      Responsible for retrieving and building org.eclipse.corona.server feature.
    • allElements.org.eclipse.corona.doc-feature
      Responsible for retrieving and creating Corona's documentation feature.
    • allElements.org.eclipse.corona.test.client
      Responsible for retrieving and creating Corona's client test feature.
    • allElements.org.eclipse.corona.test.server
      Responsible for retrieving and creating Corona's server test feature.
    If you'd like to add new feature please have in mind following things
    • Create/update proper map file located in org.eclipse.corona/maps
      feature@org.eclipse.corona.<feature name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify feature location>
      plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>
      plugin@org.eclipse.corona.<plugin name>={fetchTag},:pserver:anonymous@206.191.52.50:/cvsroot/technology,,org.eclipse.corona/<specify plugin location>
    • Add new section within allElements.xml file:
      • in target allElementsDelegator
        <antcall target="allElements.org.eclipse.corona.<feature name>"/>
      • create new target allElements.org.eclipse.corona.<feature name>
        <target name="allElements.org.eclipse.corona.<feature name>">
          <ant antfile="${genericTargets}" target="${target}" >
            <property name="type" value="feature" />
            <property name="id" value="org.eclipse.corona.<feature name>" />
          </ant>
        </target>
      • create new target assemble.org.eclipse.corona.<feature name>
          <target name="assemble.org.eclipse.corona.client">
            <ant antfile="${assembleScriptName}" dir="${buildDirectory}"/>
          </target>


Test Plug-ins (test task)

Publish Plug-ins (publish task)

Important tasks/files for publishing corona to the web site:

index.html generation

  • <build directory>/testManifest.xml
    The file contain two types of top level elements:
    • <zipType>
      Example
        <zipType name="zipTypeName">
          <platform id="ID" name="applicable platforms" fileName='<a href="file.zip">file.zip</a>'></platform>
        </zipType>
      • zipType is a category of binary distributable element.
        Zip type name should be an unique identifier. It will be used in index.html.template file as a placeholder where platform information will be put.
      • platform is a description of a platform-specific or non-platform specific binary distributable.
        id - unique identifier for the platform relevant to the zip file referenced in fileName attribute. Used to map compile logs and test logs listed in the manifest to a specific platform of a zipType.
        name - text that should appear on download page to describe where the binary can be installed, ie. "ALL" or "Windows 98/ME/2000/XP".
        fileName - html source inserted on download index page to produce link to the distributable.
    • <logFile>
      Example
        <logFile name="pluginname/myjar.bin.log">
          <effectedFile id="ID"></effectedFile>
        </logFile>
      • logFile - maps compile logs and JUnit test results to a given download.
        effectedFile - references the value of the id attribute for a platform element. Used to mark a <platform> with a status depending on the log content.
  • <build directory>/templateFiles/index.html.template
    The template for creating index.html page, this file should contain placeHolders for rows of links to platform-specific and/or platform-independant runtimes for a given download type. Place holders are defined in testManifext.xml file as a zipType name attribute and should be also defined in buildAll.xml file, in publish task as a dropTokenList property.
    <property name="dropTokenList" value="%sdk%,%runtime%,%examples%,%tests%" />
  • <build directory>/templateFiles/index.php.template
  • <build directory>/templateFiles/testResults.php.template

Push Plug-ins (push task)

Build Corona Documentation Plugin

Corona's build process is able to build corona's documentation feature, following prerequisites are required:

  • Linux machine.
  • Installed ANT build tool.
  • JAVA_HOME environment variable pointing to Java home directory e.g. 'export JAVA_HOME=/etc/jdk1.5.0_09'.
  • ANT_HOME environment variable pointing to ANT home directory e.g. 'export ANT_HOME=/etc/apache-ant-1.6.5'.
  • javaDoc property set in build.properties file, as follows
 # Default for java doc generation.
 javaDoc=true

See in HINTS section how to generate JavaDoc without a documentation plugin

Results of successful build process

Stages of build process

  • preBuild - fetching and concatenating map files (in corona.map dynamic setting of cvs tag name is made on the basis of fetchTag property value defined inside build.properties)
  • fetch - retrieving project resources on the basis of the map files definition
  • generate - generating build scripts for fetched resources
  • process - compiling sources, coping results to build output directory or creating jar files containing the build output
  • assemble - creating zip archive containg all build results

Useful links concerning eclipse build process

Hints

JavaDoc generation

Corona's JavaDoc always will be generated and integrated within documentation plugin. There is a possibility to generate javadoc without documentation plugin. Following prerequisites has to be set.

  • LINUX
    • Installed ANT.
    • JAVA_HOME environment variable pointing to Java home directory.
    • ANT_HOME envitonment variable pointing to ANT home directory.
    • Documentation plugin should be on the same level as other corona's plugins

To start generation run script antJavadoc.sh located in org.eclipse.corona.doc\build.

  • WINDOWS
    • Installed cygwin.
    • JAVA_HOME environment variable pointing to Java home directory.
    • ANT_HOME envitonment variable pointing to ANT home directory.
    • Documentation plugin should be on the same level as other corona's plugins

To start generation run under cygwin script antJavadoc.sh located in org.eclipse.corona.doc\build.