Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "E4/Builds"

< E4
(CVS)
Line 1: Line 1:
 +
 +
 +
*** OLD INFORMATION ***
 +
 +
 +
 
= Builds  =
 
= Builds  =
  

Revision as of 16:04, 8 May 2014


      • OLD INFORMATION ***


Builds

In e4 (like in the other projects under the Eclipse top-level project), we produce regular builds. One important kind of build is called integration build - usually run once a week to produce a version that is good enough to for purposes of integrating with other teams' work, and for self-hosting. Integration builds do not just take whatever is in HEAD at the time of running the build - given the amount of changes produced by the committers, this is not predictable enough. Instead, each team needs to make a build submission, which basically amounts to tagging their projects at a known good state with a tag that then gets put into the team's map file. The map files are located in so-called releng projects. These projects are checked out from HEAD by the build system, followed by checking out the referenced projects using the specified tags. This means that if a team forgets to submit to the integration build, the build will just check out whatever was submitted last time.

To make a build submission, one team member would first check out (or update) all the team's projects, and run their tests locally. Then, to tag all changed projects, and updating the map file to contain those tags, they would use the releng tools (see below for installation instructions). By using the releng tools instead of tagging and updating the map file manually, the process is made less error-prone, and as an additional benefit, change log information will be generated for you at the same time. The change log is produced by searching for bugzilla ids in commit comments, and doing a lookup in the Bugzilla system to retrieve the bugs' descriptions. Sending the change log information to the mailing list makes everyone aware of what the team has worked on, and can help later if you find regressions and would like to be able to map back to what has been changed and why.

Plug-in Versioning

Our current release, e4 0.11, has all of our plugins marked as (Incubation) and new plugins should have a version 0.11.0.qualifier.

Now that HEAD is open for our 0.12 release we are following the standard Version Numbering guidelines. For example, for existing bundles:

  • if you contribute bug fixes for a bundle, please increment the number to 0.9.1.qualifier
  • if you change the public API or public classes (to add new API or to refactor API for the 0.12 release), please increment the number to 0.10.0.qualifier

For new bundles, please mark them as 0.12.0.qualifier

When we graduate bundles so they are ready for Eclipse 4.0 they will retain their <1.0 version. When we vet the API and promote it, they will change their version number to 1.0.0.qualifier

Milestone Builds

Milestone Delivery Date
0.11 06/22/2011

Integration Builds

E4 Integration Builds
Date Time Frequency
Thursdays 22:00 EST Weekly

e4 Downloads Page

Submitting For the Build

In General:

Checking in code: The checkin comment should at a minimum include "bug #", and preferably be of the form "bug # summary"

For the build:

Tool: Teams should use the "Eclipse Releng Tools" to update their releng project for each build. The releng tools can be installed from the update sites already included in 3.7 and/or 4.1. If you cannot find it, try http://download.eclipse.org/eclipse/updates/3.7.

Install-Releng-Tools.png

Releng tools are in the eclipse updates site now that Helios is released and can be installed from the director app as well:

eclipse/eclipse -application org.eclipse.equinox.p2.director \
-consoleLog -noSplash \
-repository http://download.eclipse.org/eclipse/updates/3.7 \
-installIU org.eclipse.releng.tools.feature.group


Tags: tags should be of the form v<date>, so v20081120 or v20081120-1300

Procedure:

  1. check out your team releng project from e4/releng
  2. use the psf files to check out your core plugins, tests, and examples
  3. Run Team>Release from the context menu:
    1. Select your team's releng project, like org.eclipse.e4.ui.releng.
      Release-1.png
    2. Select the projects you want to release from your map file.
      Release-2.png
    3. You can review the changed projects in this pane. Make sure 'Generate Build Notes' is selected.
      Release-3.png
    4. Copy out the build note changes so you can send an email to e4-dev@eclipse.org, with a subject that starts with [releng]. You don not have to select a build notes file to continue.
      Release-4.png
    5. Enter the release tag for you build. This would normally be the build time and date with a v in front, v20090101-1930
      Release-5.png
    6. You can review the changes to your map file here, or simply continue.
      Release-6.png
    7. Commit the changes so they will be picked up by the I-build


Versioning Plugins: Teams should try and follow the standard versioning rules posted at Version Numbering. If you need to copy over an existing eclipse project to make changes, please increment the minor number by 100. i.e. 3.5.0 becomes 3.105.0. When you need to break API, then update the major number. i.e. 3.105.0 becomes 4.0.0

Build Infrastructure

We run our builds on build.eclipse.org (a ppc linux machine) with a local userid, e4Build.

Build Scripts

The main build project is in e4/releng/org.eclipse.e4.builder. It contains the masterBuild.sh script that can be modified to run a test build or a real build (tagged and uploaded to download.eclipse.org). It also contains the PDE build directory, e4/releng/org.eclipse.e4.builder/builder/general, that builds our master feature. Our PDE build directory also contains modified code/XML to support using p2 to run our automated tests.


Machine Setup

There are a number of directories that need to be available to run a build.

  • /opt/public/common: the java installs can be appropriate for your architecture.
    apache-ant-1.7.0/         ibm-java2-ppc-50@      ibm-java-jdk-ppc-60@

flex_sdk_3.2.0.3794_mpl/ ibm-java2-ppc-50-old@ ibm-java-ppc-60@ ibm-java2-142@ ibm-java2-ppc64-50@

  • /shared/eclipse/e4/flex_sdk_3.2.0.3794_mpl
  • /shared/eclipse/e4/build/e4

To run the tests, we need a virtual X server on :8:

Xvfb :8 -screen 0 1280x1024x24 -auth auth.cfg
metacity --display=:8.0 --replace --sm-disable 

Where auth.cfg contains at least:

localhost

Build Requirements

The build is currently controlled from the masterBuild.sh script. There are a number of dependencies that make this a linux only build, and the machine has to be set up the same way as build.eclipse.org. This section will capture the requirements/process so that all of the needed steps are executed. See bug releng - Run a build from any platform for any discussions about this section.

Setup

In masterBuild.sh

We have a set of variables that are needed in order to make the system go.

Variable Comments
writableBuildRoot

The base directory that contains all build related files except the JREs.

projRelengBranch

The branch to check out the releng project (this contains subprojects that contain all our maps). Default is HEAD.

projRoot

The CVS root used to check out the releng project. For anonymous test builds, it is :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse

arch

Build architecture, currently x86 or ppc.

supportDir

The directory that contains the eclipse install used to build (by default we use the eclipse basebuilder), and usually org.eclipse.e4.builder directory that we launch the build against.

builderDir

Points to the org.eclipse.e4.builder so that it can be in a different place than the supportDir.

builddate

Build date in the format YYYYMMDD, ex: 20090625

buildtime

Build time in the 24 hour clock format HHMM, ex: 1930

tagMaps

Controls if we tag the map files after we check out our releng project.

publishDir

Controls if we should upload the finished build. Currently this is the form of an SCP destination directory.

basebuilderBranch

The tag that determines which base builder we use for the build.

eclipseIBuild

The ID of the eclipse build we use as a target platform. With the changes to use a zipped repo for the eclipse SDK, this ID is only used to generate URLs on the download page. It should potentially be used to collect the correct p2 repo for the eclipse/equinox SDK.

javaHome

The java that is used to launch JVMs from the masterBuild.sh script.

buildTimestamp

The date and time together: 20090625-1930

buildDir

The directory to build in, by default $writableBuildRoot/build/e4/downloads/drops/4.0.0.

targetDir

A directory that contains "target platforms" to work against. It contains a zip download cache, any eclipse target platforms, an untransformed repo directory (a directory with zipped repos to use in the build), and the runnable repo generated by the build.

buildDirectory

The directory for the current build, $buildDir/I$buildTimestamp by default.

testDir

The directory used for running the automated tests.

buildResults

The directory for the build results. This will contain the index.html, compile logs, p2 repo, platform zips, etc. Defaults to $buildDirectory/I$buildTimestamp. So for example, .../4.0.0/I20090625-1930/I20090625-1930.

relengBaseBuilderDir

The root directory for the eclipse install that will run the build. By default we use the eclipse basebuilder.

writableBuildRoot


In general/build.properties

We also code/re-generate a number of properties in the PDE builder.

Variable Comments
buildType

Set to I, but not currently used to do more than create the matching build results directory and correct build "tag".

eclipseBaseURL

The URL of the eclipse SDK base used for this build. This was used to download targets and automated test frameworks, but is now only used to generate links in the download page.

emfBaseURL

Now only used to generate links in the download page.

gefBaseURL

Now only used to generate links in the download page.

wstBaseURL

Now only used to generate links in the download page.

JAVA60_HOME

A base java 6 install, set to /opt/public/common/ibm-java-jdk-ppc-60 to match build.eclipse.org.

JAVA50_64_HOME

A base java 5 64-bit install, set to /opt/public/common/ibm-java2-ppc64-50 to match build.eclipse.org.

JAVA50_HOME

A base java 5 install, set to /opt/public/common/ibm-java2-ppc-50 to match build.eclipse.org.

JAVA14_HOME

A base java 1.4.2 install, set to /opt/public/common/ibm-java2-142 to match build.eclipse.org.

JavaSE-1.6

Definition of the jars that make up the JSE 1.6 BREE.

J2SE-1.5

Definition of the jars that make up the J2SE 1.5 BREE.

J2SE-1.4

Definition of the jars that make up the J2SE 1.4 BREE.

flex.sdk

A property to the base directory for a flex SDK, needed to build the actionScript supporting tools.

repoBaseLocation

This property points to the directory of zipped repos to be used by the built. It is processed through the repo2runnable ant task.

transformedRepoLocation

The runnable repo produced by repo2runnable. Defaults to ${repoBaseLocation}-trans

Manual Setup

A lot of the directory structures must be set up before you can run a build. Perhaps the directory structure needs to be flattened so that it can be more easily created.

The basebuilder (eclipse install used to run the build) and org.eclipse.e4.builder project have to be on disk somewhere. A build updates them to the latest required versions and links to them using symlinks so that they can be used to run the build, drive the build, and provide the testing instructions.

The runnable repo is used for the build as well as to produce the platform zips and install supporting bundles for the automated tests. The untransformed repo location must be created, and populated with repos downloaded by hand. Currently, the eclipse SDK repo must be hand rolled as well. It would be nice to have the runnable repo populated by downloading the needed zipped repos.

The other possibility is to use the repo2runnable task directly and not through repoBaseLocation. Then multiple repo locations can be specified either as local files or URLs (to zipped repos or p2 repos).

There must be at least 3 JREs/JDKs installed on a machine used to build, 1.4.2, 1.5, and 1.6. A bundles BREE instructs PDE build which set of libraries to use during compile.

Steps

In masterBuild.sh

The steps we go through to run a build are controlled by the masterBuild.sh script.

Step Description
realBuildProperties

Set all of the build properties so that the build will be tagged and copied over to download.eclipse.org when complete.

testBuildProperties

Set the build properties to run a test build. Complete with tests, this is not tagged or copied to download.eclipse.org. Only one of testBuildProperties or realBuildProperties should be set.

commonProperties

Just common properties that are set for both types of builds.

updateBaseBuilder

Update the basebuilder to the correct version. Not done in certain types of test builds, like when using the basebuilder checked out into your workspace.

updateE4Builder

Get the latest of the templates and general builder. This is not done in certain types of test builds, when testing changes that are not checked in.

updateBaseBuilderInfo

Collect info on the basebuilder to run the build. This is done in such a way that the launcher jar can be used to launch other eclipse applications (like the publisher or director) as part of the PDE build.

buildMasterFeature

This runs the PDE build. It runs repo2runnable, builds all of the features and plugins in e4 and generates the e4 p2 repository, and generates platform zips.

copyCompileLogs

Script that generates an html page that links to any compile logs generated during the compile.

generateRepoHtml

Generates a default index for the repo URL ... to stop 404 complaints when people click on it in their browser.

generateSwtZip

Generates the 2 SWT action script projects, that can be imported into the workspace for use with the ActionScript tools. Checks out 2 projects, fiddles with some of the text files, and then zips them up into the results directory.

runTheTests

Copy the modified test.xml/runtests (linux) scripts to the testing directory along with one of the generated platform zips. Run the automated tests on the virtual X machine, Xvfb.

CVS

The resources, ui, and swt teams now have plugins in their team area in the e4 project. Our project is under :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse - module e4

Copyright © Eclipse Foundation, Inc. All Rights Reserved.