Skip to main content
Jump to: navigation, search

Difference between revisions of "Platform-releng-faq-obsolete"

m (How do I run a test build on the hudson install at ?)
m (Connecting to the derby database using ij)
Line 462: Line 462:
connect 'jdbc:derby://localhost:1528/perfDb36';
connect 'jdbc:derby://localhost:1528/perfDb36';
== Are the Eclipse and Equinox projects converting from CVS to git ==
Yes, here is the planning document
[[Platform-releng/Juno_Git_Migration Juno Git Migration]]
==Eclipsecon presentations==
==Eclipsecon presentations==

Revision as of 09:02, 13 June 2011

Eclipse Platform Release Engineering FAQ

If you would like additional content added to this FAQ, please send a note to the platform releng developers mailing list.


What hardware comprises the platform-releng build infrastructure?

The eclipse platform build infrastructure consists of

  1. junit test machines,
    1. ejwin1 (winxp with 1.5 vm) 5 on KVM on table
    2. ejwin2 (winxp with 1.6 vm 6 on KBM on table
    3. lb6mac (macosx Leopard) mac on table on LHS of lab
    4. ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack
    5. ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack
  2. performance machines
    1. epwin2 (winxp with 1.5 vm) G on large KVM in rack
    2. epwin3 (winxp with 1.6 vm) 7 on large KVM in rack
    3. eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack
    4. eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack
  3. Other
    1. minsky (database server with apache derby installed to store performance test results), (cvs test server)
    2. eclipsebuildserv (build machine) Linux machine on far back table of the room

The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.

Is the Eclipse platform build process completely automated?

Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.

  • org.eclipse.releng.eclipsebuilder -> scripts to build each type of drop
  • org.eclipse.releng.basebuilder -> a subset of eclipse plugins required to run the build.
  • we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks

Fetch and build scripts are generated automatically by the bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder. After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.

What's the difference between an integration and nightly build?

With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here

How do I use the releng plugin?

The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.

This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. Please use the Release feature of this plug-in to do your build submissions.

  • Release to the Team menu. This action will Tag selected projects with the specified version and update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.
  • 'Load Map Projects to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.
  • Tag Map Projects to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.
  • Compared with Released to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.
  • Replace with Released to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.
  • Fix Copyright to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.

How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?

This document provides an overview of how to build eclipse components using the releng scripts. However, it is really out of date. Your best bet is to use the source build to recompile the SDK.

What is the latest version of org.eclipse.releng.basebuilder?

See this document which describes correct tag of org.eclipse.releng.basebuilder for use in your builds.

How do I automate my builds with PDE Build?

See the PDE/Build wiki page.

How do I contribute a JUnit Plug-in Test to the build?

  1. The tests need to be contributed as one or more plugins that use the org.eclipse.test automated testing framework. JUnit tests can also be written as performance tests. Click here for the latest instructions.
  2. Open bug for for PMC approval for new plug-in projects on CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.
  3. Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.
  4. Install the org.eclipse.releng tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit map file.
  5. Open a bug against platform releng to add the plugins to the build process. Include the following information:
  • the plug-in id(s)
  • the plug-ins that contain a test.xml
  • update the to include the test.xml
  • additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK

The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.

How do I contribute an example plug-in to the build?

Once you have your plug-in ready and available in CVS, install the plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.

Next, open a bug against platform releng with the plug-in's id.

The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.

I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?

The best approach is use the source build drop and modify the scripts to support that you are interested in. Then open a bug with product Platform and component Releng attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the procedure.

How does the platform team create the source build?

See Platform-releng-sourcebuild

How do you run the source build for 3.5 builds?

See Platform-releng-sourcebuild35 (document still in progress)

How does the platform team sign their builds?

See Platform-releng-signedbuild

How long does the build take to complete?

It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.

When is the next build?

Please refer to the build schedule.

When is the next milestone?

Please refer to the Eclipse Platform Project Plan.

I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?

See bug 134413.

We have a number of tests that intermittently fail. The reasons are

  • issues with the tests themselves
  • tests subject to timing issues
  • tests that fail intermittently due to various conditions

The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.

Scripts to promote a build

  • Disable rsync
  • cd /home/users/releng/buildTools/extraTools on eclipsebuildserv
  • See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild
    • Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName>
  • Promote eclipse milestone build
    • ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4
  • Promote equinox build
    • ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4
  • Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.
  • Enable rsync, wait until build is synched to (Takes 1-2 hours)
  • Send out message to eclipse-dev,
  • Enjoy post-milestone chocolate.

I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?

There are several ways to approach this issue.

  • Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.
  • In addition, with every maintenance and integration build, the org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.
  • Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.

I'm looking for an older build but can't find them on the download page?

Check out archive site the for vintage builds.

How do I run a test build on the hudson install at ?

Login to this web page with your committer id and password.

On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).

This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug

How do I run the JUnit tests used in the build?

With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can follow the instructions associated with this zip to run the JUnit tests on the platform of your choice.

How do you run the tests on the Windows machine via rsh

To run the windows tests from the build machine,

rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt

Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)

The default Mac install only includes a JRE, not a JDK. The JDK is listed under Apple Connect under J2SE 5.0 Release 5 Developer Documentation. Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home


How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?

This document outlines the steps required to setup a test CVS repository on Linux.

How do I set up performance tests?

Refer to the performance tests how-to or the even better Performance First talk at Eclipse Con 2007.

Baseline tests are run from a branch of the builder.

  • 3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng
  • 3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng
  • 3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng

How do I find data for old performance results on existing build page?

Refer to the "Raw data and Stats" link for a specific test.

For instance for results, click here

Then look at the jdt ui tests.

Then click on a specific test.

Then click "Raw data and Stats".

You will see the data for previous builds.

How do I run the performance tests from the zips provided on the download page?

See here.

Process to implement a new baseline

Implement new performance baseline after a release.

How to regenerate performance results from build artifacts

This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.

  • cd to build directory on build machine
  • cd org.eclipse.releng.eclipsebuilder
  • apply patch to builder in bug [256297]
  • rerun generation on hacked builder
  • on build machine
    • at now
    • cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt
    • ctrl d

The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.

Why should I package plugins and features as jars?

See the Running Eclipse from JARs document from the Core team.

Why should I use qualifiers?

See the Versioning Guidelines

How do I incorporate qualifiers into the build process?

Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.

Additional steps that the platform releng team uses to incorporate qualifiers into the build process.

For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.

In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build

<condition property="forceContextQualifier" value="${buildId}">
    <equals arg1="${buildType}" arg2="N" />

Also, in the file which kicks off our build, we specify -DgenerateFeatureVersionSuffix=true

    -buildfile buildAll.xml

How can I run the update manager from a command line to mirror a remote site?

Refer to running update manager from command line in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.

I have questions about PDEBuild - where should I go for answers?

Consult the PDE Build faq or ask on the org.eclipse.platform newsgroup.

Debugging pde build with missing dependancies

Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint In display view, state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false)) print the errors for the bundle in question.

I'd like to incoporate source bundles in my build, how do I do it?

See Individual Source Bundles

Are there RSS feeds for builds?

Yes, for I-Builds only. They are available here.

If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.

See the adding javadoc document.

Troubleshooting test failures

If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.

To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.

        value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/>


  • create a remote debugging launch target in your dev environment
  • put a breakpoint somewhere in your code
  • launch the tests from the command line, using the -noclean option to preserve the modified test.xml
  • launch the debug target

I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?

Please cc the generic mail alias Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.

Eclipse Release checklist

Release checklist

New JUnit/performance machine deployment checklist

Steps to take after OS install from IT

All platforms

  • verify hostname is set on machine
  • verify hostname is registered in DNS and both forward and reverse lookups resolve correctly
  • disable screen saver
  • disable all power management options so that the machine, disk, monitor etc is always on
  • create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it
  • install ganglia and configure to interact with appropriate eclipse build node
  • verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to,
  • disable real time virus scanning of build directories
  • ensure that machines starts as build user automatically upon reboot


  • install rshd
    • set rshd user equivalency to build user
    • ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup
  • ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them


  • copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention
  • ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests
  • update /etc/hosts to include localhost
  • disable iptables in chkconfig and stop the service

Process to release builds to Simultaneous Release

  • Check out from cvsroot/callisto. For eclipse platform, update versions in features-ep.xml
  • Copy features and plugins directories from eclipsebuildserv:/builds/transfer/files/testUpdates to updates/3.3milestones

in /builds/transfer/files/testUpdates to updates/3.3milestones, update site.xml with content from site.xml in testUpdates

  • Create new digest (instructions to follow) On eclipsebuilderv /home/users/releng/buildTools/scripts/
  • Send note to cross-project and planning council mailing lists that the features-ep.xml has been updated.

Process to release builds to Simultaneous Release (maintenance stream)

  • Check out from cvsroot/callisto. For eclipse platform, update versions in features-ep.xml
  • Copy maintenance updates from eclipsebuildserv to fullmoon, on eclipsebuildserv /home/users/releng/buildTools/scripts/
  • Copy maintenance updates from fullmoon to, on fullmoon run /home/users/releng/buildTools/scripts/
  • Send a note to cross-project and planning council mailing lists that the features-ep.xml has been updated.

How do I verify the signatures of packed jars on an update site?

See bug 193568 for the tool and instructions.

Where are the p2 update sites for the Eclipse Project?

See the list

How do I use the p2 zipped repos on the build page to provision my install or a pde target?

Where can I download foundation libraries?

Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.

Members can access the content or the OSGi R4.1 specification at The latest builds of the OSGi R4.2 specification can be accessed by members at

Platform Releng Helios Plan

Helios plan

Platform Indigo Plan

Indigo plan

How to assemble the deltapack using the p2 slicer

Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.

<project default="build">
	<target name="build">
		<property name="repo" value="" />
		<property name="tempDir" value="D:\\deltapack" />

		<p2.mirror source="${repo}">
			<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" />
			<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" />
			<iu id="" version="" />
			<iu id="" version="" />
			<iu id="" version="" />
			<iu id="org.eclipse.equinox.executable" version="" />
			<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" />

Unzip 3.5RC2 to a directory and run the script using the AntRunner.

D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200. v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml

The resulting files will be in the d:\deltapack directory on your file system.

How to verify the packed jars in a repo

java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder>

Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4

[Junit4 Changes]

How to avoid breaking the build

Avoid breaking the build

Basic overview of Platform releng tasks and troubleshooting tips

Platform releng basics

Submitting content to the Helios build

The helios build files are in in /cvsroot/callisto. We update the and files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.


I just ls the features directory of the current milestone, update the build files, then commit.

Invoking the signing process from the command line

  1. Login via ssh to and ensure you're in the signer's group (groups myuserid | grep signers)
  2. Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse
  3. Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/ mail /home/data/httpd/download-staging.priv/eclipse/myOutput

Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.

You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput

You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.

Connecting to the derby database using ij

export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre

export DERBY_HOME=/builds/db-derby-

cd /builds/db-derby-

connect 'jdbc:derby://localhost:1528/perfDb36';

Are the Eclipse and Equinox projects converting from CVS to git

Yes, here is the planning document

Platform-releng/Juno_Git_Migration Juno Git Migration

Eclipsecon presentations





Best releng practices from our Eclipsecon 2005 poster:

  1. Automate the build process from end-to-end and automate early.
  2. Use PDE Build in Eclipse to generate build scripts.
  3. Automate JUnit testing and performance monitoring.
  4. Automate build notifications (email).
  5. Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)
  6. Ensure stability in the build process by thorough testing of builder changes.
  7. Schedule builds at regular intervals and enforce rebuild policies.
  8. Use map files in a central CVS repository.
  9. Build on Linux.
  10. Have fun!

Useful reference documents

Who are the platform-releng committers?

Kim Moir

As the holiday season approaches, what gifts are appropriate for your favourite release engineer?

We enjoy fine chocolate, Shaan, and apple juice.

What does the platform-releng team do when they're not running the build farm and wrangling bugs?

We herd cats. s/eds/ibm/g

What factors contribute to the low percentage of women participating in free software/open source?

Cambridge University recently completed a study on this very topic with interesting results.

Here is another one written by Val Henson, a Linux kernel committer.

2008 report from Catalyst

Deprecated contents from Eclipse Platform Release Engineering FAQ

Old content from the faq can be found here

Back to the top