Skip to main content
Jump to: navigation, search

Papyrus/Papyrus Developer Guide/Release Process: Doc

< Papyrus‎ | Papyrus Developer Guide
Revision as of 05:24, 20 May 2015 by (Talk | contribs) (Created page with "== Requirements to manage a build == * be a Papyrus committer (of course) * have a non-restricted shell access to (log a bug to Community/Server to obtain...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Requirements to manage a build

  • be a Papyrus committer (of course)
  • have a non-restricted shell access to (log a bug to Community/Server to obtain SSH access: see bug 366699). To know if your shell is restricted, try to execute linux commands and see if you get disconnected.
  • have write rights on (log a bug to Community/CVS to get it, see bug 366727)
    • use extssh and not pserver for this connection
  • have rights to launch this job : (should come with the previous rights)
  • have write access to /home/data/httpd/ (should come with the Papyrus committer rights)
  • you must subscribe to the cross-project-issues-dev mailing list, on which discussions relevant to the releng activities take place

configure password-less ssh access

To run scripts that access the server without having to type the ssh password for each operation, you can use ssh-keygen to create a key and put your public key "~/.ssh/" in "~/.ssh/authorized_keys" (see this documentation).

Releng Project

The papyrus releng project is called : org.eclipse.mdt.papyrus.releng.buckminster


It contains sub-folders corresponding to each build:

  • papyrus-trunk-nightly
  • papyrus-trunk-nightly-tests
  • papyrus-trunk-nightly-3.8-tests
  • papyrus-trunk-extra-nightly
  • papyrus-trunk-extra-nightly-tests

And each folder contains the configuration for that build:

  • build.xml: the main ant build script : installs Buckminster, imports the dependencies and the Papyrus projects from SVN, builds the workspace, creates the update site and runs the tests
  • does a bit of cleanup and initializes variables for the build
  • packages the result in a zip, and triggers the publishing of this zip
  • buckminster.cspec : describes the top-level dependencies of the build
  • build.cquery : describes the root request for the Buckminster import, properties for the import (for example, it indicates that the Papyrus source plug-ins should not be materialized during the import, as they are generated by Buckminster during the build), and references the Buckminster rmap
  • build.mspec : description for the Buckminster materialization (i.e. import) of artifacts (plug-ins and features), when building on *
  • local.mspec : description for the Buckminster materialization when building locally
  • build.rmap : should be updated for each release. An EMF-Facet plug-in allows to update it easily using the b3 file. The <!-- updateFrom --> tags are used by this script.
  • : properties for the build (when building on *
  • : properties for the build (when building locally)

And these scripts are common to all the builds:

  • resources
    • : defines an empty workspace with settings suitable for the Papyrus build (disables the missing PDE API baseline errors)
  • serverConfig : (after each modification, the scripts in this directory must be copied into "/opt/public/modeling/mdt/papyrus" where they run : commit the script and run /opt/public/modeling/mdt/papyrus/
    • : script that enables download statistics on an update site
    • addDownloadStats-main.xsl : XSL transformation that enables download statistics on a "main" update site (it contains the list of significant plug-ins for the stats)
    • addDownloadStats-extra.xsl : XSL transformation that enables download statistics on a "extra" update site (it contains the list of significant plug-ins for the stats)
  • : script to add a child update site to a composite update site definition (calls addToComposite.xsl)
  • addToComposite.xsl : XSL transformation to add a child update site to a composite update site definition
  • : retrieves the configuration files for all the Papyrus Hudson jobs, and copies them into the hudsonJobs folder (configure a password-less ssh login to avoid having to type your password 10 times in a row). Run this script every time you make changes to the Hudson job configurations, in order to keep a versioned backup of these changes.
    • : a script that runs as a cron on to publish the latest nightlies (main, extra and tests). It runs and sends a mail if this script failed.
    • : the script that does the actual work for
    • : helper functions for the publishing scripts
    • : this is an interactive script that can be started by a Papyrus committer to publish a build into the Papyrus downloads and update site
    • : used for publishing 0.8 releases
    • : run this script from /opt/public/modeling/mdt/papyrus/ to update the scripts on the server after committing your changes
  • xsl :
    • psf : to build the psf from a Buckminster bill of materials. The BOM is produced by Buckminster when importing all the dependencies, and contains a list of these dependencies.
      • bom2repository.xsl : transforms a Buckminster BOM into a list of dependent repositories
      • build.xml : used for testing the generation of the PSFs
      • papyrus_bom.xml: used for testing the generation of the PSFs
      • repository2subclipse.xsl: creates a Subclipse PSF from a repository.xml
      • repository2subversive.xsl: creates a Subversive PSF from a repository.xml
    • content2html.xsl: creates an index.html with a user-viewable list of all installable Papyrus plug-ins and features
    • content2xml.xsl: creates an index.html with a machine-readable list of all installable Papyrus plug-ins and features

Build process

Taking the example of the job papyrus-trunk-nightly:

  • When the Hudson job papyrus-trunk-nightly starts, Hudson begins by doing a checkout of the required folders (which are specified in the Hudson job's configuration) from the SVN : the "plugins", "features" and "releng" folders are retrieved, and placed under a directory named "sourceTree" in the Hudson workspace.
  • Then the first bash script (from the releng project that was just retrieved from the SVN) is executed : "sourceTree/releng/org.eclipse.mdt.papyrus.releng.buckminster/papyrus-trunk-nightly/". This script does a bit of clean-up and initializes variables for the build.
  • Then the main build script is executed: "sourceTree/releng/org.eclipse.mdt.papyrus.releng.buckminster/papyrus-trunk-nightly/build.xml":
    • it reads the property file, which defines settings for the build
    • it cleans the workspace, target platform and previous build result (depending on the user options specified on Hudson)
    • it installs the p2 director, which is used for installing Buckminster
    • then it installs Buckminster using the p2 director
    • then it materializes the workspace and target platform (using the buckminster import command). While doing this, it also saves the description of what was materialized into a file named papyrus_bom.xml (this is called the "bill of materials" in Buckminster's jargon).
    • then it builds the workspace using the Buckminster build command
    • then it creates the update site, using the Buckminster perform command with the site.p2 action on the component "", which is the feature that contains all the Papyrus features that must be installable from the produced update site
    • then it creates an index.html file in the update site, using a XSLT transformation on the update site's content.xml. This index is useful for when users try to display the update site in their web browser : they will then see the plug-ins and features contained by the update site, with an explanation on how they're supposed to be installed
    • then it generates PSF files (one for Subclipse and one for Subversive), using XSLT transformations on the papyrus_bom.xml that was created by Buckminster during the import step
  • Then the second bash script is executed : "sourceTree/releng/org.eclipse.mdt.papyrus.releng.buckminster/papyrus-trunk-nightly/". This script creates the result zip (that contains the zipped update site), and triggers the publication of the result by touching a file that acts as a signal for the cronPromote script that does the publication from

The test builds do one more step: they run the tests using the Buckminster test command, using an Eclipse launch configuration that references the tests to run.


The feature is not visible on the Papyrus Update Site. It is only useful for the Buckminster build. It contains the main Papyrus features :

  • org.eclipse.papyrus.sdk.feature (binaries)
  • org.eclipse.papyrus.sdk.source.feature (sources + binaries)

The feature org.eclipse.papyrus.sdk.feature allows to provide the Papyrus binaries on the Update Site. This feature aggregates all the main Papyrus features.


The feature org.eclipse.papyrus.sdk.sources.feature allows to provide the Papyrus binaries and the sources on the Update Site. This feature should aggregate the same features as org.eclipse.papyrus.sdk.feature, but each feature is called xxx.source.feature instead of xxx.feature. You should have warnings in this file, because the xxx.source.feature don't exist in your workspace; they will be created during the build. This feature contains org.eclipse.papyrus.sdk.feature too.


Automatic publishing to the downloads server

Nightly builds are published automatically to

This part is a bit tricky, as the build server ( and the download server are two separate servers, with different access rights. The download server can be accessed by all committers, but the Hudson jobs are launched by a Hudson user that does not have write access to the download server. Thus, publishing results is done in two parts:

  • the Hudson task gathers all results and sends a signal to the download server (basically, writing a signal file on the download server, with information on what to publish and where).
  • A cron task runs every 5 minutes on (currently setup under user "vlorenzo"), and launches the script located in /opt/public/modeling/mdt/papyrus/ on This script compares the timestamp of the signal file to a reference file. When the signal file is more recent than the reference file, it proceeds to cleaning the download area (leaving for example only the last 5 nightly builds for each version of the plug-ins), and then getting the results provided by the hudson job to copy them in the right places in the download area.

configuration of the cron script

The publishing script is started every five minutes ("*/5") to see if there are new builds to publish and publish them.

Type this command to edit your crontab on

crontab -e

Then put this single line in the file to start the publishing script every five minutes:

*/5 * * * * /opt/public/modeling/mdt/papyrus/

Build chaining

The publication of papyrus-trunk-nightly triggers the build of papyrus-trunk-extra-nightly, papyrus-trunk-nightly-tests and papyrus-trunk-nightly-3.8-tests, by using the Hudson Remote Access API, passing it the build id and version as parameters, so that the cronPromote can publish the results of papyrus-trunk-extra-nightly and papyrus-trunk-nightly-tests to the same folder as for the first build.


Build dependencies

The Hudson jobs papyrus-trunk-nightly-tests, papyrus-trunk-nightly-3.8-tests and papyrus-trunk-extra-nightly depend on the update site created by the Hudson job papyrus-trunk-nightly.

The dependency appears in the rmap:

<rm:provider componentTypes="eclipse.feature,osgi.bundle" mutable="false" readerType="p2" source="false">
  <rm:uri format="{0}">
    <bc:propertyRef key=""/>

And the variable "" is defined in the Hudson job configuration:


Finding dependencies

If you don't know where a particular plug-in or feature is located, the easiest way to find it is to search the Eclipse download area using "find" on For example, to look for "" :

find /home/data/httpd/ -name 'org.eclipse.emf.compare_*'

Publishing a build in the aggregator

The Eclipse simultaneous release train is built with the B3 aggregator. To be part of the aggregation, you must add your contribution to the aggregation model.

First, you need to install the B3 aggregator. Theoretically, you can edit the model with any text editor, but it is better to edit it with the B3 editor, which will check that your model is valid.

It is suggested that you install the B3 aggregator in a separate Eclipse, dedicated to releng activities. For the Indigo and Juno simultaneous releases, you will need two different B3 installations :

It is important to use the right version for each, because you will break the build if you commit changes that depend on the wrong version.

  • checkout the project org.eclipse.<releaseName>.build from the Eclipse CVS ":pserver:<committerID>"
  • open "mdt-papyrus.b3aggrcon"
  • change the location on this line to point to the build you want to publish in the aggregator:
<repositories location="" description="Papyrus">
  • validate the build model :
    • open "simrel.b3aggr"
    • right-click on the root and choose Validate Aggregation
    • open the error log and see if everything went OK
  • commit your changes
  • optionally start an aggregation build (the build should start automatically after you commit changes)

contact information

The people responsible for the build should be listed as contacts in the build model:


What do you select in your files ?

The licence.html file should be selected for both binary and source builds. The source folder ("src/") should not be selected for the binary nor the source build.

optimizing the rmap

The Buckminster import is very slow (more than one hour and a half) when it does the SVN checkout itself. To optimize it, we let Hudson do the checkout, and we then tell Buckminster to use the locally checked out files.

So, instead of this configuration in the rmap:

<rm:provider componentTypes="osgi.bundle" readerType="svn">
  <rm:uri format="{0}/modeling/org.eclipse.mdt.papyrus/trunk/plugins/core/{1}">
    <bc:propertyRef key="svn.repository"/>
    <bc:propertyRef key="buckminster.component"/>

We now use the "local" reader:

<rm:provider componentTypes="osgi.bundle" readerType="local">
  <rm:uri format="{0}/modeling/org.eclipse.mdt.papyrus/trunk/plugins/core/{1}">
    <bc:propertyRef key="local.repository"/>
    <bc:propertyRef key="buckminster.component"/>

Creating a composite update site

A composite update site contains child update sites that appear as if their contents were in the composite update site. It is a way to aggregate update sites without having to physically move all the plug-ins in the same "plugins" directory (and the features in the "features" directory).

To create a composite update site, add two files named compositeArtifacts.xml and compositeContent.xml at the root of the composite update site. Here is an example of the content of these files, for a composite update site that contains two children named "main" and "extra".

"p2.timestamp" is the Unix date in milliseconds, which must be udpated everytime you change the compositeArtifacts.xml or compositeContent.xml files. To obtain the right value, use the command:

date +%s000


<?xml version="1.0" encoding="UTF-8"?>
<repository name="Papyrus" type="org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository" version="1.0.0">
  <properties size="1">
    <property name="p2.timestamp" value="1327488131000"/>
  <children size="2">
    <child location="main"/>
    <child location="extra"/>


<?xml version="1.0" encoding="UTF-8"?>
<repository name="Papyrus" type="org.eclipse.equinox.internal.p2.metadata.repository.CompositeMetadataRepository" version="1.0.0">
  <properties size="1">
    <property name="p2.timestamp" value="1327488131000"/>
  <children size="2">
    <child location="main"/>
    <child location="extra"/>

The update site directory looks like this:


You can also make a composite child point to a different location. This is useful for moving update sites to transparently (see bug 349141).

For example:

<child location=""/> 

see also [1]

Papyrus update sites

The Papyrus update sites are under:

corresponding to this directory mounted on


The Papyrus update sites follow this layout:

releases (folder)
  helios (composite update site)
  indigo (composite update site)
    0.8.0 (update site)
    0.8.1 (update site)
  juno (composite update site)
    0.9.0 (composite update site)
      main (update site)
      extra (update site)
  0.8 (composite update site)
    SR2_RC1 (update site)
    SR2_RC2 (update site)
  0.9 (composite update site)
    M4 (composite update site)
    M5 (composite update site)
      main (update site)
      extra (update site)
  indigo (composite update site)
  juno (composite update site)
    main (update site)
    extra (update site)

Papyrus downloads

The Papyrus download page is:

This page is generated from downloads-common.php and downloads-scripts.php that are defined in the folder "/www/modeling/mdt/papyrus/downloads/" on CVS "".

Download statistics

The statistics are enabled on repositories that are published with This script calls, and it uses:

  • addDownloadStats-main.xsl for statistics on the main update site
  • addDownloadStats-extra.xsl for statistics on the extra update site

These XSL transformations add the property "p2.statsURI" to the update site metadata, and add a "download.stats" property on a few bundles that are significant for the project's statistics.

The statistics can then be seen on the Eclipse portal (see the documentation below)


Archiving old builds

The server only has so much space. So, the webmaster set a 2 GiB quota per project.

To stay under this limit, old builds should be either deleted or moved to

This must be done in accordance to the Papyrus retention policy. TODO: Papyrus should define a retention policy, and link to it in the portal.

See :

Promoting a RC to a release

On the day of the release, the last release candidate build should be renamed. You should rename your release candidate build in "/modeling/mdt/papyrus/downloads/drops/x.y.z/" For example:

  • S201106151305 becomes R201106151305
  • becomes

There is nothing to rename in the update site, but you should copy the contents of the release candidate update site to the release update site.

For example, copy the contents of:





Hudson job's SVN configuration

Hudson SVN Config

Interpreting build errors

The error can sometimes be found at the end of the log, but is often burried deep within megabytes of output (especially when Buckminster is run with the DEBUG log level). So, to find an error, you can search for patterns:

  • "ERROR" for Buckminster import errors
  • "Error: file " for compilation errors

import errors

ERROR   [0009] : No suitable provider for component was found in resourceMap [...]

It means that the import phase done by Buckminster failed because Buckminster could not find plug-in "" in the rmap. If this is one of your plug-ins, you can fix this error by:

  • adding it to the rmap if it was not already matched by a pattern (regular expression) there
  • making sure that the plug-in has the right name and is in the right location on SVN
  • making sure that the plug-in is contained (indirectly) by the build feature

If this is a plug-in from another project you depend on, you can fix this error by:

  • making sure you have defined in your rmap the update site from which this plug-in must be found
  • making sure that the locator pattern points Buckminster to the right update site for this plug-in
  • making sure that you depend on the right version of the plug-in (in the referencing project's MANIFEST.MF)

Notice the "[0009]" tag before the error message: it refers to more information at the end of the build log. Search for this number, and you will find something like:

INFO:  TAG-ID 0009 = Query for org.eclipse.mdt.papyrus.releng.buckminster:buckminster, path:
org.eclipse.mdt.papyrus.releng.buckminster:buckminster$0.9.0.qualifier ->$0.9.0.qualifier -> 
org.eclipse.yyy:osgi.bundle$0.9.0.qualifier -> 

It gives you the dependency chain leading to the plug-in that cannot be resolved.

compilation errors

Error: file /opt/public/jobs/<jobName>/workspace/buildroot/buckminster.workspace/plugins/<pluginName>/src/<javaClassPath>, line xyz: <javaErrorMessage>

This is a Java compilation error.

  • Make sure that the code compiles in your workspace
  • Make sure that your development environment contains the same plug-ins with the same versions as the Buckminster build (see buildroot/result/targetPlatform in the Hudson workspace)

Test launching error

java.lang.IllegalArgumentException: Export has failed because no test session is available

When you see this error message, it means the tests failed to start. This is most often because one of the test plug-ins could not be resolved. To get more information about this problem, you can look in the Hudson workspace:


If you see messages like:

Bundle was not resolved.

Then look at the messages that follow, and you should see:

Missing required bundle xyz


Missing imported package xyz

Which indicates the dependency that was not resolved. Then you can search again for references to "xyz" in this log to determine why the dependency was not resolved. Continue to iterate like this until you get down to the root of the problem.

Once you have found the root dependency that is not resolved, then you can:

  • If the dependency is already specified in one of your plug-ins, then you maybe need to modify constraints (required version for example) so that it can be resolved
  • If the dependency is not already specified in one of your plug-ins, then you can either:
    • Add it to one of your plug-ins if it makes sense
    • If it does not make sense to specify the dependency on one of your plug-ins, because the dependency is incidental (inherited from a project you depend on for example), then you can add it to the buckminster.cspec of the test build

Sometimes, the error message says that the tests failed to launch, and to "see the log", but it doesn't indicate which log. The log for a launch failure is located in:


The Hudson build's workspace

You can see the contents of the Hudson build's workspace on Hudson. For example:

At the end of the build, the Hudson workspace contains these files:

  • buildroot : the working directory for the Buckminster build
    • result
      • output : Buckminster puts what it buildt in this directory
      • targetPlatform : the target platform that was materialized by Buckminster during the import phase, and that was used during the build. All the
    • tools
      • buckminster : the installation of Buckminster that was used for building
      • director : the p2 director that was used to install Buckminster
  • sourceTree : the Papyrus sources that were checked out from SVN by Hudson
  • tmp/N201201260428 : the temporary folder that was used to create the result zip
  • updateSite : a copy of the update site that was produced by Buckminster. The other Papyrus builds (for extra and tests) use this update site
  • : this is the result of the build, packaged in a zip ready to be published by the cron publishing script

Eclipse simultaneous release plan

The page for each project's simultaneous release is usually:<release_name>/Simultaneous_Release_Plan

For example:


It contains the date each milestone is expected to be available, depending on whether the project has a +0, +1, +2 or +3 offset. Each project determines its offset depending on its dependencies : if it depends on +2 projects, then it should be +3. And if it depends on +3 projects, then it is still +3, because there is no +4 offset. In this case, communication between the projects is essential, to make sure that each build depends on the right version of its dependencies.

Each project in the simultaneous release should also respect:

  • API Freeze at M6 : no API changes should occur after M6
  • UI freeze at M6 : no major UI changes should occur after M6, in order to give Babel translators enough time to translate everything before the release
  • Feature Freeze at M7 : no new features should be developped after M7 (only bug fixes, documentation and releng changes should be committed for the release candidates)

Final Daze

As the time for the big release in june each year approaches, each project in the simultaneous release must keep informed about the plan for the release, and perform some actions at the right time to ensure the release goes smoothly (such as cleaning up, archiving, publishing the documentation, putting the release bits in the right place for mirroring, etc.).

This is documented in a page conventionally named:<release_name>/Final_Daze

for example:

It is also very important to read the cross-project-issues-dev mailing list regularly, to stay informed about everything concerning the simultaneous release.

Entering the simultaneous release train

You should indicate that your project follows the Eclipse Train. Go to the Eclipse portal [2]. In Eclipse Projects, click on view :


click on maintain


click on edit for simultaneousrelease


add 1 to your new release


Simultaneous release metadata

You must respect the rules for being part of the simultaneous release, and fill in metadata on the portal. Most of this metadata is to document your compliance with the rules. If you don't comply with some rules, you should explain why, either with an in-place explanation if appropriate, or by linking to a web page with more information.


Copyright © Eclipse Foundation, Inc. All Rights Reserved.