Difference between revisions of "Papyrus/Papyrus Developer Guide/Release Process: How To"

Revision as of 04:26, 1 September 2015

The Papyrus project is built using Tycho with this releng project: Releng

Hudson Jobs

The Papyrus builds are started from Hudson. The following jobs on the Mars branch exist currently. There should always be the following jobs on each release branch:

• papyrus-Master : builds the main plug-ins of Papyrus
• papyrus-Master-Analysis : static analysis of the code, i.e. findBugs, sonar...
• papyrus-Master-Tests : builds the tests for the main plugins of Papyrus
• papyrus-Master-Tests-Failures : tests identified as failed, not regressions, kept to keep the main tests clean and without spams
• papyrus-Master-Developer : developer tools, such as debug views in eclipse, papyrus diagram code generator and building tools
• papyrus-Master-Extra : builds the "extra" plug-ins of Papyrus
• papyrus-Master-Extra-Tests : builds the tests for the "extra" plug-ins

See all the currently defined Papyrus jobs on the page : https://hudson.eclipse.org/papyrus/view/<Eclipse version> You can add the name of the eclipse release to find the associated jobs, i.e. Mars, Luna...

What is built?

Taking the example of "papyrus-Master", we use the releng's pom.xml files.

A good/nice practice is to also update Oomph's papyrus.setup (just select it as well) while we are at it.

These files must include all features, plugins and update sites that must be built, either directly or indirectly.

The releng's pom.xml files also specifies how to get the plug-ins and features that are to be built and specifies the update sites from which to retrieve the binary dependencies from the dependent projects.

Updating the releng's pom.xml

The file reflects the dependencies on the other projects. So, it must be updated at least before each milestone, to make sure that the code that is built works with the latest versions of all the dependencies.

The easiest way to update the dependencies is to copy them from the aggregation build model, which all projects must fill in with the correct reference to their update site(s) and feature(s).

This model is in a project with the release name on the Eclipse Git:

• git://git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git

Clone the simrel repository

Import the simrel project

Warning: don't forget to pull the last changes and check for recent updates in order to build the correct state and verify that the branch built is the correct one (master or <Eclipse version>_maintenance, e.g. Luna_maintenance)

update the local repository

Update the dependencies of the selected poms

Choose the build model from the simrel project

The simrel model used to update the dependencies

A few dependencies are not in the aggregation build model though, so they must be updated manually:

• Orbit : you can find the latest version listed on : http://download.eclipse.org/tools/orbit/downloads/

Get all the occurrences of this case and their context

Inside the releng's pom.xml

As a reminder:

• find . -name "pom.xml" searches from the current (.) folder all the pom.xml files
• | passes the output of one command to another for further processing
• 'xargs constructs a command from a standard input and prints it in the command window
• grep searches for the occurrence of the name in the found file
• -n -ai shows the line number (-n) of the occurrence and the i next lines (the i lines after the occurrence)
• {} placeholder for the found file

Finding dependencies

If you don't know where a particular plug-in or feature is located, the easiest way to find it is to search the Eclipse download area using "find" on the ssh of build.eclipse.org. For example, using the ssh connection to the eclipse repository, to look for "org.eclipse.emf.compare" :

find /home/data/httpd/download.eclipse.org/modeling/emf/ -name 'org.eclipse.emf.compare_*'

Updating the versions (optional)

During a release you can have to update the plugin versions and their linked features and categories. If that is the case you will have to use the project management's menus to change the dependencies versions in every plugin of the git repository.

• Be carefull however as this will only modify opened plugins in you workspace and wont touch all the other ones. Therefore you will have to import all the plugins you want to update.
• Also you can have discrepancies between different branches (Mars, Neon,...) hence if you have to update different branches delete the previous plugins (from the old branch) and import the all the new ones (from the new branch).

Versioning tool.xml

When updating the Versions the category.xml, feature.xml and remaining pom.xml included in some of the plugins and features will need to be updated as well.

find . -type f -name "<file.xml>" -exec sed -i -e 's/<old occurence>/<new occurence>/g' {} \;

As an example:

find . -type f -name "pom.xml" -exec sed -i -e 's/<version>1.1.1-SNAPSHOT/<versiob>1.1.2-SNAPSHOT/g' {} \;
find . -type f -name "category.xml" -exec sed -i -e 's/1.1.1.qualifier/1.1.2.qualifier/g' {} \;
find . -type f -name "feature.xml" -exec sed -i -e 's/1.1.1.qualifier/1.1.2.qualifier/g' {} \;

As a reminder:

• find . -name "file.xml" searches from the current (.) folder all the file.xml files
• -type f limits the replacement to regular files
• '-exec executes the command sed
• sed -i 's/old/new/g' stream editor, in place, substitutes, regular expression representing the old value, the new value, replace all the occurences in a file not just the first one.

Be careful however as this command does not like, if your filter is a string, "/" in the names as it will interfere with the separations (s/old/new/g).

• {} placeholder for the found file
• \; ends the -exec command. All arguments following a find are taken into account until it sees the shell character to signal the end of the -exec command, e.g. ; . for the same reason, such a character must be escaped when inserting it through the shell, e.g. \ .

Push

When pushing to the papyrus' gerrit you should use the standardized message :

• [Releng] Update dependencies for <Eclipse Version> <Milestone> (e.g.: [Releng] Update dependencies for Mars M7)

Then wait for a committer to merge the changes brought by this commit.

How to start a build?

Builds, master and maintenances, are run automatically every night, every day and each hour if there has been a commit.

Manually

Only committers or release engineers in the Papyrus project can launch Papyrus build jobs from Hudson.

Go to the wanted page if you want to launch a selected build (e.g.: Mars page)

Enter the build parameters

Then:

• In Hudson, click on Build Now, change the #Build parameters as needed, and click on Build.
• You can then click on the job name in the Build History section in the left column, and then on Console Output, to follow build progress in real time.

Build parameters

Hudson builds expect these parameters:

• BUILD_ALIAS : the name of the release (0.9.0, 1.1.0M6, etc.); leave it blank for nightly and integration builds
• SIGN : whether to sign the update site (can take up to an hour on the Eclipse build server); mandatory for all builds except nightlies

Warning: Don't forget to do the same manipulation for the Master-Extra. Tip: As the signing of the build can take quite a long time, it is recommended to launch a non-signed build first as the tests are automatically launched at the end of a master branch build. As the tests roughly take the same amount of time you can see if the current build passes the tests without having to wait the extra time needed by the signing step.

How to publish a build?

Successful Nightly builds are automatically published to :

• http://download.eclipse.org/modeling/mdt/papyrus/updates/nightly/mars/

With the script

To publish a milestone or release from a build, you can use the manualPromote.sh for Luna, and manualPromote-mars.sh for Mars, script on: /opt/public/modeling/mdt/papyrus/manualPromote.sh Start this script, and fill in the parameters in the interactive script:

• the signed builds numbers
• the corresponding version
• the update site in which to publish it

Be careful however as Hudson does not make, as of yet, a zip for the tests and the current script will crash if a build number is provided and it does not find any corresponding zip.

For example:

example

In the previous example, the script will:

• publish the main Papyrus plug-ins from Papyrus-Master build #1600 to the Papyrus download in "drops/1.1.0" and the update site in "milestones/1.1/RC3/main"
• publish the extra plug-ins from Papyrus-Master-Extra build #1237 to the Papyrus download in "drops/1.1.0" in the same build folder as the main build, and the update site in "milestones/1.1/RC3/extra"
• do not publish the test results from Papyrus-Master-Tests build
• make "milestones/1.1/RC3" a composite update site, which contains "milestones/1.1/RC3/main" and "milestones/1.1/RC3/extra"
• enable p2 download statistics both on "milestones/1.1/RC3/main" and "milestones/1.1/RC3/extra"
• set the correct linux access rights on the new folders

Once the script is done, all you have left to do is to:

• Tag the build: Releases should have a tag like 1.1.0, and milestones like 1.1.0_M6
• update the aggregation model if this build must be part of the simultaneous release

Milestones / Release Candidate

Stable builds are not automatically published. They should be first tested internally before publishing. For example, to publish the 1.1.0M6 milestone build:

• First, the build is on the update site nightly http://download.eclipse.org/modeling/mdt/papyrus/updates/nightly/mars/
• Then, install the build and test it manually
• Then, publish it using the script manualPromote.sh

Add a new update site with the new build to the parent composite (~/downloads/modeling/mdt/papyrus/updates/milestones/1.1/):

• update both the compositeContent.xml and compositeArtifacts.xml files of the update site (that are located in the parent of the folder to which you extracted the update site) to add a reference to your newly added update site
  • set the value of p2.timestamp to the result of "date +%s000"
  • increase the "size" attribute of the children element
  • add a "child" element inside the "children" element with a "location" set to the name of the folder (e.g "M6")

The values to edit

• Check that the new build appears on http://www.eclipse.org/papyrus/downloads/.
• Check the edited composites on http://download.eclipse.org/modeling/mdt/papyrus/updates/milestones/?d.
  • and add browse through the directory content to check your version, e.g.: /1.1 will give you the edited root composites, and 1.1/M6 will give you the main and extra composites

Builds can be hidden from this page before a release by modifying downloads-scripts.php in /downloads/ on git://git.eclipse.org/gitroot/www.eclipse.org/papyrus.git

Simultaneous Release

Install the b3Aggregator plugin by selecting the wanted version from the directory.

Aggregator tools

If the build must be part of the simultaneous release, you must also:

• Use the B3 Aggregator (or a text editor if the modification is trivial) and push to the simrel's gerrit which path is : ssh://username@git.eclipse.org:29418/simrel/org.eclipse.simrel.build.git)

Usually, the only thing to change is the location (for example "milestones/1.1/M6/main"):

<repositories location="http://download.eclipse.org/modeling/mdt/papyrus/updates/milestones/1.1/M6/main" description="Papyrus">

Previous composite address

Updated composite address

Tip: You can check the status of the contribution at simrel's gerrit page

Tagging

Releases should have a tag like 1.1.0, and milestones should have a tag like 1.1.0_M6.

Checking

Check simrel reports

After the simrel build, you can check the reports at :

• https://hudson.eclipse.org/simrel/job/simrel.build.repoReportsAndTests/lastSuccessfulBuild/artifact/test-report/reports-staging/reporeports/index.html

Check installations

Check installation from the simrel/staging update site at :

• http://download.eclipse.org/releases/staging (Master)
• http://download.eclipse.org/releases/maintenance (Maintenance)