Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
EMF Compare/Build
Contents
Build job locations
The latest version can always be built with the master-nightly job. The current master contains version 2.2.*.
- Master : https://hudson.eclipse.org/emfcompare/job/master-nightly/
- 2.1.* : https://hudson.eclipse.org/emfcompare/job/maintenance-2.1/
- 1.3.* : https://hudson.eclipse.org/emfcompare/job/archived-1.3/
- 1.2.* : https://hudson.eclipse.org/emfcompare/job/archived-1.2/
- 1.1.* : https://hudson.eclipse.org/emfcompare/job/archived-1.1/
p2 repositories (a.k.a. update sites) locations
EMF Compare builds are stored in p2 repositories that are produced as part of the build process. This section provides an overview of the different repositories maintained by the EMF Compare project, and their corresponding location and retention policy.
EMF Compare update sites are located on the server build.eclipse.org
within the folder /home/data/httpd/download.eclipse.org/modeling/emf/compare
. There are two main folders below that: updates and simrel.
/updates
The updates
folder contains three sub-folders, each corresponding to one kind of build.
/nightly
Contains the nightly builds. The nightly builds are automatically published by the Hudson jobs. See
/integration
Contains the integration builds. Integration builds are nightly builds that are believed to in a stable, consistent state.
/releases
Contains all of the releases.
// FIXME Each of these sub-folders are further divided in the versions of the builds they contain. Furthermore, the "release" sub-folder is itself an aggregate update site that contains all of the released versions.
There is one aggregate update site for each of the "type/version" subfolder that will contain all of the corresponding versions, and one update site for each of the builds. For example, if we promote the "2.1.0M6" milestone, which qualifier was "201303191750", an update site containing only that version will be created in milestones/2.1/S201303191750/, and it will be accessible from p2 at http://download.eclipse.org/modeling/emf/compare/updates/milestones/2.1/S201303191750 . Furthermore, it will be aggregated in the milestones/2.1 update site, accessible from p2 at http://download.eclipse.org/modeling/emf/compare/updates/milestones/2.1. // END FIXME
/simrel
The simrel
contains the update sites related to the Eclipse Simultaneous Releases.
- /staging, temporary staging area to test the preliminary candidates for simultaneous release, i.e. the latest nightly build for the next simultaneous release (Luna at the time of writing). This should be the URL in the b3aggr of the next simrel most of the time. Use the proper milestones otherwise.
- /maintenance, temporary staging area to test preliminary candidates for maintenance release, i.e. the latest nightly build for the version to be contributed to the currently released simrel (Kepler at the time of writing). This should be the URL in the b3aggr of the current simrel most of the time. Use the proper milestones otherwise.
- /milestones, temporary staging area where integration builds targeted to integrate a simrel are promoted. It contains milestones (e.g., 2.1.0M4, 3.0.0M5, 2.1.2.RC1, etc.) to be aggregated to the release repository.
Launching a Build
Members of the EMF Compare team are the only persons that can launch builds. You will need to be connected to hudson and on the page of the job you want to run (see above).
There are four parameters to a build :
- Type
- N => Nightly. Unstable builds that won't be tested in-depth and are not meant to be largely distributed. One is launched every night.
- I => Integration. Should be used for builds that are meant to be tested by users, for example if a build has been launched to fix a given bug which resolution must still be verified.
- M => Maintenance. Not used by EMF Compare.
- S => Stable. Should be used for every Milestone or Release Candidate (RC) builds.
- R => Release. Must never be used from hudson. The "R" or "Release" version is in fact a renaming of the latest "Stable" RC build; refer to the promotion instructions below.
- Alias
- Must always be filled for "S" type builds. The alias format must follow the rule <major>.<minor>.<micro><build alias>. For example, when building the M7 milestone of version 2.1.0, the alias must be filled to "2.1.0M7" or to "2.1.0RC2" when building the second release candidate.
- Sign
- Mandatory for all "S" type builds. This can be left unchecked for N and I builds.
- Platform
- Kepler => Default platform at the time of writing. This will mean that the build must be run with Kepler (Eclipse 4.3) dependencies.
- Juno => If the build must be run against Juno (Eclipse 4.2) dependencies.
- Indigo => For builds that need Indigo (Eclipse 3.7) dependencies.
Promotion Instructions
Note that you need access rights on build.eclipse.org so that you can connect via ssh build.eclipse.org. If you need to promote a build but lack a shell access, open a bug such as 386379 to request an unrestricted shell access explaining "why" you need it.
For release and milestone builds, you will also need write access on
- the EMF Compare web site ( http://git.eclipse.org/c/www.eclipse.org/emf/compare.git/ ), and
- the aggregator's repository ( http://git.eclipse.org/c/simrel/org.eclipse.simrel.build.git/ )
For all builds
- Launch build on the required hudson job. Let's assume here that you build the latest version through job emf-compare-master.
- Once the build has passed
- Connect to build.eclipse.org (ssh build.eclipse.org)
- Go to the appropriate folder for promotion (here, cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/nightly/)
- Find the correct build (either in latest or in the directory <version>.<qualifier> (example 3.1.0.201502030705) then cd inside
- Create a folder with the <qualifier> above inside /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/milestones/3.1/ (change version accordingly) named S+qualifier, for example S201502030705. (mkdir /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/milestones/3.1/S201502030705)
- Copy all content of the current folder in the one we just created (cp -r features/ plugins/ artifacts.jar content.jar /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/milestones/3.1/S201502030705)
- You can now cd in that target folder (cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/milestones/3.1/S201502030705)
- copy the file p2.index from any other previous build in this folder (for example cp ../S201412161011/p2.index .)
- We need to aggregate this new build into the composite for its version. Go back to the folder above : cd ..
- ant -f /shared/modeling/tools/promotion/manage-composite.xml add -Dchild.repository=S201502030705 -Dcomposite.name="EMF Compare 3.1 milestones"
- Check that the two following update sites both exist and contain a build with the proper qualifier
- The aggregate site for the build type and version (for example when building EMF Compare 2.1.0M6 : http://download.eclipse.org/modeling/emf/compare/updates/milestones/2.1)
- The update site that contains solely that build (for example for EMF Compare 2.1.0M6 : http://download.eclipse.org/modeling/emf/compare/updates/milestones/2.1/S201303191750)
- (All update site locations are explained in the dedicated section above.)
- We need to create the "archived repository" download.
- create a folder with the same qualifier as above in the appropriate "version" folder in /home/data/httpd/download.eclipse.org/modeling/emf/compare/downloads/drops/ named as above : S'+<qualifier>. (/home/data/httpd/download.eclipse.org/modeling/emf/compare/downloads/drops/3.1.0/S201502030705)
- zip the content of our build's update site (except p2.index) and copy it to that new one:
- zip -2 -r emf-compare-update-3.1.0<alias>.zip artifacts.jar content.jar features/ plugins/ (replace <alias> with the current milestone or release candidate name. For example emf-compare-update-3.1.0M5.zip)
- mv emf-compare-update-3.1.0<alias>.zip /home/data/httpd/download.eclipse.org/modeling/emf/compare/downloads/drops/3.1.0/S201502030705
- we can now move over there cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/downloads/drops/3.1.0/S201502030705
- We need to create the checksum for this zip file:
- md5sum emf-compare-update-3.1.0<alias>.zip > emf-compare-update-3.1.0<alias>.zip.md5
- Check the md5sum : md5sum -c emf-compare-update-3.1.0<alias>.zip.md5 (should say "OK" on the command line)
For the release
The R build will be a copy of the latest RC that was promoted. Promoting the release will be made in two steps : creating the release build, then making it available ; the latter of which should only be done the day of the official release train's release.
Creating the release build
Let's assume "2.1.0" is the released version here, assume the qualifier of the latest RC, "2.1.0RC4", is 201306110000. All of the following is done on build.eclipse.org.
zips
- Go to the "drops" folder corresponding to your release
- cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/downloads/drops/2.1.0
- copy the folder of the latest RC into a folder with the same qualifier, but with the "R" prefix instead of "S"
- cp -r S201306110000 R201306110000
- Go into this new folder
- cd R201306110000
- Rename all zips and md5 to remove their "RC" alias
- mv emf-compare-update-2.1.0RC4.zip emf-compare-update-2.1.0.zip
- mv emf-compare-update-2.1.0RC4.zip.md5 emf-compare-update-2.1.0.zip.md5
- Change the md5 to remove the alias from the file names
- vi emf-compare-update-2.1.0.zip.md5
- Check all md5, it must be OK
- md5sum -c *.md5
- Hide the zip from the download page (this should be done on your own machine, not on build.eclipse.org)
- git clone http://git.eclipse.org/c/www.eclipse.org/emf/compare.git/ org.eclipse.emf.compare.website
- cd org.eclipse.emf.compare.website/downloads
- vi index.php
- locate the "$hiddenBuilds" variable, and add the zip you wish to hide to that array : "2.1.0" => array("R201306110000")
- Make sure that the zip does not appear on http://www.eclipse.org/emf/compare/downloads/
update site
- Go to the "releases" folder of the project and create the version's folder if it does not exist yet :
- cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/releases/
- mkdir 2.1
- cd 2.1
- Create a folder for the release, with the same qualifier as above
- mkdir R201306110000
- cd R201306110000
- Use the mirroring script to replicate the latest RC in this folder
- ant -f /shared/modeling/tools/promotion/mirror-repository.xml -Drepository=/home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/milestones/2.1/S201306110000/
- you might get "unable to satisfy dependency" warnings while mirroring, this isn't an issue at this point.
Making the build available
This must not be done before the day of the official release.
- Unhide the zip from the download page (this should be done on your own machine, not on build.eclipse.org)
- cd org.eclipse.emf.compare.website/downloads
- vi index.php
- locate the "$hiddenBuilds" variable, and remove the zip you wish to make available from that array : "2.1.0" => array("R201306110000")
- Add the release update site to the composite update site of the released version
- ssh build.eclipse.org
- cd /home/data/httpd/download.eclipse.org/modeling/emf/compare/updates/releases/2.1
- ant -f /shared/modeling/tools/promotion/manage-composite.xml add -Dchild.repository=R201306110000 -Dcomposite.name="EMF Compare 2.1 Releases"
Changing the aggregator (simultaneous release)
- Cloner le repository de la release simultanée
- git clone ssh://<user>@git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git/
- Install the b3 editor in the eclipse where you'll change the aggregation (for an Eclipse 4.2, it can be installed from http://download.eclipse.org/modeling/emft/b3/updates-4.2)
- Checkout the simrel project in your eclipse, on the accurate branch (Juno_maintenance for Juno, master for the current release train)
- Open the main simrel file (simrel.b3aggr) with the b3 editor
- Locate the EMF Compare entry, and change the update site it references so that it points toward the latest milestone (either through the aggregate update site or through the "milestone only" one).
- Make sure that the file validates (right-click within the b3 editor, then select "Validate Aggregation")
- Commit and push the modification
- There should only be changes in the emf-compare.b3aggrcon file.
Checklist
Standard build
- Launch the build from https://hudson.eclipse.org/emfcompare/job/master-nightly/
- Promote the build
- Check that the two impacted update sites properly contain the build
- http://download.eclipse.org/modeling/emf/compare/updates/<type>/<version> and
- http://download.eclipse.org/modeling/emf/compare/updates/<type>/<version>/<qualifier>
- Check that the build can be installed, via p2, from the aggregate update site
- http://download.eclipse.org/modeling/emf/compare/updates/<type>/<version>
- Check that the build appears on the download page, with the appropriate alias if any
Milestone build
First follow all steps of the standard builds. Then :
- Create a tag for the build, named after the alias and containing the qualifier in the message. For example our 2.1.0M7 was tagged with :
- git tag -a '2.1.0M7' -m 'EMF Compare 2.1.0M7 - Luna M7 - 201305060839'
- (git tag -a '<alias>' -m '<message>' <optional SHA-1 of commit to tag>)
- Check that the commit is properly tagged
- git show 2.1.0M7
- Share the tag
- git push origin 2.1.0M7
- Retrieve the simultaneous release's repository
- git clone ssh://<login>@git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build
- Import the org.eclipse.simrel.build project from that repository
- Install the b3 editor in your eclipse from http://download.eclipse.org/modeling/emft/b3/updates-4.4
- Make sure that the aggregation references the aggregate milestones update site
- And validate the model (right click > Validate aggregation)
- Check the aggregation (example for Kepler)
- https://hudson.eclipse.org/hudson/view/Repository%20Aggregation/job/simrel.kepler.runaggregator/
- If no build is running, launch one manually... Otherwise wait for the next one.
- Within the aggregation log, make sure that there are no line mentionning "emf.compare" with a qualifier older than what you just promoted.
- (The lines of interest mostly start with "mirroring artifact osgi.bundle".)
- If an older qualifier appears
- Re-open the simrel.b3aggr file, and change it so that it references the "latest milestone only" update site. This will prevent the aggregation from "finding" older builds.
- Re-launch the aggregation (you can cancel it manually if the one running was one you launched).
- It will most likely fail, but this time the logs will show "who" requires an older version of EMF Compare.
- Send a mail to the incriminated project's team or on the cross-projects list to see if they can update their EMF Compare dependency.
- If the aggregation job fails
- The most likely reason is a validation failure of the simrel.b3aggr file.
- If you cannot debug the issue, re-open the simrel.b3aggr file and disable the EMF Compare contribution until the problem can be fixed. Mail the cross-project list for help.
Release
First follow all steps of the standard builds. Then :
- Create a tag for the build, named after the version and containing the qualifier in the message. For example to tag the EMF Compare 2.1.0 release :
- git tag -a '2.1.0' -m 'EMF Compare 2.1.0 - 201311060000'
- (git tag -a '<alias>' -m '<message>' <optional SHA-1 of commit to tag>)
- Check that the commit is properly tagged
- git show 2.1.0
- Share the tag
- git push origin 2.1.0
- Make sure you've followed the instructions to create the release zips and update site
- The zips must be ready in the downloads/drops folder
- The download must remain hidden on the download page
- The update site must be ready at http://download.eclipse.org/modeling/emf/compare/updates/releases/<version>/<qualifier>
- The release must not be aggregated on http://download.eclipse.org/modeling/emf/compare/updates/releases/<version> yet.
- Make sure that the aggregation references the update site containing only the latest release
- http://download.eclipse.org/modeling/emf/compare/updates/releases/<version>/<qualifier>
- Check the aggregation (example for Kepler)
- https://hudson.eclipse.org/hudson/view/Repository%20Aggregation/job/simrel.kepler.runaggregator/
- If no build is running, launch one manually... Otherwise wait for the next one.
- Within the aggregation log, make sure that there are no line mentionning "emf.compare" with a qualifier older than what you just promoted.
- (The lines of interest mostly start with "mirroring artifact osgi.bundle".)
- If an older qualifier appears
- Re-open the simrel.b3aggr file, and change it so that it references the "latest release only" update site. This will prevent the aggregation from "finding" older builds.
- Re-launch the aggregation (you can cancel it manually if the one running was one you launched).
- It will most likely fail, but this time the logs will show "who" requires an older version of EMF Compare.
- Send a mail to the incriminated project's team or on the cross-projects list to see if they can update their EMF Compare dependency.
- If the aggregation job fails
- The most likely reason is a validation failure of the simrel.b3aggr file.
- If you cannot debug the issue, re-open the simrel.b3aggr file and disable the EMF Compare contribution until the problem can be fixed. Mail the cross-project list for help.
- On the day of the official release of the release train, follow the instructions to make the release available.