Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Papyrus/Papyrus Developer Guide/Release Process: How To"
(→How to start a build?) |
(→D-day of the GA release) |
||
| (252 intermediate revisions by 7 users not shown) | |||
| Line 1: | Line 1: | ||
| − | The Papyrus project is built using [ | + | The Papyrus project is built using [https://eclipse.org/tycho/ Tycho] and orchestrated by this [https://git.eclipse.org/c/papyrus/org.eclipse.papyrus.git/tree/releng releng project]. |
| − | + | ||
| − | ( | + | The release engineer must have: |
| + | * Unrestricted shell access to build.eclipse.org (log a bug to Community/Server to obtain SSH access: see [http://bugs.eclipse.org/bugs/show_bug.cgi?id=366699 bug 366699]). To know if your shell is restricted, try to execute linux commands and see if you get disconnected. | ||
| + | * The rights to launch jobs on the integration platform : https://hudson.eclipse.org/hudson/ (should come with the Papyrus committer rights, but can be granted by the platform admin) | ||
| + | * Write access to /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus (should come with the Papyrus committer rights) | ||
| + | * Subscribed to the [https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev cross-project-issues-dev] mailing list, on which discussions relevant to the releng activities take place | ||
| + | * Be aware of the release plan and the step he needs to contribute to, +3 for papyrus, which can be found [https://wiki.eclipse.org/SimRel/Simultaneous_Release_Cycle_FAQ#What_is_the_schedule_of_the_next_releases_.3F here] | ||
| + | |||
| − | |||
== Hudson Jobs == | == Hudson Jobs == | ||
| − | The Papyrus builds are started from | + | |
| − | There should always be the following jobs on each | + | The Papyrus builds are started from Jenkins. The following jobs on the [[https://ci.eclipse.org/papyrus/view/Master/ | Master branch]] exist currently. |
| + | There should always be the following jobs on each release branch: | ||
* papyrus-Master : builds the main plug-ins of Papyrus | * papyrus-Master : builds the main plug-ins of Papyrus | ||
* papyrus-Master-Analysis : static analysis of the code, i.e. findBugs, sonar... | * papyrus-Master-Analysis : static analysis of the code, i.e. findBugs, sonar... | ||
* papyrus-Master-Tests : builds the tests for the main plugins of Papyrus | * papyrus-Master-Tests : builds the tests for the main plugins of Papyrus | ||
| − | * papyrus-Master-Tests-Failures : tests | + | * papyrus-Master-Tests-Failures : tests identified as failed, not regressions, kept to keep the main tests clean and without spams |
| − | * papyrus-Master- | + | * papyrus-Master-Toolsmiths : developer tools, such as debug views in eclipse, papyrus diagram code generator, building and customization tools |
| − | + | ||
| − | + | ||
| − | + | ||
| − | See all the currently defined Papyrus jobs on | + | See all the currently defined Papyrus jobs on the page : |
| − | https://hudson.eclipse.org/papyrus/view/ | + | <nowiki>https://hudson.eclipse.org/papyrus/view/<release_name></nowiki> |
You can add the name of the eclipse release to find the associated jobs, i.e. Mars, Luna... | You can add the name of the eclipse release to find the associated jobs, i.e. Mars, Luna... | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | == Before starting a build == | |
| − | + | ||
| − | + | === General Information === | |
| − | + | Unlike Nightlies, Milestone builds, contributed to the release train, are publicly available to anyone downloading the latest Developer Build of Eclipse. This implies a few differences in terms of quality of the release: | |
| − | + | ||
| − | + | * All artifacts must be signed, using the Eclipse Certificate | |
| − | + | * The build must be as stable as possible. Especially, any Blocker or Critical bug must be fixed before the release | |
| − | + | * All License files must be available and up-to-date | |
| − | + | * All plug-in metadata must be defined (Especially, bundle name, bundle vendor and feature description) | |
| + | * The Papyrus update site must contain everything that is not already contained in the Eclipse Simrel repository (i.e. any plug-in from an Eclipse Project that is not in the release train) | ||
| − | |||
| − | === | + | === Must Have === |
| − | + | As a rule of thumbs, before starting your build journey it is preferable to have a clean install of the Eclipse package you want to test your build against and install the necessary tools and dependencies. | |
| + | You will need to install the following (some may require you to fetch previously build components from other projects that contributes earlier in the train): | ||
| − | + | Use an Eclipse with Papyrus developer tools installed from the following update site, eg for the XYZ version: | |
| + | <nowiki>https://hudson.eclipse.org/papyrus/job/Papyrus-XYZ-Toolsmiths/lastSuccessfulBuild/artifact/repository</nowiki> | ||
| − | + | This install might need you to posses the following papyrus nightly build address in your available update sites as the developper tools depends on some plugins contained in it: | |
| + | <nowiki>https://hudson.eclipse.org/papyrus/job/Papyrus-XYZ/lastSuccessfulBuild/artifact/repository/</nowiki> | ||
| + | Install the tpd updater tool from this [https://github.com/mbarbero/fr.obeo.releng.targetplatform Github project] from this update site: | ||
| + | <nowiki>https://dl.bintray.com/mbarbero/fr.obeo.releng.targetplatform/latest</nowiki> (4.1.0 and before) | ||
| + | <nowiki>http://download.eclipse.org/cbi/tpd/**</nowiki> (4.2.0 and after) | ||
| − | + | Papyrus (a clean clone is also useful to have for the builds as it will be much less susceptible to be corrupted by your daily work) up to date: | |
| + | <nowiki>git://git.eclipse.org/gitroot/papyrus/org.eclipse.papyrus.git</nowiki> | ||
| + | Simrel on which you will later post your updated aggregator file: | ||
| + | <nowiki>git://git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git</nowiki> | ||
| − | + | Another should have, not mandatory but useful, is the b3 aggregator editor to get a clearer view of the aggregator files in the simrel repository: | |
| − | + | https://wiki.eclipse.org/Eclipse_b3/aggregator/manual#Installation | |
| − | + | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | == Update Papyrus Dependencies and References == | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | You need to [[Papyrus Developer Guide/Release Process: How To/Update Papyrus Dependencies | prepare the content]] for the next release by checking that everything depends on the latest releases of the train and that the versions are all up to date. | |
| − | |||
| − | |||
| − | + | == Publish The artifacts == | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | When the produced artifacts are successfully produced by the Jenkins job and that you tested their installation and functionalities you will be able to [[Papyrus Developer Guide/Release Process: How To/Publish Papyrus Artifacts | publish them]]. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | |||
| − | |||
| − | |||
| − | === | + | === Checking the reports === |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | At the end of each milestone you will need to check the installation from the staging update site and check the reports generated by the simrel aggregation job. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | The simrel reports are available at : | |
| − | + | * <nowiki>https://ci.eclipse.org/simrel/job/simrel.runaggregator.pipeline/**BuildNumber**/artifact/target/repository/final/buildInfo/reporeports/index.html</nowiki> | |
| − | * | + | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | ** | + | |
| − | * | + | |
| − | * | + | |
| − | + | ||
| − | + | And the simrel/staging update site at : | |
| − | + | * http://download.eclipse.org/releases/staging (Master) | |
| + | * http://download.eclipse.org/releases/maintenance (Maintenance) | ||
| − | == | + | == Preparing the GA release == |
| − | + | When the final artifacts have been contributed and each milestones have been tagged in the repository, it is time to [[Papyrus Developer Guide/Release Process: How To/Prepare the GA Release | prepare the for the GA release]]. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | === | + | == D-day of the GA release == |
| − | + | ||
| − | + | ||
| − | + | Keep in mind that depending on the date of your release review, you might have an official release day after the GA of the built Eclipse Package. | |
| − | + | ||
| − | + | ||
| − | + | When the day comes however (meaning that the release is reviewed and the Eclipse has officialy declared the GA on the cross-project mailing list), you will have to: | |
| − | + | # Run the [https://ci.eclipse.org/papyrus/view/MasterRelease/ GA publishing script], | |
| − | + | # Amend the composites in the release folder to point to the new release and remove the dummy index.html file, | |
| − | + | # Merge the prepared patches for the Website and your new development branch, | |
| − | + | # Update the simrel file. | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | Finally, do not forget to remind the Papyrus Lead, again annoy as needed, to communicate about the release: | |
| − | + | # mail in the mailing list dev, | |
| − | + | # post on the forum. | |
| − | + | ||
| − | + | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | This will end the release process. Good job! :) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | == Maintenance == | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | After a while you will need to move some milestones and releases to the archives, meaning moving them from downloads.eclipse.org to archive.eclipse.org that is not mirrored. It is a good practice as it will save time, space and resources server-wise. | |
| − | + | The corresponding paths on the server are: | |
| − | + | /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/downloads/drops/ | |
| − | + | /home/data/httpd/archive.eclipse.org/modeling/mdt/papyrus/downloads/drops/ | |
| − | + | ||
| − | + | This can be done manually, until a better solution has been implemented, but be aware of the following guidelines: | |
| − | + | * d.e.o - downloads/drops: keep the last 2 or 3 releases depending on the span you want to cover and that the removed content has been copied in archive of course | |
| − | + | * d.e.o - updates/releases: everything here should be kept. Very old releases could be moved to the archive server depending on usage. | |
| − | + | * d.e.o - updates/milestones: this can be handled more loosely, but a good rule of thumb is to keep the last 2 or 3 milestones of what you want to keep in order to | |
| − | + | allow eclipse to 'revert to previous installation' if needed. | |
| + | * a.e.o - downloads/drops: this '''needs''' to contain '''every''' releases published. There are no constraint on how recent or old they need to be so if you are not sure you will remember you can copy the release as soon as its out ;). You can also put milestones in here but that is not required. | ||
Latest revision as of 10:38, 10 July 2020
The Papyrus project is built using Tycho and orchestrated by this releng project.
The release engineer must have:
- Unrestricted shell access to build.eclipse.org (log a bug to Community/Server to obtain SSH access: see bug 366699). To know if your shell is restricted, try to execute linux commands and see if you get disconnected.
- The rights to launch jobs on the integration platform : https://hudson.eclipse.org/hudson/ (should come with the Papyrus committer rights, but can be granted by the platform admin)
- Write access to /home/data/httpd/download.eclipse.org/modeling/mdt/papyrus (should come with the Papyrus committer rights)
- Subscribed to the cross-project-issues-dev mailing list, on which discussions relevant to the releng activities take place
- Be aware of the release plan and the step he needs to contribute to, +3 for papyrus, which can be found here
Contents
Hudson Jobs
The Papyrus builds are started from Jenkins. The following jobs on the [| Master branch] exist currently. There should always be the following jobs on each release branch:
- papyrus-Master : builds the main plug-ins of Papyrus
- papyrus-Master-Analysis : static analysis of the code, i.e. findBugs, sonar...
- papyrus-Master-Tests : builds the tests for the main plugins of Papyrus
- papyrus-Master-Tests-Failures : tests identified as failed, not regressions, kept to keep the main tests clean and without spams
- papyrus-Master-Toolsmiths : developer tools, such as debug views in eclipse, papyrus diagram code generator, building and customization tools
See all the currently defined Papyrus jobs on the page : https://hudson.eclipse.org/papyrus/view/<release_name> You can add the name of the eclipse release to find the associated jobs, i.e. Mars, Luna...
Before starting a build
General Information
Unlike Nightlies, Milestone builds, contributed to the release train, are publicly available to anyone downloading the latest Developer Build of Eclipse. This implies a few differences in terms of quality of the release:
- All artifacts must be signed, using the Eclipse Certificate
- The build must be as stable as possible. Especially, any Blocker or Critical bug must be fixed before the release
- All License files must be available and up-to-date
- All plug-in metadata must be defined (Especially, bundle name, bundle vendor and feature description)
- The Papyrus update site must contain everything that is not already contained in the Eclipse Simrel repository (i.e. any plug-in from an Eclipse Project that is not in the release train)
Must Have
As a rule of thumbs, before starting your build journey it is preferable to have a clean install of the Eclipse package you want to test your build against and install the necessary tools and dependencies. You will need to install the following (some may require you to fetch previously build components from other projects that contributes earlier in the train):
Use an Eclipse with Papyrus developer tools installed from the following update site, eg for the XYZ version:
https://hudson.eclipse.org/papyrus/job/Papyrus-XYZ-Toolsmiths/lastSuccessfulBuild/artifact/repository
This install might need you to posses the following papyrus nightly build address in your available update sites as the developper tools depends on some plugins contained in it:
https://hudson.eclipse.org/papyrus/job/Papyrus-XYZ/lastSuccessfulBuild/artifact/repository/
Install the tpd updater tool from this Github project from this update site:
https://dl.bintray.com/mbarbero/fr.obeo.releng.targetplatform/latest (4.1.0 and before) http://download.eclipse.org/cbi/tpd/** (4.2.0 and after)
Papyrus (a clean clone is also useful to have for the builds as it will be much less susceptible to be corrupted by your daily work) up to date:
git://git.eclipse.org/gitroot/papyrus/org.eclipse.papyrus.git
Simrel on which you will later post your updated aggregator file:
git://git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git
Another should have, not mandatory but useful, is the b3 aggregator editor to get a clearer view of the aggregator files in the simrel repository:
https://wiki.eclipse.org/Eclipse_b3/aggregator/manual#Installation
Update Papyrus Dependencies and References
You need to prepare the content for the next release by checking that everything depends on the latest releases of the train and that the versions are all up to date.
Publish The artifacts
When the produced artifacts are successfully produced by the Jenkins job and that you tested their installation and functionalities you will be able to publish them.
Checking the reports
At the end of each milestone you will need to check the installation from the staging update site and check the reports generated by the simrel aggregation job.
The simrel reports are available at :
- https://ci.eclipse.org/simrel/job/simrel.runaggregator.pipeline/**BuildNumber**/artifact/target/repository/final/buildInfo/reporeports/index.html
And the simrel/staging update site at :
- http://download.eclipse.org/releases/staging (Master)
- http://download.eclipse.org/releases/maintenance (Maintenance)
Preparing the GA release
When the final artifacts have been contributed and each milestones have been tagged in the repository, it is time to prepare the for the GA release.
D-day of the GA release
Keep in mind that depending on the date of your release review, you might have an official release day after the GA of the built Eclipse Package.
When the day comes however (meaning that the release is reviewed and the Eclipse has officialy declared the GA on the cross-project mailing list), you will have to:
- Run the GA publishing script,
- Amend the composites in the release folder to point to the new release and remove the dummy index.html file,
- Merge the prepared patches for the Website and your new development branch,
- Update the simrel file.
Finally, do not forget to remind the Papyrus Lead, again annoy as needed, to communicate about the release:
- mail in the mailing list dev,
- post on the forum.
This will end the release process. Good job! :)
Maintenance
After a while you will need to move some milestones and releases to the archives, meaning moving them from downloads.eclipse.org to archive.eclipse.org that is not mirrored. It is a good practice as it will save time, space and resources server-wise. The corresponding paths on the server are:
/home/data/httpd/download.eclipse.org/modeling/mdt/papyrus/downloads/drops/ /home/data/httpd/archive.eclipse.org/modeling/mdt/papyrus/downloads/drops/
This can be done manually, until a better solution has been implemented, but be aware of the following guidelines:
- d.e.o - downloads/drops: keep the last 2 or 3 releases depending on the span you want to cover and that the removed content has been copied in archive of course
- d.e.o - updates/releases: everything here should be kept. Very old releases could be moved to the archive server depending on usage.
- d.e.o - updates/milestones: this can be handled more loosely, but a good rule of thumb is to keep the last 2 or 3 milestones of what you want to keep in order to
allow eclipse to 'revert to previous installation' if needed.
- a.e.o - downloads/drops: this needs to contain every releases published. There are no constraint on how recent or old they need to be so if you are not sure you will remember you can copy the release as soon as its out ;). You can also put milestones in here but that is not required.