Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Mylyn/Build Infrastructure"

 
(13 intermediate revisions by 2 users not shown)
Line 11: Line 11:
 
* [http://ci.mylyn.org/view/Provisioning/ Provisioning jobs] that provision the test repositories
 
* [http://ci.mylyn.org/view/Provisioning/ Provisioning jobs] that provision the test repositories
 
* <span style="color:red">'''[*]'''</span> [http://ci.mylyn.org/view/Integration/ Weekly version-specific integration builds] that build and run all integration test fixtures on all supported Eclipse versions
 
* <span style="color:red">'''[*]'''</span> [http://ci.mylyn.org/view/Integration/ Weekly version-specific integration builds] that build and run all integration test fixtures on all supported Eclipse versions
* [http://ci.mylyn.org/view/Snapshots/ Nightly test builds] that run one integration test fixture (1 build per connector)
+
* [http://ci.mylyn.org/view/Snapshots/ Nightly test builds] that run one integration test fixture (1 build per connector) (exception: the reviews build runs all fixtures)
* [http://ci.mylyn.org/job/mylyn-all-snapshot/ Nightly mylyn-all-snapshot] that runs all integration test fixtures
+
* <s>[http://ci.mylyn.org/job/mylyn-all-snapshot/ Nightly mylyn-all-snapshot] that runs all integration test fixtures</s> Disabled.
 +
 
 +
==== Maintenance ====
 +
 
 +
To start Hudson if it has stopped:
 +
sudo su hudson
 +
cd /home/hudson
 +
./start.sh
 +
 
 +
To stop or restart Hudson if it is still running, go to <nowiki>http://ci.mylyn.org/safeExit</nowiki> or <nowiki>http://ci.mylyn.org/safeRestart</nowiki>.
 +
 
 +
It may be necessary to delete /home/hudson/nohup.out if it has grown very large.
 +
 
 +
The test servers listed at mylyn.org are under /home/tools.
  
 
=== https://hudson.eclipse.org/mylyn/ ===
 
=== https://hudson.eclipse.org/mylyn/ ===
Line 19: Line 32:
 
==== Build Jobs ====
 
==== Build Jobs ====
 
* [https://hudson.eclipse.org/mylyn/view/Gerrit/ Gerrit builds] that verify Gerrit reviews. See [[Mylyn/Contributor_Reference#Hudson]] for information about the contributor whitelist.
 
* [https://hudson.eclipse.org/mylyn/view/Gerrit/ Gerrit builds] that verify Gerrit reviews. See [[Mylyn/Contributor_Reference#Hudson]] for information about the contributor whitelist.
* [https://hudson.eclipse.org/mylyn/view/Nightlies/ Nightly snapshot builds] that run one integration test fixture and publish nightly snapshots (1 build per connector)
+
* [https://hudson.eclipse.org/mylyn/view/Nightlies/ Nightly snapshot builds] that run one integration test fixture and publish nightly snapshots
 
* <span style="color:red">'''[*]'''</span> [https://hudson.eclipse.org/mylyn/job/mylyn-integration/ Integration build] that builds and runs unit tests (no integration fixtures) against all supported Eclipse versions
 
* <span style="color:red">'''[*]'''</span> [https://hudson.eclipse.org/mylyn/job/mylyn-integration/ Integration build] that builds and runs unit tests (no integration fixtures) against all supported Eclipse versions
 
* [https://hudson.eclipse.org/mylyn/job/mylyn-integration-connectors/ Integration-connectors build] that runs all integration test fixtures
 
* [https://hudson.eclipse.org/mylyn/job/mylyn-integration-connectors/ Integration-connectors build] that runs all integration test fixtures
Line 32: Line 45:
 
== Snapshots ==
 
== Snapshots ==
  
Nightly snapshots are published to http://download.eclipse.org/mylyn/snapshots/nightly/ by the [https://hudson.eclipse.org/mylyn/view/Nightlies/ Nightly builds]. The [https://hudson.eclipse.org/mylyn/view/Releases/ Release builds] publish releases and snapshots to http://download.eclipse.org/mylyn/drops. These snapshots are used to build SRs via composite sites like http://download.eclipse.org/mylyn/snapshots/3.14/.
+
Nightly snapshots are published to http://download.eclipse.org/mylyn/snapshots/nightly/ by the [https://hudson.eclipse.org/mylyn/view/Nightlies/ Nightly builds]. The [https://hudson.eclipse.org/mylyn/view/Releases/ Release builds] publish releases and weekly snapshots to http://download.eclipse.org/mylyn/drops. These snapshots are used to build SRs via composite sites like http://download.eclipse.org/mylyn/snapshots/3.14/.
  
 
== Build Configuration ==
 
== Build Configuration ==
Line 44: Line 57:
 
The matrix builds build Mylyn against different Eclipse versions. The mylyn-eX.Y targets are used to build against the latest Eclipse release in the given stream (e.g. 4.4.2, 4.5) or the latest milestone build of the upcoming release; mylyn-emaintenance is used to build against the latest candidate for the next Eclipse update release (e.g. 4.5.1); mylyn-estaging is used to build against the latest candidate for the next Eclipse major release (e.g. 4.6). The maintenance, staging, and latest X.Y targets build against the latest plugin versions available on the Eclipse platform update site, whereas the released X.Y targets build against specific versions in the release.
 
The matrix builds build Mylyn against different Eclipse versions. The mylyn-eX.Y targets are used to build against the latest Eclipse release in the given stream (e.g. 4.4.2, 4.5) or the latest milestone build of the upcoming release; mylyn-emaintenance is used to build against the latest candidate for the next Eclipse update release (e.g. 4.5.1); mylyn-estaging is used to build against the latest candidate for the next Eclipse major release (e.g. 4.6). The maintenance, staging, and latest X.Y targets build against the latest plugin versions available on the Eclipse platform update site, whereas the released X.Y targets build against specific versions in the release.
  
Once an update site is available for the upcoming major release, a new target should be created for that release, and axes should be added for that target to the matrix builds marked with <span style="color:red">'''[*]'''</span> above:
+
The targets should be updated at several points during the Eclipse release cycle.
* create target by copying the latest and changing the target name and file name
+
 
 +
=== When a New Eclipse Version Becomes Available ===
 +
 
 +
Once the update sites are available for the upcoming major release, a new target should be created for that release, and axes should be added for that target to the matrix builds marked with <span style="color:red">'''[*]'''</span> above:
 +
 
 +
* create target by copying the latest X.Y target and changing the file name (eg. mylyn-e4.5.target -> mylyn-e4.6.target)
 +
* change the target name in the file to match the release (eg. mars -> neon)
 +
* change the update site URL to the release URL of the relevant version (eg. http://download.eclipse.org/releases/neon)
 
* change Eclipse Platform dependency versions to 0.0.0
 
* change Eclipse Platform dependency versions to 0.0.0
Old axes should be removed once they are no longer supported, except for the Indigo axis because it is still the basis for a number of Eclipse-based products.
+
* update target pom to attach the newly created target as a build artifact
 +
* update the parent pom to have a profile for the new version which specifies the new target
 +
* update the mylyn-estaging.target to refer to the staging site for the upcoming release (eg. http://download.eclipse.org/staging/neon)
 +
* update matrix build jobs to have staging, maintenance, latest release (eg. mars), upcoming release (eg. neon) as their axes
 +
 
 +
=== When a New Eclipse Version is Released ===
 +
 
 +
Once a major release occurs the default and alternate targets in the mylyn parent pom should be updated to be the newly released versions.
  
 +
=== Updating Eclipse Platform Dependency Versions ===
  
After RC4 (of a major or an update release), update versions of Eclipse Platform dependencies:
+
After RC4 (of a major or an update release), update versions of Eclipse Platform dependencies to match those in the SimRel, for build reproducibility. If preparing a Mylyn release that will go out with a SimRel, this will need to be done after the final build (i.e. in preparation for the next release).
 
  cd ~/releng/bin
 
  cd ~/releng/bin
 
  ./extractVersionsFromUpdateSite.sh ../../org.eclipse.mylyn-target/mylyn-e4.4.target ~/downloads/releases/luna/201409261001/ junit
 
  ./extractVersionsFromUpdateSite.sh ../../org.eclipse.mylyn-target/mylyn-e4.4.target ~/downloads/releases/luna/201409261001/ junit
 
Copy the desired suggestions to the target file. To get version for org.eclipse.sdk.ide (a product IU)
 
Copy the desired suggestions to the target file. To get version for org.eclipse.sdk.ide (a product IU)
 
  unzip ~/downloads/releases/luna/201409261001/content.jar; grep "id='org.eclipse.sdk.ide'" content.xml; rm content.xml
 
  unzip ~/downloads/releases/luna/201409261001/content.jar; grep "id='org.eclipse.sdk.ide'" content.xml; rm content.xml

Latest revision as of 20:49, 17 December 2018


This page describes how Mylyn is built and tested and how snapshot and release builds are published.

The Mylyn project uses two Hudson servers with the following major classes of build jobs:

http://ci.mylyn.org/

Hudson instance running on the same server that hosts the test repositories.

Build Jobs

Maintenance

To start Hudson if it has stopped:

sudo su hudson
cd /home/hudson
./start.sh

To stop or restart Hudson if it is still running, go to http://ci.mylyn.org/safeExit or http://ci.mylyn.org/safeRestart.

It may be necessary to delete /home/hudson/nohup.out if it has grown very large.

The test servers listed at mylyn.org are under /home/tools.

https://hudson.eclipse.org/mylyn/

The Mylyn HIPP. There is some duplication with jobs on ci.mylyn.org because while those jobs are faster (due to co-location with test repositories), keep much longer histories, and tend to be more reliable, they cannot publish artifacts to download.eclipse.org and their test results are not linked from release builds.

Build Jobs

Scheduling

The release build for the current release runs weekly, and thus the integration and integration-connectors builds run weekly.

[*] indicates a matrix build

Snapshots

Nightly snapshots are published to http://download.eclipse.org/mylyn/snapshots/nightly/ by the Nightly builds. The Release builds publish releases and weekly snapshots to http://download.eclipse.org/mylyn/drops. These snapshots are used to build SRs via composite sites like http://download.eclipse.org/mylyn/snapshots/3.14/.

Build Configuration

See Mylyn/FAQ#System_Properties for system properties used to control whether all fixtures are run and whether any are excluded.

Integration and release builds run against the parent git repository that includes all sub-projects as submodules.

Build Target Platforms

The matrix builds build Mylyn against different Eclipse versions. The mylyn-eX.Y targets are used to build against the latest Eclipse release in the given stream (e.g. 4.4.2, 4.5) or the latest milestone build of the upcoming release; mylyn-emaintenance is used to build against the latest candidate for the next Eclipse update release (e.g. 4.5.1); mylyn-estaging is used to build against the latest candidate for the next Eclipse major release (e.g. 4.6). The maintenance, staging, and latest X.Y targets build against the latest plugin versions available on the Eclipse platform update site, whereas the released X.Y targets build against specific versions in the release.

The targets should be updated at several points during the Eclipse release cycle.

When a New Eclipse Version Becomes Available

Once the update sites are available for the upcoming major release, a new target should be created for that release, and axes should be added for that target to the matrix builds marked with [*] above:

  • create target by copying the latest X.Y target and changing the file name (eg. mylyn-e4.5.target -> mylyn-e4.6.target)
  • change the target name in the file to match the release (eg. mars -> neon)
  • change the update site URL to the release URL of the relevant version (eg. http://download.eclipse.org/releases/neon)
  • change Eclipse Platform dependency versions to 0.0.0
  • update target pom to attach the newly created target as a build artifact
  • update the parent pom to have a profile for the new version which specifies the new target
  • update the mylyn-estaging.target to refer to the staging site for the upcoming release (eg. http://download.eclipse.org/staging/neon)
  • update matrix build jobs to have staging, maintenance, latest release (eg. mars), upcoming release (eg. neon) as their axes

When a New Eclipse Version is Released

Once a major release occurs the default and alternate targets in the mylyn parent pom should be updated to be the newly released versions.

Updating Eclipse Platform Dependency Versions

After RC4 (of a major or an update release), update versions of Eclipse Platform dependencies to match those in the SimRel, for build reproducibility. If preparing a Mylyn release that will go out with a SimRel, this will need to be done after the final build (i.e. in preparation for the next release).

cd ~/releng/bin
./extractVersionsFromUpdateSite.sh ../../org.eclipse.mylyn-target/mylyn-e4.4.target ~/downloads/releases/luna/201409261001/ junit

Copy the desired suggestions to the target file. To get version for org.eclipse.sdk.ide (a product IU)

unzip ~/downloads/releases/luna/201409261001/content.jar; grep "id='org.eclipse.sdk.ide'" content.xml; rm content.xml

Back to the top