Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Orbit/Adding Bundles To Orbit In 5 Minutes"

m
(15 intermediate revisions by 5 users not shown)
Line 3: Line 3:
 
This is for primarily meant as a guide for contributing a bundle to make its way into a release. This guide assumes all the steps to get approval for the bundle itself have already been performed.
 
This is for primarily meant as a guide for contributing a bundle to make its way into a release. This guide assumes all the steps to get approval for the bundle itself have already been performed.
  
See http://git.eclipse.org/c/gerrit/orbit/orbit-recipes.git/tree/README.md for a more thorough explanation
+
See https://git.eclipse.org/c/orbit/orbit-recipes.git/tree/README.md for a more thorough explanation
  
 
== Setup ==
 
== Setup ==
Line 13: Line 13:
 
== Creation/Preparation ==
 
== Creation/Preparation ==
  
* Choose a folder under which to create the recipe and then create it there
+
* Choose a folder under which to create the recipe and then create it there using the below mvn command or the ''create-*-recipes.sh'' if any:
  
 
  $ cd github/mongodb
 
  $ cd github/mongodb
Line 24: Line 24:
 
  $ $EDITOR osgi.bnd
 
  $ $EDITOR osgi.bnd
  
* Perform a test build of the package to have an ip_log.xml file generated. Fill in the missing information (Name, Email, Company, Ipzilla Bug Id), into the ip_log.xml file. Note that the Ipzilla Bug Id must be an Add-To-Orbit (ATO) CQ.
+
* Perform a test build of the package to have an ip_log.xml file generated. Fill in the missing information (Name, Email, Company, Ipzilla Bug Id or Clearly Defined URL), into the ip_log.xml file.
  
 
  $ mvn -U clean package -DdirtyWorkingTree=warning
 
  $ mvn -U clean package -DdirtyWorkingTree=warning
Line 44: Line 44:
 
(eg. <code><plugin id="org.mongodb.mongo-java-driver" version="3.4.1.qualifier"/></code>)
 
(eg. <code><plugin id="org.mongodb.mongo-java-driver" version="3.4.1.qualifier"/></code>)
  
* Add and commit the new recipe and changes to the aggregation feature. Be sure to reference the ATO CQ in your commit message.
+
* Add and commit the new recipe and changes to the aggregation feature. Be sure to reference the CQ or clearly defined in your commit message.
  
 
  $ git add github/mongodb/org.mongodb.mongo-java-driver_3.4.1 releng/aggregationfeature/feature.xml
 
  $ git add github/mongodb/org.mongodb.mongo-java-driver_3.4.1 releng/aggregationfeature/feature.xml
Line 53: Line 53:
 
* We should also test that our new recipe builds successfully prior to submitting for review
 
* We should also test that our new recipe builds successfully prior to submitting for review
  
First we test that our recipe builds alongside all other recipes and install them locally to be discovered later by our aggregation featue
+
First we test that our recipe builds alongside all other recipes and install them locally to be discovered later by our aggregation feature
  
 
  $ mvn clean install
 
  $ mvn clean install
Line 70: Line 70:
  
 
  $ git push origin HEAD:refs/for/master
 
  $ git push origin HEAD:refs/for/master
 +
 +
== Consumption ==
 +
 +
Once a change has been reviewed and approved, a build should be triggered within about an hour at https://ci.eclipse.org/orbit/job/orbit-recipes/ . Once the build succeeds, information about where to find the build page, as well as the generated p2 repository can be found at the bottom of the build's logs.
 +
 +
The automated builds are analogous to nightly builds, in that they are not usually retained for long and are generally used merely for testing purposes. For more stable repositories, committers have the ability to push integration, milestone, and recommended (I, S, R) builds that can also be promoted to the [http://download.eclipse.org/tools/orbit/downloads/ downloads page] for visibility.
  
 
See [[Orbit_Builds#Orbit_Builds_for_Orbit_Committers | Orbit Builds for Orbit Committers]].
 
See [[Orbit_Builds#Orbit_Builds_for_Orbit_Committers | Orbit Builds for Orbit Committers]].
 +
 +
== Manual Build And Promotion for Milestones and Releases ==
 +
 +
Currently, I-builds (Integration builds) are generated periodically for testing and early adoption.
 +
 +
However, manual builds/promotion are still needed for Milestones and Releases.
 +
 +
=== Make and promote the build to the Orbit downloads page ===
 +
 +
To manually trigger/promote a build, you must be logged in with your committer account on the [https://ci.eclipse.org/orbit/ Orbit JIPP page].
 +
 +
Go to the [https://ci.eclipse.org/orbit/job/orbit-recipes/ orbit-recipes] job on the JIPP, and while logged in with your committer account, click "Build With Parameters". Change the <code>BUILD_LABEL</code> to whichever build label is desired ('I', 'S', 'M', 'R' are recognized). Set the build parameter SIMREL_NAME to the current simultaneous release name (eg. 2021-03). Add a description (eg. 2021-03 M2 for a Milestone or 2021-03 for a Release) and mark the build as "Keep this build forever". Now wait for the build to complete.
 +
 +
To promote this newly created build to the [http://download.eclipse.org/tools/orbit/downloads/ downloads page], go to the [https://ci.eclipse.org/orbit/job/promote-to-downloads/ promote-to-downloads] job on the JIPP, and while logged in with your committer account, click "Build Now". Change the <code>SRC_LOCATION</code> parameter to the path to your builds directory relative to <code>http://download.eclipse.org/tools/orbit/</code>. For example, if your build was <code>http://download.eclipse.org/tools/orbit/I-builds/I20170425174753/</code>, then the SRC_LOCATION would be <code>I-builds/I20170425174753/</code>
 +
 +
Once this is completed, you should be able to see your build on the [http://download.eclipse.org/tools/orbit/downloads Orbit downloads page].
 +
 +
=== Set the "Notes" value ===
 +
 +
At this point the Notes entry on the Orbit downloads page for your build will be blank.
 +
 +
To set the note, edit tools/orbit/downloads/notes.php to set the milestone or release message.
 +
You can use sshfs to mount the directory and edit this file, e.g.,
 +
$ sshfs -o reconnect ${commiterID}@build.eclipse.org:/home/data/httpd/download.eclipse.org/tools/orbit/ ~/my-mount
 +
''~/my-mount can be any empty folder''
 +
 +
'''Use caution''' - this is the Orbit folder on the public download server.
 +
 +
Look inside the notes.php to see the format of the existing entries and add an entry for your build.
 +
After you've edited the file, unmount with
 +
$ umount ~/my-mount
 +
 +
=== Update the SimRel ===
 +
 +
Now update the SimRel aggrcon for Orbit, referring to [https://git.eclipse.org/r/c/simrel/org.eclipse.simrel.build/+/175651 this commit] for the change to make.
 +
 +
=== Notify orbit-dev ===
 +
 +
At this point the build is published, so it's time to let the Orbit community know.
 +
 +
Send a notification email to orbit-dev (refer to past “Declaring Build” emails for the format, for example [https://www.eclipse.org/lists/orbit-dev/msg05419.html this one]).
 +
There are two diff sections. The first can be produced with git log, e.g.,
 +
$ git log --oneline aad048099bfaa7d5de78fba817d1bd40b88b478e..HEAD
 +
 +
''aad048099bfaa7d5de78fba817d1bd40b88b478e is the last commit from the previous milestone or release''
 +
 +
Edit the output to match the format from previous “Declaring Build” emails.
 +
 +
The second diff section is produced using a custom tool which is currently not well-documented.  This section is nice to have but optional.  Some rough notes about the tool used to produce this diff follow.
 +
 +
https://github.com/rgrunber/org.fedoraproject.p2query
 +
Once you build it you can just do
 +
$ p2ql diff P2_REPO1 P2_REPO2
 +
 +
[[Some possible gotchas]]
 +
* depends on eclipse being on the path (added symlink)
 +
* expects checkout directory to be under ~/git for discovery of p2qJar and fpp2Jar (added symlink)
 +
* fpp2Jar needs fedoraproject-p2 to have been checked out and built (checked it out and built it)
 +
** the fpp2Jar should be optional. It has a useful feature that allows you to treat any folder with jars as a p2 repository, but otherwise not required.
 +
* fedoraproject-p2 does not build in its current state due to obsolete reference to missing S-repo (locally updated to the R-repo)
 +
** This project uses build-classpath, which may be a Fedora specific tool. If unavailable, that call can be replaced with hardcoded classpaths.
 +
* after fixing above, fedoraproject-p2 still errored for me with 'mvn clean verify', but adding '-DskipTests' worked
  
 
[[Category: Orbit]]
 
[[Category: Orbit]]

Revision as of 17:00, 2 February 2021

For the long version See Adding Bundles to Orbit

This is for primarily meant as a guide for contributing a bundle to make its way into a release. This guide assumes all the steps to get approval for the bundle itself have already been performed.

See https://git.eclipse.org/c/orbit/orbit-recipes.git/tree/README.md for a more thorough explanation

Setup

  • Clone the orbit-recipes git repository
$ git clone ssh://USERNAME@git.eclipse.org:29418/orbit/orbit-recipes

Creation/Preparation

  • Choose a folder under which to create the recipe and then create it there using the below mvn command or the create-*-recipes.sh if any:
$ cd github/mongodb
$ mvn ebr:create-recipe -DgroupId=org.mongodb -DartifactId=mongo-java-driver -Dversion=3.4.1 -DbundleSymbolicName=org.mongodb.mongo-java-driver
  • Modify recipe pom and osgi.bnd to suite the needs of the bundle. If the bundles has dependencies, make sure to specify them under the Import-Package header in the osgi.bnd as opposed to just leaving it as '*;resolution:=optional'.
$ cd org.mongodb.mongo-java-driver_3.4.1
$ $EDITOR pom.xml
$ $EDITOR osgi.bnd
  • Perform a test build of the package to have an ip_log.xml file generated. Fill in the missing information (Name, Email, Company, Ipzilla Bug Id or Clearly Defined URL), into the ip_log.xml file.
$ mvn -U clean package -DdirtyWorkingTree=warning
$ $EDITOR src/eclipse/ip_log.xml

Add To Build

  • Add the new recipe to the category pom as a module

(eg. <module>org.mongodb.mongo-java-driver_3.4.1</module>)

$ cd ../
$ $EDITOR pom.xml
  • Add the recipe bundle to the aggregation feature to ensure it ends up in the generated repository
$ cd ../../
$ EDITOR releng/aggregationfeature/feature.xml

(eg. <plugin id="org.mongodb.mongo-java-driver" version="3.4.1.qualifier"/>)

  • Add and commit the new recipe and changes to the aggregation feature. Be sure to reference the CQ or clearly defined in your commit message.
$ git add github/mongodb/org.mongodb.mongo-java-driver_3.4.1 releng/aggregationfeature/feature.xml
$ git commit

Building Locally (Recommended)

  • We should also test that our new recipe builds successfully prior to submitting for review

First we test that our recipe builds alongside all other recipes and install them locally to be discovered later by our aggregation feature

$ mvn clean install

Next we test that the aggregation feature builds successfully

$ mvn clean install -f releng/aggregationfeature/pom.xml

Finally, we test that the generated repository containing all bundle recipes builds successfully

$ mvn clean package -f releng/repository/pom.xml

Contributing

  • Lastly, we push to review
$ git push origin HEAD:refs/for/master

Consumption

Once a change has been reviewed and approved, a build should be triggered within about an hour at https://ci.eclipse.org/orbit/job/orbit-recipes/ . Once the build succeeds, information about where to find the build page, as well as the generated p2 repository can be found at the bottom of the build's logs.

The automated builds are analogous to nightly builds, in that they are not usually retained for long and are generally used merely for testing purposes. For more stable repositories, committers have the ability to push integration, milestone, and recommended (I, S, R) builds that can also be promoted to the downloads page for visibility.

See Orbit Builds for Orbit Committers.

Manual Build And Promotion for Milestones and Releases

Currently, I-builds (Integration builds) are generated periodically for testing and early adoption.

However, manual builds/promotion are still needed for Milestones and Releases.

Make and promote the build to the Orbit downloads page

To manually trigger/promote a build, you must be logged in with your committer account on the Orbit JIPP page.

Go to the orbit-recipes job on the JIPP, and while logged in with your committer account, click "Build With Parameters". Change the BUILD_LABEL to whichever build label is desired ('I', 'S', 'M', 'R' are recognized). Set the build parameter SIMREL_NAME to the current simultaneous release name (eg. 2021-03). Add a description (eg. 2021-03 M2 for a Milestone or 2021-03 for a Release) and mark the build as "Keep this build forever". Now wait for the build to complete.

To promote this newly created build to the downloads page, go to the promote-to-downloads job on the JIPP, and while logged in with your committer account, click "Build Now". Change the SRC_LOCATION parameter to the path to your builds directory relative to http://download.eclipse.org/tools/orbit/. For example, if your build was http://download.eclipse.org/tools/orbit/I-builds/I20170425174753/, then the SRC_LOCATION would be I-builds/I20170425174753/

Once this is completed, you should be able to see your build on the Orbit downloads page.

Set the "Notes" value

At this point the Notes entry on the Orbit downloads page for your build will be blank.

To set the note, edit tools/orbit/downloads/notes.php to set the milestone or release message. You can use sshfs to mount the directory and edit this file, e.g.,

$ sshfs -o reconnect ${commiterID}@build.eclipse.org:/home/data/httpd/download.eclipse.org/tools/orbit/ ~/my-mount

~/my-mount can be any empty folder

Use caution - this is the Orbit folder on the public download server.

Look inside the notes.php to see the format of the existing entries and add an entry for your build. After you've edited the file, unmount with

$ umount ~/my-mount

Update the SimRel

Now update the SimRel aggrcon for Orbit, referring to this commit for the change to make.

Notify orbit-dev

At this point the build is published, so it's time to let the Orbit community know.

Send a notification email to orbit-dev (refer to past “Declaring Build” emails for the format, for example this one). There are two diff sections. The first can be produced with git log, e.g.,

$ git log --oneline aad048099bfaa7d5de78fba817d1bd40b88b478e..HEAD

aad048099bfaa7d5de78fba817d1bd40b88b478e is the last commit from the previous milestone or release

Edit the output to match the format from previous “Declaring Build” emails.

The second diff section is produced using a custom tool which is currently not well-documented. This section is nice to have but optional. Some rough notes about the tool used to produce this diff follow.

https://github.com/rgrunber/org.fedoraproject.p2query Once you build it you can just do

$ p2ql diff P2_REPO1 P2_REPO2

Some possible gotchas

  • depends on eclipse being on the path (added symlink)
  • expects checkout directory to be under ~/git for discovery of p2qJar and fpp2Jar (added symlink)
  • fpp2Jar needs fedoraproject-p2 to have been checked out and built (checked it out and built it)
    • the fpp2Jar should be optional. It has a useful feature that allows you to treat any folder with jars as a p2 repository, but otherwise not required.
  • fedoraproject-p2 does not build in its current state due to obsolete reference to missing S-repo (locally updated to the R-repo)
    • This project uses build-classpath, which may be a Fedora specific tool. If unavailable, that call can be replaced with hardcoded classpaths.
  • after fixing above, fedoraproject-p2 still errored for me with 'mvn clean verify', but adding '-DskipTests' worked

Back to the top