Linux Tools Project/Releng

{{#eclipseproject:technology.linux-distros}}

Linux Tools Release Engineering Procedures

Releases

We have nightly builds happening from trunk. The input source code for the build is taken from what is listed in the map file in org.eclipse.linuxtools.releng/maps. When it comes time for a release, we will:

1. Create branches for each sub-project (ex. 0.2.0); the branch should be the same name across all sub-projects
2. Branch releng/trunk/org.eclipse.linuxtools.releng
3. Change the map file to build from the head of the branch
4. Continue to have automated builds with test runs from the branches
5. Continue to commit any work not targetted at the release in trunk
6. Commit changes specifically targetted towards the release only on the branch
7. Commit changes targetted at both the release and trunk on both the branch and trunk
8. When all sub-project leads are satisfied that the branch is stable, they will tag the branch and update the map file on the release branch of releng to use the tag as input to the build
9. Once all sub-projects have tagged, a release candidate build will be performed to ensure content from the tags was used
   1. This release candidate build with have "-buildAlias <release number optionally followed by RC or something>" <-- the quotes are important!
10. If further fixes are necessary, multiple release candidate builds may happen
11. Once all sub-project leads are satisfied with the results of the latest release candidate, it is promoted to the release build
12. The releng project will be tagged for consistency
13. The Hudson job will then be switched back to use trunk of the releng project which takes input entirely from trunk
14. The Hudson job that will become the build should be locked to prevent automatic deletion

The input to the map file -- and thus the automated builds -- looks a bit like the following picture:

New and Noteworthy

New and noteworthy (N&N) items should be written as soon as possible after the item has been committed. This will prevent a frantic scramble at the end of the release cycle. For example, for the 0.3.0 release, we have http://www.eclipse.org/linuxtools/new-0.3/ in CVS. Please add new items under the various headings. Feel free to add new headings as necessary. The parenthesized numbers in the pseudo-index at the top of the file indicate the number of N&N items in that section. There is a template to be used for each release at new-template. Copy this for releases and be sure all recent releases are listed in the table and list at the top.

N&N images

Screenshots should be cropped to show only the pertinent region of the screen. Use the gimp to add drop shadows (pick the default values for radius and opacity). Save images as PNGs.

Accepting Patches

Please ensure that bugs which have patches contributed by non-committers have the "contributed" keyword set on them as well as the iplog flag set to +1. The flag should be set on the patch attachment itself and not the bug.

0.2.0

1. We are basing 0.2.0 on top of Ganymede dependencies (Eclipse SDK 3.4.2, CDT 5.0.x, BIRT 2.3.1, EMF 2.4.1)
2. Please run smoke tests of installation and usage of your components by end of your day Wednesday 8 April 2009.
3. For the 0.2.0 release, aim to have N&N items completed by 2100 UTC (5 PM North America Eastern) on Thursday 9 April 2009.
4. Please ensure any patches that have gone into 0.2.0 have the "contributed" keyword set so that the IP log is complete.

0.2.0 Ramp-down

Builds will continue to happen every 6 hours if changes have been made in SVN. Don't forget that the 0.2.0 builds that will get replicated to the updates-nightly update site are pulling from 0.2.0 branches.

Once everyone is satisfied with the contents of updates-nightly, we will:

1. turn off automated builds
2. tag the head of the 0.2.0 branches
3. update linuxtools.map in releng/trunk/org.eclipse.linuxtools.releng to pull from the tag
4. do a signed build
5. put the signed build on updates-nightly
6. verify that things install and run as we expect
7. verify that the JARs are signed by the Eclipse Foundation (no broken certificate icons in the "installed software" dialog)
8. if there are issues that need to be fixed, we will repeat the above steps

When we are satisfied with a build, we can move on to finalizing the N&N, Andrew will do release review slides, and the release process from the eclipse.org side will begin. At that point, updates-nightly will switch back to nightly builds from trunk. Once we have the go-ahead from the Foundation, our release update site will host the 0.2.0 release.

0.2.0 Status

Sub-project / Item	Ready to go? Date	Sign-off
Autotools	yes, 2009-04-09	jjohnstn
ChangeLog	yes, 2009-04-09	jjohnstn
Libhover	yes, 2009-04-09	jjohnstn
OProfile	yes, 2009-04-09	ksebasti
RPM Stubby	yes, 2009-04-09	akurtakov
Specfile Editor	yes, 2009-04-09	akurtakov
SystemTap	yes, 2009-04-09	overholt
Valgrind	yes, 2009-04-09	ebaron
Signed JARs	yes, 2009-04-13	overholt

Release complete and review scheduled for 6 May 2009.

Adding a new component

Like most Eclipse projects, our build process is feature-driven. A top-level dummy feature, org.eclipse.linuxtools.all, acts as a container which lists all of our sub-project's top-level features. These then cascade down into their plugins and the whole thing gets wrapped up into our update site and zips. In order for a new component to be built, it needs to fit into the hierarchy somewhere. When adding a new top-level sub-project feature (sub-features of existing features should get added to their containing feature), follow these steps:

Code-level checklist

1. Create your feature and containing plugins
   1. Name the feature(s) org.eclipse.linuxtools.mycoolstuff and put it in SVN as org.eclipse.linuxtools.mycoolstuff-feature (replacing "mycoolstuff", obviously)
   2. Name the plugin(s) org.eclipse.linuxtools.mycoolstuff.core, org.eclipse.linuxtools.mycoolstuff.ui, etc.
   3. Ensure your packages are all in the org.eclipse.linuxtools namespace (and in the .mycoolstuff.core, .mycoolstuff.ui packages where appropriate)
2. Create your JUnit test plugins
   1. Use JUnit 3 and not JUnit 4
   2. Name your test plugin the same as your functional plugin but with a ".tests" tacked onto the end
   3. Create a test.xml for use in the automated builds (see an example here) *OR* add a property like the example in bug 276574 does.

SVN-level checklist

1. If this is a new sub-project, create a directory to contain it, like "oprofile" or "autotools"
2. Ensure that the standard directory structure of "branches", "tags", and "trunk" exists under your containing directory
3. Check your features and plugins into SVN under trunk
4. Create a tag called "initial-checkin" which is a snapshot of your initial work

Build-related checklist

1. Add any new top-level feature to org.eclipse.linuxtools.all
   1. If you are adding a new sub-feature, ensure it is in a containing feature which itself is a part of the org.eclipse.linuxtools.all tree
2. If your sub-project has dependencies outside the existing ones (BIRT, EMF, DTP, CDT), notify Andrew Overholt
   1. Hopefully this will have been caught in the CQ (legal review)
   2. Add dependency zips to build.properties
3. Ensure your BREEs are correct in your plugin MANIFEST.MF files
4. Ensure the version on your feature ends with ".qualifier" (without the quotation marks)
5. Add your test feature to the org.eclipse.linuxtools.test feature
   1. Alternatively, add to an existing test feature that is already in the org.eclipse.linuxtools.test feature hierarchy
6. Add new features to our update site's category.xml file
7. Add all new features, plugins, and fragments to our map file
8. Modify or create a new PSF file and put it here

Building locally

• check out releng/{trunk,branches/Galileo}/org.eclipse.linuxtools.releng and Athena (the Eclipse common builder) using this PSF
  • File -> Import -> Team Project Set ... give it the location where you saved the above .psf
• ensure the dependencyURLs are correct
  • specifically the eclipse sdk tarball is architecture dependent, so you will need the version that corresponds to your architecture (for instance on x86, you'd simply remove "-ppc" from the filename and on x86_64 replace it with '-x86_64')
  • if you have any/all of these locally, you can create the variable "downloadsDir" and put the dependency zips or symlinks to them in this directory
• Within the build.properties file in org.eclipse.linuxtools.releng: uncomment the JAVA60_HOME and JavaSE-1.6 variables and adjust them so they point to the correct locations on your system.
• set the variables JAVA50_HOME and J2SE-1.5 variables to either:
  • the values of JAVA60_HOME and JavaSE-1.6 variables respectively
  • or if you want to build using a separate 1.5 JDK you can specify the corresponding paths for it
• right-click on org.eclipse.linuxtools.releng/build.xml and run it as an ant build

Building using a local checkout

If you want to avoid checking everything out of SVN, you can build using a local source checkout. This can be useful if you have made local changes and want to verify that they will work.

• check out org.eclipse.linuxtools.releng and Athena as above
• if you do not already have them checked out, check out all of the Linux Tools projects using the Import Project Set command on org.eclipse.linuxtools.releng/psfs/linuxtools.psf
• open the build.properties file in org.eclipse.linuxtools.releng
• uncomment localSourceCheckoutDir and set it to a directory on your filesystem
  • this will be used to create the necessary directory hierarchy to perform the build
• once build.properties is complete as above, run the script org.eclipse.linuxtools.releng/tools/localSourceSetup.sh from a terminal
  • this will create symbolic links to your local checkout in your workspace in the directory specified in localSourceCheckoutDir
• right-click on org.eclipse.linuxtools.releng/build.xml and run it as an ant build

If you have subversion-javahl installed you may encounter JVM crash when trying to fetch sources due to incompatible ABI. The easiest fix is to uninstall/remove your local javahl.

Adding a new test plugin to those that are run during the automated build

• Create test plugin(s) (ex. org.eclipse.linuxtools.mycoolfeature.ui.tests)
  • Add a test.xml like the other suites (this is used when the automated build is run)
  • Ensure test.xml and any resources are listed in build.properties
  • Check your plugin(s) into SVN
• If you are adding an entire test feature, add it to the included features in org.eclipse.linuxtools.test-feature
• If it's just a single test plugin (or more than one), add it/them to the included plugins in org.eclipse.linuxtools.test-feature
• Ensure the test plugin(s) do not get packed in the including feature.xml (ie. don't have "unpack=false" in the relevant plugin section of feature.xml)
• Add the test plugin(s) to the list of those to be run in testing.properties in org.eclipse.linuxtools.releng
• Add the test plugin(s)/feature(s) to the map file (be sure to update both the Galileo branch and trunk)
• Add the test plugin(s)/feature(s) to your sub-project's PSF (again, be sure to update both Galileo branch and trunk)
• Verify that your tests are built/run with a local build (see instructions on this page)

The next time a build happens, your test plugin(s) will be built and run. If you need a build pushed sooner than the next 6 hour mark when our scheduled builds happen, speak with Andrew Overholt.