Revision as of 10:47, 6 April 2009

Linux Tools Release Engineering Procedures

Adding a new component

Like most Eclipse projects, our build process is feature-driven. A top-level dummy feature, org.eclipse.linuxtools.all, acts as a container which lists all of our sub-project's top-level features. These then cascade down into their plugins and the whole thing gets wrapped up into our update site and zips. In order for a new component to be built, it needs to fit into the hierarchy somewhere. When adding a new top-level sub-project feature (sub-features of existing features should get added to their containing feature), follow these steps:

Code-level checklist

1. Create your feature and containing plugins
   1. Name the feature(s) org.eclipse.linuxtools.mycoolstuff-feature (replacing "mycoolstuff", obviously)
   2. Name the plugin(s) org.eclipse.linuxtools.mycoolstuff.core, org.eclipse.linuxtools.mycoolstuff.ui, etc.
   3. Ensure your packages are all in the org.eclipse.linuxtools namespace (and in the .mycoolstuff.core, .mycoolstuff.ui packages where appropriate)
2. Create your JUnit test plugins
   1. Use JUnit 3 and not JUnit 4
   2. Name your test plugin the same as your functional plugin but with a ".tests" tacked onto the end
   3. Create a test.xml for use in the automated builds (see an example here)

SVN-level checklist

1. If this is a new sub-project, create a directory to contain it, like "oprofile" or "autotools"
2. Ensure that the standard directory structure of "branches", "tags", and "trunk" exists under your containing directory
3. Check your features and plugins into SVN under trunk
4. Create a tag called "initial-checkin" which is a snapshot of your initial work

Build-related checklist

1. Add any new top-level feature to org.eclipse.linuxtools.all
   1. If you are adding a new sub-feature, ensure it is in a containing feature which itself is a part of the org.eclipse.linuxtools.all tree
2. If your sub-project has dependencies outside the existing ones (BIRT, EMF, DTP, CDT), notify Andrew Overholt
   1. Hopefully this will have been caught in the CQ (legal review)
   2. Add dependency zips to build.properties
3. Ensure your BREEs are correct in your plugin MANIFEST.MF files
4. Add your test feature to the org.eclipse.linuxtools.test feature
   1. Alternatively, add to an existing test feature that is already in the org.eclipse.linuxtools.test feature hierarchy
5. Add new features to our update site's multiple site.xml files
   1. Be sure to add dependency sites to associateSites.xml
6. Add all new features, plugins, and fragments to our map file (Galileo branch has a separate one so please add it here, too)
7. Modify or create a new PSF file and put it here

Galileo or Ganymede?

At present (2009-04-01), we are currently targetting Ganymede (2008 / 3.4.x) as we have existing users that we wish to support. As soon as Galileo is released, we will switch over to full-time Galileo-based development. In the meantime, we run automated builds and tests against both Ganymede and Galileo dependencies.

What does this mean for me, a contributor/committer?

If your code needs to have Galileo-specific functionality, create a 'Galileo' branch and commit the work there. Until we fully switch over to Galileo-based development, trunk should remain Ganymede-specific. Of course, it would be incredibly beneficial if you as a plugin developer could try some non-JUnit usage testing with Galileo dependencies :)

Adding a new feature to the build

• add the feature to the org.eclipse.linuxtools.all feature.xml

Building locally

• check out releng/{trunk,branches/Galileo}/org.eclipse.linuxtools.releng
• right-click on build.xml and run it as an ant build

Building using a local checkout

• check out releng/{trunk,branches/Galileo}/org.eclipse.linuxtools.releng
  • follow the instructions at the top of the build.xml file
• check out the Linux Tools projects using the Import Project Set command on org.eclipse.linuxtools.releng/psfs/linuxtools.psf
• open the build.properties file in org.eclipse.linuxtools.releng
• uncomment localSourceCheckoutDir and set it to a directory on your filesystem
  • this will be used to create the necessary directory hierarchy to perform the build
• ensure the dependencyURLs are correct
  • specifically the eclipse sdk tarball is architecture dependent, so you will need the version that corresponds to your architecture (for instance on x86, you'd simply remove "-ppc" from the filename)
• uncomment the JAVA60_HOME and JavaSE-1.6 variables and adjust them so they point to the correct locations on your system.
• set the variables JAVA50_HOME and J2SE-1.5 variables to either:
  • the values of JAVA60_HOME and JavaSE-1.6 variables respectively
  • or if you want to build using a separate 1.5 JDK you can specify the corresponding paths for it
• now that build.properties is complete, run the script org.eclipse.linuxtools.releng/tools/localSourceSetup.sh
  • this will create symbolic links to your local checkout in your workspace in the directory specified in localSourceCheckoutDir
• right-click on org.eclipse.linuxtools.releng/build.xml and run it as an ant build

Adding a new test plugin to those that are run during the automated build

• Create test plugin(s) (ex. org.eclipse.linuxtools.mycoolfeature.ui.tests)
  • Add a test.xml like the other suites (this is used when the automated build is run)
  • Ensure test.xml and any resources are listed in build.properties
  • Check your plugin(s) into SVN
• If you are adding an entire test feature, add it to the included features in org.eclipse.linuxtools.test-feature
• If it's just a single test plugin (or more than one), add it/them to the included plugins in org.eclipse.linuxtools.test-feature
• Ensure the test plugin(s) do not get packed in the including feature.xml (ie. don't have "unpack=false" in the relevant plugin section of feature.xml)
• Add the test plugin(s) to the list of those to be run in testing.properties in org.eclipse.linuxtools.releng
• Add the test plugin(s)/feature(s) to the map file (be sure to update both the Galileo branch and trunk)
• Add the test plugin(s)/feature(s) to your sub-project's PSF (again, be sure to update both Galileo branch and trunk)
• Verify that your tests are built/run with a local build (see instructions on this page)

The next time a build happens, your test plugin(s) will be built and run. If you need a build pushed sooner than the next 6 hour mark when our scheduled builds happen, speak with Andrew Overholt.

Releases

We have nightly builds happening from trunk. The input source code for the build is taken from what is listed in the map file in org.eclipse.linuxtools.releng/maps. When it comes time for a release, we will:

1. Create branches for each sub-project (ex. 0.2.0); the branch should be the same name across all sub-projects
2. Change the map file to build from the head of the branch
3. Continue to have automated builds with test runs from the branches
4. Continue to commit any work not targetted at the release in trunk
5. Commit changes specifically targetted towards the release only on the branch
6. Commit changes targetted at both the release and trunk on both the branch and trunk
7. When all sub-project leads are satisfied that the branch is stable, they will tag the branch and update the map file to use the tag as input to the build
8. Once all sub-projects have tagged, a release candidate build will be performed to ensure content from the tags was used
9. If further fixes are necessary, multiple release candidate builds may happen
10. Once all sub-project leads are satisfied with the results of the latest release candidate, it is promoted to the release build
11. The map file is then switched back to take input entirely from trunk

The input to the map file -- and thus the automated builds -- looks a bit like the following picture: