Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Linux Tools Project/Releng

< Linux Tools Project
Revision as of 09:57, 19 May 2010 by Charley.wang.gmail.com (Talk | contribs) (→‎Building locally)

{{#eclipseproject:technology.linux-distros}}

Linux Tools Release Engineering Procedures

Update sites

Hudson builds

build.eclipse.org runs Hudson which runs builds of our project every 6 hours (0 */6 * * *) if something has changed in SVN.

Releases

We have nightly builds happening from trunk. The input source code for the build is taken from what is listed in the map file in org.eclipse.linuxtools.releng/maps. When it comes time for a release, we will:

  1. Create branches for each sub-project (ex. 0.5.0); the branch should be the same name across all sub-projects
    1. for f in autotools changelog eclipse-build gcov gprof libhover lttng man oprofile profiling releng rpm rpmstubby systemtap valgrind; do svn cp svn+ssh://commiterId@dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/$f/trunk svn+ssh://commiterId@dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/$f/branches/0.5.0 -m "Branch for 0.5.0"; done
  2. Change the map file to build from the head of the branch
    1. cd org.eclipse.linuxtools.releng
    2. sed -i "s|trunk|branches/0.5.0|g" maps/linuxtools.map
    3. cd psfs
    4. for f in `ls`; do sed -i "s|trunk|branches/0.5.0|g" $f; done
  3. Update the .all feature to have an appropriate version
  4. Alter the Hudson job configuration to use the branch of the releng project
  5. Continue to have automated builds with test runs from the branches
  6. Continue to commit any work not targetted at the release to trunk
  7. Commit changes specifically targetted towards the release only on the branch
  8. Commit changes targetted at both the release and trunk on both the branch and trunk
  9. When all sub-project leads are satisfied that the branch is stable, they will tag the branch with the release version number (ex. 0.5.0) and update the map file on the release branch of releng to use the tag as input to the build
  10. Once all sub-projects have tagged, a release candidate build will be performed to ensure content from the tags was used
    1. This release candidate build will be of type "S" have "-buildAlias <release number optionally followed by RC or something>" <-- the quotes are important!
  11. If further fixes are necessary, multiple release candidate builds may happen
  12. Once all sub-project leads are satisfied with the results of the latest release candidate, it is promoted to the release build by:
    1. Tagging the releng project with its map file containing tags
    2. Updating the Hudson job configuration to build from the tag of the releng project
    3. Performing a build with a -buildAlias of the release number alone (ex. "-buildAlias 0.5.0")
  13. The Hudson job will then be switched back to use trunk of the releng project which takes input entirely from trunk
  14. The Hudson job that will become the build should be locked to prevent automatic deletion
  15. Ensure the resulting -Update zip is put into the downloads directory (Andrew Overholt and Alex Kurtakov are currently the only committers with access to this area)
  16. Update linuxtools/{index.php,downloads.php,new} to reflect the release
  17. Add a news item to the main wiki page
  18. Copy our update site to a versioned copy to archive it
  19. Verify the update site and all download links (including zip of update site) work
  20. Add release to list of versions in bugzilla (in portal)
  21. Add release.1 to list of target milestones in bugzilla (in portal)
  22. Announce release on mailing list, newsgroup and blog(s)

The input to the map file -- and thus the automated builds -- looks a bit like the following picture: LinuxToolsReleng.png

New and Noteworthy

New and noteworthy (N&N) items should be written as soon as possible after the item has been committed. This will prevent a frantic scramble at the end of the release cycle. For example, for the 0.3.0 release, we have http://www.eclipse.org/linuxtools/new-0.3/ in CVS (:ext:youruserid@dev.eclipse.org:/cvsroot/org.eclipse and module: www/linuxtools). Please add new items under the various headings. Feel free to add new headings as necessary. The parenthesized numbers in the pseudo-index at the top of the file indicate the number of N&N items in that section. There is a template to be used for each release at new-template. Copy this for releases and be sure all recent releases are listed in the table and list at the top.

N&N images

Screenshots should be cropped to show only the pertinent region of the screen. Use the gimp to add drop shadows (pick the default values for radius and opacity). Save images as PNGs.

Accepting Patches

Please ensure that bugs which have patches contributed by non-committers have the "contributed" keyword set on them as well as the iplog flag set to +1. The flag should be set on the patch attachment itself and not the bug.

Adding a new component

Like most Eclipse projects, our build process is feature-driven. A top-level dummy feature, org.eclipse.linuxtools.all, acts as a container which lists all of our sub-project's top-level features. These then cascade down into their plugins and the whole thing gets wrapped up into our update site and zips. In order for a new component to be built, it needs to fit into the hierarchy somewhere. When adding a new top-level sub-project feature (sub-features of existing features should get added to their containing feature), follow these steps:

Code-level checklist

  1. Create your feature and containing plugins
    1. Name the feature(s) org.eclipse.linuxtools.mycoolstuff and put it in SVN as org.eclipse.linuxtools.mycoolstuff-feature (replacing "mycoolstuff", obviously)
    2. Name the plugin(s) org.eclipse.linuxtools.mycoolstuff.core, org.eclipse.linuxtools.mycoolstuff.ui, etc.
    3. Ensure your packages are all in the org.eclipse.linuxtools namespace (and in the .mycoolstuff.core, .mycoolstuff.ui packages where appropriate)
  2. Create your JUnit test plugins
    1. Use JUnit 3 and not JUnit 4
    2. Name your test plugin the same as your functional plugin but with a ".tests" tacked onto the end
    3. Create a test.xml for use in the automated builds (see an example here) *OR* add a property like the example in bug 276574 does.

SVN-level checklist

  1. If this is a new sub-project, create a directory to contain it, like "oprofile" or "autotools"
  2. Ensure that the standard directory structure of "branches", "tags", and "trunk" exists under your containing directory
  3. Check your features and plugins into SVN under trunk
  4. Create a tag called "initial-checkin" which is a snapshot of your initial work

Build-related checklist

  1. Add any new top-level feature to org.eclipse.linuxtools.all
    1. If you are adding a new sub-feature, ensure it is in a containing feature which itself is a part of the org.eclipse.linuxtools.all tree
  2. If your sub-project has dependencies outside the existing ones (BIRT, EMF, DTP, CDT, GEF), notify Andrew Overholt
    1. Hopefully this will have been caught in the CQ (legal review)
    2. Add dependency zips to build.properties
  3. Ensure your BREEs are correct in your plugin MANIFEST.MF files
  4. Ensure the version on your feature ends with ".qualifier" (without the quotation marks)
  5. Add your test feature to the org.eclipse.linuxtools.test feature
    1. Alternatively, add to an existing test feature that is already in the org.eclipse.linuxtools.test feature hierarchy
  6. Add new features to our update site's category.xml file
  7. Add all new features, plugins, and fragments to our map file
  8. Modify or create a new PSF file and put it here
  9. To make Hudson do nice things like show which SVN commits when into which build, add your SVN location to the Hudson job configuration
    1. Ensure an existing committer has added your eclipse.org userid to the list of users and groups allowed to modify the job
      1. After logging in to Hudson, pick the job you want to modify and click "Configure"
      2. Under "Enable project-based security", add the new committer userid(s)
      3. Finally, check the Configure and Update checkboxes next to the new userid(s)
    2. Galileo builds (SVN trunk as of 2009-12-22):
      1. Go to https://build.eclipse.org/hudson/job/cbi-linuxtools-Galileo/configure
      2. For Repository URL, enter your new component's SVN trunk URL (ex. http://dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/valgrind/trunk)
      3. For Local module directory (optional), enter your new component's name followed by "-trunk" (ex. valgrind-trunk)
    3. Helios builds (SVN trunk or a Helios branch depending upon the plugin -- see linuxtools.map for details):
      1. Go to https://build.eclipse.org/hudson/job/cbi-linuxtools-Helios/configure
      2. Under Source Code Management, click the Add more locations... button
      3. For Repository URL, enter your new component's SVN trunk URL (ex. http://dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/valgrind/trunk)
      4. For Local module directory (optional), enter your new component's name followed by "-trunk" (ex. valgrind-trunk)
      5. If you have Helios-specific branches, enter them (ex. http://dev.eclipse.org/svnroot/technology/org.eclipse.linuxtools/valgrind/branches/Helios and valgrind-Helios)

Building locally

  • check out releng/{trunk,branches/Galileo}/org.eclipse.linuxtools.releng using this PSF and Athena (the Eclipse common builder) using this PSF
    • File -> Import -> Team Project Set ... give it the location where you saved the above .psf

Build errors in Athena's basebuilder can be safely ignored. Please note that you will need CVS and SVN to complete the Team Project Set checkouts.

Within the build.properties file in org.eclipse.linuxtools.releng:

  • ensure the dependencyURLs are correct
    • specifically the eclipse sdk tarball is architecture dependent, so you will need the version that corresponds to your architecture (for instance on x86, you'd simply remove "-ppc" from the filename and on x86_64 replace it with '-x86_64')
    • if you have any/all of these locally, you can create the directory "/tmp/build/downloads" and put the dependency zips or symlinks to them in this directory. This location can be changed with the downloadsDir variable.
  • uncomment the JAVA60_HOME and JavaSE-1.6 variables and adjust them so they point to the correct locations on your system.
  • set the variables JAVA50_HOME and J2SE-1.5 variables to either:
    • the values of JAVA60_HOME and JavaSE-1.6 variables respectively
    • or if you want to build using a separate 1.5 JDK you can specify the corresponding paths for it
  • right-click on org.eclipse.linuxtools.releng/build.xml and run it as an ant build

Building using a local checkout

If you want to avoid checking everything out of SVN, you can build using a local source checkout. This can be useful if you have made local changes and want to verify that they will work.

  • check out org.eclipse.linuxtools.releng and Athena as above
  • if you do not already have them checked out, check out all of the Linux Tools projects using the Import Project Set command on this PSF.
  • open the build.properties file in org.eclipse.linuxtools.releng
  • uncomment localSourceCheckoutDir and set it to a directory on your filesystem
    • this will be used to create the necessary directory hierarchy to perform the build
  • once build.properties is complete as above, run the script org.eclipse.linuxtools.releng/tools/localSourceSetup.sh from a terminal
    • this will create symbolic links to your local checkout in your workspace in the directory specified in localSourceCheckoutDir
  • right-click on org.eclipse.linuxtools.releng/build.xml and run it as an ant build


If you have subversion-javahl installed you may encounter JVM crash when trying to fetch sources due to incompatible ABI. The easiest fix is to uninstall/remove your local javahl.

Adding a new test plugin to those that are run during the automated build

  • Create test plugin(s) (ex. org.eclipse.linuxtools.mycoolfeature.ui.tests)
    • Add a test.xml like the other suites (this is used when the automated build is run)
    • Ensure test.xml and any resources are listed in build.properties
    • Check your plugin(s) into SVN
  • If you are adding an entire test feature, add it to the included features in org.eclipse.linuxtools.test-feature
  • If it's just a single test plugin (or more than one), add it/them to the included plugins in org.eclipse.linuxtools.test-feature
  • Ensure the test plugin(s) do not get packed in the including feature.xml (ie. don't have "unpack=false" in the relevant plugin section of feature.xml)
  • Add the test plugin(s) to the list of those to be run in testing.properties in org.eclipse.linuxtools.releng
  • Add the test plugin(s)/feature(s) to the map file (be sure to update both the Galileo branch and trunk)
  • Add the test plugin(s)/feature(s) to your sub-project's PSF (again, be sure to update both Galileo branch and trunk)
  • Verify that your tests are built/run with a local build (see instructions on this page)

The next time a build happens, your test plugin(s) will be built and run. If you need a build pushed sooner than the next 6 hour mark when our scheduled builds happen, speak with Andrew Overholt.

eclipse-build tarballs

Retrieved from "https://wiki.eclipse.org/index.php?title=Linux_Tools_Project/Releng&oldid=201234"

Back to the top