WTP/Build/WTP Patches for Release 3x streams

What is an "official patch"?

Under some circumstance the WTP project may find it desirable to produce a "feature patch" which typically includes exactly one jar, that has a fix for exactly one bug. These are done for bugs which happen to be considered extremely severe or blocking -- but may not even apply to all installs, or all users. So, as a general rule, they are not to be installed by everyone ... but any bugs fixed by patches will be fixed in the next maintenance release, if there is one ... though the exact fix might be different.

While few, it is important to create and apply these in a controlled, repeatable process in order to maintain product level professional quality. For example, it is important to compile with exactly the same compiler, and exactly the same "pre-reqs" that was used to create the original build.

Technical Notes for Developers

Normally, only the "releng team" need to mess with patches. There should be very few, very special purpose, handled and produced on a case-by-case bases. But, occasionally developers will need to commit code or create patch features.

Branches

This description uses "3.2.1" and "3.2.1 patches" for a concrete example, but the same patterns hold for other 3.x streams.

For WTP 3.2.1 these fixes and related files will be released to the R3_2_1_patches stream. (If there is not already such a stream for a component, it will of course have to be created, by branching from the R3_2_maintenance stream). Projects at other levels, such as Dali at 2.3.1, the same pattern is used, namely R2_3_1_patches stream.

We do it this way so the main maintenance branch remains exactly like it was for release. Any fixes in a patch stream should also be merged into the maintenance branch if (and only if) there will be a subsequent maintenance release.

The map files used during a build will to be created and pulled from the branches corresponding to the above, R3_2_1_patches branch (or R2_3_1_patches). All the maps files are branched at same time (unlike the code) since that have to be be named in the build scripts that does the patch builds.

Note that it is important that any particular patch stream be created from the released version of any the map files. We've found it easiest to literally create new patch streams after a release ... rather than, say, have a generic "patches" stream.

Creating a Patch Feature

The patch feature is typically a brand new feature, and must be added to a map file (in the patches stream). Of course, if that feature already exists, and another plugin is being added to it, then it only needs the new version released to the patches branch.

These patch features must correspond to the "build component features" since those are the assumed lowest unit by which anyone consumes our features (for example, org.eclipse.wst.xml_core.feature.patch ... usually not lower, such as org.eclipse.wst.xsd.feature.patch, nor higher, such as org.eclipse.wst.feature.patch).

For consistent end-user readability, the name of the patch feature should be the same as the feature it is patching, with the word "Patches" added. For example, if original was named "WST Server Core", the patch feature should be named "WST Server Core Patches".

The end-result is

• a compressed archive (zip file) on patches download site.
• a feature and plugin suitable for an update site. These update jars are actually in the same directory as the zip download, but normally not visible from the download web page, but when appropriate will be copied to the official /webtools/updates site.

Conventions

1. Remember to increment the service field of any plugins by +1.
2. The version of the feature patch should "match" the feature version it corresponds to (e.g. 3.2.1.qualifier). But, if subsequent versions of a patch feature are needed (to contain cumulative fixes) then the service field may need to be incremented. This is important for those that have already applied the previous patch feature to be offered the new patch feature ... a qualifier only increment may not enough. [Is this valid? I'm not sure we do it this way in practice?]
3. update the features.properties file so it lists the bug that is fixed by the patch.
4. update or provide a buildnotes.html file that lists the bug fixed by the patch.
5. All patches are produced in a cumulative fashion -- cumulative from the release, that is, not from any previous patches.
6. In the event that different patches depend on each other (that is, several must be applied together) be sure to document it in the build notes and bugzilla.

Reminders

These are tips on avoiding problems that have been hit before. They are especially problematic, since everything may appear to work fine, but then various install or update scenarios will not work.

1. The patch feature must "requires" the feature that it is patching, including all 4 fields of the version -- the 'qualifier' field is not computed automatically during the batch build, as other qualifiers are. This won't show up as a problem just unzipping the features and plugins, but will when updates

are attempted.

1. Remember to "match" features and patch features ... that is, if a plugin is in another feature, besides the one that the patch feature 'requires', then various paths of updating won't work. For WTP, this is most easily accomplished by having patch features which parallel the build component features.
2. Be positive that the "unpack" attribute of the include plugin element is exactly the same in the original feature, and the patch feature. If it was unpacked in the original, it must be unpacked in the patch. Again, this won't often show up as a problem until much later in the process.

Examples/Samples

You can check out modules from the R3_0_5_patches branch to see existing patch features to use as a starting point.

Technical Notes for Release Engineers

There is a special component

wtp-patches3

in the releng.wtpbuilder cvs project that along with special distribution targets

patches3.build
patches3.site
patches3.tests

will produce patch features that contains a special feature to "host" the plugin. See Eclipse Help for details on patch features.

Web Tools Repository Site

If it is desired to be generally available to all WTP users (in the webtools/repository site), then the repository must be updated. Remember to

1. run pack200
2. recreate (or append?) to the metadata
3. add (back) repo properties, such as mirrorsURL and download stats
4. update the category.xml and republish the CategoryPublisher

Bugs which will be made public on update site must go through PMC Review and typically some regression testing before being put in webtools repository site, since in theory any such fixes could effect other adopters/products, so they deserve extra review.

Changes to Unit Tests

The JUnit's are ran as-is (or, as-was) in the released version, except for the releng tests, which should have its versioning reference updated, so there is a clear list of what's changed, what's the same as the reference release). So, in other words, some may fail if fixes cause them to fail, but typically we never update the tests for patch builds, and just live with the broken tests.

Getting the feature patch in to a build

There is a special feature that drives the patch build:

org.eclipse.wtp.patch.assembly.feature

The patch feature is added to this assembly feature by the releng team.and released to the releng project map in the patches branch.

In addition to adding the patch feature to the assembly feature, there is a "category.xml" file in releng/components/wtp-patches32x that needs to be updated for any new patch features going into the build. Just needs to be added once, when a patch feature first added, and use the category being used for the patch stream. For example:

  <feature
       id="org.eclipse.wst.jsdt.feature.patch">
       <category
           name="Web Tools Platform (WTP) Patches @buildversionid@"/>
  </feature>

In addition, to get the new patch features to be installed during the unit tests, there is a variable, testFeatureIUs, in build.cfg that needs to updated, to name patch features that are part of the build repository, to be installed for the unit tests. Once again, just needs to be updated once, when there is a new patch feature added to the patch build. For example,

wtpFeatureIUs=org.eclipse.wst.jsdt.feature.patch.feature.group,org.eclipse.jst.web_core.feature.patch.feature.group

Doing the build

The patch build is never done automatically but must be started manually from the manual, "schedule build" button on the dashboard.

Testing the patch

Best to use Eclipse "install" function to install the patch into a released distribution.

Questions?

Feel free to ask questions on wtp-releng. And, please, update this page when you find the answer!