Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Modeling Project Releng/Plugin Version Auditing
Contents
Overview
A project or component is typically laid out with a number of plugins and features, which normally have several interdependencies. Among other issues that arises from said interdependencies is the fact that features and plugins are versioned independently, but those versions must be incremented in tandem when a feature includes a plugin. In a related manner, when a commit is made to a plugin, the version of the plugin must be incremented, although only once per release cycle. Both of these issues tend to give developers headaches (particularly when they forget about them), and as such versionaudit.php is meant to be a way of automagically validating both of these conditions, as well as several other related conditions.
User guide
As a user, you should know about the various things that versionaudit.php can automate for you, and the details of this are in this section.
Commit checking
One of the things that versionaudit.php checks is that when a plugin is changed after a release, its' version is incremented coorespondingly. After the first commit and version increment, commits to that plugin are straightforward until another release is made. versionaudit.php knows the versions of the plugin at last release and in current CVS by checking the META-INF/MANIFEST.MF and/or plugin.xml files for that plugin.
For example, if EMF releases 2.0.5 (which contains org.eclipse.emf.codegen.ecore 2.0.2), and then a commit is made to org.eclipse.emf.codegen.ecore, it can no longer be called 2.0.2, and must be incremented to 2.0.3. Once this increment has been made, the number of commits made afterwards to org.eclipse.emf.codegen.ecore doesn't matter, it will remain at version 2.0.3 until EMF 2.0.6 is released, at which point it may or may not be incremented again, based on if commits are made to it or not.
Plugin/feature interdependencies
The other primary feature of versionaudit.php is that of validating the version of a feature based upon the plugins it contains and when they have been incremented. Feature dependencies upon plugins are discovered by parsing the <include> and <plugin> tags in a feature's feature.xml file. Once the dependencies have been found, versionaudit.php makes sure that that plugin's increment results in feature increments as well, but only once per feature per release cycle.
For example, in EMF 2.3.0, all of the following depend on org.eclipse.emf.codegen.ecore:
- org.eclipse.emf-feature
- org.eclipse.emf.codegen.ecore-feature
- org.eclipse.emf.sdk-feature
- org.eclipse.emf.doc-feature
This means that if the version of org.eclipse.emf.codegen.ecore is incremented, then the versions of all four of the above features must also be incremented, but only once per feature per release cycle. Many of the above features depend on more than just org.eclipse.emf.codegen.ecore, which means that a developer previously would need to check several conditions before incrementing the versions of the features, now versionaudit.php automates all of this work.
Other sanity checking
One of many possible problems that may arise with plugin versioning is that of double (or triple, or quadruple, or ...) incrementing, which is undesirable, as it would mean that one release of a project or component would contain a plugin or feature at x.y.z, and then in the very next release that plugin or feature would be at x.y.(z+n), where n >= 2. versionaudit.php makes sure that both features and plugins are incremented a maximum of once per release cycle.
Another possible problem that only arises in older branches is that a feature requests a specific version of a plugin in it's feature.xml file (such as 2.0.3) instead of any version (0.0.0, and let PDE figure out the actual version automagically). Obviously when a plugin is incremented, the version requested in the feature.xml file must also be incremented, but this is an easy detail to forget about. versionaudit.php makes sure the version is 0.0.0, or that the version matches the actual version of the plugin. This is generally only a problem in older branches as they use older versions of PDE, which did not support using 0.0.0 as a version.
An additional inconsistency that may arise is specific to the .doc plugin as it stores its' version information in both build.xml and META-INF/MANIFEST.MF. If these versions don't match, then problems will likely arise. versionaudit.php makes sure these versions match and will complain appropriately if they do not.
Technical notes
versionaudit.php runs on each desired branch separately. This implies that you must have all of those branches checked out separately (plus the last R build for each), although you only need to run parsecvs.sh on one of them, as CVS provides (mostly) the same log data regardless of the current branch checked out.
It should be noted that the CVS logs will be missing the information for files and directories that do not exist in the working copy. Assuming that HEAD is normally a superset of all other branches, it is recommended to (only) run parsecvs.sh against the HEAD checkout.
Documentation plugins and features (that is, those that end in .doc or .doc-feature) are considered specially. Instead of being incremented in tandem with the rest of the plugins and features, they only need to match the major version of the related plugins and features.
Server setup
If you've already setup Search CVS, the amount of additional setup required for versionaudit.php is largely limited to doing a few CVS checkouts. For details, see below.
Database
versionaudit.php requires access to the same database used for Search CVS, no additional setup is required in this respect.
CVS checkouts
versionaudit.php requires copies of the latest CVS version and the last R build checked out onto a locally accessable filesystem for every branch that is actively being developed or maintained. More specifically, the latest CVS HEAD is expected to be found in cvssrc/ while the rest of the branches and R builds are expected to be found in cvssrc_branches/. In the case of EMF, the layout is as follows:
- cvssrc_branches/emf-org.eclipse.emf-R2_0_maintenance (most recent CVS)
- cvssrc_branches/emf-org.eclipse.emf-R2_0_maintenance-latest (most recent R build)
- cvssrc_branches/emf-org.eclipse.emf-R2_1_maintenance (most recent CVS)
- cvssrc_branches/emf-org.eclipse.emf-R2_1_maintenance-latest (most recent R build)
- cvssrc_branches/emf-org.eclipse.emf-R2_2_maintenance (most recent CVS)
- cvssrc_branches/emf-org.eclipse.emf-R2_2_maintenance-latest (most recent R build)
- cvssrc/emf-org.eclipse.emf (most recent CVS)
- cvssrc_branches/emf-org.eclipse.emf-latest (most recent R build)
emf-org.eclipse.emf is the exception to the rule here, as it's expected to already exist there for use in Search CVS, whereas the rest are only necessary for versionaudit.php.
To automate the checkout process (and to show from where the "-latest" folders are sourced), see setup_branches.sh and setup_latest_releases.sh.
Web UI
The code consists of a versionaudit-common.php file and one or more versionaudit.php files. The versionaudit.php files simply define the branches and locations of those branches, then includes the versionaudit-common.php file, something like so:
<?php $dirs = array( "HEAD" => "/opt/public/modeling/searchcvs/cvssrc/emf-org.eclipse.emf", "R2_2_maintenance" => "/opt/public/modeling/searchcvs/cvssrc_branches/emf-org.eclipse.emf-R2_2_maintenance", "R2_1_maintenance" => "/opt/public/modeling/searchcvs/cvssrc_branches/emf-org.eclipse.emf-R2_1_maintenance", "R2_0_maintenance" => "/opt/public/modeling/searchcvs/cvssrc_branches/emf-org.eclipse.emf-R2_0_maintenance" ); include_once($_SERVER["DOCUMENT_ROOT"] . "/modeling/includes/versionaudit-common.php"); ?>
The advantage with this approach is that you can have multiple versionaudit.php placeholder files in different locations which all use the same code. The modeling project alone has at least 3:
If you only want to audit a single branch (or want more infomation, you can use these querystring arguments:
- ?debug=1 - debug set to level N (default 0)
- ?html -- html output instead of text
- ?branch=R2_4_maintenance - specify a single branch
Note that the tool is only as good as the database's data is current. Running a build will update the database; but if you're impatient, you can also Update Search CVS by hand.
JUnit Integration
Because the versionaudit.php file provides a convenient and consistent API for checking and reporting version compliance, it can be integrated into JUnit tests run during a build, and should anything other than a status of 'OK' be returned, fail the test (and therefore the build).
See VersionAuditTest.java for implementation details.