Equinox/p2/Omni Version

Under Construction

Introduction

This page describes a proposal for adding support for non OSGi version and version ranges in Equinox p2. This page was created as a result of the discussion on the p2 call on Dec 1, 2008.

Background

There are other versioning schemes in wide use that are not compatible with OSGi version and version ranges. The problem is both syntactic and semantic.

Example of semantic issue

Many open source projects do their versioning in a fashion similar to OSGi but with one very significant difference. For two versions that are otherwise equal, a lack of qualifier signifies a higher version then when a qualifier is present. I.e.

1.0.0.alpha 
1.0.0.beta
1.0.0.rc1
1.0.0

The 1.0.0 is the final release. The qualifier happens to be in alphabetical order here but that's not always true.

Example of syntax issue

Here are some examples of versions used in Red Had Fedora distributions.

KDE Admin version 7:4.0.3-3.fc9
Compat libstdc version 33-3.2.3-63
Automake 1.4p6-15.fc7

These are not syntactically compatible with OSGi versions as they use colon, and dash as leading separators.

Some versioning schemes place the qualifier first.

Current implementation in p2

The current implementation in p2 uses the classes Version and VersionRange to describe the two concepts and these are implementations handling only OSGi version type.

Proposed Solution

Equinox p2 should support a set of "built in" version types. After a lengthy discussion about various "chicken and egg" type of problems relating to dynamic version type specifications and when and how the need for a particular version type is detected, and when it needs to be installed the meeting came to the conclusion that support for a "handful of built in types" would be sufficient as a starting point.

• The interfaces IVersion and IVersionRange should be used throughout the code instead of directly using the corresponding Version and VersionRange classes.
• An IVersion is obtained by calling a factory method such as VersionFactory.create(String versionString)
• An IVersionRange is obtained by a similar factory method
• The version string and version range has a URI scheme like prefix to indicate the version type
• The factory API can naturally contain some options where scheme and version strings are either separate or canonical
• When a version or version range is present without the version type prefix, the default is to use OSGi version type (this preserves backwards compatibility).

Proposed Version Types

type name	description
alpha	A alhpa numeric sequence of arbitrary depth separated by '.' e.g. 'mango.banana.0003.smoothie'. This type primarily exists to enable encoding of complex types. Segments may not contain reserved version range delimiters. A complex type is one that requires: more than one alhpa compared segment more than three numerical segments in combination with alpha segement(s) alpha segment(s) in combination with less than three numerical segments non '.' segment separators As it is difficult to know how many zeros to pad with when a "numeric" comparison is required, a variation could be to have a special reserved character for a '"pseudo numeric" segment. As an example let's pick '-' which would mean that the previous example could be written 'mango.banana.-3.smoothie'. Comparison is still alphanumeric but it is always zero padded to the same length before comparison. (i.e. '2' becomes '002' when compared against '123' and '00002' when compared against 'mango'). Special cases needs to be worked out (i.e. does '-mango' become '-000mango' when compared against '12345678', and how can a segment be started with the special character? ('..' could be used as that would otherwise be an illegal combination, or we need an escape mechanism when '-' is needed in the starting position)). See further at "Exploring a possible pattern based type" below.
numeric	A numeric sequence of arbitrary depth separated by '.' e.g. 1.1.1.1.1.1
osgi	The default OSGi version type.
string	A free form string version that may contain any character except the reserved version range delimiters. Can imagine calling this "text" which perhaps makes it easier to differentiate from "alpha".
timestamp	Time stamps compared in ascending order.
triplet	A variation on OSGi, with the same syntax, but where the a lack of qualifier > any qualifier.

The version range delimiters are: '(', ')', '[', ']' and , ',' (comma).

Exploring a possible pattern based type

A "pattern based" type would be of value where it is possible to specify number of segments, their significance, use of numeric or string comparison, and what delimiters to use. This section discusses options and possible solutions (a proposal is found after the discussion),

Candidates for pattern description:

• Java Message Format
• Regular expression
• Custom format

The message format has the benefits of being bidirectional and being easier to read by a human, but it can however not be used to describe optional parts and is therefore ruled out. The regular expression is the most powerful transformation, but is somewhat complicated to write and to understand by a human. A custom format can be made more compact and easier to use.

If the pattern is short it can be included in the version type string itself as a parameter. A regular expression even for a quite simple type like OSGi is quite long and complicated and needs to either be kept in a separate property, or the presentation and input of versions and ranges needs to deal with the pattern in a special way.

pattern embedded in version type

pattern(<patternstring>):<version string>

separate pattern

pattern:<version string>
org.eclipse.equinox.p2.versionpattern=<pattern string>

Alternatives to the type name "pattern" could be "external", "foreign", "raw", or "original".

Even if comparisons are internally made using the canonical form (the result of transformation), the original version string is needed for presentation purposes (as opposed to formatting the canonical form back to the original - this as the original may have surplus information that is not made use of in the canonical form, but would confuse a human accustomed with the versioning scheme if not present).

An IU publisher would create the IU with a "pattern" version type, the regular expression pattern is stored in a IU property, and the original version string is kept in a "original.version" (or similar property). The canonical form for the "pattern" version type can be the same as for the "alpha" type. The p2 engine does not make use of the pattern, all version comparisons are performed using the canonical form.

When presenting the version in a UI, the original version string value can be used.

Specifying a dependency on a specific version can be done by looking up the IU and then picking the version type pattern from that IU. A problem is when a user needs to specify a dependency on a range as the pattern is not known unless picked up first from the IU that specifies it. Once the pattern is known, the boundaries could be entered in the original form, and then parsed by the pattern.

When comparing two pattern based versions the pattern does not have to be taken into account. One could argue that patterns would need to be the same, but there are many ways to write a regular expression that parses the version format into the canonical form.

A more relaxed way of handling this would be to not encode the version at the required capability end. The comparison would instead be based on the format in the candidate IU. It would then not be possible to validate the correctness of the version format at the required capability end, but seems like a small price to pay for getting rid of/having keep track of the transformation rule. All responsibility is placed on the publisher.

Using Regexp

Regular expressions get quite horrible. In addition to the regular expression itself, there is the need to specify the segments significance and use of string or numeric comparison.

It could be specified like this:

/<regexp>/<group descriptions>/<version>

where <regexp> is a standard regular expression applied to <version> - this produces the segments. The <group descriptions> is a comma separated list of group descriptors. The group description specifies three things:

1. Position 2. Ascending or descending order 3. Numeric or Alphanumeric comparison

Position can be specified with a digit, or a quoted digit to indicate that the argument is a string. A '-' specifies descending order.

The OSGi type can then be described like this:

/^(\d+)(?:\.(\d+)(?:\.(\d+)(?:\.([a-zA-Z0-9_+-]+))?)?)?$/0,1,2,'3'/

Which is quite powerful, but absolutely horrible to read.

Using a custom format

A custom format can be made much more compact and easy to read.

If [...] means 'optional, and %d means 'digit group', and %s means 'alpha group'. An initial 'argument reordering' can be used if significance is not from left to right as elements are parsed.

OSGi can then be described as:

%d[.%d[.%d[.%s]]]

If the last parsed segment should be the most significant in the comparison

{3,0,1,2}%d[.%d[.%d[.%s]]]

Although not as powerful as a regular expression it may be sufficient. This scheme could use additional scanf syntax to add more capabilities (min/max chars, etc.).

This pattern is short enough to enable it being used directly in the type specification. This could be done by a declaration such as:

pattern( %d[.%d[.%d[.%s]]]):1.0.0

or a range

pattern(%d[.%d[.%d[.%s]]]):[1.0.0.r12345, 2.0.0]

Pattern Type Proposal

type name	description
raw	A version type of "raw" can be used when the format of the version is unknown. When compared against another raw type the following rules apply: If none of the types have a pattern, the comparison is a string comparison. If one of the types have a pattern, both compared types are transformed using the pattern and are then compared. If both types have a pattern, the pattern is applied to the respective type and the results are compared A raw version may not contain version range delimiters - if present in the original version string they must be escaped with a \. (And thus a \ must be escaped to be included in the version string).
raw(<pattern>)	Specifies a raw version type with a transformation pattern. The pattern can contain: characters  %s - an alpha group  %d - a numeric (integer) group \ - escape (must be used when a ')' or '%' is needed as a character, or if a '{' is needed as the first character and no reordering is wanted), \\ is needed to include a '\' [...] - indicates an optional part {<segment order>} - optional, when placed first in the string it defines the significance of segments - the default is {0,1,2,...}

Exploring Internationalization

The proposed types using alphanumerical segments are assumed to use vanilla string comparison. This does not work so well if versions are expressed in a language were lexical ordering is different. Language specific collation could be supported by combining version type name with the name of a ISO 639 Language code (see java.util.Locale) and where the default would be english. The language could be encoded with a separating '-' e.g. 'string-pt' for collation in portuguese.

This opens up another can of worms (decomposition strength, comparison of locale and non locale specified types, etc.), and it is probably best to implement just basic string comparison in the first release. It is also questionable if internationalization is needed at all, as "known tools" does not support this, and "correct collation" would thus yield a different result.

Support for internationalized collation is not recommended.

Version Range

Version range uses the osgi syntax, but prefixed with version type name.

Examples:

• [1.0.0,2.0.0] equal to osgi:[1.0.0,2.0.0]
• string:[titanic,andrea doria]
• alpha:[-7.-4.-0.-3.-3.fc.9,-8] - an ecoding of example KDE Admin version 7:4.0.3-3.fc9 to 8:
• triplet:[1.0.0.RC1,1.0.0]

Factory API

The factory API could be something simple like this:

public class VersionFactory
{
    IVersion createVersion(String versionString);
    IVersion createVersion(String versionType, String versionString);
}
public class VersionRangeFactory
{
    IVersionRange createVersionRange(String versionRangeString);
    IVersionRange createVersionRange(String versionType, String versionString);
}

Hard to say how much indirection is required - methods could just be static to keep things simple.

If we want to support the pattern based type, the factory methods needs the pattern as well. To make this generic, it could be seen as a paramter to the version type.

public class VersionFactory
{
    IVersion createVersion(String versionString);
    IVersion createVersion(String versionType, String versionString);
    IVersion createVersionWithArg(String versionTypeParameter, String versionString);
    IVersion createVersionWithArg(String versionType, String versionTypeParameter, String versionString);
}
public class VersionRangeFactory
{
    IVersionRange createVersionRange(String versionRangeString);
    IVersionRange createVersionRange(String versionType, String versionString);
    IVersionRange createVersionRangeWithArg(String versionTypeParameter, String versionString);
    IVersionRange createVersionRangeWithArg(String versionType, String versionTypeParameter, String versionString);
}

When creating a pattern based version, the versionTypeParameter must be supplied. When creating a pattern based version range, the pattern is optional - the pattern of the individual candidates would then be used to create the canonical form of the upper and lower bounds.

IVersion and IVersionRange API

Basically follows the current Version and VersionRange classes.

Implementation Steps

1. Enablement
   1. Change use of Version and VersionRange to use IVersion and IVersionRange
   2. Implement the VersionFactory and VersionRangeFactory by just creating instances of Version and VersionRange.
   3. Make sure UI validates human input based on factory result
2. Basic Extensions
   1. Implement the non pattern based version types
   2. Modify the factories to create correct instances of the types
3. Pattern based implementation
   1. Implement the pattern based version types
   2. Modify the factory
   3. Implement support for display of original version strings in UI code
   4. Implement support for pattern handling in UI (don't know all the detailed requirements).