A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.

In order to create a Maven-conformant aggregate repo all that is required is to set the property Maven Result property of the Aggregation to true. The aggregation deployed to the Build Root location will be a Maven repo. In this repo all .source artifacts will be co-located with their main artifact, sharing the same artifactId but with qualifier -sources, as expected by Maven.

Additionally, the property Version Format of the Aggregation allows to select between three strategies for version numbers:

Normal

Versions numbers are kept unmodified

Strict Maven

Use a '-' (dash) to separate three-part versions from any qualifier, to make the resulting versions parsable by Maven (so 3.12.2.v20161117-1814 will be converted to 3.12.2-v20161117-1814).

In previous versions this strategy was selected via property Strict Maven Versions which is now deprecated in favor of Version Format.

Maven Release

Cut off any qualifier, i.e., produce three-part version numbers to tell Maven that all artifacts are "releases".

If more fine grained control over version formats is needed, each Maven Mapping can be fine tuned with a pair of Version Pattern and Version Template, which will only be applied to artifacts matching the corresponding Maven Mapping. In this case, the existing version will be matched against the pattern, and if it matches, the version will be replaced by applying the corresponding template, where $1, $2 ... represent match groups from the pattern. As an example consider the case where the micro digit of a version should be omitted if it is equal to '0', so "4.12.0.v201504281640" will be translated to "4.12":

• Version Pattern: ([^.]+)\.([^.]+)\.0(?:\..*)?
• Version Template: $1.$2

If no version pattern is defined for any Maven Mapping, or if a defined version pattern does not match, the aggregator will fall back to the global Version Format strategy.

A sample aggr file, SDK4Mvn.aggr, shows an easy-to-use example of how to create a Maven-conformant p2 repository (i.e. one that works with Maven or p2). The sample has a short description file that explains some of the details.

Headless support

You will need a headless installation of CBI with the Aggregator feature installed.

Running from the command line

cbiAggr aggregate <options>

Note: if you install the aggregator into an Eclipse SDK or Platform (rather than a pure headless installation), you can run from the command line, by using

eclipse -application org.eclipse.cbi.p2repo.cli.headless aggregate <options>

For a detailed listing of the available options consult the next section.

Command line options

Hudson integration

Currently there is no support for Hudson.

Aggregation model components and specific actions

This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.

Global actions

The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:

• Clean Aggregation - cleans all traces of previous aggregations in the specified target location (Build Root)
• Validate Aggregation - verifies model validity and resolves dependencies; no artifacts are copied or created
• Build Aggregation - performs the aggregation and creates the aggregated repository in the target location (Build Root)
• Clean then Build Aggregation - performs a Clean followed by a Build

Aggregation

The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the Build Root (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). There are several child components some of which can be reference in other parts of the model: Configuration, Contact,Validation Set, Custom Category, Maven Mapping.

Configuration

An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual Contributions.

A Configuration is a combination of the following properties:

Validation Set

The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.

A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.

A validation set can have two types of children: Contribution and Validation Repository.

Versions prior to 0.2.0 did not have the validation set node. Older aggr files will therefore be converted and given one validation set called main

Contribution

Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. A contribution definition may consist of several Mapped Repository and Maven Mapping components.

Mapped Repository

A Contribution may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to Products, Bundles, Features, Categories, Exclusion Rules, Valid Configuration Rules.

Product

Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given Mapped Repository. This naturally requires that products are present in the repositories being mapped.

Category

Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given Mapped Repository.

Bundle

Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given Mapped Repository).

Feature

Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given Mapped Repository). The features to include must be present in the mapped repository.

Furthermore, this component provides the means to group features implicitly into Custom Categories.

Exclusion Rule

The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.

Valid Configuration Rule

By default all contributed contents of a Mapped Repository will be verified for all Configurations defined for the aggregation. A Valid Configuration Rule provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).

Maven Mapping (Contribution)

See Maven Mapping for a detailed description and list of properties.

Contact

Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.

Custom Category

A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by Features defined for inclusion from a Mapped Repository. The relationship to between Custom Category and a Feature is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the Feature definition, and vice versa.

Validation Repository

A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository. It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).

Maven Mapping

The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the Aggregator and the Contribution level.

This only applies when the Maven Result property of the Aggregator model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a Contribution adds to the aggregated repository:

1. first Maven Mappings defined as children of a Contribution are applied in the order in which they appear as children of the parent Contribution node
2. second Maven Mappings defined as children of the Aggregator model are applied in the order in which they appear as children of the parent Aggregator node
3. finally the default Maven Mapping is applied

The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:

^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$

The default maven mapping use the replacement $1 for groupId and $0 for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like org.eclipse.cbi.p2repo.aggregator the groupId would be org.eclipse.cbi.p2repo and the artifactId org.eclipse.cbi.p2repo.aggregator. The resulting Maven repo will have the folder structure:

org
   |
   eclipse
      |
      cbi 
         |
         org.eclipse.cbi.p2repo.aggregator
            |
            <folder name after version>
               |
               org.eclipse.cbi.p2repo.aggregator-<version>.jar
               ...

Legacy format support

[Note: this section is not completely accurate, with the current version of the CBI aggregator. It is certainly desired to offer this level of support, but that is not currently the case.]

As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.

The transformation is fully automated when you run in headless mode but you will see a message like The build model file is obsolete, using the default transformation indicating that a transformation took place. The new model only exists in memory while the aggregation runs and is never stored on disk.

When you are opening an old model in the IDE, a transformation wizard will be presented where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.

The behavior of the transformation wizard is:

• If the name is unchanged, also retain the detached contributions
• If a new name is given, embed all contributions in the new file

If you want a new name and still retain the detached contributions then you have two options

Either:

1. Do the transformation using the same name
2. Manually rename the main aggr file
3. Manually search/replace and change the name in all your contributions

Or:

1. Do the transformation using a different name
2. Remove all your detached contributions
3. Detach all contributions again

What else should be documented

1. version editor (version formats...)
2. how enable/disable on the context menu works
3. drag&drop support
4. 'available versions' tree
5. context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)

Questions

• How is blame-mail handled when running in the UI? Are the options "production" etc. available then?

When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.

• How do you know what the "log output url" is? How can it be set?

You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.

• Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??

The Hudson Support article was copied from the old buckminster documentation. It can be removed.

• Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there?

Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.

• It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??

Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).