https://wiki.eclipse.org/api.php?action=feedcontributions&user=Thomas.tada.se&feedformat=atomEclipsepedia - User contributions [en]2024-03-28T13:30:03ZUser contributionsMediaWiki 1.26.4https://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=371391CBI/aggregator/manual2014-10-02T11:03:40Z<p>Thomas.tada.se: </p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 4.4 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-4.4/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-4.4</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
The *.b3aggr model files shown be below can be created via File > New > b3 > Repository Aggregation.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project Git and opened in an appropriately set up workbench: [http://git.eclipse.org/c/b3/b3.git/tree/org.eclipse.b3.aggregator.samples/buckminster-indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
Note: if you install the aggregator into an Eclipse SDK or Platform (rather than a pure headless installation), you can run from the command line, by using<br />
<pre>eclipse -application org.eclipse.b3.cli.headless aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| Mirror meta-data repository references from the contributed repositories. Note the current recommendation is to not have update sites in feature definitions, but instead in "products", that is, to allow adopters or consumers to specify where to get updates from and what "extras" to offer. See {{bug|358415}}. But, this option can still useful for some situations.<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like ''The build model file is obsolete, using the default transformation'' indicating that a transformation took place. The new model only exists in memory while the aggregation runs and is never stored on disk.<br />
<br />
When you are opening an old model in the IDE, a transformation wizard will be presented where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster.json&diff=348181Buckminster.json2013-10-02T08:08:11Z<p>Thomas.tada.se: Adds 4.2 and 4.3 versions</p>
<hr />
<div>Below you will find a valid buckminster.json file for hudson. Stick in .hudson/userContent/buckminster/buckminster.json<br />
<br />
{"buckminsters":[<br />
{"id":"4.3",<br />
"name":"Buckminster 4.3",<br />
"url":"http://ftp.halifax.rwth-aachen.de/eclipse/tools/buckminster/products/director_latest.zip",<br />
"iu":"org.eclipse.buckminster.cmdline.product",<br />
"repositoryURL":"http://download.eclipse.org/tools/buckminster/headless-4.3",<br />
"repositories":[<br />
{ "url":"http://download.eclipse.org/tools/buckminster/headless-4.3",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.core.headless.feature"},<br />
{"id":"org.eclipse.buckminster.cvs.headless.feature"},<br />
{"id":"org.eclipse.buckminster.emma.headless.feature"},<br />
{"id":"org.eclipse.buckminster.git.headless.feature"},<br />
{"id":"org.eclipse.buckminster.maven.headless.feature"},<br />
{"id":"org.eclipse.buckminster.pde.headless.feature"}<br />
]},<br />
<br />
{"url":"http://download.cloudsmith.com/buckminster/external-4.3",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.subclipse.headless.feature"}<br />
]<br />
}<br />
]<br />
},<br />
{"id":"4.2",<br />
"name":"Buckminster 4.2",<br />
"url":"http://ftp.halifax.rwth-aachen.de/eclipse/tools/buckminster/products/director_latest.zip",<br />
"iu":"org.eclipse.buckminster.cmdline.product",<br />
"repositoryURL":"http://download.eclipse.org/tools/buckminster/headless-4.2",<br />
"repositories":[<br />
{ "url":"http://download.eclipse.org/tools/buckminster/headless-4.2",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.core.headless.feature"},<br />
{"id":"org.eclipse.buckminster.cvs.headless.feature"},<br />
{"id":"org.eclipse.buckminster.emma.headless.feature"},<br />
{"id":"org.eclipse.buckminster.git.headless.feature"},<br />
{"id":"org.eclipse.buckminster.maven.headless.feature"},<br />
{"id":"org.eclipse.buckminster.pde.headless.feature"}<br />
]},<br />
<br />
{"url":"http://download.cloudsmith.com/buckminster/external-4.2",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.subclipse.headless.feature"}<br />
]<br />
}<br />
]<br />
},<br />
{"id":"3.7",<br />
"name":"Buckminster 3.7",<br />
"url":"http://ftp.halifax.rwth-aachen.de/eclipse/tools/buckminster/products/director_latest.zip",<br />
"iu":"org.eclipse.buckminster.cmdline.product",<br />
"repositoryURL":"http://download.eclipse.org/tools/buckminster/headless-3.7",<br />
"repositories":[<br />
{ "url":"http://download.eclipse.org/tools/buckminster/headless-3.7",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.core.headless.feature"},<br />
{"id":"org.eclipse.buckminster.cvs.headless.feature"},<br />
{"id":"org.eclipse.buckminster.emma.headless.feature"},<br />
{"id":"org.eclipse.buckminster.git.headless.feature"},<br />
{"id":"org.eclipse.buckminster.maven.headless.feature"},<br />
{"id":"org.eclipse.buckminster.pde.headless.feature"}<br />
]},<br />
<br />
{"url":"http://download.cloudsmith.com/buckminster/external-3.7",<br />
"features":[<br />
{"id":"org.eclipse.buckminster.subclipse.headless.feature"}<br />
]<br />
}<br />
]<br />
}<br />
]<br />
}<br />
[[Category:Buckminster|JSON]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=347636CBI/aggregator/manual2013-09-20T09:53:48Z<p>Thomas.tada.se: </p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 4.3 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-4.3/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-4.3</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://http://git.eclipse.org/c/b3/b3.git/tree/org.eclipse.b3.aggregator.samples/buckminster-indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
Note: if you install the aggregator into an Eclipse SDK or Platform (rather than a pure headless installation), you can run from the command line, by using<br />
<pre>eclipse -application org.eclipse.b3.cli.headless aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| Mirror meta-data repository references from the contributed repositories. Note the current recommendation is to not have update sites in feature definitions, but instead in "products", that is, to allow adopters or consumers to specify where to get updates from and what "extras" to offer. See {{bug|358415}}. But, this option can still useful for some situations.<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like ''The build model file is obsolete, using the default transformation'' indicating that a transformation took place. The new model only exists in memory while the aggregation runs and is never stored on disk.<br />
<br />
When you are opening an old model in the IDE, a transformation wizard will be presented where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Simrel/Contributing_to_Simrel_Aggregation_Build&diff=331283Simrel/Contributing to Simrel Aggregation Build2013-03-15T07:44:56Z<p>Thomas.tada.se: </p>
<hr />
<div>__TOC__<br />
<br />
These instructions outline how to contribute to the aggregation build for the common repository. <br />
<br />
These instructions were substantially changed in August of 2012, to accommodate migration to new source repository, and at the same time a rename of the main project needed from that repository. The instructions and process for Juno (SR1 and SR2) are very similar to those for Kepler (SR0, in June 2013) except for the branch to use to for modifications/updates; Juno_maintenance for the former, master for the latter. For history of migration and change of project names, see {{bug|359240}}. <br />
<br />
If at anytime, there are questions, issues or problems, don't hesitate to ask on [mailto://cross-project-issues-dev@eclipse.org cross-project list], or [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=Cross-Project open a cross-project bug]. <br />
<br />
== Get the simrel.build project == <br />
<br />
If you don't have it already, you'll need to install Eclipse EGit, from common repository or their own repository at <br />
<br />
<code><nowiki>http://download.eclipse.org/egit/updates/</nowiki></code><br />
<br />
Clone the repository named <code>org.eclipse.simrel.build.git</code> and import the project of similar name, <code>org.eclipse.simrel.build</code>. There is only the one project in that repository. An appropriate URL would be similar to<br />
<br />
<code><nowiki>ssh://yourCommitterUserId@git.eclipse.org/gitroot/simrel/org.eclipse.simrel.build.git</nowiki></code> <br />
<br />
Make sure that your remote branch specifies '''rebase=true'''. ex:<br />
<br />
branch.master.remote=origin<br />
branch.master.merge=refs/heads/master<br />
branch.master.rebase=true<br />
<br />
See the [[#Configuration]] section below for more details on the best way to set up your workspace and cloned repository. <br />
<br />
You can set this up as the default for cloning/tracking remote branches by adding the user setting:<br />
<br />
branch.autosetuprebase=always<br />
<br />
Or, for casual browsing, see <br />
{{Git|simrel|org.eclipse.simrel.build.git}}<br />
<br />
If you do not have write access to this repository location, then [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=Cross-Project open a bugzilla entry] or send an email to the [mailto:webmaster@eclipse.org webmaster], explaining which project you are working with, and CC the Planning Council chairperson (currently david_williams@us.ibm.com).<br />
<br />
Note: To control "inactive committers", once per year, usually around SR1, anyone who has not committed anything to the "build" project (governed by callisto-dev Linux group) will simply be removed from that group. So, if you need write access again (after not needing it for a year) simply re-request to be added, again stating which project you are committing changes for. [Reminder: there is at least one id, 'genie', that is a member of callisto-dev that should not be removed, even though it will show no "commits", as it is required to be there for signing purposes. See {{bug|362436}} for problems that result from removing it.]<br />
<br />
The 'master' branch of that repo is used for the most forward looking release (namely Kepler, as of this writing) and maintenance for SR1 and SR2 will be named following the pattern of '<Release>_maintenance' (so, Juno_maintenance for Juno SR1 and Juno SR2).<br />
<br />
== Install the b3 Aggregator == <br />
<br />
* Use the 0.2.x version of the b3 aggregator Editor, running on Eclipse 4.2 (Juno SR2), installed from b3's "4.2 repo", at following URL: <br />
<br />
<code><nowiki>http://download.eclipse.org/modeling/emft/b3/updates-4.2/</nowiki></code><br />
<br />
* Follow instructions to install the [[Eclipse_b3/aggregator/manual | b3 Aggregator editor]] (and get the above mentioned project in your workspace). Tip: you may prefer to have a separate workspace for this aggregation work, as it works with many p2 repositories and can add a lot of background processing as it works with all those repositories. <br />
<br />
* Open the file <code>simrel.b3aggr</code> using the '''Aggregator Model Editor'''<br />
<br />
: Tip: Having the model "loaded" and allowing it to update itself (staying current and caching the latest contents of referenced repos) and syncing up (pulling) the simrel.build project on a regular basis, can often help you spot errors or inconsistencies with your own contributions, and allow fixes to be made before you commit and push your changes to the repository and risk breaking the build. <br />
<br />
== For new project contributions == <br />
<br />
* Create the following elements ('''New Child''') under top '''Aggregation: ''' node or '''Validation set:'''.<br />
** One or more '''Contacts''' (show Property View to specify both '''Email''' and '''Name'''). [It must be a real email, not dev list]. <br />
** A '''Contribution''' (specify '''Label''' and link to '''Contact''')<br />
*** A '''Mapped Repository''' (specify '''Location''': URL of your p2 repository)<br />
**** Your '''Feature'''s (select name from features found in your repository, select '''Categories''' from pre-defined set, specify exact version to be included in aggregation under '''Version Range''')<br />
* To create your b3aggrcon file, select your specific '''Contribution''' and invoke '''Detach Resource''' from the context menu. Choose a filename like <code>''projectname''.b3aggrcon</code> (renaming this file at a later stage is not supported). For ease of bookkeeping, it is advisable to use the exact name of your project as it appears in the Eclipse Foundation databases, including top level and subproject names, as appropriate, for example, <code>emf-validation.b3aggrcon</code> is preferable to <code>validation.b3aggrcon</code> or <code>webtools.b3aggrcon</code> preferable to <code>wtp.b3aggrcon</code>. <br />
<br />
* Verify. Be sure to "pull" to be sure you have current contents of every thing (with no conflicts). To ensure that your contribution will not break the build, right-click on top-most Aggregation: node and <br />
#'''Validate''' checks the general XML and EMF Model validity (short running), then<br />
#'''Validate Aggregation''' checks the whole model specifies correct and valid repo locations and compatibility dependencies (long running). <br />
<br />
* Commit and Push. At this point, you are ready to commit and push your contribution. You will need to check in changes to the <code>simrel.b3aggr</code> file, as well as your <code>''<projectname>''.b3aggrcon</code> file.<br />
<br />
== Updating contributions == <br />
<br />
* To change things like Contributors or Categories, or adding or removing features, you should use the b3 Aggregator with the top level <code>simrel.b3aggr</code> file ... as these things often have relationships that span multiple files and you need to update, synchronize and checkin all effected files. Note: the Categories are normally only added or edited by Planning Council, so be sure large changes there have been discussed via bugzilla, etc. (You can do this via a bugzilla entry in cross-project category). <br />
<br />
* To change values of feature versions, or repository URLs, you can directly change your <code>''projectname''.b3aggrcon</code> file with text editor (or build scripts) and check those in, in isolation. Of course, you can and should still use the b3 aggregator editor, and it is often desirable to do so, as it will do a "validate aggregation" and will tell you if something is wrong. For example, if there is a typo, and the repository URL does not point to a valid repository, you'll know about it right away, if you use the b3 Aggregator Editor. <br />
<br />
* Note that contributions, features, and repositories can be enabled or disabled, via property page. This allows temporary changes with minimal disruption. For example, if you disable a contribution or feature, it will be left out of a category, without having to also edit the category). This is especially useful if there is a leaf component that is "broken" and should temporarily be omitted from the build. '''Important: ''' One implication of this is you will need to sometimes re-enable your contribution or feature, even if you did not disable it. These are sometimes disabled by others -- without notice -- especially if a contribution or feature is causing build breaks for an extended period of time especially if there's been no communication explaining it or describing status or outlook on cross-project list. Of course, fixing the issue is the desired first choice, as disabling one contribution or feature will often require other contributions or features to be disabled simply because they depend on the broken one. <br />
<br />
== Categories == <br />
<br />
The overall categories used in the common repository are the responsibility of the Planning Council (in that they have the final say about any new ones, removals, etc.). So ... please open a cross-project bug if you'd like to propose new categories or some reorganization. But otherwise, feel free to add or remove your features to what ever categories you think are appropriate (using the full aggregator editor, since two files are changed when doing so) and others will open bugs if something seems wrong, or in the wrong category. <br />
<br />
== Runtime Target Platform Category == <br />
<br />
Some features (or bundles) are not intended to be installed into an IDE ... they do not contribute to the IDE (such as menu items, etc.). By convention, such features should be placed in the "EclipseRt Target Platform Category". This would be the case for, say, a "server" that someone was coding and testing for. In some cases, a runtime feature might "cause harm" (or, change behavior) if a user mistakenly installed it into their IDE. To prevent a feature (or bundle) from being installed into an IDE, the current "process" is for that feature or bundle to specify a negative requirement on a "magic IU". This is usually done in a p2.inf file, with contents of <br />
<br />
# this bundle should not be installed into IDE<br />
requires.0.namespace = A.PDE.Target.Platform<br />
requires.0.name = Cannot be installed into the IDE<br />
requires.0.range = 0.0.0<br />
<br />
The details of the "magic" solution may change in Juno, as a cleaner solution is being discussed in {{bug|365004}} ... it would be a similar "negative requirement" but just may be on a different (non magic) IU. <br />
<br />
<br />
== Configuration ==<br />
<br />
=== Configure the workspace ===<br />
<br />
[This section originally copied from [[Platform-releng/Git_Workflows#Configure_the_workspace]].<br />
<br />
Open the '''Team > Git > Configuration''' preference page and select the '''User Settings''' tab.<br />
* Add entries for '''user.name''' and '''user.email'''. If you don't want to share your e-mail you can also use your committer account ID. Note that you will not be able to push changes to the repository if the latter property is not matching with your records at the Eclipse Foundation.<br />
* Add entry '''branch.autosetuprebase = always'''<br />
<br />
On the '''General > Workspace''' preference page, set '''New text file line delimiter''' to '''Unix'''.<br />
<br />
=== Configuring the repo ===<br />
<br />
This section originally copied from [[Platform-releng/Git_Workflows#Configuring_the_repo]].<br />
<br />
Make sure that '''core.autocrlf = false''' and on Windows '''core.filemode = false'''. <br />
<br />
[Note: some people have found if they use other tools in addition to Eclipse, that '''core.autocrlf = false''' does not work well for them. The most important thing to avoid is files with "mixed line delimiters" since "autocrlf" will never be able to "fix up" files like that. You may want to turn on the Eclipse preference for "visible white space" so you can see if you or your tools introduce mixed line delimiters and if so correct them with "File, Convert Line Delimiters To ..." (after setting content type, described below) or use command line tools such as dos2unix to fix them to have consistent delimiters.] <br />
<br />
Unless you are working on topic branches, we work in a fairly linear history. Please make sure branch.'''branchname'''.rebase = true is set. If you've set '''branch.autosetuprebase = always''' as explained in [[#Configure_the_workspace]], then this is done automatically.<br />
<br />
Otherwise, once you've cloned a repository, you can go to the '''Preferences &gt; Team &gt; Git &gt; Configuration''' page. Select your repository, select the branch you picked when you cloned the repository, and click '''New Entry...'''. Append "rebase" to the text in the 'Key' field and enter "true" as value. <br />
<br />
[[Image:RepositoryConfigurationSettings.png]]<br />
<br />
=== Configuring the workspace content types ===<br />
<br />
By default, the b3aggr file and b3aggrcon file types will not be recognized by Eclipse and thus treated as "binary" files. This, for example, will prevent you from using the "Convert Line Endings" function on these files. Because of several issues with Git, EGit, and mixed line endings, we follow the convention of trying to always have only "LF" in the repository version of the files. It is strongly recommended to have only the "LF" version of the file in your workspace too. But, in some cases (due to mistakes or your own settings) you may have to "manually" change the CRLF back to LF and then commit and push those changes. <br />
<br />
One way to enable these file types to be seen as "text files" is to go into content type preferences, and associate *.b3aggr and *.b3aggrcon files with the content type of "XML". After doing this you may have to go back into "associated editors" and reset b3 Aggregator Editor to be the default editor for *.b3aggr files (or, else, the XML editor will be the default, and you really should not edit that file with XML editor, except in very specialized circumstances). <br />
<br />
<br />
<br />
<br />
<br />
[[Category:Juno| ]][[Category:Kepler| ]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Extending_or_Contributing_to_Buckminster&diff=315514Extending or Contributing to Buckminster2012-09-14T06:25:27Z<p>Thomas.tada.se: </p>
<hr />
<div>{{Backlink|Buckminster Project}}<br />
<br />
If you are going to be working with Buckminster itself, writing your own extensions, or provide patches, you should first install Buckminster as described in [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/doc/BuckyBook.pdf The Definitive Guide]. As Buckminster source is stored in a Git repository, you need to install the EGit and JGit features from the release train.<br />
<br />
==Initial clone of the repository==<br />
#Ensure that the '''Git Repositories''' view is visible. You should be able to find it under ''"Window"'' => ''"Show View"'' => ''"Other..."'' => ''"Git"'' if its not.<br />
#Click on ''"Clone a Git repository"''<br />
#Paste the following URL into the URI field in the wizard that pops up <pre>git://git.eclipse.org/gitroot/buckminster/buckminster.git</pre><br />
#Click ''"Next>"'' twice, then ''"Finish"''<br />
<br />
==Populating the workspace==<br />
===Create the local clone and check out the releng project===<br />
#Expand ''"buckminster"'' => ''"Working Directory"'' in the '''Git Repositories''' view.<br />
#Select ''"org.eclipse.buckminster.releng"'', right click, and then select ''"Import projects..."''<br />
#Click ''"Next>"'' in the list of projects. A page where only the selected project is visible shows up.<br />
#Click ''"Finish"''<br />
<br />
===Prepare an empty target platform===<br />
#Create a new general project named '''TP''' (or some name of your preference)<br />
#Open ''"Window"'' => ''"Preferences"'' and expand ''"Plug-in Development"''<br />
#Select ''"Target Platform"''<br />
#Click ''"Add..."''<br />
#Click ''"Next>"'' (i.e. start with nothing)<br />
#Enter '''TP''' in the ''Name:'' field<br />
#Click ''"Add..."''<br />
#Select ''"Directory"'' and click ''"Next"''<br />
#Click on ''"Variables..."''<br />
#Scroll down and select ''"workspace_loc"'' and then type '''TP''' in the ''Argument:'' field.<br />
#Click ''"OK"'' and then ''"Finish"'' and then ''"Finish"'' again.<br />
#Set your new target definition active by checking it in the list of ''Target definitions:'' and click ''"OK"''<br />
<br />
===Run the cquery to populate the workspace===<br />
#Expand the '''org.eclipse.buckminster.releng''' project and open the '''build.cquery''' file. It should open using the Buckminster cquery editor.<br />
# Press "Resolve & Materialize" in the CQUERY editor that appears<br />
# Buckminster will now fetch and build Buckminster from source.<br />
<br />
==Submitting a patch==<br />
#Make necessary changes<br />
#Ideally, write a JUnit test case that test your change and add it to the set of JUnit tests in the corresponding test project<br />
#Commit your changes to your local Buckminster clone<br />
#Use EGit to create a patch for your commit. This can be done by viewing the repository history, right clicking a commit, and then select ''"Create Patch..."''<br />
#Submit this patch as an attachment to a bug in our [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Buckminster Bugzilla]<br />
[[Category:Buckminster]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster_Aggregator_User_Guide&diff=275841Buckminster Aggregator User Guide2011-11-03T07:50:33Z<p>Thomas.tada.se: Replacing page with 'The Buckmintster aggregator has move to b3. Please visit the [http://wiki.eclipse.org/Eclipse_b3/aggregator/manual B3 Aggregator Manual] page instead.'</p>
<hr />
<div>The Buckmintster aggregator has move to b3. Please visit the [http://wiki.eclipse.org/Eclipse_b3/aggregator/manual B3 Aggregator Manual] page instead.</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Eclipse_b3/contributing/Setting_Up_Development_Environment&diff=264866Eclipse b3/contributing/Setting Up Development Environment2011-08-17T14:07:08Z<p>Thomas.tada.se: /* Getting The Source Code */</p>
<hr />
<div>If you intend to hack b3, here is how to get started.<br />
<br />
==Installing Prerequisites==<br />
In order to work on b3 you'll need to install prerequisites described in the following paragraphs. (In the order they are presented in this document.)<br />
<br />
===Eclipse===<br />
The first thing you'll need is the Eclipse 3.6 IDE for your platform. Here is a pointer (if you don't get it by heart yet) to the download page: http://www.eclipse.org/downloads/<br /> Grab the Classic version at the very bottom and unpack the archive into a convenient path.<br />
<br />
===Git support===<br />
Git support is needed simply because the b3 sources are kept in an Git repository.<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the Indigo repository<br />
* expand the ''Collaboration'' category<br />
* select the ''Eclipse EGit'' and ''Eclipse JGit'' entries<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===EMF===<br />
EMF is the first of dependencies of b3, to install it follow these instructions:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the Indigo repository<br />
* expand the ''Modeling'' category<br />
* select the 'Ecore Tools SDK''<br />
* select the ''EMF - Eclipse Modeling Framework SDK''<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Xtext===<br />
Xtext is the next dependency of b3. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* Install Xtext and mwe2 from http://download.itemis.com/updates/releases/<br />
<br />
Before Helios and Xtext release 1.0 it was required to use particular versions. Here are the older instructions for milestones and nightly builds:<br />
* browse the Xtext milestones repository (latest release won't do as of writing of this document): http://download.itemis.com/updates/milestones<br />
* select entire following categories:<br />
** ''tmf-xtext '' use version 1.0 M6 or later (see below)<br />
** ''xtext-antlr '' use version 1.0 M6 or later (see below)<br />
* click through the other pages of the install wizard to finish the installation (you will need to acknowledge installation of unsigned software as some of the installed components are not signed)<br />
<br /><br />
The Xtext completes the installations so please restart the workbench as recommended this time!<br />
<br />
If you are running on a 64 bit machine, a build after 20100503 will be required (soon) as there are pending changes that will cause "static intitializer exceeds size limit" errors otherwise. You can get nightly builds at http://download.itemis.com/updates/nightly/<br />
<br />
==Preparing The Workspace==<br />
In order to keep the code clean and to avoid merge problems, b3 projects should be organized with unified code style and formatting rules. Unfortunately not all the settings are available as project specific. This is why a workspace template should be used before any source code is checked out. Follow the steps below to initialize your workspace properly:<br />
* download the [[Media:b3 workspace template archive.zip|b3 workspace template archive]]<br />
* create an empty folder for the new workspace<br />
* extract the template archive into that folder<br />
* start Eclipse IDE using the new workspace<br />
* refresh the initial project (select the project and press F5)<br />
* follow the steps described in the README.txt file (which is initially opened in the Eclipse IDE text editor)<br />
<br/><br />
<br />
==Getting The Source Code==<br />
B3 source code is kept in the [http://git.eclipse.org/c/b3/b3.git/ b3.git] repository.<br />
<br />
To get the b3 sources, follow these instructions:<br />
* run Eclipse, open the ''File'' -> ''Import'' wizard<br />
* Select ''Git'' -> ''Projects from Git'' and click ''Next''<br />
* Click ''Clone...''<br />
* Enter the URI ''<nowiki>git://git.eclipse.org/gitroot/b3/b3.git</nowiki>'' and click ''Next''<br />
* Verify that the ''master'' branch is checked and click ''Next''<br />
* Select ''Import Existing Projects'' and click ''Next''<br />
* Check the projects listed below and click ''Finish''<br />
** org.eclipse.b3 [ .backend.*, .beelang.*, .build, .core ]<br />
<br /><br />
Done!<br />
<br />
==Running It==<br />
Once you have all the b3 projects in your workspace you are ready to launch a b3 enabled workbench. You do it like this:<br />
* bring up the ''Run / Run Configurations...'' dialog<br />
* create a new ''Eclipse Application'' launch configuration (or use the /org.eclipse.b3.beelang.tests/B3InteractiveTest.launch in the beelang.tests project mentioned below)<br />
* rename it to something suitable, like ''B3''<br />
* launch it<br />
<br /><br />
A new workbench should start, perform the following actions in the new workbench:<br />
<ul><br />
<li>checkout the following project: http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.beelang.tests</li><br />
<li>switch to the ''Package Explorer'' view and open the ''/src-b3/smoketestFunctionArithmetic.b3'' file in the checked out project<br />
<ul><br />
<li>the file should open in an editor supporting syntax highlighting</li><br />
<li>there should be an outline view with a root node of ''BeeModel''</li><br />
</ul></li><br />
<li>right click the ''BeeModel'' node in the outline view a select the ''Run Main Function'' option<br />
<ul><li>a console window with the following content should show up:<pre><br />
Running the main function...<br />
Result = true</pre></li><br />
</ul></li><br />
</ul><br />
<br /><br />
Congratulations, you have just executed your first b3 code!<br />
<br />
==Running b3 JUnit tests==<br />
B3 has a JUnit runner that runs b3 scripts as JUnit tests. There are many tests written in b3. There is a launch configuration for running the tests - it is found here /org.eclipse.b3.beelang.tests/AllB3Tests - headless.launch - simply use this launch configuration to get the correct configuration.<br />
<br />
At the time this instruction was updated there are three outstanding failures in the test (for which there are bugzillas logged) - one failure because repository syntax has changed and impl is not yet working, and two are related to b3's java integration.<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Eclipse_b3/contributing/Setting_Up_Development_Environment&diff=264860Eclipse b3/contributing/Setting Up Development Environment2011-08-17T13:43:26Z<p>Thomas.tada.se: </p>
<hr />
<div>If you intend to hack b3, here is how to get started.<br />
<br />
==Installing Prerequisites==<br />
In order to work on b3 you'll need to install prerequisites described in the following paragraphs. (In the order they are presented in this document.)<br />
<br />
===Eclipse===<br />
The first thing you'll need is the Eclipse 3.6 IDE for your platform. Here is a pointer (if you don't get it by heart yet) to the download page: http://www.eclipse.org/downloads/<br /> Grab the Classic version at the very bottom and unpack the archive into a convenient path.<br />
<br />
===Git support===<br />
Git support is needed simply because the b3 sources are kept in an Git repository.<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the Indigo repository<br />
* expand the ''Collaboration'' category<br />
* select the ''Eclipse EGit'' and ''Eclipse JGit'' entries<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===EMF===<br />
EMF is the first of dependencies of b3, to install it follow these instructions:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the Indigo repository<br />
* expand the ''Modeling'' category<br />
* select the 'Ecore Tools SDK''<br />
* select the ''EMF - Eclipse Modeling Framework SDK''<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Xtext===<br />
Xtext is the next dependency of b3. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* Install Xtext and mwe2 from http://download.itemis.com/updates/releases/<br />
<br />
Before Helios and Xtext release 1.0 it was required to use particular versions. Here are the older instructions for milestones and nightly builds:<br />
* browse the Xtext milestones repository (latest release won't do as of writing of this document): http://download.itemis.com/updates/milestones<br />
* select entire following categories:<br />
** ''tmf-xtext '' use version 1.0 M6 or later (see below)<br />
** ''xtext-antlr '' use version 1.0 M6 or later (see below)<br />
* click through the other pages of the install wizard to finish the installation (you will need to acknowledge installation of unsigned software as some of the installed components are not signed)<br />
<br /><br />
The Xtext completes the installations so please restart the workbench as recommended this time!<br />
<br />
If you are running on a 64 bit machine, a build after 20100503 will be required (soon) as there are pending changes that will cause "static intitializer exceeds size limit" errors otherwise. You can get nightly builds at http://download.itemis.com/updates/nightly/<br />
<br />
==Preparing The Workspace==<br />
In order to keep the code clean and to avoid merge problems, b3 projects should be organized with unified code style and formatting rules. Unfortunately not all the settings are available as project specific. This is why a workspace template should be used before any source code is checked out. Follow the steps below to initialize your workspace properly:<br />
* download the [[Media:b3 workspace template archive.zip|b3 workspace template archive]]<br />
* create an empty folder for the new workspace<br />
* extract the template archive into that folder<br />
* start Eclipse IDE using the new workspace<br />
* refresh the initial project (select the project and press F5)<br />
* follow the steps described in the README.txt file (which is initially opened in the Eclipse IDE text editor)<br />
<br/><br />
<br />
==Getting The Source Code==<br />
B3 source code is kept in the [http://git.eclipse.org/c/b3/b3.git/ b3.git] repository.<br />
<br />
To get the b3 sources, follow these instructions:<br />
* run Eclipse, open the ''SVN Repository Exploring'' perspective<br />
* add the chosen repository to the the list of SVN repositories (which should be empty at this time)<br />
* expand the repository node, then expand the ''trunk'' node<br />
* checkout the projects listed below from ''trunk'' as projects into your workspace<br />
** org.eclipse.b3 [ .backend.*, .beelang.*, .build, .core ]<br />
<br /><br />
Done!<br />
<br />
==Running It==<br />
Once you have all the b3 projects in your workspace you are ready to launch a b3 enabled workbench. You do it like this:<br />
* bring up the ''Run / Run Configurations...'' dialog<br />
* create a new ''Eclipse Application'' launch configuration (or use the /org.eclipse.b3.beelang.tests/B3InteractiveTest.launch in the beelang.tests project mentioned below)<br />
* rename it to something suitable, like ''B3''<br />
* launch it<br />
<br /><br />
A new workbench should start, perform the following actions in the new workbench:<br />
<ul><br />
<li>checkout the following project: http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.beelang.tests</li><br />
<li>switch to the ''Package Explorer'' view and open the ''/src-b3/smoketestFunctionArithmetic.b3'' file in the checked out project<br />
<ul><br />
<li>the file should open in an editor supporting syntax highlighting</li><br />
<li>there should be an outline view with a root node of ''BeeModel''</li><br />
</ul></li><br />
<li>right click the ''BeeModel'' node in the outline view a select the ''Run Main Function'' option<br />
<ul><li>a console window with the following content should show up:<pre><br />
Running the main function...<br />
Result = true</pre></li><br />
</ul></li><br />
</ul><br />
<br /><br />
Congratulations, you have just executed your first b3 code!<br />
<br />
==Running b3 JUnit tests==<br />
B3 has a JUnit runner that runs b3 scripts as JUnit tests. There are many tests written in b3. There is a launch configuration for running the tests - it is found here /org.eclipse.b3.beelang.tests/AllB3Tests - headless.launch - simply use this launch configuration to get the correct configuration.<br />
<br />
At the time this instruction was updated there are three outstanding failures in the test (for which there are bugzillas logged) - one failure because repository syntax has changed and impl is not yet working, and two are related to b3's java integration.<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Eclipse_b3/contributing/Setting_Up_Development_Environment&diff=264854Eclipse b3/contributing/Setting Up Development Environment2011-08-17T13:27:41Z<p>Thomas.tada.se: </p>
<hr />
<div>If you intend to hack b3, here is how to get started.<br />
<br />
==Installing Prerequisites==<br />
In order to work on b3 you'll need to install prerequisites described in the following paragraphs. (In the order they are presented in this document.)<br />
<br />
===Eclipse===<br />
The first thing you'll need is the Eclipse 3.6 IDE for your platform. Here is a pointer (if you don't get it by heart yet) to the download page: http://www.eclipse.org/downloads/<br /> Grab the Classic version at the very bottom and unpack the archive into a convenient path.<br />
<br />
===Git support===<br />
Git support is needed simply because the b3 sources are kept in an Git repository.<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the Git releases repository: http://download.eclipse.org/egit/updates/<br />
* uncheck the ''Group items by category'' check box<br />
* select the ''Eclipse EGit'' and ''Eclipse JGit'' entries<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===EMF===<br />
EMF is the first of dependencies of b3, to install it follow these instructions:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the EMF releases repository: http://download.eclipse.org/modeling/emf/updates/releases<br />
* expand the ''EMF SDK 2.5.0 (EMF + XSD)'' category (you may experiment with later releases if available)<br /><br />
* select all entries in the expanded category which names start with ''EMF'', except ''EMF/XSD All-In-One SDK'' (you may also omit Examples)<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Ecore Tools===<br />
(Installing and using Ecore Tools from Helios (not that is released) should probably work fine, but not yet tested).<br />
<br />
B3 works without this, but the tools include, among others, graphical editors for certain b3 language definitions which would otherwise have to be edited in plain text, which would be a pain. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the EMFT releases repository: http://download.eclipse.org/modeling/emft/updates/releases<br />
* expand the ''EMFT ECORETOOLS SDK (Incubation) 0.9.0'' category (you may experiment with later releases if available)<br /> '''Subject to change now that 3.6M7 is just about to be available'''<br />
* select all entries in the expanded category, except ''Ecore Tools All-In-One Feature (Incubation)'' (you may also omit Examples)<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Xtext===<br />
Xtext is the next dependency of b3. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* Install Xtext and mwe2 from http://download.itemis.com/updates/releases/<br />
<br />
Before Helios and Xtext release 1.0 it was required to use particular versions. Here are the older instructions for milestones and nightly builds:<br />
* browse the Xtext milestones repository (latest release won't do as of writing of this document): http://download.itemis.com/updates/milestones<br />
* select entire following categories:<br />
** ''tmf-xtext '' use version 1.0 M6 or later (see below)<br />
** ''xtext-antlr '' use version 1.0 M6 or later (see below)<br />
* click through the other pages of the install wizard to finish the installation (you will need to acknowledge installation of unsigned software as some of the installed components are not signed)<br />
<br /><br />
The Xtext completes the installations so please restart the workbench as recommended this time!<br />
<br />
If you are running on a 64 bit machine, a build after 20100503 will be required (soon) as there are pending changes that will cause "static intitializer exceeds size limit" errors otherwise. You can get nightly builds at http://download.itemis.com/updates/nightly/<br />
<br />
==Preparing The Workspace==<br />
In order to keep the code clean and to avoid merge problems, b3 projects should be organized with unified code style and formatting rules. Unfortunately not all the settings are available as project specific. This is why a workspace template should be used before any source code is checked out. Follow the steps below to initialize your workspace properly:<br />
* download the [[Media:b3 workspace template archive.zip|b3 workspace template archive]]<br />
* create an empty folder for the new workspace<br />
* extract the template archive into that folder<br />
* start Eclipse IDE using the new workspace<br />
* refresh the initial project (select the project and press F5)<br />
* follow the steps described in the README.txt file (which is initially opened in the Eclipse IDE text editor)<br />
<br/><br />
<br />
==Getting The Source Code==<br />
B3 source code is kept in the Eclipse SVN repository. There are two ways to access the repository:<br />
* anonymous access, through the http protocol, at: '''<nowiki>http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/</nowiki>'''<br />
* commiters' access, through the svn+ssh protocol, at: '''svn+ssh://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/'''<br />
<br /><br />
choose whichever is appropriate for you.<br />
<br />
To get the b3 sources, follow these instructions:<br />
* run Eclipse, open the ''SVN Repository Exploring'' perspective<br />
* add the chosen repository to the the list of SVN repositories (which should be empty at this time)<br />
* expand the repository node, then expand the ''trunk'' node<br />
* checkout the projects listed below from ''trunk'' as projects into your workspace<br />
** org.eclipse.b3 [ .backend.*, .beelang.*, .build, .core ]<br />
<br /><br />
Done!<br />
<br />
==Running It==<br />
Once you have all the b3 projects in your workspace you are ready to launch a b3 enabled workbench. You do it like this:<br />
* bring up the ''Run / Run Configurations...'' dialog<br />
* create a new ''Eclipse Application'' launch configuration (or use the /org.eclipse.b3.beelang.tests/B3InteractiveTest.launch in the beelang.tests project mentioned below)<br />
* rename it to something suitable, like ''B3''<br />
* launch it<br />
<br /><br />
A new workbench should start, perform the following actions in the new workbench:<br />
<ul><br />
<li>checkout the following project: http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.beelang.tests</li><br />
<li>switch to the ''Package Explorer'' view and open the ''/src-b3/smoketestFunctionArithmetic.b3'' file in the checked out project<br />
<ul><br />
<li>the file should open in an editor supporting syntax highlighting</li><br />
<li>there should be an outline view with a root node of ''BeeModel''</li><br />
</ul></li><br />
<li>right click the ''BeeModel'' node in the outline view a select the ''Run Main Function'' option<br />
<ul><li>a console window with the following content should show up:<pre><br />
Running the main function...<br />
Result = true</pre></li><br />
</ul></li><br />
</ul><br />
<br /><br />
Congratulations, you have just executed your first b3 code!<br />
<br />
==Running b3 JUnit tests==<br />
B3 has a JUnit runner that runs b3 scripts as JUnit tests. There are many tests written in b3. There is a launch configuration for running the tests - it is found here /org.eclipse.b3.beelang.tests/AllB3Tests - headless.launch - simply use this launch configuration to get the correct configuration.<br />
<br />
At the time this instruction was updated there are three outstanding failures in the test (for which there are bugzillas logged) - one failure because repository syntax has changed and impl is not yet working, and two are related to b3's java integration.<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Eclipse_b3/contributing/Setting_Up_Development_Environment&diff=264853Eclipse b3/contributing/Setting Up Development Environment2011-08-17T13:23:18Z<p>Thomas.tada.se: /* Subversion support */</p>
<hr />
<div>If you intend to hack b3, here is how to get started.<br />
<br />
==Installing Prerequisites==<br />
In order to work on b3 you'll need to install prerequisites described in the following paragraphs. (In the order they are presented in this document.)<br />
<br />
===Eclipse===<br />
The first thing you'll need is the Eclipse 3.6 IDE for your platform. Here is a pointer (if you don't get it by heart yet) to the download page: http://www.eclipse.org/downloads/<br /> Grab the Classic version at the very bottom and unpack the archive into a convenient path.<br />
<br />
===Git support===<br />
Git support is needed simply because the b3 sources are kept in an Git repository.<br />
<br />
===EMF===<br />
EMF is the first of dependencies of b3, to install it follow these instructions:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the EMF releases repository: http://download.eclipse.org/modeling/emf/updates/releases<br />
* expand the ''EMF SDK 2.5.0 (EMF + XSD)'' category (you may experiment with later releases if available)<br /><br />
* select all entries in the expanded category which names start with ''EMF'', except ''EMF/XSD All-In-One SDK'' (you may also omit Examples)<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Ecore Tools===<br />
(Installing and using Ecore Tools from Helios (not that is released) should probably work fine, but not yet tested).<br />
<br />
B3 works without this, but the tools include, among others, graphical editors for certain b3 language definitions which would otherwise have to be edited in plain text, which would be a pain. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* browse the EMFT releases repository: http://download.eclipse.org/modeling/emft/updates/releases<br />
* expand the ''EMFT ECORETOOLS SDK (Incubation) 0.9.0'' category (you may experiment with later releases if available)<br /> '''Subject to change now that 3.6M7 is just about to be available'''<br />
* select all entries in the expanded category, except ''Ecore Tools All-In-One Feature (Incubation)'' (you may also omit Examples)<br />
* click through the other pages of the install wizard to finish the installation<br />
<br /><br />
<br />
===Xtext===<br />
Xtext is the next dependency of b3. Installation instructions follow:<br />
<br />
* open the ''Help / Install New Software'' wizard<br />
* Install Xtext and mwe2 from http://download.itemis.com/updates/releases/<br />
<br />
Before Helios and Xtext release 1.0 it was required to use particular versions. Here are the older instructions for milestones and nightly builds:<br />
* browse the Xtext milestones repository (latest release won't do as of writing of this document): http://download.itemis.com/updates/milestones<br />
* select entire following categories:<br />
** ''tmf-xtext '' use version 1.0 M6 or later (see below)<br />
** ''xtext-antlr '' use version 1.0 M6 or later (see below)<br />
* click through the other pages of the install wizard to finish the installation (you will need to acknowledge installation of unsigned software as some of the installed components are not signed)<br />
<br /><br />
The Xtext completes the installations so please restart the workbench as recommended this time!<br />
<br />
If you are running on a 64 bit machine, a build after 20100503 will be required (soon) as there are pending changes that will cause "static intitializer exceeds size limit" errors otherwise. You can get nightly builds at http://download.itemis.com/updates/nightly/<br />
<br />
==Preparing The Workspace==<br />
In order to keep the code clean and to avoid merge problems, b3 projects should be organized with unified code style and formatting rules. Unfortunately not all the settings are available as project specific. This is why a workspace template should be used before any source code is checked out. Follow the steps below to initialize your workspace properly:<br />
* download the [[Media:b3 workspace template archive.zip|b3 workspace template archive]]<br />
* create an empty folder for the new workspace<br />
* extract the template archive into that folder<br />
* start Eclipse IDE using the new workspace<br />
* refresh the initial project (select the project and press F5)<br />
* follow the steps described in the README.txt file (which is initially opened in the Eclipse IDE text editor)<br />
<br/><br />
<br />
==Getting The Source Code==<br />
B3 source code is kept in the Eclipse SVN repository. There are two ways to access the repository:<br />
* anonymous access, through the http protocol, at: '''<nowiki>http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/</nowiki>'''<br />
* commiters' access, through the svn+ssh protocol, at: '''svn+ssh://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/'''<br />
<br /><br />
choose whichever is appropriate for you.<br />
<br />
To get the b3 sources, follow these instructions:<br />
* run Eclipse, open the ''SVN Repository Exploring'' perspective<br />
* add the chosen repository to the the list of SVN repositories (which should be empty at this time)<br />
* expand the repository node, then expand the ''trunk'' node<br />
* checkout the projects listed below from ''trunk'' as projects into your workspace<br />
** org.eclipse.b3 [ .backend.*, .beelang.*, .build, .core ]<br />
<br /><br />
Done!<br />
<br />
==Running It==<br />
Once you have all the b3 projects in your workspace you are ready to launch a b3 enabled workbench. You do it like this:<br />
* bring up the ''Run / Run Configurations...'' dialog<br />
* create a new ''Eclipse Application'' launch configuration (or use the /org.eclipse.b3.beelang.tests/B3InteractiveTest.launch in the beelang.tests project mentioned below)<br />
* rename it to something suitable, like ''B3''<br />
* launch it<br />
<br /><br />
A new workbench should start, perform the following actions in the new workbench:<br />
<ul><br />
<li>checkout the following project: http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.beelang.tests</li><br />
<li>switch to the ''Package Explorer'' view and open the ''/src-b3/smoketestFunctionArithmetic.b3'' file in the checked out project<br />
<ul><br />
<li>the file should open in an editor supporting syntax highlighting</li><br />
<li>there should be an outline view with a root node of ''BeeModel''</li><br />
</ul></li><br />
<li>right click the ''BeeModel'' node in the outline view a select the ''Run Main Function'' option<br />
<ul><li>a console window with the following content should show up:<pre><br />
Running the main function...<br />
Result = true</pre></li><br />
</ul></li><br />
</ul><br />
<br /><br />
Congratulations, you have just executed your first b3 code!<br />
<br />
==Running b3 JUnit tests==<br />
B3 has a JUnit runner that runs b3 scripts as JUnit tests. There are many tests written in b3. There is a launch configuration for running the tests - it is found here /org.eclipse.b3.beelang.tests/AllB3Tests - headless.launch - simply use this launch configuration to get the correct configuration.<br />
<br />
At the time this instruction was updated there are three outstanding failures in the test (for which there are bugzillas logged) - one failure because repository syntax has changed and impl is not yet working, and two are related to b3's java integration.<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264677CBI/aggregator/manual2011-08-16T06:14:28Z<p>Thomas.tada.se: /* Legacy format support */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like ''The build model file is obsolete, using the default transformation'' indicating that a transformation took place. The new model only exists in memory while the aggregation runs and is never stored on disk.<br />
<br />
When you are opening an old model in the IDE, a transformation wizard will be presented where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264676CBI/aggregator/manual2011-08-16T06:13:43Z<p>Thomas.tada.se: /* Legacy format support */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like ''The build model file is obsolete, using the default transformation'' indicating that a transformation took place.<br />
<br />
When you are opening an old model in the IDE, a transformation wizard will be presented where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264675CBI/aggregator/manual2011-08-16T06:12:44Z<p>Thomas.tada.se: /* Legacy format support */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like ''The build model file is obsolete, using the default transformation'' indicating that a transformation took place.<br />
<br />
The IDE will present a transformation wizard where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264674CBI/aggregator/manual2011-08-16T06:11:53Z<p>Thomas.tada.se: /* What else should be documented */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=Legacy format support=<br />
As time passes, the aggregator model will undergo changes. All such changes will result in a new XML schema describing the aggregator model. The aggregator will always detect the schema in use by a model and, if needed, transform it into the current schema.<br />
<br />
The transformation is fully automated when you run in headless mode but you will see a message like '''The build model file is obsolete, using the default transformation''' indicating that a transformation took place.<br />
<br />
The IDE will present a transformation wizard where you are given a chance to specify a new file name for the transformed model. Please note that if you are using detached contributions (contributions that live in files of their own), and you want to keep it that way, then you should not change the name. If the name is changed, the aggregator has no choice but to create a new file that embeds all the contributions. This is because a detached contribution is referencing the aggregation file by name.<br />
<br />
The behavior of the transformation wizard is:<br />
* If the name is unchanged, also retain the detached contributions<br />
* If a new name is given, embed all contributions in the new file<br />
<br />
If you want a new name and still retain the detached contributions then you have two options<br />
<br />
Either:<br />
# Do the transformation using the same name<br />
# Manually rename the main b3aggr file<br />
# Manually search/replace and change the name in all your contributions<br />
<br />
Or:<br />
# Do the transformation using a different name<br />
# Remove all your detached contributions<br />
# Detach all contributions again<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264673CBI/aggregator/manual2011-08-16T05:57:06Z<p>Thomas.tada.se: /* What else should be documented */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264672CBI/aggregator/manual2011-08-16T05:54:16Z<p>Thomas.tada.se: /* Configuration */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
|'''X86'''<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
|'''Win32'''<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
|'''Win32'''<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264671CBI/aggregator/manual2011-08-16T05:52:47Z<p>Thomas.tada.se: /* Configuration */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''Photon'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264670CBI/aggregator/manual2011-08-16T05:49:46Z<p>Thomas.tada.se: /* Configuration */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
<br />
'''PPC64'''<br />
<br />
'''S390'''<br />
<br />
'''S390x'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
<br />
'''QNX'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
<br />
'''PHOTON'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264669CBI/aggregator/manual2011-08-16T05:46:51Z<p>Thomas.tada.se: /* Command line options */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--agentLocation'''<br />
| <path to directory><br />
| '''<buildRoot>'''/p2<br />
| Controls the location of the p2 agent. When specified as outside of the build root, the agent will not be cleaned out by the aggregator and thus retain its cache.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264668CBI/aggregator/manual2011-08-16T05:42:48Z<p>Thomas.tada.se: /* Functional Overview */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). ''Contributions'' and ''Validation Repositories'' are grouped into ''Validation Sets''. Everything in a ''Validation Set'' will be validated as one unit, i.e. it must be possible to install everything in a ''Validation Set'' together. The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated '''Validation Sets''' to ensure that everything in them is installable at the same time. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264613CBI/aggregator/manual2011-08-15T09:03:42Z<p>Thomas.tada.se: /* Command line options */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE (VERIFY in 0.1)<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264612CBI/aggregator/manual2011-08-15T09:03:16Z<p>Thomas.tada.se: /* Running from the command line */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=264611CBI/aggregator/manual2011-08-15T09:02:09Z<p>Thomas.tada.se: /* Command line options */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 validationSet <options></pre> ( if you use version 0.1, just type ''b3 aggregate <options>'' )<br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VALIDATE<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VALIDATE''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260663CBI/aggregator/manual2011-07-11T13:16:36Z<p>Thomas.tada.se: /* Aggregator */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregation==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260662CBI/aggregator/manual2011-07-11T13:16:00Z<p>Thomas.tada.se: </p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregation model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregation model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregation model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregation model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregation model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregation model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260660CBI/aggregator/manual2011-07-11T13:13:37Z<p>Thomas.tada.se: /* Aggregator */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregation node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contact|Contact]],[[#Validation Set|Validation Set]], [[#Custom Category|Custom Category]], [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Validation Set===<br />
The aggregator uses the p2 planner tool to determine what needs to be copied. The validation set determines the scope of one such planning session per valid configuration. This vouches for that everything that is contained within a validation set will be installable into the same target. Sometimes it is desirable to have more than one version of a feature in a repository even though multiple versions cannot be installed. This can be done using one validation set for each version.<br />
<br />
A validation set can extend other validation set and thereby provide a convenient way for sharing common things that multiple validation sets will make use of. An extended validation set will not be validated by its own. It will only be validated as part of the sets that extend it.<br />
<br />
A validation set can have two types of children: [[#Contribution|Contribution]] and [[#Validation Repository|Validation Repository]].<br />
<br />
Versions prior to 0.2.0 did not have the validation set node. Older b3aggr files will therefore be converted and given one validation set called ''main''<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the validation set. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the validation set to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
<br />
|Extends<br />
| '''<[[#Validation Set|Validation Set]]>'''*<br />
| -<br />
| Zero or more references to [[#Validation Set|Validation Set]]s defined in the given Aggregator model that this validation set extends.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this validation set.<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregation model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260650CBI/aggregator/manual2011-07-11T12:41:53Z<p>Thomas.tada.se: /* Global actions */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Aggregation''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Validate Aggregation''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Aggregation''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Aggregation''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260649CBI/aggregator/manual2011-07-11T12:40:52Z<p>Thomas.tada.se: /* Creating a Maven-conformant p2 repo */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260648CBI/aggregator/manual2011-07-11T12:40:08Z<p>Thomas.tada.se: /* Providing a repo indirection */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_indigo_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260646CBI/aggregator/manual2011-07-11T12:36:48Z<p>Thomas.tada.se: /* Mirroring contributions */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo". This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=File:B3_aggregator_sample_1.png&diff=260644File:B3 aggregator sample 1.png2011-07-11T12:34:07Z<p>Thomas.tada.se: uploaded a new version of "Image:B3 aggregator sample 1.png"</p>
<hr />
<div></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260643CBI/aggregator/manual2011-07-11T12:33:14Z<p>Thomas.tada.se: /* Mirroring contributions */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
=== Mirroring contributions ===<br />
<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_indigo.b3aggr buckminster_indigo.b3aggr]). <br />
<br />
The background is that Buckminster provides support for Subclipse. In addition to all components hosted at Eclipse, a complete installation will also require Subclipse components from Tigris.org (http://subclipse.tigris.org/update_1.6.x). We want to create a repo that combines these components and makes them accessible from one location. We want to make several platform configurations available. <br />
<br />
[[Image:B3 aggregator sample 1.png|thumb|center|800x750px|b3 Aggregator - Sample 1]] <br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified: <br />
<br />
#The top node '''''Aggregation''''' groups all model definitions regarding '''''ValidationSet'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that: <br />
#*the top node has been provided with a mandatory ''Label'' with the value "Indigo + Buckminster for Subclipse"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc) <br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below) <br />
#We have one ValidationSet labeled "main". A ValidationSet constitutes everything that will be validated as one unit by the p2 planner.<br />
#The ''main'' ValidationSet contains three different contributions.<br />
#The first '''''Contribution''''' to the aggregation is labeled "Indigo".<br />
This contribution includes binary configuration-specific artifacts which are only available for linux. If a simple contribution would be defined the aggregation would fail for all non-linux configurations, and hence the aggregation would fail as a whole. <br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions <br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for linux-based configurations<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included <br />
#*the result of this definition is that all categories, products, and features from Indigo p2 repo will be included in the aggregated repo.<br />
#The second '''''Contribution''''' is labeled "Subclipse" and deals with the inclusion of bundles provided from the Subclipse project. #*this contribution represents the simplest example of a contribution <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''. <br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subclipse support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subversive from the aggregated repo to make it easier for the user. <br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded <br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos <br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Aggregation'''''. This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''. <br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=File:B3_aggregator_sample_1.png&diff=260610File:B3 aggregator sample 1.png2011-07-11T09:37:29Z<p>Thomas.tada.se: uploaded a new version of "Image:B3 aggregator sample 1.png"</p>
<hr />
<div></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=File:B3_Install_Aggregator.png&diff=260602File:B3 Install Aggregator.png2011-07-11T09:22:45Z<p>Thomas.tada.se: uploaded a new version of "Image:B3 Install Aggregator.png"</p>
<hr />
<div></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=File:B3_Install_New_Software.png&diff=260601File:B3 Install New Software.png2011-07-11T09:22:18Z<p>Thomas.tada.se: uploaded a new version of "Image:B3 Install New Software.png"</p>
<hr />
<div></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=260600CBI/aggregator/manual2011-07-11T09:10:49Z<p>Thomas.tada.se: /* Installation */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.7 SDK from http://download.eclipse.org/eclipse/downloads<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.7/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.7</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.b3.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.6".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=246300CBI/aggregator/manual2011-04-12T12:33:17Z<p>Thomas.tada.se: /* Headless installation */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/headless-3.6</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.6".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=246299CBI/aggregator/manual2011-04-12T12:32:51Z<p>Thomas.tada.se: /* Headless installation */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current stable version is: '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6</nowiki>''''', and our latest build can be found at '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>'''''<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.6".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=246298CBI/aggregator/manual2011-04-12T12:30:54Z<p>Thomas.tada.se: /* Eclipse SDK installation */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6/</nowiki>''''' in the ''Location'' field. Or alternatively, if you feel adventurous and want to use the latest build, '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>'''''.<br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current version is: '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>''''' (Check the [http://eclipse.org/modeling/emft/b3/download/ download page] for changes of the headless update site.)<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.6".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=246296CBI/aggregator/manual2011-04-12T12:25:06Z<p>Thomas.tada.se: </p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to from a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just an EMF model", it can be produced via EMF APIs, transformation tools, etc., and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions'' - specifications of what to include from different repositories, and ''Validation Repositories'' - repositories that are used when validating, but which are not included in the produced aggregation (i.e., they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of ''Contacts'' - individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built-in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location.<br />
* '''Selective mirroring''' — an aggregation can produce an aggregate consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation, the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation, which is useful in large multi-layer project structures where issues may be related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation, including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
== Eclipse SDK installation ==<br />
<br />
{|<br />
|-<br />
| colspan="2" | <br />
*Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help'' <br />
[[Image:B3 Install New Software.png|thumb|center|580x492px|Install New Software wizard]] <br />
*Click the ''Add...'' button and enter the URL '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.p2.repository/</nowiki>''''' in the ''Location'' field. <br />
*Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
[[Image:B3 Install Aggregator.png|thumb|center|580x492px|b3 Aggregator Editor selection]]<br />
*Accept the Eclipse Public License and click ''Finish'' <br />
*Restart the IDE once the installation is finished.<br />
|-<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current version is: '''''<nowiki>https://hudson.eclipse.org/hudson/job/emft-b3-build/lastSuccessfulBuild/artifact/b3.headless.p2.repository</nowiki>''''' (Check the [http://eclipse.org/modeling/emft/b3/download/ download page] for changes of the headless update site.)<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.6".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--stacktrace'''<br />
| ---<br />
| ''no value''<br />
| Display stack trace on uncaught error.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference.<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode) - see [[Eclipse_b3/aggregator/Migration_From_buckminster]]<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Equinox/p2/Query_Language_for_p2&diff=207834Equinox/p2/Query Language for p22010-06-21T08:10:48Z<p>Thomas.tada.se: Changed requiredCapabilities to requirements</p>
<hr />
<div>[[Category:Equinox_p2]] <br />
===Background===<br />
As p2 gets more widely used, we predict that repositories will grow larger and larger over time. The Galileo repository alone contains 3866 IU's (in one single service release). Starting with Helios, we plan to keep everything over time which will put the IU count close to 10.000. And that's just one year of Eclipse IP approved material. OSGi is gaining in popularity and p2 is getting increasingly popular. Some companies plan to map the central maven repository as p2. We can soon expect to see p2 repositories with over a 100.000 IU's in them.<br />
<br />
====The problem====<br />
p2 has a query mechanism today that makes it hard to create a repository implementation that is based on a database. It is also diffiult to create an efficient client/server solution. The reason for this is that most of the queries are written as implementations of the IQuery interface, either as ContextQuery derivates that need access to all rows in the database, or as MatchQuery derivates with a java method to which all rows need to be passed.<br />
<br />
There is no uniform way to translate these queries into an expression that can be understood by a database or by something running in another process.<br />
<br />
Some attempts have been made to rectify this and a repository implementation could potentially make some intelligent decisions based on information derived from looking at the query (what class it is, and in some cases, what properties that are exposed). While this is an improvement, the general problem remains; A large amount of the queries are still more or less completely black-boxed and it's enough with one single such query to force a full selection of everything from the database/remote server.<br />
<br />
The p2QL expression language discussed here is an attempt to rectify this problem. It can be used in two ways:<br />
* Coexisting with current IQuery / IQueryable<br />
*: Using the ExpressionContextQuery (implements IQuery), or the ExpressionQuery (implements IMatchQuery), it can coexist with todays IQuery/IQueryable. The queries are created based on an expression and an array of parameters. See section [[#IQuery examples]].<br />
* With more explicit separation of concerns<br />
*: It can also be used in an alternative solution where an expression returns an Iterator. This usage scenario is particularly useful when implementing remote repositories or implementations on top of a database. It is also a very efficient way of querying for things that you don't want to collect in memory.<br />
<br />
====Bugzilla====<br />
We discussed the use of a query language at length on the p2 meeting on November 9 and November 16. The issue is captured in bugzilla [https://bugs.eclipse.org/bugs/show_bug.cgi?id=294691 Create a QueryLanguage for p2].<br />
<br />
===Language Design===<br />
TODO: Add some text here to explain where the ideas stem from (xtend, xtext, Scala, Java) and other motivation behind the choice of syntax.<br />
====Boolean Queries====<br />
Queries can be written as simple predicates such as "id == $0". When executed, the predicate will be evaluated once for each row in the queried material. This form is used for the IMatchQuery implementation. The current value is available in a predefined variable named ''this''. Like in Java, ''this'' is normally implicit so that:<br />
<pre>id == $0</pre><br />
is actually the same as:<br />
<pre>this.id == $0</pre><br />
<br />
====Collection Queries====<br />
A query can also be declared as something that acts on "everything". This is particularly useful when things like search for latest or doing requirements traversal. These queries make use of a predefined variable named ''everything''. When accessed, that variable returns an iterator over all queried material. As with ''this'' in a predicate query, ''everything'' is something that you don't normally need to specify, i.e.:<br />
<pre>select(x | x.id == $0)</pre><br />
is the same as:<br />
<pre>everything.select(x | x.id == $0)</pre><br />
A boolean query can always be written as a collection query. These two queries are fully equivalent:<br />
<pre>Boolean: id == $0<br />
Collection: select(x | x.id == $0)</pre><br />
<br />
====Special operators====<br />
<ul><li><b>Matches operator '~='</b><br/><br />
<p>A large amount of queries involve versions, version ranges, and capability matching. So managing that is important. Another thing that is important is to be able to support filtering based on ID's. All of this is now also built in to the language through the matches operator '''~='''. It can be though of as the 'satisfies' method on an installable unit or provided capability, as 'is included in' when used with a version and version range, as a plain matches when used with a string and a pattern, or as 'instanceof' when comparing with a class.</p><br />
<p>The following things can be matched using the operator</p><br />
{| border="1" cellpadding="3"<br />
!LHS<br />
!RHS<br />
!Implemented as<br />
|-<br />
|IInstallableUnit<br />
|IRequirement<br />
|lhs.satisfies(rhs)<br />
|-<br />
|IInstallableUnit<br />
|Filter<br />
|rhs.matches(lhs.properties)<br />
|-<br />
|IInstallableUnit<br />
|IUpdateDescriptor<br />
|rhs.isUpdateOf(lhs)<br />
|-<br />
|Version<br />
|VersionRange<br />
|rhs.isIncluded(lhs)<br />
|-<br />
|Map<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|Dictionary<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|String<br />
|SimplePattern<br />
|rhs.isMatch(lhs)<br />
|-<br />
|String<br />
|VersionRange<br />
|rhs.isIncluded(version(lhs))<br />
|-<br />
|<any><br />
|Class<br />
|rhs.isInstance(lhs)<br />
|-<br />
|Class<br />
|Class<br />
|rhs.isAssignableFrom(lhs)<br />
|}</li><br />
<li><b>And operator '&&'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''true'' if all its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''intersect''' operator and the result consists those elements that could be found in all collections.<br />
</li><br />
<li><b>Or operator '||'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''false'' if none of its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''union''' operator and the result is the unique sum of all elements from all collections.<br />
</li><br />
</ul><br />
<br />
====Functions====<br />
A small number of functions are available to assist with some common problems. Example:<br />
An IRequiredCapability.getFilter() returns a String. I only want the capabilities that has a filter that match certain properties. Consequently, I need an instance of an OSGi Filter. Not a string. I can do this using the ''filter'' function. Like this:<br />
<pre>requirements.exists(rc | rc.filter == null || $1 ~= filter(rc.filter))</pre><br />
The currently available constructors are:<br />
{| border="1" cellpadding="3"<br />
!name<br />
!arguments<br />
!creates<br />
|-<br />
|filter<br />
|string<br />
|an instance of org.osgi.framework.Filter<br />
|-<br />
|version<br />
|string<br />
|a p2 Version<br />
|-<br />
|range<br />
|string<br />
|a p2 VersionRange<br />
|-<br />
|class<br />
|string<br />
|a Class<br />
|-<br />
|boolean<br />
|string<br />
|a boolean<br />
|-<br />
|set<br />
|comma separated list of expressions<br />
|an instance of java.util.HashSet<br />
|-<br />
|iquery<br />
|IQuery [, variable [, collector]]<br />
|The result of evaluating the query<br />
|}<br />
<p>A note about the <b>iquery</b> constructor.<br/><br />
The constructor operates in one of two modes depending on its IQuery argument.</p><br />
<p>If the argument implements IMatchQuery, the the constructor will return the boolean result of invoking the isMatch(value) method on that query. The value is picked from the variable and the default variable is 'item'. It is considered an error to pass a third argument in combination with an IMatchQuery.</p><br />
<p>When the query does not implement the IMatchQuery interface, it is considered a context query and its method perform(iterator, collector) will be called. The iterator is picked from the variable and the default variable is 'everything'. The collector can be passed in as the third argument. If no third argument is present, a new collector will be created.</p><br />
<br />
====Collection functions====<br />
A number of functions was added to enable various manipulations of collections. A common way of writing collection functions is:<br />
<pre>elements.someFunction(element | <expression>)</pre><br />
Here are the functions that are available in the p2QL language:<br />
<ul><br />
<li>'''Select'''<br/><br />
The '''select''' function will evaluate its expression, once for each element, and return a new collection containing all elements for which the evaluation result was ''true''. Example:<br />
<pre>select(x | x.id == 'org.eclipse.equinox.p2.ql')</pre><br />
</li><br />
<li>'''Collect'''<br/><br />
The '''collect''' collects the result of evaluating each element in a new collection. Example:<br />
<pre>collect(x | x.id)</pre><br />
returns a collection consisting of the value of the id property of each element in the original collection.<br />
</li><br />
<li>'''Exists'''<br/><br />
The '''exists''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields ''true''. If no such element is found, the whole evaluation yields ''false''. Example:<br />
<pre>providedCapabilities.exists(p | p.namespace == 'osgi.bundle')</pre></li><br />
<li>'''All'''<br/><br />
The '''all''' function will evaluate an expression for each element until an evaluation yields ''false''. If that happens, the whole evaluation yields ''false''. If no such element is found, the whole evaluation yields ''true''. Example:<br />
<pre>$0.all(rc | this ~= rc)</pre><br />
Assuming $0 is a list of required capabilities, this query asks for items that fulfill all requirements.</li><br />
<li>'''First'''<br/><br />
The '''first''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields that element. If no such element is found, the whole evaluation yields ''null''. Example:<br />
<pre>providedCapabilities.first(p | p.namespace == 'java.package')</pre><br />
Returns the first provided capability with namespace 'java.package'.</li><br />
<li>'''Flatten'''<br/><br />
Intended to be applied on collections of collections. Yields a single collection with all elements from the source collections, in the order they are evaluated.Example:<br />
<pre>collect(x | x.requirements).flatten()</pre><br />
Yields a collection with all required capabilities from all iu's that the collect was applied on.</li><br />
<li>'''Latest'''<br/><br />
Some queries must make sure that the result only contains the latest version of each found IU. The special function ''latest'' to makes that possible. The function will require that the collection elements implement the IVersionedId interface. Here is an example of how to use it:<br />
<pre>select(x | x.id == $0).latest()</pre><br />
As a convenience, it is also possible to write:<br />
<pre>latest(x | x.id == $0)</pre></li><br />
<li>'''Limit'''<br/><br />
It is sometimes desirable to limit the number of rows returned by a query. The function 'limit' can be used for that:<br />
<pre>select(...).limit(100)</pre></li><br />
<li>'''Unique'''<br/><br />
This function will ensure that the resulting collection contains unique elements. The function can operate in two modes, with or without a cache argument. I.e.<br />
<pre>x.unique()</pre><br />
or<br />
<pre>x.unique(cache)</pre><br />
The latter expects the argument to be a ''java.util.Set'' that it can use to enforce the uniqueness of the element. This enables the result to be unique in a larger scope then the collection itself.</li><br />
<li>'''Traverse'''<br/><br />
A common scenario in p2 is that you want to start with a set of roots and then find all items that fulfill the root requirements. Those items in turn introduce new requirements so you want to find them too. The process continues until no more requirements can be satisfied. This type of query can be performed using the '''traverse''' function. The function will evaluate an expression, once for each element, collect elements for which the evaluation returned ''true'', then then re-evaluate using the collected result as source of elements. No element is evaluated twice. This continues until no more elements are found:<br />
<pre>$0.traverse(parent | parent.requirements.collect(rc | select(iu | ~= rc)).flatten())</pre><br />
This is of course a fairly naive slicing mechanism. See [[#Currying]] below for more advanced examples.</li><br />
</ul><br />
<br />
====Query parameters====<br />
The query must be parameterised so that expression parsing can be done once and then executed multiple times. A parameter is expressed as $''n'' where ''n'' is either the parameter index, originating from 0.<br />
<br />
===Basic IQuery examples===<br />
Here are some examples of how to use the expressions with IQuery:<br />
Query for all IU's that has an id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("id == $0", id);</pre><br />
Query for the latest IU of some specific id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0)", id);</pre><br />
Query an artifact repository for all keys with a specific classifier:<br />
<pre>IQuery&lt;IArtifactKey&gt; query = QueryUtil.createMatchQuery(IArtifactKey.class, "classifier == $0", classifier);</pre><br />
Query for the latest IU that matches a specific version range. Since the second parameter is a VersionRange, the ~= operator is interpreted as ''isIncluded'':<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0 && x.version ~= $1)", id, range);</pre><br />
Query for an IU that has a specific property set:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("properties[$0] == $1", key, value);</pre><br />
The same query, but this time for multiple possible values:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$1.exists(v | properties[$0] == v)", key, new Object[] { v1, v2, v3 });</pre><br />
Query for all categories found in the repository:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQueryx("properties[$0] == true", IInstallableUnit.PROP_TYPE_CATEGORY);</pre><br />
Query for all IU's that fulfil at least one of the requirements from another IU. Since the first parameter is a list of IRequirements, the ~= applied to each each IU using ''satisfies''.<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rc | this ~= rc)", iu.getRequirements());</pre><br />
Query for the latest version of all patches:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x ~= $0)", IInstallableUnitPatch.class);</pre><br />
Query for all IU's affected by a patch:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rcs | rcs.all(rc | this ~= rc))", patch.getApplicabilityScope());</pre><br />
<br />
===Localization===<br />
An installable unit may have locale specific properties. Such properties may be stored in the IU itself or in fragments that provide localization for the IU in the 'org.eclipse.equinox.p2.localization' namespace.<br />
p2QL will interpret the member ''translations'' applied on an IU as a map of translated properties, hiding all the details. As an example, this query:<br />
<br />
<pre>translations['license'] ~= /*kommersiellt bruk*/</pre><br />
<br />
when used with a default Locale("sv_SE") would yield all IU's that has a license, translated into Swedish, that contains the exact phrase 'kommersiellt bruk'.<br />
<br />
A special class named '''KeyWithLocale''' was added to allow queries for entries that does not match the current locale. So the above query can be written as:<br />
<br />
<pre>QueryUtil.createMatchQuery("this.translations[$0] ~= /*kommersiellt bruk*/", new KeyWithLocale("license", new Locale("sv", "SE")))</pre><br />
<br />
===Currying===<br />
Want some add some spice? Probably...<br />
<br />
Consider the traversal function and the example we had above:<br />
<pre>$0.traverse(parent | parent.requirements.collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
<p>Now let us assume that we want to perform this traversal and filter the requirements based on platform. Our first attempt could look something like this.<br />
<pre>$0.traverse(parent | parent.requirements.select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
This would however be very inefficient since many requirements exists in several IU's. During the traversal there is no point traversing a requirement more then once so it would be good to "remember" the perused requirements.<br />
</p><p>The p2QL syntax permits something generally referred to as ''currying''. Currying means that you can provide more parameters then the single one that represents the current element to the expression of a collection function. So far, we've seen examples using the syntax:<br />
<pre>select(x | <do something with x>)</pre><br />
This is actually a short form for the longer:<br />
<pre>select(_, {x | <do something with x>})</pre><br />
The select provides one parameter to each iteration. It's value is always provided reachable using the special operator '_'. In this case, the variable x maps to the parameter _ since they have the same positional offset. I can add more parameters by declaring:<br />
<pre>select(a, b, _, {x, y, z, <do something with x, y, z>})</pre><br />
Variable x will now have the value of a, y the value of b, and z the value of _.</p><br />
<p>So why is this important? Well, the initializers are just evaluated once for one call to select. The expression however, is evaluated once for each new value of _.<p><br />
<p>Let us now return to the traversal again. Because with this syntax, we can actually specify a global set that we can pass to the unique function so that no requirement is perused twice:<br />
<pre>$0.traverse(set(), _, { rqCache, parent | parent.requirements.unique(rqCache).select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten()})</pre><br />
</p><br />
<br />
===Performance comparison: p2QL traversal versus the p2 Slicer===<br />
<p>A test was performed using the query example above on the Galileo repository, using a specific version of the org.eclipse.sdk.feature.group IU as the root. The test produced a result with 411 IU's in < 12 ms average. I compared this with another test that used the p2 Slicer. The number of IU's produced was exactly the same. The slicer however, took > 22 ms (it also uses a capability cache internally). Not very scientific perhaps but repeated tests produce similar results.</p><br />
<p>I'm sure the Slicer can be optimized and achieve results that might be even slightly better then the traversal but I still think this proves a point. p2QL Performance is very good at this point.</p><br />
<br />
===Java API===<br />
The expression tree created by the parser must be well documented and easy to use so that queries can be created programmatically. Since all expressions are immutable and without context, they can be combined freely. Hence, code like this is fully possible:<br />
<pre><br />
// Create some expressions. Note the use of identifiers instead of<br />
// indexes for the parameters<br />
<br />
IExpressionFactory factory = ExpressionUtil.getFactory();<br />
IExpression item = factory.variable("this");<br />
IExpression cmp1 = factory.equals(factory.member(item, "id"), factory.parameter(0));<br />
IExpression cmp2 = factory.equals(<br />
factory.at(factory.member(item, "properties"), factory.parameter(1)),<br />
factory.parameter(2));<br />
<br />
IExpression everything = factory.variable("everything");<br />
IExpression lambda = factory.lambda(item, factory.and(new IExpression[] {cmp1, cmp2}));<br />
IExpression latest = factory.latest(factory.select(everything, lambda));<br />
<br />
// Create the query<br />
IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery&lt;IInstallableUnit&gt;(latest "test.bundle", "org.eclipse.equinox.p2.type.group", Boolean.TRUE);<br />
</pre><br />
<br />
===The [http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form BNF]===<br />
<pre><br />
condition<br />
: orExpression ( '?' orExpression ':' orExpression )?<br />
;<br />
<br />
orExpression : andExpression ( '||' andExpression )* ;<br />
<br />
andExpression : binaryExpression ( '&&' binaryExpression )* ;<br />
<br />
binaryExpression : notExpression ( op notExpression )?;<br />
<br />
op : '==' | '!=' | '>' | '>=' | '<' | '<=' | '~=' ;<br />
<br />
notExpression<br />
: '!' notExpression<br />
| collectionExpression<br />
;<br />
<br />
collectionExpression<br />
: memberExpression ( '.' collectionFunction )*<br />
;<br />
<br />
memberExpression : function ( ( '.' ID ) | ( '[' memberExpression ']' ) )* ;<br />
<br />
function<br />
: ( filter | version | range | class) '(' condition ')'<br />
| set '(' ( condition ( ',' condition )* )? ')'<br />
| unaryExpression<br />
;<br />
<br />
collectionFunction<br />
: ( select | reject | collect | exists | all | traverse | first ) '(' lambdaDefinition ')'<br />
| limit '(' memberExpression ')'<br />
| unique '(' memberExpression? ')'<br />
| latest '(' lambdaDefinition? ')'<br />
| flatten '(' lambdaDefinition? ')'<br />
;<br />
<br />
lambdaDefinition<br />
: initializer ( ',' initializer )* ( ',' '{' lambda '}' )?<br />
| '{' lambda '}'<br />
| lambda<br />
;<br />
<br />
initializer<br />
: '_'<br />
| condition<br />
;<br />
<br />
lambda<br />
: ( ID ( ',' ID )* )? '|' condition<br />
;<br />
<br />
unaryExpression<br />
: '(' condition ')'<br />
| '[' condition ( ',' condition )* ']' // #array construct<br />
| STRING<br />
| INT<br />
| parameter<br />
| 'null'<br />
| 'true'<br />
| 'false'<br />
| ID<br />
;<br />
<br />
parameter<br />
: '$' INT | ID<br />
;<br />
</pre></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Equinox/p2/Query_Language_for_p2&diff=207833Equinox/p2/Query Language for p22010-06-21T08:08:30Z<p>Thomas.tada.se: /* Functions */</p>
<hr />
<div>[[Category:Equinox_p2]] <br />
===Background===<br />
As p2 gets more widely used, we predict that repositories will grow larger and larger over time. The Galileo repository alone contains 3866 IU's (in one single service release). Starting with Helios, we plan to keep everything over time which will put the IU count close to 10.000. And that's just one year of Eclipse IP approved material. OSGi is gaining in popularity and p2 is getting increasingly popular. Some companies plan to map the central maven repository as p2. We can soon expect to see p2 repositories with over a 100.000 IU's in them.<br />
<br />
====The problem====<br />
p2 has a query mechanism today that makes it hard to create a repository implementation that is based on a database. It is also diffiult to create an efficient client/server solution. The reason for this is that most of the queries are written as implementations of the IQuery interface, either as ContextQuery derivates that need access to all rows in the database, or as MatchQuery derivates with a java method to which all rows need to be passed.<br />
<br />
There is no uniform way to translate these queries into an expression that can be understood by a database or by something running in another process.<br />
<br />
Some attempts have been made to rectify this and a repository implementation could potentially make some intelligent decisions based on information derived from looking at the query (what class it is, and in some cases, what properties that are exposed). While this is an improvement, the general problem remains; A large amount of the queries are still more or less completely black-boxed and it's enough with one single such query to force a full selection of everything from the database/remote server.<br />
<br />
The p2QL expression language discussed here is an attempt to rectify this problem. It can be used in two ways:<br />
* Coexisting with current IQuery / IQueryable<br />
*: Using the ExpressionContextQuery (implements IQuery), or the ExpressionQuery (implements IMatchQuery), it can coexist with todays IQuery/IQueryable. The queries are created based on an expression and an array of parameters. See section [[#IQuery examples]].<br />
* With more explicit separation of concerns<br />
*: It can also be used in an alternative solution where an expression returns an Iterator. This usage scenario is particularly useful when implementing remote repositories or implementations on top of a database. It is also a very efficient way of querying for things that you don't want to collect in memory.<br />
<br />
====Bugzilla====<br />
We discussed the use of a query language at length on the p2 meeting on November 9 and November 16. The issue is captured in bugzilla [https://bugs.eclipse.org/bugs/show_bug.cgi?id=294691 Create a QueryLanguage for p2].<br />
<br />
===Language Design===<br />
TODO: Add some text here to explain where the ideas stem from (xtend, xtext, Scala, Java) and other motivation behind the choice of syntax.<br />
====Boolean Queries====<br />
Queries can be written as simple predicates such as "id == $0". When executed, the predicate will be evaluated once for each row in the queried material. This form is used for the IMatchQuery implementation. The current value is available in a predefined variable named ''this''. Like in Java, ''this'' is normally implicit so that:<br />
<pre>id == $0</pre><br />
is actually the same as:<br />
<pre>this.id == $0</pre><br />
<br />
====Collection Queries====<br />
A query can also be declared as something that acts on "everything". This is particularly useful when things like search for latest or doing requirements traversal. These queries make use of a predefined variable named ''everything''. When accessed, that variable returns an iterator over all queried material. As with ''this'' in a predicate query, ''everything'' is something that you don't normally need to specify, i.e.:<br />
<pre>select(x | x.id == $0)</pre><br />
is the same as:<br />
<pre>everything.select(x | x.id == $0)</pre><br />
A boolean query can always be written as a collection query. These two queries are fully equivalent:<br />
<pre>Boolean: id == $0<br />
Collection: select(x | x.id == $0)</pre><br />
<br />
====Special operators====<br />
<ul><li><b>Matches operator '~='</b><br/><br />
<p>A large amount of queries involve versions, version ranges, and capability matching. So managing that is important. Another thing that is important is to be able to support filtering based on ID's. All of this is now also built in to the language through the matches operator '''~='''. It can be though of as the 'satisfies' method on an installable unit or provided capability, as 'is included in' when used with a version and version range, as a plain matches when used with a string and a pattern, or as 'instanceof' when comparing with a class.</p><br />
<p>The following things can be matched using the operator</p><br />
{| border="1" cellpadding="3"<br />
!LHS<br />
!RHS<br />
!Implemented as<br />
|-<br />
|IInstallableUnit<br />
|IRequirement<br />
|lhs.satisfies(rhs)<br />
|-<br />
|IInstallableUnit<br />
|Filter<br />
|rhs.matches(lhs.properties)<br />
|-<br />
|IInstallableUnit<br />
|IUpdateDescriptor<br />
|rhs.isUpdateOf(lhs)<br />
|-<br />
|Version<br />
|VersionRange<br />
|rhs.isIncluded(lhs)<br />
|-<br />
|Map<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|Dictionary<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|String<br />
|SimplePattern<br />
|rhs.isMatch(lhs)<br />
|-<br />
|String<br />
|VersionRange<br />
|rhs.isIncluded(version(lhs))<br />
|-<br />
|<any><br />
|Class<br />
|rhs.isInstance(lhs)<br />
|-<br />
|Class<br />
|Class<br />
|rhs.isAssignableFrom(lhs)<br />
|}</li><br />
<li><b>And operator '&&'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''true'' if all its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''intersect''' operator and the result consists those elements that could be found in all collections.<br />
</li><br />
<li><b>Or operator '||'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''false'' if none of its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''union''' operator and the result is the unique sum of all elements from all collections.<br />
</li><br />
</ul><br />
<br />
====Functions====<br />
A small number of functions are available to assist with some common problems. Example:<br />
An IRequiredCapability.getFilter() returns a String. I only want the capabilities that has a filter that match certain properties. Consequently, I need an instance of an OSGi Filter. Not a string. I can do this using the ''filter'' function. Like this:<br />
<pre>requirements.exists(rc | rc.filter == null || $1 ~= filter(rc.filter))</pre><br />
The currently available constructors are:<br />
{| border="1" cellpadding="3"<br />
!name<br />
!arguments<br />
!creates<br />
|-<br />
|filter<br />
|string<br />
|an instance of org.osgi.framework.Filter<br />
|-<br />
|version<br />
|string<br />
|a p2 Version<br />
|-<br />
|range<br />
|string<br />
|a p2 VersionRange<br />
|-<br />
|class<br />
|string<br />
|a Class<br />
|-<br />
|boolean<br />
|string<br />
|a boolean<br />
|-<br />
|set<br />
|comma separated list of expressions<br />
|an instance of java.util.HashSet<br />
|-<br />
|iquery<br />
|IQuery [, variable [, collector]]<br />
|The result of evaluating the query<br />
|}<br />
<p>A note about the <b>iquery</b> constructor.<br/><br />
The constructor operates in one of two modes depending on its IQuery argument.</p><br />
<p>If the argument implements IMatchQuery, the the constructor will return the boolean result of invoking the isMatch(value) method on that query. The value is picked from the variable and the default variable is 'item'. It is considered an error to pass a third argument in combination with an IMatchQuery.</p><br />
<p>When the query does not implement the IMatchQuery interface, it is considered a context query and its method perform(iterator, collector) will be called. The iterator is picked from the variable and the default variable is 'everything'. The collector can be passed in as the third argument. If no third argument is present, a new collector will be created.</p><br />
<br />
====Collection functions====<br />
A number of functions was added to enable various manipulations of collections. A common way of writing collection functions is:<br />
<pre>elements.someFunction(element | <expression>)</pre><br />
Here are the functions that are available in the p2QL language:<br />
<ul><br />
<li>'''Select'''<br/><br />
The '''select''' function will evaluate its expression, once for each element, and return a new collection containing all elements for which the evaluation result was ''true''. Example:<br />
<pre>select(x | x.id == 'org.eclipse.equinox.p2.ql')</pre><br />
</li><br />
<li>'''Collect'''<br/><br />
The '''collect''' collects the result of evaluating each element in a new collection. Example:<br />
<pre>collect(x | x.id)</pre><br />
returns a collection consisting of the value of the id property of each element in the original collection.<br />
</li><br />
<li>'''Exists'''<br/><br />
The '''exists''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields ''true''. If no such element is found, the whole evaluation yields ''false''. Example:<br />
<pre>providedCapabilities.exists(p | p.namespace == 'osgi.bundle')</pre></li><br />
<li>'''All'''<br/><br />
The '''all''' function will evaluate an expression for each element until an evaluation yields ''false''. If that happens, the whole evaluation yields ''false''. If no such element is found, the whole evaluation yields ''true''. Example:<br />
<pre>$0.all(rc | this ~= rc)</pre><br />
Assuming $0 is a list of required capabilities, this query asks for items that fulfill all requirements.</li><br />
<li>'''First'''<br/><br />
The '''first''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields that element. If no such element is found, the whole evaluation yields ''null''. Example:<br />
<pre>providedCapabilities.first(p | p.namespace == 'java.package')</pre><br />
Returns the first provided capability with namespace 'java.package'.</li><br />
<li>'''Flatten'''<br/><br />
Intended to be applied on collections of collections. Yields a single collection with all elements from the source collections, in the order they are evaluated.Example:<br />
<pre>collect(x | x.requiredCapabilities).flatten()</pre><br />
Yields a collection with all required capabilities from all iu's that the collect was applied on.</li><br />
<li>'''Latest'''<br/><br />
Some queries must make sure that the result only contains the latest version of each found IU. The special function ''latest'' to makes that possible. The function will require that the collection elements implement the IVersionedId interface. Here is an example of how to use it:<br />
<pre>select(x | x.id == $0).latest()</pre><br />
As a convenience, it is also possible to write:<br />
<pre>latest(x | x.id == $0)</pre></li><br />
<li>'''Limit'''<br/><br />
It is sometimes desirable to limit the number of rows returned by a query. The function 'limit' can be used for that:<br />
<pre>select(...).limit(100)</pre></li><br />
<li>'''Unique'''<br/><br />
This function will ensure that the resulting collection contains unique elements. The function can operate in two modes, with or without a cache argument. I.e.<br />
<pre>x.unique()</pre><br />
or<br />
<pre>x.unique(cache)</pre><br />
The latter expects the argument to be a ''java.util.Set'' that it can use to enforce the uniqueness of the element. This enables the result to be unique in a larger scope then the collection itself.</li><br />
<li>'''Traverse'''<br/><br />
A common scenario in p2 is that you want to start with a set of roots and then find all items that fulfill the root requirements. Those items in turn introduce new requirements so you want to find them too. The process continues until no more requirements can be satisfied. This type of query can be performed using the '''traverse''' function. The function will evaluate an expression, once for each element, collect elements for which the evaluation returned ''true'', then then re-evaluate using the collected result as source of elements. No element is evaluated twice. This continues until no more elements are found:<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.collect(rc | select(iu | ~= rc)).flatten())</pre><br />
This is of course a fairly naive slicing mechanism. See [[#Currying]] below for more advanced examples.</li><br />
</ul><br />
<br />
====Query parameters====<br />
The query must be parameterised so that expression parsing can be done once and then executed multiple times. A parameter is expressed as $''n'' where ''n'' is either the parameter index, originating from 0.<br />
<br />
===Basic IQuery examples===<br />
Here are some examples of how to use the expressions with IQuery:<br />
Query for all IU's that has an id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("id == $0", id);</pre><br />
Query for the latest IU of some specific id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0)", id);</pre><br />
Query an artifact repository for all keys with a specific classifier:<br />
<pre>IQuery&lt;IArtifactKey&gt; query = QueryUtil.createMatchQuery(IArtifactKey.class, "classifier == $0", classifier);</pre><br />
Query for the latest IU that matches a specific version range. Since the second parameter is a VersionRange, the ~= operator is interpreted as ''isIncluded'':<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0 && x.version ~= $1)", id, range);</pre><br />
Query for an IU that has a specific property set:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("properties[$0] == $1", key, value);</pre><br />
The same query, but this time for multiple possible values:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$1.exists(v | properties[$0] == v)", key, new Object[] { v1, v2, v3 });</pre><br />
Query for all categories found in the repository:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQueryx("properties[$0] == true", IInstallableUnit.PROP_TYPE_CATEGORY);</pre><br />
Query for all IU's that fulfil at least one of the requirements from another IU. Since the first parameter is a list of IRequirements, the ~= applied to each each IU using ''satisfies''.<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rc | this ~= rc)", iu.getRequiredCapabilities());</pre><br />
Query for the latest version of all patches:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x ~= $0)", IInstallableUnitPatch.class);</pre><br />
Query for all IU's affected by a patch:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rcs | rcs.all(rc | this ~= rc))", patch.getApplicabilityScope());</pre><br />
<br />
===Localization===<br />
An installable unit may have locale specific properties. Such properties may be stored in the IU itself or in fragments that provide localization for the IU in the 'org.eclipse.equinox.p2.localization' namespace.<br />
p2QL will interpret the member ''translations'' applied on an IU as a map of translated properties, hiding all the details. As an example, this query:<br />
<br />
<pre>translations['license'] ~= /*kommersiellt bruk*/</pre><br />
<br />
when used with a default Locale("sv_SE") would yield all IU's that has a license, translated into Swedish, that contains the exact phrase 'kommersiellt bruk'.<br />
<br />
A special class named '''KeyWithLocale''' was added to allow queries for entries that does not match the current locale. So the above query can be written as:<br />
<br />
<pre>QueryUtil.createMatchQuery("this.translations[$0] ~= /*kommersiellt bruk*/", new KeyWithLocale("license", new Locale("sv", "SE")))</pre><br />
<br />
===Currying===<br />
Want some add some spice? Probably...<br />
<br />
Consider the traversal function and the example we had above:<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
<p>Now let us assume that we want to perform this traversal and filter the requirements based on platform. Our first attempt could look something like this.<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
This would however be very inefficient since many requirements exists in several IU's. During the traversal there is no point traversing a requirement more then once so it would be good to "remember" the perused requirements.<br />
</p><p>The p2QL syntax permits something generally referred to as ''currying''. Currying means that you can provide more parameters then the single one that represents the current element to the expression of a collection function. So far, we've seen examples using the syntax:<br />
<pre>select(x | <do something with x>)</pre><br />
This is actually a short form for the longer:<br />
<pre>select(_, {x | <do something with x>})</pre><br />
The select provides one parameter to each iteration. It's value is always provided reachable using the special operator '_'. In this case, the variable x maps to the parameter _ since they have the same positional offset. I can add more parameters by declaring:<br />
<pre>select(a, b, _, {x, y, z, <do something with x, y, z>})</pre><br />
Variable x will now have the value of a, y the value of b, and z the value of _.</p><br />
<p>So why is this important? Well, the initializers are just evaluated once for one call to select. The expression however, is evaluated once for each new value of _.<p><br />
<p>Let us now return to the traversal again. Because with this syntax, we can actually specify a global set that we can pass to the unique function so that no requirement is perused twice:<br />
<pre>$0.traverse(set(), _, { rqCache, parent | parent.requiredCapabilities.unique(rqCache).select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten()})</pre><br />
</p><br />
<br />
===Performance comparison: p2QL traversal versus the p2 Slicer===<br />
<p>A test was performed using the query example above on the Galileo repository, using a specific version of the org.eclipse.sdk.feature.group IU as the root. The test produced a result with 411 IU's in < 12 ms average. I compared this with another test that used the p2 Slicer. The number of IU's produced was exactly the same. The slicer however, took > 22 ms (it also uses a capability cache internally). Not very scientific perhaps but repeated tests produce similar results.</p><br />
<p>I'm sure the Slicer can be optimized and achieve results that might be even slightly better then the traversal but I still think this proves a point. p2QL Performance is very good at this point.</p><br />
<br />
===Java API===<br />
The expression tree created by the parser must be well documented and easy to use so that queries can be created programmatically. Since all expressions are immutable and without context, they can be combined freely. Hence, code like this is fully possible:<br />
<pre><br />
// Create some expressions. Note the use of identifiers instead of<br />
// indexes for the parameters<br />
<br />
IExpressionFactory factory = ExpressionUtil.getFactory();<br />
IExpression item = factory.variable("this");<br />
IExpression cmp1 = factory.equals(factory.member(item, "id"), factory.parameter(0));<br />
IExpression cmp2 = factory.equals(<br />
factory.at(factory.member(item, "properties"), factory.parameter(1)),<br />
factory.parameter(2));<br />
<br />
IExpression everything = factory.variable("everything");<br />
IExpression lambda = factory.lambda(item, factory.and(new IExpression[] {cmp1, cmp2}));<br />
IExpression latest = factory.latest(factory.select(everything, lambda));<br />
<br />
// Create the query<br />
IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery&lt;IInstallableUnit&gt;(latest "test.bundle", "org.eclipse.equinox.p2.type.group", Boolean.TRUE);<br />
</pre><br />
<br />
===The [http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form BNF]===<br />
<pre><br />
condition<br />
: orExpression ( '?' orExpression ':' orExpression )?<br />
;<br />
<br />
orExpression : andExpression ( '||' andExpression )* ;<br />
<br />
andExpression : binaryExpression ( '&&' binaryExpression )* ;<br />
<br />
binaryExpression : notExpression ( op notExpression )?;<br />
<br />
op : '==' | '!=' | '>' | '>=' | '<' | '<=' | '~=' ;<br />
<br />
notExpression<br />
: '!' notExpression<br />
| collectionExpression<br />
;<br />
<br />
collectionExpression<br />
: memberExpression ( '.' collectionFunction )*<br />
;<br />
<br />
memberExpression : function ( ( '.' ID ) | ( '[' memberExpression ']' ) )* ;<br />
<br />
function<br />
: ( filter | version | range | class) '(' condition ')'<br />
| set '(' ( condition ( ',' condition )* )? ')'<br />
| unaryExpression<br />
;<br />
<br />
collectionFunction<br />
: ( select | reject | collect | exists | all | traverse | first ) '(' lambdaDefinition ')'<br />
| limit '(' memberExpression ')'<br />
| unique '(' memberExpression? ')'<br />
| latest '(' lambdaDefinition? ')'<br />
| flatten '(' lambdaDefinition? ')'<br />
;<br />
<br />
lambdaDefinition<br />
: initializer ( ',' initializer )* ( ',' '{' lambda '}' )?<br />
| '{' lambda '}'<br />
| lambda<br />
;<br />
<br />
initializer<br />
: '_'<br />
| condition<br />
;<br />
<br />
lambda<br />
: ( ID ( ',' ID )* )? '|' condition<br />
;<br />
<br />
unaryExpression<br />
: '(' condition ')'<br />
| '[' condition ( ',' condition )* ']' // #array construct<br />
| STRING<br />
| INT<br />
| parameter<br />
| 'null'<br />
| 'true'<br />
| 'false'<br />
| ID<br />
;<br />
<br />
parameter<br />
: '$' INT | ID<br />
;<br />
</pre></div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=CBI/aggregator/manual&diff=199839CBI/aggregator/manual2010-05-10T07:33:07Z<p>Thomas.tada.se: /* Command line options */</p>
<hr />
<div>=Introduction=<br />
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.<br />
<br />
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository. <br />
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:<br />
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install. <br />
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.<br />
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.<br />
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).<br />
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.<br />
<br />
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.<br />
<br />
Furthermore, it is worth noting that:<br />
* the Aggregator is based on EMF models<br />
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator<br />
<br />
=Functional Overview=<br />
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to form a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just and EMF model", it can be produced via EMF APIs, transformation tools, etc. and thus support advanced use cases).<br />
<br />
The model mainly consists of ''Contributions''; specifications of what to include from different repositories, and Validation Repositories; repositories that are used when validating, but that are not included in the produced aggregation (i.e. they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of Contacts; individuals/mailing-lists to inform when processing fails.<br />
<br />
Here are some of the important features supported by the b3 Aggregator:<br />
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.<br />
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built in rules. Custom rules are also supported.<br />
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location<br />
* '''Selective mirroring''' — an aggregation can produce an aggregation consisting of a mix of references to repositories and mirrored repositories.<br />
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.<br />
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.<br />
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories. <br />
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. <br />
* '''Blame Email''' — when issues are found during validation the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation which is useful in large multi layer project structures where issues may related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation including handling of mock emails when testing aggregation scripts.<br />
<br />
=Installation=<br />
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)<br />
<br />
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).<br />
<br />
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing. <br />
<br />
==Eclipse SDK installation==<br />
{|<br />
|- <br />
| colspan="2" |<br />
# Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help''<br />
# Click the ''Add...'' button and enter the URL <nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6</nowiki> in the ''Location'' field.<br />
# Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice. <br />
# Accept the Eclipse Public License and click ''Finish''<br />
# Restart the IDE once the installation is finished.<br />
<br />
|- <br />
| <br />
[[Image:b3_Install_New_Software.png|thumb|center|580x492px|'''''Install New Software wizard''''']]<br />
| <br />
[[Image:b3_Install_Aggregator.png|thumb|center|580x492px|'''''b3 Aggregator Editor''''' selection]]<br />
|}<br />
<br />
==Headless installation==<br />
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.<br />
<br />
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].<br />
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.<br />
#'''Install b3''' with the following command: <br />
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where<br />
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current version is: http://download.eclipse.org/modeling/emft/b3/headless-3.6 (Check the [http://eclipse.org/modeling/emft/b3/download/ download page] for changes of the headless update site.)<br />
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3<br />
#**'''''-p b3''''' is the name of the p2 profile<br />
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base<br />
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature<br />
<br />
<br />
<br />
=Getting started with standard examples=<br />
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator. <br />
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.<br />
<br />
==Aggregating a p2 repo==<br />
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:<br />
*provide a "one-stop shop" experience<br />
*conveniently pull in third-party components that are not hosted at Eclipse<br />
*provide this repo as an indirection mechanism if required<br />
<br />
===Mirroring contributions===<br />
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).<br />
<br />
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).<br />
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.<br />
<br />
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]<br />
<br />
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:<br />
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:<br />
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager <br />
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed <br />
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)<br />
#*any number of configurations can be defined<br />
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)<br />
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.5".<br />
#*this contribution represents the simplest example of a contribution<br />
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included<br />
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo<br />
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.<br />
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions<br />
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations <br />
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.<br />
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.<br />
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded<br />
#A list of all included repos is displayed at the bottom of the model editor view <br />
#*this list allows browsing the contents of all repos<br />
#*this part of the model is not editable<br />
<br />
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''. <br />
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.<br />
<br />
Check the next section for a slightly different approach.<br />
<br />
===Providing a repo indirection===<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.<br />
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.<br />
<br />
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]<br />
|}<br />
<br />
==Creating a Maven-conformant p2 repo==<br />
{|- width="100%"<br />
|- valign="top"<br />
|<br />
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.<br />
<br />
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.<br />
<br />
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.<br />
<br />
|<br />
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]<br />
|}<br />
<br />
=Headless support=<br />
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.<br />
<br />
==Running from the command line==<br />
Just type:<pre>b3 aggregate <options></pre><br />
<br />
For a detailed listing of the available options consult the next section.<br />
<br />
==Command line options==<br />
{| {{Greytable}} <br />
|-<br />
! Option<br />
! Value<br />
! Default<br />
! Description<br />
<br />
|- valign="top"<br />
| '''--buildModel'''<br />
| <path to build model><br />
| This value is required<br />
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).<br />
<br />
|- valign="top"<br />
| '''--action'''<br />
| VERIFY<br />
<br />
BUILD<br />
<br />
CLEAN<br />
<br />
CLEAN_BUILD<br />
<br />
| BUILD<br />
| Specifies the type of the execution.<br />
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location<br />
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location<br />
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD <br />
<br />
|- valign="top"<br />
| '''--buildId'''<br />
| <string><br />
| build-<timestamp in the format yyyyMMddHHmm><br />
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527<br />
<br />
|- valign="top"<br />
| '''--buildRoot'''<br />
| <path to directory><br />
| ''buildRoot'' declared in the aggregation model<br />
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.<br />
<br />
|- valign="top"<br />
| '''--production'''<br />
| N/A<br />
| N/A<br />
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.<br />
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''<br />
<br />
|- valign="top"<br />
| '''--emailFrom'''<br />
| <email><br />
| Address of build master<br />
| Becomes the sender of the emails sent from the aggregator.<br />
<br />
|- valign="top"<br />
| '''--emailFromName'''<br />
| <name><br />
| If --emailFrom is set then this option sets the real name of the email sender.<br />
<br />
|- valign="top"<br />
| '''--mockEmailTo'''<br />
| <email><br />
| ''no value''<br />
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.<br />
<br />
|- valign="top"<br />
| '''--mockEmailCC'''<br />
| <email><br />
| ''no value''<br />
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.<br />
<br />
|- valign="top"<br />
| '''--logURL'''<br />
| <url><br />
| N/A<br />
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.<br />
<br />
|- valign="top"<br />
| '''--logLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the console trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--eclipseLogLevel'''<br />
| DEBUG<br />
<br />
INFO<br />
<br />
WARNING<br />
<br />
ERROR<br />
| INFO<br />
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.<br />
<br />
|- valign="top"<br />
| '''--subjectPrefix'''<br />
| <string><br />
| ?<br />
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"<br />
<br />
|- valign="top"<br />
| '''--smtpHost'''<br />
| <host name><br />
| ''localhost''<br />
| The SMTP host to talk to when sending emails. Defaults to "localhost".<br />
<br />
|- valign="top"<br />
| '''--smtpPort'''<br />
| <port number><br />
| ''25''<br />
| The SMTP port number to use when talking to the SMTP host. Default is 25.<br />
<br />
|- valign="top"<br />
| '''--packedStrategy'''<br />
| COPY<br />
<br />
VERIFY<br />
<br />
UNPACK_AS_SIBLING<br />
<br />
UNPACK<br />
<br />
SKIP<br />
| ''value from the model''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.<br />
<br />
|- valign="top"<br />
| '''--trustedContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).<br />
<br />
''Note: this option is mutually exclusive with option --mavenResult (see below)''<br />
<br />
|- valign="top"<br />
| '''--validationContributions'''<br />
| <comma separated list><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
A comma separated list of contributions with repositories that will be used for aggregation validation only rather than mirrored or referenced into the final repository.<br />
<br />
|- valign="top"<br />
| '''--mavenResult'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Tells the aggregator to generate a hybrid repository that is compatible with p2 and maven2.<br />
''Note: this option is mutually exclusive with option --trustedContributions (see above)''<br />
<br />
|- valign="top"<br />
| '''--mirrorReferences'''<br />
| ---<br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Mirror meta-data repository references from the contributed repositories<br />
<br />
|- valign="top"<br />
| '''--referenceIncludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Include only those references that matches the given regular expression pattern.<br />
<br />
|- valign="top"<br />
| '''--referenceExcludePattern'''<br />
| <regexp><br />
| ''no value''<br />
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''<br />
Exclude all references that matches the given regular expression pattern. An exclusion takes precedence over an inclusion in case both patterns match a reference."<br />
|}<br />
<br />
==Hudson integration==<br />
Currently there is no support for Hudson.<br />
<br />
=Aggregator model components and specific actions=<br />
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.<br />
<br />
==Global actions==<br />
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:<br />
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')<br />
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created<br />
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')<br />
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''<br />
<br />
==Aggregator==<br />
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup). <br />
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Buildmaster<br />
| <[[#Contact|Contact]]>?<br />
| -<br />
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.<br />
<br />
|- valign="top"<br />
|Build Root<br />
| <urn><br />
| -<br />
| This is a required property specifying the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| <[[#Maven Mapping|Maven Mapping]]>*<br />
| -<br />
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.<br />
<br />
|- valign="top"<br />
|Maven Result<br />
| '''true'''<br />
'''false'''<br />
| false<br />
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven. <br />
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository. <br />
<br />
|- valign="top"<br />
|Packed Strategy<br />
| '''Copy'''<br />
'''Verify'''<br />
<br />
'''Unpack as Sibling'''<br />
<br />
'''Unpack'''<br />
<br />
'''Skip'''<br />
<br />
| Copy<br />
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:<br />
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action<br />
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result<br />
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling<br />
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form<br />
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source<br />
<br />
|- valign="top"<br />
|Sendmail<br />
| '''false'''<br />
'''true'''<br />
| false<br />
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).<br />
<br />
|- valign="top"<br />
|Type<br />
| '''S'''<br />
'''I'''<br />
<br />
'''N'''<br />
<br />
'''M'''<br />
<br />
'''C'''<br />
<br />
'''R'''<br />
| S<br />
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.<br />
*'''S''' - stable<br />
*'''I''' - integration<br />
*'''N''' - nightly<br />
*'''M''' - milestone<br />
*'''C''' - continuous<br />
*'''R''' - release<br />
<br />
|}<br />
<br />
===Configuration===<br />
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.<br />
<br />
A Configuration is a combination of the following properties:<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Architecture<br />
|'''X86'''<br />
<br />
'''PPC'''<br />
<br />
'''X86_64'''<br />
<br />
'''IA64'''<br />
<br />
'''IA64_32'''<br />
<br />
'''Sparc'''<br />
| -<br />
|Specifies the architecture for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the configuration for the aggregation process.<br />
<br />
|- valign="top"<br />
|Operating System<br />
|'''Win32'''<br />
<br />
'''Linux'''<br />
<br />
'''MacOSX'''<br />
<br />
'''AIX'''<br />
<br />
'''HPUX'''<br />
<br />
'''Solaris'''<br />
| -<br />
|Specifies the operating system for which this configuration should be verified.<br />
<br />
|- valign="top"<br />
|Window System<br />
|'''Win32'''<br />
<br />
'''GTK'''<br />
<br />
'''Carbon'''<br />
<br />
'''Cocoa'''<br />
<br />
'''Motif'''<br />
| -<br />
|Specifies the windowing system for which this configuration should be verified.<br />
<br />
|}<br />
<br />
===Contribution===<br />
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository. <br />
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Contacts<br />
| '''<[[#Contact|Contact]]>'''*<br />
| -<br />
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur. <br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.<br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Mandatory label for this contribution.<br />
<br />
|- valign="top"<br />
|Maven Mappings<br />
| '''<[[#Maven Mapping|Maven Mapping]]>'''*<br />
| -<br />
| See [[#Maven Mapping|Maven Mapping]] for details ...<br />
<br />
|}<br />
<br />
<br />
====Mapped Repository====<br />
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Category Prefix<br />
| <string><br />
| -<br />
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the Mapped Repository. For documentation purposes only.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.<br />
<br />
|- valign="top"<br />
|Mirror Artifacts<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
<br />
=====Product=====<br />
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| -<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <product IU's id><br />
| -<br />
| The id of the product to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>'''*'''<br />
| -<br />
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Category=====<br />
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <category IU id><br />
| -<br />
| The id of the category to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Label Override<br />
| <string><br />
| -<br />
| New category name used instead of the default name.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Bundle=====<br />
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <bundle IU id><br />
| -<br />
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
|<[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Feature=====<br />
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The features to include must be present in the mapped repository.<br />
<br />
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]]. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Categories<br />
| <[[#Custom Category|Custom Category]]>*<br />
| -<br />
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa. <br />
<br />
|- valign="top"<br />
|Description<br />
| <description><br />
| ''no value''<br />
| Optional description of the mapping.<br />
<br />
|- valign="top"<br />
|Enabled<br />
|'''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Name<br />
| <feature IU id><br />
| -<br />
| The id of the feature to be included in the aggregation. A drop-down of available names is provided. <br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Exclusion Rule=====<br />
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.<br />
<br />
|}<br />
<br />
=====Valid Configuration Rule=====<br />
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description for documentation purposes.<br />
<br />
|- valign="top"<br />
|Name<br />
| <IU id><br />
| -<br />
| The id of the IU to be included in the verification process. A drop-down of available names is provided.<br />
<br />
|- valign="top"<br />
|Valid Configurations<br />
| <[[#Configuration|Configuration]]>*<br />
| -<br />
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.<br />
<br />
|- valign="top"<br />
|Version Range<br />
| <version range><br />
| 0.0.0<br />
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.<br />
<br />
|}<br />
<br />
====Maven Mapping (Contribution)====<br />
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.<br />
<br />
===Contact===<br />
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Email<br />
| <email><br />
| -<br />
| The email to be used when notifying the contact.<br />
<br />
|- valign="top"<br />
|Name<br />
| <string><br />
| -<br />
| Full name of the contact. May be displayed as label when referenced in other parts of the model.<br />
<br />
|}<br />
<br />
===Custom Category===<br />
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]]. <br />
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus, adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa. <br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Description<br />
| <string><br />
| -<br />
| Description of the category as displayed when accessing the aggregated repository.<br />
<br />
|- valign="top"<br />
|Features<br />
| <[[#Feature|Feature]]>*<br />
| -<br />
| [[#Feature|Feature]]s included in this category by reference.<br />
<br />
|- valign="top"<br />
|Identifier<br />
| <string><br />
| -<br />
| Category id. Required Eclipse category property. <br />
<br />
|- valign="top"<br />
|Label<br />
| <string><br />
| -<br />
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.<br />
<br />
|}<br />
<br />
===Validation Repository===<br />
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.<br />
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Enabled<br />
| '''true'''<br />
'''false'''<br />
| true<br />
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.<br />
<br />
|- valign="top"<br />
|Location<br />
| <URL><br />
| <br />
| Specifies the location of the repository.<br />
<br />
|- valign="top"<br />
|Nature<br />
| <nature><br />
| p2<br />
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.<br />
<br />
|}<br />
<br />
===Maven Mapping===<br />
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory, <br />
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level. <br />
<br />
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:<br />
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node<br />
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node<br />
#finally the default Maven Mapping is applied<br />
<br />
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model. <br />
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids have an effect on the resulting Maven repo structure. The default pattern is:<br />
<br />
<pre><br />
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$<br />
</pre><br />
<br />
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:<br />
<br />
<pre><br />
org<br />
|<br />
eclipse<br />
|<br />
b3 <br />
|<br />
org.eclipse.b3.aggregator<br />
|<br />
<folder name after version><br />
|<br />
org.eclipse.b3.aggregator-<version>.jar<br />
...<br />
</pre><br />
<br />
<br />
<br />
{| {{Greytable}}<br />
|-<br />
!Property<br />
!Value(s)<br />
!Default Value<br />
!Comment<br />
<br />
|- valign="top"<br />
|Name Pattern<br />
| regex<br />
| -<br />
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.<br />
<br />
|- valign="top"<br />
|Group Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|- valign="top"<br />
|Artifact Id<br />
| <string> (may contain pattern group references (i.e. $1, ...))<br />
| -<br />
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).<br />
<br />
|}<br />
<br />
=What else should be documented=<br />
<br />
# version editor (version formats...)<br />
# how enable/disable on the context menu works<br />
# drag&drop support<br />
# 'available versions' tree<br />
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)<br />
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode)<br />
<br />
==Questions==<br />
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?<br />
''When launched from IDE, only --buildModel and --action options are set (all other options have default values). Perhaps it's worth adding a launching dialog which would enable setting all options that are available in headless mode.''<br />
* How do you know what the "log output url" is? How can it be set?<br />
''You can, for example, redirect a copy of the console output to a publicly available resource (a text file served by a http server). The public URL should be passed in the --logURL option so that a link to it would appear in the informational email.''<br />
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??<br />
''The Hudson Support article was copied from the old buckminster documentation. It can be removed.''<br />
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there? <br />
''Only listed configurations are supported (they are a part of the model). Adding an option means changing the model and creation of a new aggregator build.''<br />
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??<br />
''Yes, it is possible to load p2 and maven2 repositories by default (if someone adds a custom loader then he can load any type of repository). The output is always p2 compatible, optionally combined with maven2 (with no extensibility - at least now). However, if a maven2 repo is loaded and the result is also maven2 compatible, it is not identical to the original (not all attributes loaded from maven are stored in the p2 model).''<br />
<br />
[[Category:Eclipse b3]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster_Retention_Policy&diff=197160Buckminster Retention Policy2010-04-20T11:56:41Z<p>Thomas.tada.se: New page: We maintain three different update sites. Maintenance of the current release. Nightly builds of the coming release. Site for consumption of the release train. In addition to those sites,...</p>
<hr />
<div>We maintain three different update sites.<br />
<br />
Maintenance of the current release.<br />
Nightly builds of the coming release.<br />
Site for consumption of the release train.<br />
<br />
In addition to those sites, we also maintain archives of all update sites that have been produced.</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster_3.6_New_and_Noteworthy&diff=197152Buckminster 3.6 New and Noteworthy2010-04-20T11:20:40Z<p>Thomas.tada.se: </p>
<hr />
<div>==The Helios release of Buckminster has the following new and noteworthy features.<br />
*Support for Git<br />
*Headless JUnit and Eclemma (code coverage) support<br />
*Comprehensive documentation available<br />
*Graphical dependency visualizer<br />
*Much improved target platform support<br />
*Provisioning and management of API baseline<br />
*New EMF based editors for MSPEC and RMAP<br />
*Reader type for Team Project Set (.psf) files<br />
*p2 site size reduction to 1/3 using improved pack200 support<br />
*Omni Version support<br />
*Qualifier generator using Build Identifier<br />
*LDAP style filters on RMAP providers, CQUERY advisors, and MSPEC nodes<br />
*Smart version range generation for feature ‘includes'<br />
*Support for category.xml files<br />
*Headless 'install JRE' support<br />
*Better defaults often renders the MSPEC unnecessary<br />
*Using new p2 API, p2 ‘pure’ reader, and separate p2 agent</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster_3.6_New_and_Noteworthy&diff=197149Buckminster 3.6 New and Noteworthy2010-04-20T11:09:25Z<p>Thomas.tada.se: New page: ==Support for Git== A new reader type was added that enables Buckminster to use git repositories. Buckminster will always use a local repository but is also capable of creating and updatin...</p>
<hr />
<div>==Support for Git==<br />
A new reader type was added that enables Buckminster to use git repositories. Buckminster will always use a local repository but is also capable of creating and updating it as a clone from a remote source.<br />
==Headless JUnit and Eclemma (code coverage) support==<br />
New headless commands are available to start JUit tests and Emma Code Coverage<br />
==Comprehensive documentation available==<br />
<br />
==Graphical dependency visualizer==<br />
==Much improved target platform support==<br />
==provisioning and management of API baseline==<br />
==New EMF based editors for MSPEC and RMAP==<br />
==Reader type for Team Project Set (.psf) files==<br />
==p2 site size reduction to 1/3 using improved pack200 support==<br />
==Omni Version support==<br />
==Qualifier generator using Build Identifier==<br />
==LDAP style filters on RMAP providers, CQUERY advisors, and MSPEC nodes==<br />
==Smart version range generation for feature ‘includes’==<br />
==Support for category.xml files==<br />
==Headless 'install JRE' support==<br />
==Better defaults often renders the MSPEC unnecessary==<br />
==Using new p2 API, p2 ‘pure’ reader, and separate p2 agent==</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Buckminster/Helios_Ramp-Down_Policy&diff=197144Buckminster/Helios Ramp-Down Policy2010-04-20T10:25:27Z<p>Thomas.tada.se: </p>
<hr />
<div>= Buckminster Ramp-down Policy for Helios =<br />
<br />
Release Candidates:<br />
<br />
* RC1 - Build Tue, May 18 - Release Wed, May 19<br />
* RC2 - Build Tue, May 25 - Release Wed, May 26<br />
* RC3 - Build Tue, June 1 - Release Wed, June 2<br />
* RC4 - Build Tue, June 8 - Release Wed, June 9<br />
* Final - Build Tue, June 15 - Release Wed June 16<br />
<br />
No new features will be added after M6 (Tue, March 16).<br />
API freeze on M7 (Wed, May 5)<br />
<br />
Starting Tue, June 1 (after RC3 build), all commits must be accompanied with a notification to the buckminster-dev list providing a link to the bug that was fixed.<br />
<br />
The idea is to ensure we're only fixing bugs during the RC cycle. And as we get closer, it's important that we all understand what's changing and can object if too much is changing too late to ensure we stabilize by the end of the release.<br />
<br />
Project lead will also continue to send countdown e-mails as we go to track open bugs.</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Hudson&diff=193810Hudson2010-03-23T22:49:09Z<p>Thomas.tada.se: /* Who are the Adminstrators */</p>
<hr />
<div>= General Information =<br />
<br />
The Eclipse Foundation runs a Hudson continuous integration server that eclipse hosted projects can take advantage of for their builds.&nbsp; The Hudson server allows for the execution of Continuous Integration Builds, Nightly Builds, Integration Builds, Release Builds, and Testing. The Hudson instance is maintained by the eclipse community. Job creation is limited to committers and admin rights are administered by the community as a whole. <br />
<br />
== Hudson for Committers ==<br />
<br />
*[[Common Build Infrastructure/Getting Started/Build In Hudson|Build in Hudson]] - Information on requesting jobs, running jobs, setting up builds. <br />
*[[Building an RCP application with hudson (Buckminster)|RCP apps with Buckminster on Hudson]]<br />
*[[Teneo/Teneo_Build_Setup|Building an Eclipse project (Teneo) with Buckminster and Hudson]]<br />
*[[Common Build Infrastructure/Getting Started#In_Hudson|Athena Common Builder on Hudson]] <br />
*[[Hudson/Maven|Maven on Hudson]] <br />
*[[Hudson/HowTo|How to....]]<br />
<br />
== Hudson for Administrators ==<br />
<br />
*[[Common Build Infrastructure/Managing Hudson|Manage Hudson]] <br />
*[[Hudson/Admin/UpgradeHudson|Upgrade Hudson]] <br />
*[[Hudson/Admin/Installed_Plugins|Installed Plugins]]<br />
<br />
=== Duties of Administrators ===<br />
<br />
#Hudson upgrades and restarts <br />
#New Hudson accounts <br />
#Add plugins <br />
#Set policy for Hudson usage <br />
#Watch changes to this wiki page <br />
#Monitor the Hudson Inbox.<br />
<br />
=== Who are the Adminstrators ===<br />
<br />
*Nick Boldt <br />
*Kim Moir <br />
*David Carver<br />
*Thomas Hallgren<br />
<br />
Please add other known administrators.<br />
<br />
== Hudson for Distributed Builds ==<br />
<br />
*Testing on Multiple Platforms <br />
*What is the Test-Slave Node? <br />
*How do I use the Test Slave Node to run Tests?<br />
<br />
[[Category:Releng]]<br />
[[Category:Hudson]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Helios/Participating_Projects&diff=192470Helios/Participating Projects2010-03-16T10:29:44Z<p>Thomas.tada.se: Changed Buckminster from +2 to +3 since we are dependent on other +3</p>
<hr />
<div>The projects that plan to participate in the Helios Simultaneous Release are listed below, along with their leaders, release engineer, and offset category. <br />
<br />
[Note: the data that's currently there was just a quick edit of last years data to provide a starting point. As projects fill in the table, be sure to discuss with your PMC and Project Chain and Project Peers. It is desired to consolidate as many "rows" as can meaningfully and legitimately be combined. In the end, each row should correspond to one project review (that is, one set of docuware for each row).] <br />
<br />
{| cellspacing="1" cellpadding="3" border="1" align="center" style="width: 723px; height: 1839px;"<br />
|-<br />
! Top Level Project <br />
! Project <br />
! Project Lead(s) <br />
! Release Engineer <br />
! width="75" | Offset <br />
! width="174" | Portal Shortname(s) <br />
! width="174" | Aggregator<br>Build File <br />
! width="100" | M4<br />
|-<br />
| valign="top" | Business Intelligence and Reporting Tools (BIRT) <br />
| valign="top" | <br />
Business Intelligence and Reporting Tools (BIRT) <br />
<br />
| valign="top" | Wenfeng Li <br />
| valign="top" | Xiaoying Gu <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | birt <br />
| width="174" valign="top" | birt.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | Data Tools Platform (DTP) <br />
| valign="top" | <br />
Data Tools Platform (DTP) <br />
<br />
| valign="top" | Brian Payton <br />
| valign="top" | Xiaoying Gu <br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | datatools <br />
| width="174" valign="top" | dtp.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" rowspan="3" | DSDP <br />
| valign="top" | <br />
DSDP TM <br />
<br />
| valign="top" | Martin Oberhuber <br />
| valign="top" | Martin Oberhuber <br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | dsdp.tm <br />
| width="174" valign="top" | dsdp-tm.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
DSDP TmL <br />
<br />
| valign="top" | Eric Cloninger, Fabio Fantato <br />
| valign="top" | Fabio Fantato <br />
| width="75" valign="top" | +0 <br />
| width="174" valign="top" | dsdp.tml <br />
| width="174" valign="top" | dsdp-tml.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
DSDP MTJ <br />
<br />
| valign="top" | Gustavo de Paula <br />
| valign="top" | Diego Madruga Sandin <br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | dsdp.mtj <br />
| width="174" valign="top" | dsdp-mtj.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | The Eclipse Project <br />
| valign="top" | <br />
The Eclipse Project <br />
<br />
''Platform, JDT, PDE'' <br />
<br />
| valign="top" | Mike Wilson <br />
| valign="top" | Kim Moir <br />
| width="75" valign="top" | 0 <br />
| width="174" valign="top" | eclipse<br> <br />
eclipse.jdt<br> eclipse.pde<br> eclipse.platform <br />
<br />
| width="174" valign="top" | ep.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" rowspan="7" | Eclipse Runtime Project (RT)<br> <br />
| valign="top" | <br />
Equinox <br />
<br />
| valign="top" | Thomas Watson, Jeff McAffer <br />
| valign="top" | Kim Moir <br />
| width="75" valign="top" | 0 <br />
| width="174" valign="top" | rt.equinox <br />
| width="174" valign="top" | equinox.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | Eclipse Persistence Services Project <br />
(EclipseLink)<br> <br />
<br />
| valign="top" | Peter Krogh<br> <br />
| valign="top" | <br />
Doug Clarke <br />
<br />
Peter Krogh <br />
<br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | rt.eclipselink <br />
| width="174" valign="top" | eclipselink.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Swordfish <br />
<br />
| valign="top" | Zsolt Beothy-Elo, Oliver Wolf <br />
| valign="top" | Zsolt Beothy-Elo <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | rt.swordfish <br />
| width="174" valign="top" | swordfish.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Riena <br />
<br />
| valign="top" | Christian Campo <br />
| valign="top" | Christian Campo <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | &nbsp;??? <br />
| width="174" valign="top" | riena.build <br />
| width="100" valign="top" | Y (starts with M5)<br />
|-<br />
| valign="top" | <br />
Eclipse Communication Framework (ECF) <br />
<br />
| valign="top" | Scott Lewis <br />
| valign="top" | Ted Kubaska/Scott Lewis <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | rt.ecf <br />
| width="174" valign="top" | ecf.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Rich Ajax Platform (RAP) <br />
<br />
| valign="top" | Jochen Krause, Ruediger Herrmann <br />
| valign="top" | Ralf Sternberg, Ruediger Herrmann <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | rt.rap <br />
| width="174" valign="top" | rap.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | Jetty <br />
| valign="top" | Greg Wilkins <br />
| width="75" valign="top" | TBD <br />
| width="174" valign="top" | &nbsp;? <br />
| width="174" valign="top" | rt.jetty <br />
| width="100" valign="top" | TBD <br />
| M6<br />
|-<br />
| valign="top" rowspan="10" | Modeling<br />
|-<br />
| valign="top" | <br />
Amalgam<br> <br />
<br />
| valign="top" | <br />
Cédric Brun<br> <br />
<br />
| valign="top" | <br />
Cédric Brun<br> <br />
<br />
| width="75" valign="top" | <br />
+3<br> <br />
<br />
| width="174" valign="top" | <br />
modeling.amalgam<br> <br />
<br />
| width="174" valign="top" | <br />
amalgam.build<br> <br />
<br />
| width="100" valign="top" | <br />
Y<br> <br />
<br />
|-<br />
| valign="top" | <br />
EMF<br> <br />
<br />
''CDO'',<br> ''Compare'',<br> ''Core (EMF)'',<br> ''Net4j'',<br> ''Model Query'',<br> ''Model Transaction'',<br> ''Model Validation'',<br> ''Teneo'' <br />
<br />
| valign="top" | <br />
Ed Merks<br> <br />
<br />
''Eike Stepper'',<br> ''Cédric Brun'',<br> ''Ed Merks'',<br> ''Eike Stepper'',<br> ''Bernd Kolb'',<br> ''Bernd Kolb'',<br> ''Boris Gruschko'',<br> ''Martin Taal'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Eike Stepper'',<br> ''Cédric Brun'',<br> ''Ed Merks'',<br> ''Eike Stepper'',<br> ''Boris Gruschko'',<br> ''Boris Gruschko'',<br> ''Boris Gruschko'',<br> ''Martin Taal'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+2,<br> +2,<br> +1,<br> +1,<br> +2,<br> +2,<br> +2,<br> +2 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.emf.cdo,<br> modeling.emf.compare,<br> modeling.emf.emf,<br> modeling.emf.net4j,<br> modeling.emf.query,<br> modeling.emf.transaction,<br> modeling.emf.validation,<br> modeling.emf.teneo <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
emf-cdo.build,<br> emf-compare.build,<br> emf-emf.build,<br> emf-net4j.build,<br> emf-query.build,<br> emf-transaction.build,<br> emf-validation.build,<br> emf-teneo.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y,<br> Y,<br> Y,<br> Y,<br> Y,<br> Y,<br> Y,<br> Y <br />
<br />
|-<br />
| valign="top" | <br />
EMFT<br> <br />
<br />
''Ecore Tools'',<br> ''EEF'',<br> ''Mint'',<br> ''MWE'' <br />
<br />
| valign="top" | <br />
Ed Merks<br> <br />
<br />
''David Sciamma'',<br> ''Goulwen Le Fur'',<br> ''Peter Nehrer'',<br> ''Sven Efftinge'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Jacques Lescot'',<br> ''Stéphane Bouchet'',<br> ''Peter Nehrer'',<br> ''Dennis Huebner'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+3,<br> +2,<br> +2,<br> +2 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.emft.ecoretools,<br> modeling.emft.eef,<br> modeling.emft.mint,<br> modeling.emft.mwe <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
emft-ecoretools.build,<br> emft-eef.build,<br> emft-mint.build,<br> emft-mwe.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y,<br> N,<br> Y,<br> Y <br />
<br />
|-<br />
| valign="top" | <br />
GMF<br> <br />
<br />
| valign="top" | <br />
Artem Tikhomirov<br> <br />
<br />
| valign="top" | <br />
Anthony Hunter<br> <br />
<br />
| width="75" valign="top" | <br />
+2<br> <br />
<br />
| width="174" valign="top" | <br />
modeling.gmf<br> <br />
<br />
| width="174" valign="top" | <br />
gmf.build<br> <br />
<br />
| width="100" valign="top" | <br />
Y<br> <br />
<br />
|-<br />
| valign="top" | <br />
GMT<br> <br />
<br />
''MoDisco'' <br />
<br />
| valign="top" | <br />
Jean Bézevin<br> <br />
<br />
''Hugo Brunelière'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Nicolas Bros'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+3 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.gmt.modisco <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
gmt-modisco.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
N <br />
<br />
|-<br />
| valign="top" | <br />
M2M<br> <br />
<br />
''ATL'',<br> ''QVTo'' <br />
<br />
| valign="top" | <br />
Frédéric Jouault<br> <br />
<br />
''Frédéric Jouault'',<br> ''Radek Dvorak'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''William Piers'',<br> ''Radek Dvorak'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+2,<br> +2 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.m2m.atl,<br> modeling.m2m.qvt-oml <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
m2m-atl.build,<br> m2m-qvtoml.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y,<br> Y <br />
<br />
|-<br />
| valign="top" | <br />
M2T<br> <br />
<br />
''Acceleo'',<br> ''JET'',<br> ''Xpand'' <br />
<br />
| valign="top" | <br />
Paul Elder<br> <br />
<br />
''Jonathan Musset'',<br> ''Paul Elder'',<br> ''Sven Efftinge'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Cédric Brun'',<br> ''Paul Elder'',<br> ''Dennis Huebner'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+2,<br> +1,<br> +2 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.m2t.acceleo,<br> modeling.m2t.jet,<br> modeling.m2t.xpand <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
m2t-acceleo.build,<br> m2t-jet.build,<br> m2t-xpand.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y,<br> Y,<br> Y <br />
<br />
|-<br />
| valign="top" | <br />
MDT<br> <br />
<br />
''OCL'',<br> ''UML2'',<br> ''XSD'' <br />
<br />
| valign="top" | <br />
Kenn Hussey<br> <br />
<br />
''Aleksandr Igdalov'',<br> ''James Bruck'',<br> ''Ed Merks'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Aleksandr Igdalov'',<br> ''James Bruck'',<br> ''Ed Merks'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+1,<br> +1,<br> +1 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.mdt.ocl,<br> modeling.mdt.uml2,<br> modeling.mdt.xsd <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
mdt-ocl.build,<br> mdt-uml2.build,<br> mdt-xsd.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y,<br> Y,<br> Y <br />
<br />
|-<br />
| valign="top" | <br />
TMF<br> <br />
<br />
''Xtext'' <br />
<br />
| valign="top" | <br />
Sven Efftinge, Frédéric Jouault<br> <br />
<br />
''Sven Efftinge'' <br />
<br />
| valign="top" | <br />
<br> <br />
<br />
''Dennis Huebner'' <br />
<br />
| width="75" valign="top" | <br />
<br> <br />
<br />
+2 <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
modeling.tmf.xtext <br />
<br />
| width="174" valign="top" | <br />
<br> <br />
<br />
tmf-xtext.build <br />
<br />
| width="100" valign="top" | <br />
<br> <br />
<br />
Y <br />
<br />
|-<br />
| valign="top" | SOA Tools <br />
| valign="top" | <br />
SOA Tools Platform (STP) <br />
<br />
''SCA Tools'' ''BPMN'' <br />
<br />
| valign="top" | Oisin Hurley <br />
''Stéphane Drapeau'' ''Antoine Toulmé'' <br />
<br />
| valign="top" | Oisin Hurley <br> <br />
''Stéphane Drapeau'' ''Antoine Toulmé'' <br />
<br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | stp<br> <br />
stp.bpmnmodeler<br> stp.sca <br />
<br />
| width="174" valign="top" | stp.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | Test &amp; Performance Tools Platform (TPTP) <br />
| valign="top" | <br />
Test &amp; Performance Tools Platform (TPTP) <br />
<br />
''Platform, Test, Trace, Monitoring'' <br />
<br />
| valign="top" | Kathy Chan <br />
| valign="top" | Joel Cayne <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | tptp<br>tptp.performance<br>tptp.platform<br>tptp.test<br>tptp.monitoring <br />
| width="174" valign="top" | tptp.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | Web Tools Platform (WTP) <br />
| valign="top" | <br />
Web Tools Platform (WTP) <br />
<br />
| valign="top" | David Williams <br />
| valign="top" | David Williams <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | webtools<br> <br />
webtools.common<br> webtools.dali<br> webtools.ejbtools<br> webtools.jeetools<br> webtools.jsf<br> webtools.servertools<br> webtools.sourceediting<br> webtools.webservices <br />
<br />
| width="174" valign="top" | webtools.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" rowspan="6" | Tools <br />
| valign="top" | <br />
Graphical Editing Framework (GEF) <br />
<br />
| valign="top" | Anthony Hunter <br />
| valign="top" | Anthony Hunter <br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | tools.gef <br />
| width="174" valign="top" | gef.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Buckminster <br />
<br />
| valign="top" | Thomas Hallgren, Henrik Lindberg <br />
| valign="top" | Thomas Hallgren <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | tools.buckminster <br />
| width="174" valign="top" | buckminster.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
CDT <br />
<br />
| valign="top" | Doug Schaefer <br />
| valign="top" | Vivian Kong <br />
| width="75" valign="top" | +1 <br />
| width="174" valign="top" | tools.cdt <br />
| width="174" valign="top" | cdt.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Mylyn <br />
<br />
| valign="top" | Mik Kersten <br />
| valign="top" | Steffen Pingel <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | tools.mylyn <br />
| width="174" valign="top" | mylyn.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
PHP Development Tools (PDT) <br />
<br />
| valign="top" | Roy Ganor <br />
| valign="top" | Roy Ganor <br />
''Nick Boldt (backup)'' <br />
<br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | [tools.pdt] <br />
| width="174" valign="top" | pdt.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
PTP <br />
<br />
| valign="top" | Greg Watson <br />
| valign="top" | Beth Tibbitts <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | tools.ptp <br />
| width="174" valign="top" | ptp.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" rowspan="8" | Technology <br />
| valign="top" | <br />
EPP <br />
<br />
| valign="top" | Markus Knauer <br />
''Wayne Beaton'' <br />
<br />
| valign="top" | <br />
Wayne Beaton <br />
<br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | technology.packaging <br />
| width="174" valign="top" | epp-udc.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Accessibility Tools Framework (ACTF) <br />
<br />
| valign="top" | Chieko Asakawa <br />
| valign="top" | Kentarou Fukuda <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.actf <br />
| width="174" valign="top" | actf.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Memory Analyzer (MAT) <br />
<br />
| valign="top" | Andreas Buchen <br />
| valign="top" | Erwin Margewitsch <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.mat <br />
| width="174" valign="top" | mat.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
DLTK <br />
<br />
| valign="top" | Andrey Platov <br />
| valign="top" | Andrey Platov <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.dltk <br />
| width="174" valign="top" | dltk.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
Subversive <br />
<br />
| valign="top" | Igor Vinnykov <br />
| valign="top" | Igor Burilo <br />
| width="75" valign="top" | +2 <br />
| width="174" valign="top" | technology.subversive <br />
| width="174" valign="top" | subversive.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | <br />
JWT <br />
<br />
| valign="top" | Marc Dutoo, Florian Lautenbacher <br />
| valign="top" | Christian Saad <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.jwt <br />
| width="174" valign="top" | jwt.build <br />
| width="100" valign="top" | Y<br />
|-<br />
| valign="top" | EGit <br />
| valign="top" | Shawn Pearce <br />
| valign="top" | &nbsp;? <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.egit <br />
| width="174" valign="top" | &nbsp;?? <br />
| width="100" valign="top" | N<br />
|-<br />
| valign="top" | Linux Tools <br />
| valign="top" | Andrew Overholt <br />
| valign="top" | Andrew Overholt <br />
| width="75" valign="top" | +3 <br />
| width="174" valign="top" | technology.linux-distros <br />
| width="174" valign="top" | &nbsp;?? <br />
| width="100" valign="top" | N<br />
|}<br />
<br />
[[Category:Helios]] [[Category:Coordinated]]</div>Thomas.tada.sehttps://wiki.eclipse.org/index.php?title=Equinox/p2/Query_Language_for_p2&diff=191775Equinox/p2/Query Language for p22010-03-09T11:53:34Z<p>Thomas.tada.se: /* Basic IQuery examples */</p>
<hr />
<div>[[Category:Equinox_p2]] <br />
===Background===<br />
As p2 gets more widely used, we predict that repositories will grow larger and larger over time. The Galileo repository alone contains 3866 IU's (in one single service release). Starting with Helios, we plan to keep everything over time which will put the IU count close to 10.000. And that's just one year of Eclipse IP approved material. OSGi is gaining in popularity and p2 is getting increasingly popular. Some companies plan to map the central maven repository as p2. We can soon expect to see p2 repositories with over a 100.000 IU's in them.<br />
<br />
====The problem====<br />
p2 has a query mechanism today that makes it hard to create a repository implementation that is based on a database. It is also diffiult to create an efficient client/server solution. The reason for this is that most of the queries are written as implementations of the IQuery interface, either as ContextQuery derivates that need access to all rows in the database, or as MatchQuery derivates with a java method to which all rows need to be passed.<br />
<br />
There is no uniform way to translate these queries into an expression that can be understood by a database or by something running in another process.<br />
<br />
Some attempts have been made to rectify this and a repository implementation could potentially make some intelligent decisions based on information derived from looking at the query (what class it is, and in some cases, what properties that are exposed). While this is an improvement, the general problem remains; A large amount of the queries are still more or less completely black-boxed and it's enough with one single such query to force a full selection of everything from the database/remote server.<br />
<br />
The p2QL expression language discussed here is an attempt to rectify this problem. It can be used in two ways:<br />
* Coexisting with current IQuery / IQueryable<br />
*: Using the ExpressionContextQuery (implements IQuery), or the ExpressionQuery (implements IMatchQuery), it can coexist with todays IQuery/IQueryable. The queries are created based on an expression and an array of parameters. See section [[#IQuery examples]].<br />
* With more explicit separation of concerns<br />
*: It can also be used in an alternative solution where an expression returns an Iterator. This usage scenario is particularly useful when implementing remote repositories or implementations on top of a database. It is also a very efficient way of querying for things that you don't want to collect in memory.<br />
<br />
====Bugzilla====<br />
We discussed the use of a query language at length on the p2 meeting on November 9 and November 16. The issue is captured in bugzilla [https://bugs.eclipse.org/bugs/show_bug.cgi?id=294691 Create a QueryLanguage for p2].<br />
<br />
===Language Design===<br />
TODO: Add some text here to explain where the ideas stem from (xtend, xtext, Scala, Java) and other motivation behind the choice of syntax.<br />
====Boolean Queries====<br />
Queries can be written as simple predicates such as "id == $0". When executed, the predicate will be evaluated once for each row in the queried material. This form is used for the IMatchQuery implementation. The current value is available in a predefined variable named ''this''. Like in Java, ''this'' is normally implicit so that:<br />
<pre>id == $0</pre><br />
is actually the same as:<br />
<pre>this.id == $0</pre><br />
<br />
====Collection Queries====<br />
A query can also be declared as something that acts on "everything". This is particularly useful when things like search for latest or doing requirements traversal. These queries make use of a predefined variable named ''everything''. When accessed, that variable returns an iterator over all queried material. As with ''this'' in a predicate query, ''everything'' is something that you don't normally need to specify, i.e.:<br />
<pre>select(x | x.id == $0)</pre><br />
is the same as:<br />
<pre>everything.select(x | x.id == $0)</pre><br />
A boolean query can always be written as a collection query. These two queries are fully equivalent:<br />
<pre>Boolean: id == $0<br />
Collection: select(x | x.id == $0)</pre><br />
<br />
====Special operators====<br />
<ul><li><b>Matches operator '~='</b><br/><br />
<p>A large amount of queries involve versions, version ranges, and capability matching. So managing that is important. Another thing that is important is to be able to support filtering based on ID's. All of this is now also built in to the language through the matches operator '''~='''. It can be though of as the 'satisfies' method on an installable unit or provided capability, as 'is included in' when used with a version and version range, as a plain matches when used with a string and a pattern, or as 'instanceof' when comparing with a class.</p><br />
<p>The following things can be matched using the operator</p><br />
{| border="1" cellpadding="3"<br />
!LHS<br />
!RHS<br />
!Implemented as<br />
|-<br />
|IInstallableUnit<br />
|IRequirement<br />
|lhs.satisfies(rhs)<br />
|-<br />
|IInstallableUnit<br />
|Filter<br />
|rhs.matches(lhs.properties)<br />
|-<br />
|IInstallableUnit<br />
|IUpdateDescriptor<br />
|rhs.isUpdateOf(lhs)<br />
|-<br />
|Version<br />
|VersionRange<br />
|rhs.isIncluded(lhs)<br />
|-<br />
|Map<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|Dictionary<br />
|Filter<br />
|rhs.match(lhs)<br />
|-<br />
|String<br />
|SimplePattern<br />
|rhs.isMatch(lhs)<br />
|-<br />
|String<br />
|VersionRange<br />
|rhs.isIncluded(version(lhs))<br />
|-<br />
|<any><br />
|Class<br />
|rhs.isInstance(lhs)<br />
|-<br />
|Class<br />
|Class<br />
|rhs.isAssignableFrom(lhs)<br />
|}</li><br />
<li><b>And operator '&&'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''true'' if all its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''intersect''' operator and the result consists those elements that could be found in all collections.<br />
</li><br />
<li><b>Or operator '||'</b><br/><br />
This operator checks if the first operand evaluates to a boolean. If it is, the it is assumed that all other operands also are booleans and the full evaluation is ''false'' if none of its operands evaluate to ''true''. If the first result was not a boolean, then it is assumed that it is a collection and that all other operands also evaluates collections. The operator will then function as a '''union''' operator and the result is the unique sum of all elements from all collections.<br />
</li><br />
</ul><br />
<br />
====Functions====<br />
A small number of functions are available to assist with some common problems. Example:<br />
An IRequiredCapability.getFilter() returns a String. I only want the capabilities that has a filter that match certain properties. Consequently, I need an instance of an OSGi Filter. Not a string. I can do this using the ''filter'' function. Like this:<br />
<pre>requiredCapabilities.exists(rc | rc.filter == null || $1 ~= filter(rc.filter))</pre><br />
The currently available constructors are:<br />
{| border="1" cellpadding="3"<br />
!name<br />
!arguments<br />
!creates<br />
|-<br />
|filter<br />
|string<br />
|an instance of org.osgi.framework.Filter<br />
|-<br />
|version<br />
|string<br />
|a p2 Version<br />
|-<br />
|range<br />
|string<br />
|a p2 VersionRange<br />
|-<br />
|class<br />
|string<br />
|a Class<br />
|-<br />
|boolean<br />
|string<br />
|a boolean<br />
|-<br />
|set<br />
|comma separated list of expressions<br />
|an instance of java.util.HashSet<br />
|-<br />
|iquery<br />
|IQuery [, variable [, collector]]<br />
|The result of evaluating the query<br />
|}<br />
<p>A note about the <b>iquery</b> constructor.<br/><br />
The constructor operates in one of two modes depending on its IQuery argument.</p><br />
<p>If the argument implements IMatchQuery, the the constructor will return the boolean result of invoking the isMatch(value) method on that query. The value is picked from the variable and the default variable is 'item'. It is considered an error to pass a third argument in combination with an IMatchQuery.</p><br />
<p>When the query does not implement the IMatchQuery interface, it is considered a context query and its method perform(iterator, collector) will be called. The iterator is picked from the variable and the default variable is 'everything'. The collector can be passed in as the third argument. If no third argument is present, a new collector will be created.</p><br />
<br />
====Collection functions====<br />
A number of functions was added to enable various manipulations of collections. A common way of writing collection functions is:<br />
<pre>elements.someFunction(element | <expression>)</pre><br />
Here are the functions that are available in the p2QL language:<br />
<ul><br />
<li>'''Select'''<br/><br />
The '''select''' function will evaluate its expression, once for each element, and return a new collection containing all elements for which the evaluation result was ''true''. Example:<br />
<pre>select(x | x.id == 'org.eclipse.equinox.p2.ql')</pre><br />
</li><br />
<li>'''Collect'''<br/><br />
The '''collect''' collects the result of evaluating each element in a new collection. Example:<br />
<pre>collect(x | x.id)</pre><br />
returns a collection consisting of the value of the id property of each element in the original collection.<br />
</li><br />
<li>'''Exists'''<br/><br />
The '''exists''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields ''true''. If no such element is found, the whole evaluation yields ''false''. Example:<br />
<pre>providedCapabilities.exists(p | p.namespace == 'osgi.bundle')</pre></li><br />
<li>'''All'''<br/><br />
The '''all''' function will evaluate an expression for each element until an evaluation yields ''false''. If that happens, the whole evaluation yields ''false''. If no such element is found, the whole evaluation yields ''true''. Example:<br />
<pre>$0.all(rc | this ~= rc)</pre><br />
Assuming $0 is a list of required capabilities, this query asks for items that fulfill all requirements.</li><br />
<li>'''First'''<br/><br />
The '''first''' function will evaluate an expression for each element in the collection until an evaluation yields ''true''. If that happens, the result of the whole evaluation yields that element. If no such element is found, the whole evaluation yields ''null''. Example:<br />
<pre>providedCapabilities.first(p | p.namespace == 'java.package')</pre><br />
Returns the first provided capability with namespace 'java.package'.</li><br />
<li>'''Flatten'''<br/><br />
Intended to be applied on collections of collections. Yields a single collection with all elements from the source collections, in the order they are evaluated.Example:<br />
<pre>collect(x | x.requiredCapabilities).flatten()</pre><br />
Yields a collection with all required capabilities from all iu's that the collect was applied on.</li><br />
<li>'''Latest'''<br/><br />
Some queries must make sure that the result only contains the latest version of each found IU. The special function ''latest'' to makes that possible. The function will require that the collection elements implement the IVersionedId interface. Here is an example of how to use it:<br />
<pre>select(x | x.id == $0).latest()</pre><br />
As a convenience, it is also possible to write:<br />
<pre>latest(x | x.id == $0)</pre></li><br />
<li>'''Limit'''<br/><br />
It is sometimes desirable to limit the number of rows returned by a query. The function 'limit' can be used for that:<br />
<pre>select(...).limit(100)</pre></li><br />
<li>'''Unique'''<br/><br />
This function will ensure that the resulting collection contains unique elements. The function can operate in two modes, with or without a cache argument. I.e.<br />
<pre>x.unique()</pre><br />
or<br />
<pre>x.unique(cache)</pre><br />
The latter expects the argument to be a ''java.util.Set'' that it can use to enforce the uniqueness of the element. This enables the result to be unique in a larger scope then the collection itself.</li><br />
<li>'''Traverse'''<br/><br />
A common scenario in p2 is that you want to start with a set of roots and then find all items that fulfill the root requirements. Those items in turn introduce new requirements so you want to find them too. The process continues until no more requirements can be satisfied. This type of query can be performed using the '''traverse''' function. The function will evaluate an expression, once for each element, collect elements for which the evaluation returned ''true'', then then re-evaluate using the collected result as source of elements. No element is evaluated twice. This continues until no more elements are found:<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.collect(rc | select(iu | ~= rc)).flatten())</pre><br />
This is of course a fairly naive slicing mechanism. See [[#Currying]] below for more advanced examples.</li><br />
</ul><br />
<br />
====Query parameters====<br />
The query must be parameterised so that expression parsing can be done once and then executed multiple times. A parameter is expressed as $''n'' where ''n'' is either the parameter index, originating from 0.<br />
<br />
===Basic IQuery examples===<br />
Here are some examples of how to use the expressions with IQuery:<br />
Query for all IU's that has an id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("id == $0", id);</pre><br />
Query for the latest IU of some specific id:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0)", id);</pre><br />
Query an artifact repository for all keys with a specific classifier:<br />
<pre>IQuery&lt;IArtifactKey&gt; query = QueryUtil.createMatchQuery(IArtifactKey.class, "classifier == $0", classifier);</pre><br />
Query for the latest IU that matches a specific version range. Since the second parameter is a VersionRange, the ~= operator is interpreted as ''isIncluded'':<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x.id == $0 && x.version ~= $1)", id, range);</pre><br />
Query for an IU that has a specific property set:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("properties[$0] == $1", key, value);</pre><br />
The same query, but this time for multiple possible values:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$1.exists(v | properties[$0] == v)", key, new Object[] { v1, v2, v3 });</pre><br />
Query for all categories found in the repository:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQueryx("properties[$0] == true", IInstallableUnit.PROP_TYPE_CATEGORY);</pre><br />
Query for all IU's that fulfil at least one of the requirements from another IU. Since the first parameter is a list of IRequirements, the ~= applied to each each IU using ''satisfies''.<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rc | this ~= rc)", iu.getRequiredCapabilities());</pre><br />
Query for the latest version of all patches:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery("latest(x | x ~= $0)", IInstallableUnitPatch.class);</pre><br />
Query for all IU's affected by a patch:<br />
<pre>IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createMatchQuery("$0.exists(rcs | rcs.all(rc | this ~= rc))", patch.getApplicabilityScope());</pre><br />
<br />
===Localization===<br />
An installable unit may have locale specific properties. Such properties may be stored in the IU itself or in fragments that provide localization for the IU in the 'org.eclipse.equinox.p2.localization' namespace.<br />
p2QL will interpret the member ''translations'' applied on an IU as a map of translated properties, hiding all the details. As an example, this query:<br />
<br />
<pre>translations['license'] ~= /*kommersiellt bruk*/</pre><br />
<br />
when used with a default Locale("sv_SE") would yield all IU's that has a license, translated into Swedish, that contains the exact phrase 'kommersiellt bruk'.<br />
<br />
A special class named '''KeyWithLocale''' was added to allow queries for entries that does not match the current locale. So the above query can be written as:<br />
<br />
<pre>QueryUtil.createMatchQuery("this.translations[$0] ~= /*kommersiellt bruk*/", new KeyWithLocale("license", new Locale("sv", "SE")))</pre><br />
<br />
===Currying===<br />
Want some add some spice? Probably...<br />
<br />
Consider the traversal function and the example we had above:<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
<p>Now let us assume that we want to perform this traversal and filter the requirements based on platform. Our first attempt could look something like this.<br />
<pre>$0.traverse(parent | parent.requiredCapabilities.select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten())</pre><br />
This would however be very inefficient since many requirements exists in several IU's. During the traversal there is no point traversing a requirement more then once so it would be good to "remember" the perused requirements.<br />
</p><p>The p2QL syntax permits something generally referred to as ''currying''. Currying means that you can provide more parameters then the single one that represents the current element to the expression of a collection function. So far, we've seen examples using the syntax:<br />
<pre>select(x | <do something with x>)</pre><br />
This is actually a short form for the longer:<br />
<pre>select(_, {x | <do something with x>})</pre><br />
The select provides one parameter to each iteration. It's value is always provided reachable using the special operator '_'. In this case, the variable x maps to the parameter _ since they have the same positional offset. I can add more parameters by declaring:<br />
<pre>select(a, b, _, {x, y, z, <do something with x, y, z>})</pre><br />
Variable x will now have the value of a, y the value of b, and z the value of _.</p><br />
<p>So why is this important? Well, the initializers are just evaluated once for one call to select. The expression however, is evaluated once for each new value of _.<p><br />
<p>Let us now return to the traversal again. Because with this syntax, we can actually specify a global set that we can pass to the unique function so that no requirement is perused twice:<br />
<pre>$0.traverse(set(), _, { rqCache, parent | parent.requiredCapabilities.unique(rqCache).select(rc | rc.filter == null || $1 ~= rc.filter).collect(rc | select(iu | iu ~= rc)).flatten()})</pre><br />
</p><br />
<br />
===Performance comparison: p2QL traversal versus the p2 Slicer===<br />
<p>A test was performed using the query example above on the Galileo repository, using a specific version of the org.eclipse.sdk.feature.group IU as the root. The test produced a result with 411 IU's in < 12 ms average. I compared this with another test that used the p2 Slicer. The number of IU's produced was exactly the same. The slicer however, took > 22 ms (it also uses a capability cache internally). Not very scientific perhaps but repeated tests produce similar results.</p><br />
<p>I'm sure the Slicer can be optimized and achieve results that might be even slightly better then the traversal but I still think this proves a point. p2QL Performance is very good at this point.</p><br />
<br />
===Java API===<br />
The expression tree created by the parser must be well documented and easy to use so that queries can be created programmatically. Since all expressions are immutable and without context, they can be combined freely. Hence, code like this is fully possible:<br />
<pre><br />
// Create some expressions. Note the use of identifiers instead of<br />
// indexes for the parameters<br />
<br />
IExpressionFactory factory = ExpressionUtil.getFactory();<br />
IExpression item = factory.variable("this");<br />
IExpression cmp1 = factory.equals(factory.member(item, "id"), factory.parameter(0));<br />
IExpression cmp2 = factory.equals(<br />
factory.at(factory.member(item, "properties"), factory.parameter(1)),<br />
factory.parameter(2));<br />
<br />
IExpression everything = factory.variable("everything");<br />
IExpression lambda = factory.lambda(item, factory.and(new IExpression[] {cmp1, cmp2}));<br />
IExpression latest = factory.latest(factory.select(everything, lambda));<br />
<br />
// Create the query<br />
IQuery&lt;IInstallableUnit&gt; query = QueryUtil.createQuery&lt;IInstallableUnit&gt;(latest "test.bundle", "org.eclipse.equinox.p2.type.group", Boolean.TRUE);<br />
</pre><br />
<br />
===The [http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form BNF]===<br />
<pre><br />
condition<br />
: orExpression ( '?' orExpression ':' orExpression )?<br />
;<br />
<br />
orExpression : andExpression ( '||' andExpression )* ;<br />
<br />
andExpression : binaryExpression ( '&&' binaryExpression )* ;<br />
<br />
binaryExpression : notExpression ( op notExpression )?;<br />
<br />
op : '==' | '!=' | '>' | '>=' | '<' | '<=' | '~=' ;<br />
<br />
notExpression<br />
: '!' notExpression<br />
| collectionExpression<br />
;<br />
<br />
collectionExpression<br />
: memberExpression ( '.' collectionFunction )*<br />
;<br />
<br />
memberExpression : function ( ( '.' ID ) | ( '[' memberExpression ']' ) )* ;<br />
<br />
function<br />
: ( filter | version | range | class) '(' condition ')'<br />
| set '(' ( condition ( ',' condition )* )? ')'<br />
| unaryExpression<br />
;<br />
<br />
collectionFunction<br />
: ( select | reject | collect | exists | all | traverse | first ) '(' lambdaDefinition ')'<br />
| limit '(' memberExpression ')'<br />
| unique '(' memberExpression? ')'<br />
| latest '(' lambdaDefinition? ')'<br />
| flatten '(' lambdaDefinition? ')'<br />
;<br />
<br />
lambdaDefinition<br />
: initializer ( ',' initializer )* ( ',' '{' lambda '}' )?<br />
| '{' lambda '}'<br />
| lambda<br />
;<br />
<br />
initializer<br />
: '_'<br />
| condition<br />
;<br />
<br />
lambda<br />
: ( ID ( ',' ID )* )? '|' condition<br />
;<br />
<br />
unaryExpression<br />
: '(' condition ')'<br />
| '[' condition ( ',' condition )* ']' // #array construct<br />
| STRING<br />
| INT<br />
| parameter<br />
| 'null'<br />
| 'true'<br />
| 'false'<br />
| ID<br />
;<br />
<br />
parameter<br />
: '$' INT | ID<br />
;<br />
</pre></div>Thomas.tada.se