Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "CBI/aggregator/manual"

(Introduction)
(Redirected page to CBI/aggregator)
 
(86 intermediate revisions by 15 users not shown)
Line 1: Line 1:
=Introduction=
+
#REDIRECT [[CBI/aggregator]]
The Aggregator is based on and part of the ''Eclipse b3'' project. b3 provides a versatile and adaptable framework supporting build, assembly and deployment processes. It supports a rich set of use cases. One of those - the aggregation of repositories - is the focus of the ''b3 Aggregator'' tool.
+
 
+
The Aggregator combines repositories from various sources into a new aggregated p2 repository. It can also be configured to produce a hybrid p2/Maven2 repository.
+
There are many situations where using aggregated repositories is a good solution. The reasons vary from licensing issues to organizational requirements:
+
#'''Owners of a p2 repo for a given project may not be in position to host all required or recommended components due to licensing issues''' - Buckminster's SVN support can serve as an example here, as it requires components available through the main Eclipse p2 repo as well as third-party components. Hence users have to visit several repos for a complete install.
+
#'''Projects want to provide convenient access to their products''' - Installation instructions requiring the user to visit several repos for a complete install are not uncommon. An aggregated repo for all those locations provides a convenient one-stop strategy. The aggregation may support mirroring all consumed p2 repos or simply providing an indirection via a composite repo.
+
#'''Organizations or teams want control over internally used components''' - It may be necessary to have gated access to relevant p2 repos and do an organizational "healthcheck" of those before internal distribution. Furthermore, internally used aggregated repos can provide a common basis for all organizational users.
+
#'''Increase repository availability''' - by aggregating and mirroring what is used from multiple update sites into an internally controlled server (or several).
+
#'''Distributed Development Support''' - an overall product repository is produced by aggregating contributions from multiple teams.
+
 
+
The Aggregator tool is focused on supporting these specific requirements, and it plays an important role in the full scope of the b3 project. The Aggregator is however used in scenarios outside of the traditional "build domain" and this has been reflected in the user interface which does not delve into the details of "building" and should therefore be easy to use by non build experts.
+
 
+
Furthermore, it is worth noting that:
+
* the Aggregator is based on EMF models
+
* headless execution of aggregation definitions (once they have been created) is possible using a command line packaging of the Aggregator
+
 
+
=Functional Overview=
+
The b3 Aggregator performs aggregation and validation of repositories. The input to the aggregator engine (that tells it what to do) is a ''b3aggr'' EMF model. Such a model is most conveniently created by using the b3 Aggregator editor. This editor provides both editing and interactive execution of aggregation commands. The editor is based on a standard EMF "tree and properties view" style editor where nodes are added and removed to form a tree, and the details of nodes are edited in a separate properties view. Once a b3aggr model has been created it is possible to use the command line / headless aggregator to perform aggregation (and other related commands). (Note that since the b3aggr is "just and EMF model", it can be produced via EMF APIs, transformation tools, etc. and thus support advanced use cases).
+
 
+
The model mainly consists of ''Contributions''; specifications of what to include from different repositories, and Validation Repositories; repositories that are used when validating, but that are not included in the produced aggregation (i.e. they are not copied). The model also contains specification of various processing rules (exclusions, transformation of names, etc.), and specification of Contacts; individuals/mailing-lists to inform when processing fails.
+
 
+
Here are some of the important features supported by the b3 Aggregator:
+
* '''p2''' and '''maven2''' support — the aggregator can aggregate from and to both p2 and maven2 repositories.
+
* '''Maven2 name mapping support''' — names in the p2 domain are automatically mapped to maven2 names using built in rules. Custom rules are also supported.
+
* '''Mirroring''' — artifacts from repositories are mirrored/downloaded/copied to a single location
+
* '''Selective mirroring''' — an aggregation can produce an aggregation consisting of a mix of references to repositories and mirrored repositories.
+
* '''Cherry picking''' — it is possible to pick individual items when the entire content of a repository is not wanted. Detailed picking is supported as well as picking transitive closures like a product, or a category to get everything it contains/requires.
+
* '''Pruning''' — it is possible to specify mirroring based on version ranges. This can be used to reduce the size of the produced result when historical versions are not needed in the aggregated result.
+
* '''Categorization''' — categorization of installable units is important to the consumers of the aggregated repository. Categories are often choosen by repository publishers in a fashion that makes sense when looking at a particular repository in isolation, but when they are combined with others it can be very difficult for the user to understand what they relate to. An important task for the constructor of an aggregation is to be able to organize the aggregated material in an easily consumable fashion. The b3 aggregator has support for category prefixing, category renaming, addition of custom categories, as well as adding and removing features in categories.
+
* '''Validation''' — the b3 aggregator validates the aggregated result to ensure that everything in the repository is installable. 
+
* '''Blame Email''' — when issues are found during validation the aggregator supports sending emails describing the issue. This is very useful when aggregating the result of many different projects. Advanced features include specifying contacts for parts of the aggregation which is useful in large multi layer project structures where issues may related to the combination of a group of projects rather than one individual project - someone responsible for the aggregation itself should be informed about these cross-project issues. The aggregator supports detailed control over email generation including handling of mock emails when testing aggregation scripts.
+
 
+
=Installation=
+
Start by installing a fresh Eclipse 3.6 SDK from http://download.eclipse.org/eclipse/downloads (The latest version used at the time of writing was http://download.eclipse.org/eclipse/downloads/drops/S-3.6M6-201003121448/index.php)
+
 
+
The b3 aggregator can either be integrated in your Eclipse SDK or it can be installed as a standalone headless product (i.e. pure command line, without any graphical UI).
+
 
+
The instructions below show the current URLs (when this document was written). Always check the latest information on the [http://eclipse.org/modeling/emft/b3/download/ b3 download page] before installing.
+
 
+
==Eclipse SDK installation==
+
{|
+
|-
+
| colspan="2" |
+
# Start your Eclipse installation and open the '''''Install New Software wizard'''''. You'll find in under the top menu ''Help''
+
# Click the ''Add...'' button and enter the URL <nowiki>http://download.eclipse.org/modeling/emft/b3/updates-3.6</nowiki> in the ''Location'' field.
+
# Select the '''''b3 Aggregator Editor''''' and click ''Next'' twice.
+
# Accept the Eclipse Public License and click ''Finish''
+
# Restart the IDE once the installation is finished.
+
 
+
|-
+
|
+
[[Image:b3_Install_New_Software.png|thumb|center|580x492px|'''''Install New Software wizard''''']]
+
|
+
[[Image:b3_Install_Aggregator.png|thumb|center|580x492px|'''''b3 Aggregator Editor''''' selection]]
+
|}
+
 
+
==Headless installation==
+
Installation of the headless version of the Aggregator is similar to a typical headless b3 installation. The following steps focus on the installation of the headless Aggregator feature.
+
 
+
#Start by '''Downloading the (headless) director''' which can be found [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/products/director_latest.zip here].
+
#Next '''unpack the director''' to a location of your choice. You may want to set the <code>PATH</code> to include the install location and make the director accessible for additional use.
+
#'''Install b3''' with the following command:
+
#*<code>director -r <HEADLESS_REPO> -d <INSTALL_DIR> -p b3 -i org.eclipse.b3.cli.product -i org.eclipse.b3.aggregator.engine.feature.feature.group</code> where
+
#**'''''-r <HEADLESS_REPO>''''' is the headless p2 update site: Current version is: http://download.eclipse.org/modeling/emft/b3/headless-3.6 (Check the [http://eclipse.org/modeling/emft/b3/download/ download page] for changes of the headless update site.)
+
#**'''''-d <INSTALL_DIR>''''' is the chosen install location of the headless b3
+
#**'''''-p b3''''' is the name of the p2 profile
+
#**'''''-i org.eclipse.cli.product''''' is the name of the headless b3 base
+
#**'''''-i org.eclipse.b3.aggregator.engine.feature.feature.group''''' is the name of the aggregator feature
+
 
+
 
+
 
+
=Getting started with standard examples=
+
In the following sections we provide two simple examples that are easy to replicate and should highlight the most important features of the Aggregator.
+
The first example deals with the creation of two variations of a p2 repo. The second shows the Aggregator's Maven support.
+
 
+
==Aggregating a p2 repo==
+
The first sample aggregation is build around Buckminster and its support for Subversive. The objective of this aggregated repo is to:
+
*provide a "one-stop shop" experience
+
*conveniently pull in third-party components that are not hosted at Eclipse
+
*provide this repo as an indirection mechanism if required
+
 
+
===Mirroring contributions===
+
(This example aggregation can be downloaded via the b3 project SVN and opened in an appropriately set up workbench: [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_mirrored.b3aggr buckminster_helios_mirrored.b3aggr]).
+
 
+
The background is that Buckminster provides support for Subversive. In addition to the Subversive components hosted at Eclipse (http://www.eclipse.org/subversive/), a complete installation will also require Subversive SVN connectors provided by Polarion (http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/).
+
We want to create a repo that combines all these components and makes them accessible from one location. We want to make several platform configurations available.
+
 
+
[[Image:b3_aggregator_sample_1.png|thumb|center|800x750px|'''''b3 Aggregator - Sample 1''''']]
+
 
+
This example already includes some of the more advanced aggregation features. Stepping through the model view from the top node the following components can be identified:
+
#The top node '''''Aggregator''''' groups all model definitions regarding '''''Configuration'''''s and '''''Contribution'''''s. Looking at the properties section at the bottom we see that:
+
#*the top node has been provided with a mandatory ''Label'' with the value "Helios + Buckminster for Subversive"; this is also the label that is shown to users when accessing the aggregated repo via the p2 update manager
+
#*the ''Build Root'' identifies the location to which the artifacts of the aggregated repo will be deployed 
+
#The aggregation is defined for three configurations (i.e. os=win32, ws=win32, arch=x86; etc)
+
#*any number of configurations can be defined
+
#*during the aggregation process all dependencies of the ''contributed'' components will be verified for all provided configurations, unless exceptions are defined (see below)
+
#The first '''''Contribution''''' to the aggregation is labeled "Helios 3.5".
+
#*this contribution represents the simplest example of a contribution
+
#*one '''''Mapped Repository''''' is defined for this contribution (it can have multiple); all that is needed is a user-defined label and the URL of the repository that should be included
+
#*the result of this definition is that the complete Helios p2 repo will be included in the aggregated repo
+
#The second '''''Contribution''''' is labeled "Subversive SVN connectors" and deals with the inclusion of bundles provided by Polarion. This contribution includes binary configuration-specific artifacts which are only available for win32. If a simple contribution would be defined the aggregation would fail for all non-win32 configurations, and hence the aggregation would fail as a whole.
+
#*this requires a definition of '''''Valid Configurations Rule'''''s that state exceptions
+
#*the rules defined for the the three components in question essentially state that the verification process for those components should only be performed for win32-based configurations
+
#The third '''''Contribution''''' is labeled "Buckminster (latest)". It shows another advanced feature - an '''''Exclusion Rule'''''.
+
#*remember that the objective of the sample repo is to provide convenient setup of Buckminster with Subversive support, and since Buckminster's Subclipse and Subversive support are mutually exclusive, we want to exclude the features for Subclipse from the aggregated repo to make it easier for the user.
+
#*this is done using an '''''Exclusion Rule''''' defined for each Installable Unit that should be excluded
+
#A list of all included repos is displayed at the bottom of the model editor view
+
#*this list allows browsing the contents of all repos
+
#*this part of the model is not editable
+
 
+
The aggregation can be run by right-clicking any node in the model and selecting '''''Build Repository'''''.
+
This example was setup to use a mirroring approach for all contributed repos. Hence, the complete contents of all included can be found in the aggregated repos target location specified under '''''Build Root'''''.
+
 
+
Check the next section for a slightly different approach.
+
 
+
===Providing a repo indirection===
+
{|- width="100%"
+
|- valign="top"
+
|
+
Mirroring all repo artifacts of your aggregated contributions is a very valuable and important feature when performing aggregation, but there are also many cases where this is not necessary. It is possible to turn off artifact mirroring/copying by changing one property for a defined contribution.
+
Each '''''Mapped Repository''''' has a boolean property called ''Mirror Artifacts'' which can be set to <code>false</code> in order to prevent copying all artifacts of the contributed repo to the aggregated repo.
+
 
+
The following [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_redirect.b3aggr buckminster_helios_redirect.b3aggr] is a variation of the first example with the ''Mirror Artifacts'' property set to <code>false</code> for all contributed repos. Running this aggregation will result in a composite repository that provides indirection to the contributed repos.
+
 
+
|
+
[[Image:b3_aggregator_sample_not_mirrored.png|thumb|right|580x200px|'''''b3 Aggregator - mirroring disabled''''']]
+
|}
+
 
+
==Creating a Maven-conformant p2 repo==
+
{|- width="100%"
+
|- valign="top"
+
|
+
A powerful feature of the Aggregator is the ability to create aggregated repos that can be consumed as Maven repos, (i.e. providing the directory/file structure and artifacts required by Maven). An aggregated repository that supports Maven can be consumed both as a p2 and a Maven repo (at the same time). This flexibility is possible thanks to p2's separation of dependency meta-data and the actual location of the referenced artifacts.
+
 
+
In order to create a Maven-conformant aggregate repo all that is required is to set the property '''''Maven Result''''' property of the '''''Aggregator''''' to <code>true</code>. The aggregation deployed to the '''''Build Root''''' location will be a Maven repo.
+
 
+
The sample [http://dev.eclipse.org/svnroot/modeling/org.eclipse.emft.b3/trunk/org.eclipse.b3.aggregator.samples/buckminster_helios_maven.b3aggr buckminster_helios_maven.b3aggr] is a variation of the previous aggregations configured to produce a Maven repo.
+
 
+
|
+
[[Image:b3_aggregator_sample_maven.png|thumb|right|580x200px|'''''b3 Aggregator - Maven result''''']]
+
|}
+
 
+
=Headless support=
+
You will need a [[#Headless installation|headless installation of b3]] with the Aggregator feature installed.
+
 
+
==Running from the command line==
+
Just type:<pre>b3 aggregate <options></pre>
+
 
+
For a detailed listing of the available options consult the next section.
+
 
+
==Command line options==
+
{| {{Greytable}}
+
|-
+
! Option
+
! Value
+
! Default
+
! Description
+
 
+
|- valign="top"
+
| '''--buildModel'''
+
| <path to build model>
+
| This value is required
+
| Appoints the aggregation definition that drives the execution. The value must be a valid path to a file (absolute or relative to current directory).
+
 
+
|- valign="top"
+
| '''--action'''
+
| VERIFY
+
 
+
BUILD
+
 
+
CLEAN
+
 
+
CLEAN_BUILD
+
 
+
| BUILD
+
| Specifies the type of the execution.
+
*'''VERIFY''' - verifies model validity and resolves dependencies; no artifacts are copied or created
+
*'''BUILD''' - performs the aggregation and creates the aggregated repository in the target location
+
*'''CLEAN''' - cleans all traces of previous aggregations in the specified target location
+
*'''CLEAN_BUILD''' - performs a CLEAN followed by a BUILD
+
 
+
|- valign="top"
+
| '''--buildId'''
+
| <string>
+
| build-<timestamp in the format yyyyMMddHHmm>
+
| Assigns a build identifier to the aggregation. The identifier is used to identify the build in notification emails. Defaults to: build-<timestamp> where <timestamp> is formatted according as yyyyMMddHHmm, i.e. build-200911031527
+
 
+
|- valign="top"
+
| '''--buildRoot'''
+
| <path to directory>
+
| ''buildRoot'' declared in the aggregation model
+
| Controls the output. Defaults to the build root defined in the aggregation definition. The value must be a valid path to a directory (absolute or relative to current folder). If the desired directory structure does not exist then it is created.
+
 
+
|- valign="top"
+
| '''--production'''
+
| N/A
+
| N/A
+
| Indicates that the build is running in real production. That means that no mock emails will be sent. Instead, the contacts listed for each contribution will get emails when things go wrong.
+
'''CAUTION: Use this option only if you are absolutely sure that you know what you are doing, especially when using models "borrowed" from other production builds without changing the emails first.'''
+
 
+
|- valign="top"
+
| '''--emailFrom'''
+
| <email>
+
| Address of build master
+
| Becomes the sender of the emails sent from the aggregator.
+
 
+
|- valign="top"
+
| '''--emailFromName'''
+
| <name>
+
| If --emailFrom is set then this option sets the real name of the email sender.
+
 
+
|- valign="top"
+
| '''--mockEmailTo'''
+
| <email>
+
| ''no value''
+
| Becomes the receiver of the mock-emails sent from the aggregator. If not set, no mock emails are sent.
+
 
+
|- valign="top"
+
| '''--mockEmailCC'''
+
| <email>
+
| ''no value''
+
| Becomes the CC receiver of the mock-emails sent from the aggregator. If not set, no mock CC address is used.
+
 
+
|- valign="top"
+
| '''--logURL'''
+
| <url>
+
| N/A
+
| The URL that will be pasted into the emails. Should normally point to the a public URL for output log for the aggregator so that the receiver can browse the log for details on failures.
+
 
+
|- valign="top"
+
| '''--logLevel'''
+
| DEBUG
+
 
+
INFO
+
 
+
WARNING
+
 
+
ERROR
+
| INFO
+
| Controls the verbosity of the console trace output. Defaults to global b3 settings.
+
 
+
|- valign="top"
+
| '''--eclipseLogLevel'''
+
| DEBUG
+
 
+
INFO
+
 
+
WARNING
+
 
+
ERROR
+
| INFO
+
| Controls the verbosity of the eclipse log trace output. Defaults to global b3 settings.
+
 
+
|- valign="top"
+
| '''--subjectPrefix'''
+
| <string>
+
| ?
+
| The prefix to use for the subject when sending emails. Defaults to the label defined in the aggregation definition. The subject is formatted as: "[<subjectPrefix>] Failed for build <buildId>"
+
 
+
|- valign="top"
+
| '''--smtpHost'''
+
| <host name>
+
| ''localhost''
+
| The SMTP host to talk to when sending emails. Defaults to "localhost".
+
 
+
|- valign="top"
+
| '''--smtpPort'''
+
| <port number>
+
| ''25''
+
| The SMTP port number to use when talking to the SMTP host. Default is 25.
+
 
+
|- valign="top"
+
| '''--packedStrategy'''
+
| COPY
+
 
+
VERIFY
+
 
+
UNPACK_AS_SIBLING
+
 
+
UNPACK
+
 
+
SKIP
+
| ''value from the model''
+
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''
+
Controls how mirroring is done of packed artifacts found in the source repository. Defaults to the setting in the aggregation definition.
+
 
+
|- valign="top"
+
| '''--trustedContributions'''
+
| <comma separated list>
+
| ''no value''
+
| ''Deprecated (Use only with on-the-fly transformation from deprecated build models)''
+
A comma separated list of contributions with repositories that will be referenced directly (through a composite repository) rather than mirrored into the final repository (even if the repository is set to mirror artifacts by default).
+
 
+
|}
+
 
+
==Hudson integration==
+
Currently there is no support for Hudson.
+
 
+
=Aggregator model components and specific actions=
+
This section provides an in-depth description and reference of the Aggregator model, listing all model components, properties and available actions.
+
 
+
==Global actions==
+
The following aggregator-specific actions are available via the context menu that can be invoked on any node in the Aggregator model editor:
+
*'''Clean Repository''' - cleans all traces of previous aggregations in the specified target location (''Build Root'')
+
*'''Verify Repository''' - verifies model validity and resolves dependencies; no artifacts are copied or created
+
*'''Build Repository''' - performs the aggregation and creates the aggregated repository in the target location (''Build Root'')
+
*'''Clean then Build Repository''' - performs a ''Clean'' followed by a ''Build''
+
 
+
==Aggregator==
+
The root node of any aggregation model is the Aggregator node. It specifies a number of global properties including the ''Build Root'' (the target location of the aggregated repository) as well as the repo structure (maven-conformant or classic p2 setup).
+
There are several child components some of which can be reference in other parts of the model: [[#Configuration|Configuration]], [[#Contribution|Contribution]], [[#Contact|Contact]], [[#Custom Category|Custom Category]], [[#Validation Repository|Validation Repository]], and [[#Maven Mapping|Maven Mapping]].
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Buildmaster
+
| <[[#Contact|Contact]]>?
+
| -
+
| Specifies an optional build master that will be notified of progress and problems if sending of mail is enabled. This is a reference to any [[#Contact|Contact]] that has been added to this Aggregator model.
+
 
+
|- valign="top"
+
|Build Root
+
| <urn>
+
| -
+
| This is a required property specifying the target location of the aggregated repository.
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| An optional description of the aggregated repository that will be shown when accessing the aggregated repository via the p2 update manager.
+
 
+
|- valign="top"
+
|Label
+
| <string>
+
| -
+
| A required label that will be used for the aggregated repository. This will be shown as the repo label when accessing the aggregated repository via the p2 update manager.
+
 
+
|- valign="top"
+
|Maven Mappings
+
| <[[#Maven Mapping|Maven Mapping]]>*
+
| -
+
| References [[#Maven Mapping|Maven Mapping]]s added as children to the Aggregator model root node. See [[#Maven Mapping|Maven Mapping]] for details.
+
 
+
|- valign="top"
+
|Maven Result
+
| '''true'''
+
'''false'''
+
| false
+
| Controls the output structure of the aggregated repo. If '''true''', the aggregated repo will be Maven-conformant. Both the structure and meta-data of the aggregated repository will follow the conventions required by Maven.
+
NOTE that due to the flexibility of p2 (separation of meta-data about dependencies and location of artifacts) the aggregated repo will at the same time also function as a valid p2 repository.
+
 
+
|- valign="top"
+
|Packed Strategy
+
| '''Copy'''
+
'''Verify'''
+
 
+
'''Unpack as Sibling'''
+
 
+
'''Unpack'''
+
 
+
'''Skip'''
+
 
+
| Copy
+
| This property controls how packed artifacts found in contributed repositories are handled when building the Aggregation:
+
*'''Copy''' - if the source contains packed artifacts, copy and store them verbatim. No further action
+
*'''Verify''' - same as ''copy'' but unpack the artifact and then discard the result
+
*'''Unpack as Sibling''' - same as ''copy'' but unpack the artifact and store the result as a sibling
+
*'''Unpack''' - use the packed artifact for data transfer but store it in unpacked form
+
*'''Skip''' - do not consider packed artifacts. This option will require unpacked artifacts in the source
+
 
+
|- valign="top"
+
|Sendmail
+
| '''false'''
+
'''true'''
+
| false
+
| Controls whether or not emails are sent when errors are detected during the aggregation process. A value of <code>false</code> disables sending of emails (including mock emails).
+
 
+
|- valign="top"
+
|Type
+
| '''S'''
+
'''I'''
+
 
+
'''N'''
+
 
+
'''M'''
+
 
+
'''C'''
+
 
+
'''R'''
+
| S
+
| Indicates the Aggregation type. This is an annotation merely for the benefit of the build master. It is not visible in the resulting repo.
+
*'''S''' - stable
+
*'''I''' - integration
+
*'''N''' - nightly
+
*'''M''' - milestone
+
*'''C''' - continuous
+
*'''R''' - release
+
 
+
|}
+
 
+
===Configuration===
+
An Aggregation may have one or more Configuration definitions. The aggregated repo will be verified for all added configurations. If dependencies for any of the given configurations fails the aggregation as a whole fails. It is however possible to specify exceptions for individual [[#Contribution|Contribution]]s.
+
 
+
A Configuration is a combination of the following properties:
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Architecture
+
|'''X86'''
+
 
+
'''PPC'''
+
 
+
'''X86_64'''
+
 
+
'''IA64'''
+
| -
+
|Specifies the architecture for which this configuration should be verified.
+
 
+
|- valign="top"
+
|Enabled
+
| '''true'''
+
'''false'''
+
| true
+
| Disables (false) or enables (true) the configuration for the aggregation process.
+
 
+
|- valign="top"
+
|Operating System
+
|'''Win32'''
+
 
+
'''Linux'''
+
 
+
'''MacOSX'''
+
| -
+
|Specifies the operating system for which this configuration should be verified.
+
 
+
|- valign="top"
+
|Window System
+
|'''Win32'''
+
 
+
'''GTK'''
+
 
+
'''Carbon'''
+
 
+
'''cocoa'''
+
| -
+
|Specifies the windowing system for which this configuration should be verified.
+
 
+
|}
+
 
+
===Contribution===
+
Contributions are the key element of any aggregation. Contributions specify which repositories (or parts thereof (category, feature, product, IU)) to include, and the constraints controlling their inclusion in the aggregated repository.
+
A contribution definition may consist of several [[#Mapped Repository|Mapped Repository]] and [[#Maven Mapping|Maven Mapping]] components.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Contacts
+
| '''<[[#Contact|Contact]]>'''*
+
| -
+
| Zero or more references to [[#Contact|Contact]]s defined in the given Aggregator model. The referenced contacts will be notified if aggregation errors related to the contribution as a whole occur.
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| Optional description of the contribution. This is for documentation purposes only and will not end up in the aggregated repository.
+
 
+
|- valign="top"
+
|Enabled
+
| '''true'''
+
'''false'''
+
| true
+
| Disables (false) or enables (true) the contribution to be considered in the aggregation process. Note that this may lead to missing dependencies and hence verification errors.
+
 
+
|- valign="top"
+
|Label
+
| <string>
+
| -
+
| Mandatory label for this contribution.
+
 
+
|- valign="top"
+
|Maven Mappings
+
| '''<[[#Maven Mapping|Maven Mapping]]>'''*
+
| -
+
| See [[#Maven Mapping|Maven Mapping]] for details ...
+
 
+
|}
+
 
+
 
+
====Mapped Repository====
+
A [[#Contribution|Contribution]] may define several Mapped Repositories defining the actual content of the contribution. The Aggregator provides fine-grained control over the contribution from each Mapped Repository through references to [[#Product|Product]]s, [[#Bundle|Bundle]]s, [[#Feature|Feature]]s, [[#Category|Categories]], [[#Exclusion Rule|Exclusion Rule]]s, [[#Valid Configuration Rule|Valid Configuration Rule]]s.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Category Prefix
+
| <string>
+
| -
+
| A prefix added to the label of this repository when displayed in the p2 update manager. In the absence of custom categories this allows a useful grouping of repositories in an aggregated repository to be specified.
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| Description of the Mapped Repository. For documentation purposes only.
+
 
+
|- valign="top"
+
|Enabled
+
| '''true'''
+
'''false'''
+
| true
+
| Controls whether a Mapped Repository is considered as part of the Contribution. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Location
+
| <URL>
+
|
+
| This (required property) specifies the location of the repository that should be added to the enclosing Contribution.
+
 
+
|- valign="top"
+
|Mirror Artifacts
+
| '''true'''
+
'''false'''
+
| true
+
| Controls whether the contents (artifacts) of the specified repository will be copied to the target location of the aggregated repository.
+
 
+
|- valign="top"
+
|Nature
+
| <nature>
+
| p2
+
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.
+
 
+
|}
+
 
+
 
+
=====Product=====
+
Defining Product components allows the addition of individual Eclipse products to the aggregation to be specified (as opposed to mapping the complete contents of a given [[#Mapped Repository|Mapped Repository]]. This naturally requires that products are present in the repositories being mapped.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <description>
+
| -
+
| Optional description of the mapping.
+
 
+
|- valign="top"
+
|Enabled
+
| '''true'''
+
'''false'''
+
| true
+
| Controls whether a Product is considered as part of the Contribution. Setting this property to <code>false</code> excludes the product from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Name
+
| <product IU's id>
+
| -
+
| The id of the product to be included in the aggregation. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Valid Configurations
+
| <[[#Configuration|Configuration]]>'''*'''
+
| -
+
| References to zero or more configurations for which this product should be verified. If no references are given the product is verified for all [[#Configuration|Configuration]]s defined for the aggregation.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.
+
 
+
|}
+
 
+
=====Category=====
+
Defining Category components allows the addition of the content in specific categories (provided by the contributed repository) rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]].
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <description>
+
| ''no value''
+
| Optional description of the mapping.
+
 
+
|- valign="top"
+
|Enabled
+
|'''true'''
+
'''false'''
+
| true
+
| Controls whether a repository Category is considered as part of the Contribution. Setting this property to <code>false</code> excludes the category from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Name
+
| <category IU id>
+
| -
+
| The id of the category to be included in the aggregation. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Label Override
+
| <string>
+
| -
+
| New category name used instead of the default name.
+
 
+
|- valign="top"
+
|Valid Configurations
+
| <[[#Configuration|Configuration]]>*
+
| -
+
| References to zero or more configurations for which the category contents should be verified. If no references are given the category is verified for all [[#Configuration|Configuration]]s defined for the aggregation.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.
+
 
+
|}
+
 
+
=====Bundle=====
+
Defining Bundle components allows addition of individual Eclipse bundles to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]).
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <description>
+
| ''no value''
+
| Optional description of the mapping.
+
 
+
|- valign="top"
+
|Enabled
+
|'''true'''
+
'''false'''
+
| true
+
| Controls whether a referenced bundle is considered as part of the Contribution. Setting this property to <code>false</code> excludes the bundle from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Name
+
| <bundle IU id>
+
| -
+
| The id of the bundle to be included in the aggregation. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Valid Configurations
+
|<[[#Configuration|Configuration]]>*
+
| -
+
| References to zero or more configurations for which the referenced bundle has to be verified. If no references are given the bundle has to be verified for all [[#Configuration|Configuration]]s defined for the aggregation.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.
+
 
+
|}
+
 
+
=====Feature=====
+
Defining Feature components allows the addition of individual Eclipse features to the aggregation to be specified (rather than the complete contents of a given [[#Mapped Repository|Mapped Repository]]). The  features to include must be present in the mapped repository.
+
 
+
Furthermore, this component provides the means to group features implicitly into [[#Custom Category|Custom Categories]].
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Categories
+
| <[[#Custom Category|Custom Category]]>*
+
| -
+
| Optionally references the [[#Custom Category|Custom Category]] definitions into which the feature should be placed upon aggregation. The relationship to the custom category is bi-directional so adding the feature to a custom category will update this property automatically in the [[#Custom Category|Custom Category]] definition, and vice versa.
+
 
+
|- valign="top"
+
|Description
+
| <description>
+
| ''no value''
+
| Optional description of the mapping.
+
 
+
|- valign="top"
+
|Enabled
+
|'''true'''
+
'''false'''
+
| true
+
| Controls whether a referenced feature is considered as part of the Contribution. Setting this property to <code>false</code> excludes the feature from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Name
+
| <feature IU id>
+
| -
+
| The id of the feature to be included in the aggregation. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Valid Configurations
+
| <[[#Configuration|Configuration]]>*
+
| -
+
| References to zero or more configurations for which the referenced feature should be verified. If no references are given the feature is verified for all [[#Configuration|Configuration]]s defined for the aggregation.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies acceptable versions for this installable unit. A pop-up editor is available.
+
 
+
|}
+
 
+
=====Exclusion Rule=====
+
The Exclusion Rules provides an alternative way to control the content of the aggregated repository. An exclusion rule may specify exclusion of any bundle, feature or product. The excluded IU will not be considered in the aggregation and verification process. Each exclusion rule can only reference one IU id.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| Description for documentation purposes.
+
 
+
|- valign="top"
+
|Name
+
| <IU id>
+
| -
+
| The id of the installable unit to be excluded from the aggregation. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies versions of this IU to be excluded. A pop-up editor is available.
+
 
+
|}
+
 
+
=====Valid Configuration Rule=====
+
By default all contributed contents of a [[#Mapped Repository|Mapped Repository]] will be verified for all [[#Configuration|Configuration]]s defined for the aggregation. A [[#Valid_Configuration_Rule|Valid Configuration Rule]] provides more control over validation. When using a Valid Configuration Rule, the referenced IUs (product, feature, or bundle) will only be verified and aggregated for the configurations specified in the rule. The rule only applies if the whole repository is mapped (i.e. when no explicit features, products, bundles or categories are mapped, regardless if they are enabled or disabled).
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| Description for documentation purposes.
+
 
+
|- valign="top"
+
|Name
+
| <IU id>
+
| -
+
| The id of the IU to be included in the verification process. A drop-down of available names is provided.
+
 
+
|- valign="top"
+
|Valid Configurations
+
| <[[#Configuration|Configuration]]>*
+
| -
+
| References to one or more configurations for which the referenced IU should be verified. This implicitly excludes verification and aggregation for all other [[#Configuration|Configuration]]s defined as part of the aggregation model.
+
 
+
|- valign="top"
+
|Version Range
+
| <version range>
+
| 0.0.0
+
| A version range that specifies versions of this installable unit for which the rule should apply. A pop-up editor is available.
+
 
+
|}
+
 
+
====Maven Mapping (Contribution)====
+
See [[#Maven Mapping|Maven Mapping]] for a detailed description and list of properties.
+
 
+
===Contact===
+
Defines a resuseable contact element which can be referenced in other parts of the model and may be used to send notifications about the aggregation process.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Email
+
| <email>
+
| -
+
| The email to be used when notifying the contact.
+
 
+
|- valign="top"
+
|Name
+
| <string>
+
| -
+
| Full name of the contact. May be displayed as label when referenced in other parts of the model.
+
 
+
|}
+
 
+
===Custom Category===
+
A Custom Category provides a grouping mechanism for features in the aggregated repository. A custom category can be referenced by [[#Feature|Feature]]s defined for inclusion from a [[#Mapped Repository|Mapped Repository]].
+
The relationship to between Custom Category and a [[#Feature|Feature]] is bi-directional. Thus,  adding the feature to a custom category will update this property automatically in the [[#Feature|Feature]] definition, and vice versa.
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Description
+
| <string>
+
| -
+
| Description of the category as displayed when accessing the aggregated repository.
+
 
+
|- valign="top"
+
|Features
+
| <[[#Feature|Feature]]>*
+
| -
+
| [[#Feature|Feature]]s included in this category by reference.
+
 
+
|- valign="top"
+
|Identifier
+
| <string>
+
| -
+
| Category id. Required Eclipse category property.
+
 
+
|- valign="top"
+
|Label
+
| <string>
+
| -
+
| Label displayed for the category when accessing the aggregated repository via the p2 update manager.
+
 
+
|}
+
 
+
===Validation Repository===
+
A Validation Repository is used to define that a repository should be used when validating dependencies for the aggregation but whose contents should not be included in the aggregated repository.
+
It supports the cases where the objective is to create a repository that is not self sufficient, but rather a complement to something else (typical use case is an aggregation of everything produced by an organization with validation against the main Eclipse repository).
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Enabled
+
| '''true'''
+
'''false'''
+
| true
+
| Controls whether a Validation Repository is considered during the verification and aggregation process. Setting this property to <code>false</code> excludes the repository from the verification and aggregation process.
+
 
+
|- valign="top"
+
|Location
+
| <URL>
+
|
+
| Specifies the location of the repository.
+
 
+
|- valign="top"
+
|Nature
+
| <nature>
+
| p2
+
| This specifies the nature of the repository, i.e. which loader should be used for loading the repository. The number of available repository loaders depends on installed extensions. By default, '''p2''' and '''maven2''' loaders are present in the installation.
+
 
+
|}
+
 
+
===Maven Mapping===
+
The Aggregator supports the creation of Maven-conformant repositories. A Maven repository requires a structure and use of naming conventions that may have to be achieved by a transformation of the Bundle-SymbolicName (BSN) when working with Eclipse bundles. There is a default translation from BSN naming standard to Maven naming. If that is not satisfactory,
+
custom transformations are supported by the definition of one or more Maven Mappings which can be defined at the [[#Aggregator|Aggregator]] and the [[#Contribution|Contribution]] level.
+
 
+
This only applies when the ''Maven Result'' property of the [[#Aggregator|Aggregator]] model is set to true. In that case all defined Maven Mappings are applied in the order in which they appear in the model starting from the most specific to the most generic. That means for each artifact that a [[#Contribution|Contribution]] adds to the aggregated repository:
+
#first Maven Mappings defined as children of a [[#Contribution|Contribution]] are applied in the order in which they appear as children of the parent [[#Contribution|Contribution]] node
+
#second Maven Mappings defined as children of the [[#Aggregator|Aggregator]] model are applied in the order in which they appear as children of the parent [[#Aggregator|Aggregator]] node
+
#finally the default Maven Mapping is applied
+
 
+
The most generic mapping is a default pattern that is applied whenever a Maven is created. It does not need to be added explicitly to the model.
+
A mapping is specified using a regular expression that is applied to each BSN. The regular expression must specify two replacements; one for the maven groupId, and one for the maven artifactId. The group and artifact ids  have an effect on the resulting Maven repo structure. The default pattern is:
+
 
+
<pre>
+
^([^.]+(?:\.[^.]+(?:\.[^.]+)?)?)(?:\.[^.]+)*$
+
</pre>
+
 
+
The default maven mapping use the replacement <code>'''$1'''</code> for groupId and <code>'''$0'''</code> for artifactId. Hence, when applying the default maven mapping regular expression to a BSN, up to 3 segments (with dots as segment delimiters) are taken as the group id, and the entire BSN is taken as the artifact name. If this is a applied to something like <code>org.eclipse.b3.aggregator</code> the groupId would be <code>org.eclipse.b3</code> and the artifactId <code>org.eclipse.b3.aggregator</code>. The resulting Maven repo will have the folder structure:
+
 
+
<pre>
+
org
+
  |
+
  eclipse
+
      |
+
      b3
+
        |
+
        org.eclipse.b3.aggregator
+
            |
+
            <folder name after version>
+
              |
+
              org.eclipse.b3.aggregator-<version>.jar
+
              ...
+
</pre>
+
 
+
 
+
 
+
{| {{Greytable}}
+
|-
+
!Property
+
!Value(s)
+
!Default Value
+
!Comment
+
 
+
|- valign="top"
+
|Name Pattern
+
| regex
+
| -
+
| A regular expression which is applied to the Bundle-SymbolicName (BSN). Pattern groups may be referenced in the replacement properties.
+
 
+
|- valign="top"
+
|Group Id
+
| <string> (may contain pattern group references (i.e. $1, ...))
+
| -
+
| Group Id applied in a maven mapping structure (groupId/artifactId). The Group Id replacement may contain pattern group references (i.e. $1, ...).
+
 
+
|- valign="top"
+
|Artifact Id
+
| <string> (may contain pattern group references (i.e. $1, ...))
+
| -
+
| Artifact Id applied in a maven mapping structure (groupId/artifactId). The Artifact Id replacement may contain pattern group references (i.e. $1, ...).
+
 
+
|}
+
 
+
=What else should be documented=
+
 
+
# version editor (version formats...)
+
# how enable/disable on the context menu works
+
# drag&drop support
+
# 'available versions' tree
+
# context menu actions (mapping IUs from repository view, searching for IUs from 'available version' tree...)
+
# legacy format support (transformation wizard in the UI, on-the-fly transformation in the headless mode)
+
 
+
==Questions==
+
* How is blame-mail handled when running in the UI? Are the options "production" etc. available then?
+
* How do you know what the "log output url" is? How can it be set?
+
* Why is there a section that says "there is no hudson support" - is hudson support needed/required in any way??
+
* Some configurations seems to be missing - or is the list open ended? (Sparc, Motif, etc..) - I assume user can put anything there?
+
* It seems that it is possible to load maven repositories. I suppose the result of aggregation is a valid p2 repository (or a combination of p2/maven)??
+
 
+
[[Category:Eclipse b3]]
+

Latest revision as of 15:01, 7 November 2018

Redirect to:

Back to the top