Jump to: navigation, search

Difference between revisions of "Equinox p2 Repository Mirroring"

m (Getting the Mirroring Tools)
(what to invoke)
 
(14 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
 +
 +
p2 manages all of its data in repositories.  There are two types of repos, artifact and metadata.
  
 
p2 artifact repositories can hold many different forms of the same artifact. For example, a JAR artifact may exist as
 
p2 artifact repositories can hold many different forms of the same artifact. For example, a JAR artifact may exist as
Line 8: Line 10:
 
* ...  
 
* ...  
  
p2 metadata repositories are like artifact profiles that describe dependencies needed to function.  
+
p2 metadata repositories describe inter-component dependencies and identify artifacts to install.  
  
This Repository Mirroring application can be used to mirror artifact and metadata repositories. In addition, this application can perform selective mirroring where the user can specify parts of a repository and create a more specific mirror (e.g. only mirror latest code).  
+
The Repository Mirroring applications can be used to mirror artifact and metadata repositories. In addition, users can do selective mirroring of artifacts or metadata either to create a more specific mirror (e.g. only mirror latest code) or merge content into an existing mirror.  
  
This example demonstrates how to mirror a repository using the repository mirror application available in the equinox p2 tools.
+
This example demonstrates how to mirror a repository using the repository mirror applications available in the equinox p2 tools.
 
+
==Getting the Mirroring Tools==
+
Download the p2 agent from the provisioning section of: [http://download.eclipse.org/eclipse/equinox/drops/S-3.4RC3-200805301730/index.php#Provisioning Equinox Stable Build: 3.4RC3].
+
  
 
==Running the Mirroring Tools==
 
==Running the Mirroring Tools==
  
Run from the command line by running the Admin tool with the args below (e.g. eclipse.exe -application  org.eclipse.equinox.p2.tools.MirrorApplication -source ...).  
+
Currently, there are two different mirror applications: one for artifacts and one for metadata.
  
The attached example performs the basic mirroring of an artifact and metadata repository into a target repository. The target repository will be an exact mirror of the source repositories.  This means any information on the target repository prior to mirroring will be lost, and if there is no target repository to begin with then a new repository will be created.
+
===Mirroring Metadata===
  
   -console -consolelog -application org.eclipse.equinox.p2.tools.MirrorApplication
+
To make an exact mirror of a metadata repository, use the following arguments.  Note that if the target repository does not exist, a new repository is created with the same properties as the source.
 +
 
 +
   eclipse -nosplash -verbose
 +
  -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication
 
   -source Insert Source URL (e.g. <nowiki>http://download.eclipse.org/eclipse/updates/3.4milestones/</nowiki>)
 
   -source Insert Source URL (e.g. <nowiki>http://download.eclipse.org/eclipse/updates/3.4milestones/</nowiki>)
 
   -destination Insert Destination URL (e.g. file:/tmp/3.4milestonesMirror/)
 
   -destination Insert Destination URL (e.g. file:/tmp/3.4milestonesMirror/)
  -ius all
 
  -artifacts all
 
  -overwrite
 
  
If only metadata are desired in the destination then you can remove the line: '''-ius all'''
+
The application will be "eclipsec.exe" (instead of "eclipse") for Windows installations.
 +
 
 +
By adding the argument '''-writeMode clean''', all installable units in the target destination will be removed before the mirroring is performed.
 +
 
 +
Adding the argument '''-destinationName &#60;destination name&#62;''' will set the destination repository's name to the specified destination name. If this argument is not included the destination will use the source repository's name if no repository exists at the destination.
 +
 
 +
===Mirroring Artifacts===
 +
 
 +
To make an exact mirror of an artifact repository, use the following arguments.  Note that if the target repository does not exist, a new repository is created with the same properties as the source.
 +
 
 +
  eclipse -nosplash -verbose
 +
  -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication
 +
  -source Insert Source URL (e.g. <nowiki>http://download.eclipse.org/eclipse/updates/3.4milestones/</nowiki>)
 +
  -destination Insert Destination URL (e.g. file:/tmp/3.4milestonesMirror/)
 +
 
 +
Again, the application will be "eclipsec.exe" (instead of "eclipse") for Windows installations.
 +
 
 +
By adding the argument '''-writeMode clean''', all artifacts in the target destination will be removed before the mirroring is performed.
  
or if only the artifacts are desired then remove the line: '''-artifacts all'''
+
Adding the argument '''-destinationName <destination name>''' will set the destination repository's name to the specified destination name. If this argument is not included the destination will use the source repository's name if no repository exists at the destination.
  
For a compressed destination repository, add the line: '''-compressed'''
+
Adding the argument '''-verbose''' will enable verbose error reporting and logging. This will write errors to the <eclipse workspace>/.metadata/.log file.
  
and the resulting destination artifact or content xml will be in compressed jar format.
+
Adding the argument '''-ignoreErrors''' will ensure the mirror application does not fail in the event of an error. Note: while using this argument the mirror application may complete without errors but the destination repository may not include all artifacts from the source repository.
  
===Selective Mirroring===
+
Adding the argument '''-raw''' instructs the mirroring application to copy the exact artifact descriptors from source into the destination instead of initializing new artifact descriptors with properties from the source descriptors.
  
If only a piece of the source repository should be mirrored, then you can specify the individual components as arguments. For ius the format is
+
Adding the argument '''-compare''' instructs the mirroring application to perform a comparison when a duplicate artifact descriptor is found.
  
  -ius IU,IU2/Version,IU3/[VersionRange]
+
Adding the argument '''-comparator &#60;comparator ID&#62;''' specifies the mirroring application should use an Artifact Comparator with an ID of "comparator ID" to compare artifact descriptors. The mirroring application uses the "MD5 Comparator" to compare the MD5 hash property of the artifact descriptors if no comparator is defined.
  or
+
  -ius org.eclipse.sdk.examples.source,org.eclipse.ui.examples.undo.source/3.2.100,org.eclipse.jdt.ui.examples.projects.source /[3.1.0,3.2.0]
+
  
For artifacts, the format is  
+
Adding the argument '''-baseline &#60;baseline location&#62;''' will compare all artifacts in the source against a known good baseline repository specified by the baseline location. In the event of conflicting artifacts, precedence is given to the baseline repository. Can be combined with '''-comparator &#60;comparator ID&#62;'''.
  
  -artifacts {classifier,iu,version},{classifier,iu,version}
+
[[Category:Equinox_p2|Repository Mirroring]]
  or
+
  -artifacts {osgi.bundle,org.eclipse.swt.examples.browser,3.1.100.v3320c},{osgi.bundle,org.eclipse.sdk.examples.source,3.1.100.v20070605}
+

Latest revision as of 15:06, 11 March 2011

Introduction

p2 manages all of its data in repositories. There are two types of repos, artifact and metadata.

p2 artifact repositories can hold many different forms of the same artifact. For example, a JAR artifact may exist as

  • a canonical (unmolested) JAR
  • a compressed JAR (assuming the original was not compressed)
  • a pack200 JAR
  • a delta relative to another version of the artifact
  • ...

p2 metadata repositories describe inter-component dependencies and identify artifacts to install.

The Repository Mirroring applications can be used to mirror artifact and metadata repositories. In addition, users can do selective mirroring of artifacts or metadata either to create a more specific mirror (e.g. only mirror latest code) or merge content into an existing mirror.

This example demonstrates how to mirror a repository using the repository mirror applications available in the equinox p2 tools.

Running the Mirroring Tools

Currently, there are two different mirror applications: one for artifacts and one for metadata.

Mirroring Metadata

To make an exact mirror of a metadata repository, use the following arguments. Note that if the target repository does not exist, a new repository is created with the same properties as the source.

 eclipse -nosplash -verbose
 -application org.eclipse.equinox.p2.metadata.repository.mirrorApplication
 -source Insert Source URL (e.g. http://download.eclipse.org/eclipse/updates/3.4milestones/)
 -destination Insert Destination URL (e.g. file:/tmp/3.4milestonesMirror/)

The application will be "eclipsec.exe" (instead of "eclipse") for Windows installations.

By adding the argument -writeMode clean, all installable units in the target destination will be removed before the mirroring is performed.

Adding the argument -destinationName <destination name> will set the destination repository's name to the specified destination name. If this argument is not included the destination will use the source repository's name if no repository exists at the destination.

Mirroring Artifacts

To make an exact mirror of an artifact repository, use the following arguments. Note that if the target repository does not exist, a new repository is created with the same properties as the source.

 eclipse -nosplash -verbose
 -application org.eclipse.equinox.p2.artifact.repository.mirrorApplication
 -source Insert Source URL (e.g. http://download.eclipse.org/eclipse/updates/3.4milestones/)
 -destination Insert Destination URL (e.g. file:/tmp/3.4milestonesMirror/)

Again, the application will be "eclipsec.exe" (instead of "eclipse") for Windows installations.

By adding the argument -writeMode clean, all artifacts in the target destination will be removed before the mirroring is performed.

Adding the argument -destinationName <destination name> will set the destination repository's name to the specified destination name. If this argument is not included the destination will use the source repository's name if no repository exists at the destination.

Adding the argument -verbose will enable verbose error reporting and logging. This will write errors to the <eclipse workspace>/.metadata/.log file.

Adding the argument -ignoreErrors will ensure the mirror application does not fail in the event of an error. Note: while using this argument the mirror application may complete without errors but the destination repository may not include all artifacts from the source repository.

Adding the argument -raw instructs the mirroring application to copy the exact artifact descriptors from source into the destination instead of initializing new artifact descriptors with properties from the source descriptors.

Adding the argument -compare instructs the mirroring application to perform a comparison when a duplicate artifact descriptor is found.

Adding the argument -comparator <comparator ID> specifies the mirroring application should use an Artifact Comparator with an ID of "comparator ID" to compare artifact descriptors. The mirroring application uses the "MD5 Comparator" to compare the MD5 hash property of the artifact descriptors if no comparator is defined.

Adding the argument -baseline <baseline location> will compare all artifacts in the source against a known good baseline repository specified by the baseline location. In the event of conflicting artifacts, precedence is given to the baseline repository. Can be combined with -comparator <comparator ID>.