Difference between revisions of "Common Build Infrastructure/Defining Binary Dependencies"

From Eclipsepedia

Jump to: navigation, search
(Update Sites)
 
(11 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
{{:DocumentationGuidelines/DraftHeader}}
 +
 
There are multiple ways to define your dependencies when doing an Athena build. Below are examples which can be used in your build.properties file.
 
There are multiple ways to define your dependencies when doing an Athena build. Below are examples which can be used in your build.properties file.
  
Line 17: Line 19:
  
 
=Update Sites=
 
=Update Sites=
 +
 +
In the past, you needed to specify your platform's SDK in order to build. Now, you can provision the entire build and test runtime environment using p2 repos so that your build can be run on multiple platforms without having to change the input dependencies. You will still have to set a different path for JAVA_HOME and other filesystem constants in <code>build.properties</code>, however.
 +
 +
You can install a mix of features, plugins, and feature.groups like this:
 +
 +
repositoryURLs=http://path/to/eclipse-Update-R-3.5.1-200909170800.zip
 +
IUsToInstall=org.eclipse.sdk.feature.group+org.eclipse.sdk.ide
 +
 +
:''In the above example, Eclipse 3.5+ is required to install both IUs in a single operation from a remote update site zip using the "+" operator. If the remote update site zip is missing its p2 metadata, it will be regenerated from an included site.xml in the zip, or else fail. Some of zipped eclipse SDK update sites can be found under http://download.eclipse.org/athena/repos/''
 +
 +
repositoryURLs=jar:file:/tmp/build/downloads/eclipse-Update-R-3.5.1-200909170800.zip!/
 +
IUsToInstall=org.eclipse.sdk.feature.group,org.eclipse.sdk.ide
 +
 +
:''In the above example, Eclipse 3.4+ is required to install each IUs in separate operations from a local update site zip using the "," operator.''
 +
 +
repositoryURLs=file:/tmp/build/downloads/eclipse-Update-R-3.5.1-200909170800/
 +
featureIDsToInstall=org.eclipse.sdk
 +
pluginIDsToInstall=org.eclipse.sdk.ide
 +
 +
:''In the above example, Eclipse 3.4+ is required to install each IUs in separate operations from a local update site.''
 +
 +
More examples follow below.
  
 
==Remote Site==
 
==Remote Site==
 +
 +
===Eclipse 3.4 (Ganymede) ===
  
 
Provisioning directly from an update site may fail due to socket timeouts; however, this method can work when the network is cooperative.
 
Provisioning directly from an update site may fail due to socket timeouts; however, this method can work when the network is cooperative.
Line 37: Line 63:
 
  featureIDsToInstall=org.eclipse.emf,org.eclipse.birt,org.eclipse.cdt
 
  featureIDsToInstall=org.eclipse.emf,org.eclipse.birt,org.eclipse.cdt
  
With Eclipse 3.5M7+, you can do multiple provisioning installs in a single p2 operation. Note the use of "+" to define which feature.groups to install together. You can still use commas to define sequential operations.
+
===Eclipse 3.5 (Galileo) ===
 +
 
 +
With Eclipse 3.5, you can do multiple provisioning installs in a single p2 operation. Note the use of "+" to define which feature.groups to install together. You can still use commas to define sequential operations.
  
 
  dependencyURLs=\
 
  dependencyURLs=\
   http://download.eclipse.org/eclipse/downloads/drops/S-3.5M7-200904302300/eclipse-SDK-3.5M7-linux-gtk.tar.gz
+
   http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/eclipse-SDK-3.5-linux-gtk.tar.gz
 
  repositoryURLs=http://download.eclipse.org/releases/galileo
 
  repositoryURLs=http://download.eclipse.org/releases/galileo
 
  featureIDsToInstall=org.eclipse.emf+org.eclipse.birt+org.eclipse.cdt
 
  featureIDsToInstall=org.eclipse.emf+org.eclipse.birt+org.eclipse.cdt
 +
 +
You can also install plugins from, for example, an Orbit update site ('''MUST BE A p2 REPO''', not simply a "Classic" update site):
 +
 +
dependencyURLs=\
 +
  http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/eclipse-SDK-3.5-linux-gtk.tar.gz
 +
repositoryURLs=http://download.eclipse.org/tools/orbit/downloads/drops/R20090825191606/updateSite
 +
pluginIDsToInstall=org.apache.xml.resolver+javax.xml+org.apache.xml.serializer+org.apache.xerces+org.apache.xalan
  
 
==Remote Zipped Site==
 
==Remote Zipped Site==
Line 54: Line 89:
 
   http://www.eclipse.org/downloads/download.php?r=1&amp;file=/tools/gef/downloads/drops/3.4.2/R200902171642/GEF-Update-3.4.2.zip
 
   http://www.eclipse.org/downloads/download.php?r=1&amp;file=/tools/gef/downloads/drops/3.4.2/R200902171642/GEF-Update-3.4.2.zip
 
  featureIDsToInstall=org.eclipse.emf,org.eclipse.gef
 
  featureIDsToInstall=org.eclipse.emf,org.eclipse.gef
 +
 +
=Snapshot Dependencies=
 +
 +
In most builds, the resulting zips contain the timestamp of the build. As such, each build will produce a uniquely named set of zips. This is perfect for ensuring you know what's in the zip, and how old it is, but not as useful for creating cascades of builds which depend on the output of others.
 +
 +
https://.../cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/build/N200909180843/m2t-xpand-Update-incubation-N200909180843.zip
 +
 +
So, to solve this problem, you can build and consume snapshot zips. These are identical to the output of a normal build except that they are named more generically so that as they are replaced hourly or daily, their NAME does not change.
 +
 +
https://.../cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/snapshot/m2t-xpand-SDK-incubation-N-Snapshot.zip
 +
 +
==Creating Snapshots==
 +
 +
To provide others with SNAPSHOT builds (usually nightlies), set the following property when running your Athena build (via Hudson or commandline):
 +
 +
export SNAPSHOT="true"
 +
. /opt/public/cbi/build/org.eclipse.dash.common.releng/hudson/run.sh
 +
 +
==Using Snapshots ==
 +
 +
To consume others' SNAPSHOT builds, explore their Hudson jobs to find their "Last Successful Build" then "Build Artifacts". Browse their workspace for a generically named zip such as this one:
 +
 +
https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/snapshot/m2t-xpand-SDK-incubation-N-Snapshot.zip
 +
 +
Then, add this zip to your <code>build.properties</code> file via the <code>dependencyURLs</code> or <code>repositoryURLs</code> as appropriate, depending on whether the snapshot zip contains an SDK/runtime or p2 repository (update site).
  
 
= More information =
 
= More information =
Line 64: Line 124:
 
[[Category:Athena Common Build]]
 
[[Category:Athena Common Build]]
 
[[Category:Releng]]
 
[[Category:Releng]]
 +
[[Category:Draft_Documentation]]

Latest revision as of 15:02, 2 December 2009

Warning2.png
Draft Content
This page is currently under construction. Community members are encouraged to maintain the page, and make sure the information is accurate.


There are multiple ways to define your dependencies when doing an Athena build. Below are examples which can be used in your build.properties file.

Note that instead of http:// or ftp:// paths, you can also specify /path/to/some.zip. Once a zip has been fetched from remote, it will be cached in your downloadsDir for reuse - but you need not reference it by a local path since the build system will check the downloadsDir for a matching zip before attempting to re-download it from remote.

By default, downloadsDir will be /tmp/build/downloads/ (c:\tmp\build\downloads\ on Windows); to set a different location, use the downloadsDir property in your build.properties to point to an alternate path.

Contents

[edit] SDK or Runtime Zips

By default, everything listed in dependencyURLs will be installed automatically. Should you want to override this behaviour, you can define your own buildExtra.xml or testExtra.xml and inject custom steps for unpacking dependencyURLs. Zips containing "master" or "update" in their name will be treated as update sites; all others will be unpacked into eclipse/dropins/.

# CDT master zip is an update site zip w/ p2 metadata and site.xml
dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.4.2-200902111700/eclipse-SDK-3.4.2-linux-gtk-ppc.tar.gz,\
 http://download.eclipse.org/tools/cdt/releases/ganymede/dist/cdt-master-5.0.2.zip,\
 http://download.eclipse.org/birt/downloads/drops/R-R1-2_3_2_1-200904141342/birt-report-framework-2_3_2_1.zip,\
 http://download.eclipse.org/modeling/emf/emf/downloads/drops/2.4.2/R200902171115/emf-runtime-2.4.2.zip

[edit] Update Sites

In the past, you needed to specify your platform's SDK in order to build. Now, you can provision the entire build and test runtime environment using p2 repos so that your build can be run on multiple platforms without having to change the input dependencies. You will still have to set a different path for JAVA_HOME and other filesystem constants in build.properties, however.

You can install a mix of features, plugins, and feature.groups like this:

repositoryURLs=http://path/to/eclipse-Update-R-3.5.1-200909170800.zip
IUsToInstall=org.eclipse.sdk.feature.group+org.eclipse.sdk.ide
In the above example, Eclipse 3.5+ is required to install both IUs in a single operation from a remote update site zip using the "+" operator. If the remote update site zip is missing its p2 metadata, it will be regenerated from an included site.xml in the zip, or else fail. Some of zipped eclipse SDK update sites can be found under http://download.eclipse.org/athena/repos/
repositoryURLs=jar:file:/tmp/build/downloads/eclipse-Update-R-3.5.1-200909170800.zip!/
IUsToInstall=org.eclipse.sdk.feature.group,org.eclipse.sdk.ide
In the above example, Eclipse 3.4+ is required to install each IUs in separate operations from a local update site zip using the "," operator.
repositoryURLs=file:/tmp/build/downloads/eclipse-Update-R-3.5.1-200909170800/
featureIDsToInstall=org.eclipse.sdk
pluginIDsToInstall=org.eclipse.sdk.ide
In the above example, Eclipse 3.4+ is required to install each IUs in separate operations from a local update site.

More examples follow below.

[edit] Remote Site

[edit] Eclipse 3.4 (Ganymede)

Provisioning directly from an update site may fail due to socket timeouts; however, this method can work when the network is cooperative.

dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.4.2-200902111700/eclipse-SDK-3.4.2-linux-gtk.tar.gz
repositoryURLs=http://download.eclipse.org/releases/ganymede/
featureIDsToInstall=org.eclipse.emf,org.eclipse.birt,org.eclipse.cdt

Or, specify multiple update sites:

dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.4.2-200902111700/eclipse-SDK-3.4.2-linux-gtk.tar.gz
repositoryURLs=\
 http://download.eclipse.org/tools/cdt/updates/ganymede/,\
 http://download.eclipse.org/modeling/emf/updates/releases/,\
 http://download.eclipse.org/birt/update-site/2.3/
featureIDsToInstall=org.eclipse.emf,org.eclipse.birt,org.eclipse.cdt

[edit] Eclipse 3.5 (Galileo)

With Eclipse 3.5, you can do multiple provisioning installs in a single p2 operation. Note the use of "+" to define which feature.groups to install together. You can still use commas to define sequential operations.

dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/eclipse-SDK-3.5-linux-gtk.tar.gz
repositoryURLs=http://download.eclipse.org/releases/galileo
featureIDsToInstall=org.eclipse.emf+org.eclipse.birt+org.eclipse.cdt

You can also install plugins from, for example, an Orbit update site (MUST BE A p2 REPO, not simply a "Classic" update site):

dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/eclipse-SDK-3.5-linux-gtk.tar.gz
repositoryURLs=http://download.eclipse.org/tools/orbit/downloads/drops/R20090825191606/updateSite
pluginIDsToInstall=org.apache.xml.resolver+javax.xml+org.apache.xml.serializer+org.apache.xerces+org.apache.xalan

[edit] Remote Zipped Site

As noted above, you can use update site zips as dependencyURLs. However, it's probably better to set them as repositoryURLs since that is what they are. Using the http://www.eclipse.org/downloads/download.php?r=1&file= prefix will fetch the specified zip from the mirror deemed to be closest to your IP address, unpack it, and if necessary, generate p2 metadata from the included site.xml file.

dependencyURLs=\
 http://download.eclipse.org/eclipse/downloads/drops/R-3.4.2-200902111700/eclipse-SDK-3.4.2-linux-gtk.tar.gz
repositoryURLs=\
 http://www.eclipse.org/downloads/download.php?r=1&file=/modeling/emf/emf/downloads/drops/2.4.2/R200902171115/emf-sdo-xsd-Update-2.4.2.zip,\
 http://www.eclipse.org/downloads/download.php?r=1&file=/tools/gef/downloads/drops/3.4.2/R200902171642/GEF-Update-3.4.2.zip
featureIDsToInstall=org.eclipse.emf,org.eclipse.gef

[edit] Snapshot Dependencies

In most builds, the resulting zips contain the timestamp of the build. As such, each build will produce a uniquely named set of zips. This is perfect for ensuring you know what's in the zip, and how old it is, but not as useful for creating cascades of builds which depend on the output of others.

https://.../cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/build/N200909180843/m2t-xpand-Update-incubation-N200909180843.zip

So, to solve this problem, you can build and consume snapshot zips. These are identical to the output of a normal build except that they are named more generically so that as they are replaced hourly or daily, their NAME does not change.

https://.../cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/snapshot/m2t-xpand-SDK-incubation-N-Snapshot.zip

[edit] Creating Snapshots

To provide others with SNAPSHOT builds (usually nightlies), set the following property when running your Athena build (via Hudson or commandline):

export SNAPSHOT="true"
. /opt/public/cbi/build/org.eclipse.dash.common.releng/hudson/run.sh

[edit] Using Snapshots

To consume others' SNAPSHOT builds, explore their Hudson jobs to find their "Last Successful Build" then "Build Artifacts". Browse their workspace for a generically named zip such as this one:

https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-m2t-xpand-0.7/lastSuccessfulBuild/artifact/snapshot/m2t-xpand-SDK-incubation-N-Snapshot.zip

Then, add this zip to your build.properties file via the dependencyURLs or repositoryURLs as appropriate, depending on whether the snapshot zip contains an SDK/runtime or p2 repository (update site).

[edit] More information