Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "CBI"

(Tutorials and other resources)
(p2 repo aggregator)
(44 intermediate revisions by 11 users not shown)
Line 1: Line 1:
The Eclipse Common Build Infrastructure (CBI) is an initiative combining technologies and practices for building Eclipse Software.
+
The Eclipse Common Build Infrastructure (CBI) is an initiative combining infrastructure, technologies and practices for building, testing and delivering Eclipse software.
  
==Initiative Goals==
+
= Goals =
 
+
Primary goals of the CBI initiative are:
===Primary===
+
 
* Make it really easy to contribute Eclipse projects
 
* Make it really easy to contribute Eclipse projects
 
** Make it really easy to copy & modify source
 
** Make it really easy to copy & modify source
Line 11: Line 10:
 
** Make it really easy to sign software
 
** Make it really easy to sign software
  
===Secondary===
+
Secondary goals are:
 
* Get all Eclipse projects building their software on Eclipse Foundation hardware.
 
* Get all Eclipse projects building their software on Eclipse Foundation hardware.
* Enable the [http://wiki.eclipse.org/EclipseLTS Long Term Support Program].
+
* Make it easy for people to build custom Eclipse distributions.
  
 +
= Asking for Help =
  
There is a strong link between CBI and the [http://wiki.eclipse.org/EclipseLTS Long Term Support Program] which enables a marketplace of companies providing maintenance and support for Eclipse technologies for durations far beyond typical community support. Please NOTE: CBI features will be available to community.
+
* Need help actually building your code: ask your project mentors, or ask on the Common Build mailing list (cbi-dev). There are no dumb questions.
 +
* Subscribe to cbi-dev here: https://dev.eclipse.org/mailman/listinfo/cbi-dev
  
It is our hope that this project develops an offering that is compelling so that many projects will move to use it.
+
= Service Level Agreement (SLA) =
  
==Next meeting==
+
Most CBI services are Tier 2 - Best Effort, which means they are expected to be available at all times, and rapid restoration can be expected in the event of an outage.  Eclipse Strategic Members can contact Webmaster in certain cases of off-hours support.
  
* Bi-weekly conference call, Tuesdays 10:00 to 11:00 Eastern timezone. [http://wiki.eclipse.org/CBI/Conference Conference bridge details]. Contact andrew dot ross at eclipse dot org if you would like to be added to the Calendar reminder. The dates for each month are as follows:
+
Please see [[IT_SLA]] for more information on the Eclipse Foundation IT Services SLA.
** July 2012: 10th & 24th
+
** August 2012: 7th & 21st
+
* Code Sprint #2 - in July, 2012
+
  
 +
= Preferred Build Technologies =
  
 +
== CI Environment (Jenkins) ==
  
==Resources==
+
We provide dedicated Jenkins instances to projects. See also [[Jenkins]].
* mailing list [https://dev.eclipse.org/mailman/listinfo/cbi-dev cbi-dev]
+
  
===Bugs===
+
Jenkins is a continuous integration (CI) server used on Eclipse servers for Eclipse projects as part of the [[CBI|Common Build Infrastructure (CBI)]]. Jenkins instances are maintained by the Eclipse Webmasters/Release Engineer.
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap;product=CBI;version=1.0;list_id=2249872 CBI 1.0]
+
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap;product=CBI;version=2.0;list_id=2249872 CBI 2.0]
+
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap&product=CBI&list_id=38248 List of All Bugs] (Product = CBI)
+
  
==Tutorials, News, and other resources==
+
* List of Jenkins Instances Per Project (JIPP):
* [http://www.vogella.com/articles/EclipseTycho/article.html Tycho tutorial by Lars Vogel]
+
** https://ci.eclipse.org/ (standalone)
* [http://www.fosslc.org/drupal/content/tycho-good-bad-and-ugly Video discussing JBoss tools use of Tycho]
+
** https://jenkins.eclipse.org/ (Kubernetes cluster)
* [http://wiki.eclipse.org/CBI/Workshops Workshops being developed]
+
* [http://www.vogella.com/blog/2012/10/08/building-eclipse-sdk-locally-with-maven/ Building Eclipse SDK locally with Maven]
+
  
==Eclipse platform CBI build==
+
NOTE: JIPP instances are being migrated from a standalone implementation to a Kubernetes cluster implementation.
* [[CBI/Eclipse Platform Build]] see the [[CBI/Eclipse Platform Build Roadmap]]
+
  
==Preferred Build Technologies==
+
=== Requesting a JIPP instance ===
  
===Hudson===
+
Please [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins&short_desc=JIPP%20Request%20for%20project%20XXX file a bug] against Eclipse Foundation > Community > CI-Jenkins to request your project's own instance. Please ensure your project lead can +1 the request. Please specify if you wish to grant write access to your download area and/or code repositories.
  
* The Eclipse [http://hudson.eclipse.org Hudson instance]; and
+
{{important|Security issues| There may be security issues related to using the Gerrit plugin, more especially when allowing the CI system to write directly to the code repos and downloads area. If you request plugins other than those installed by default, webmasters may not be able to help troubleshoot any issues that you may encounter with your instance.}}
* Hudson projects using [https://hudson.eclipse.org/hudson/view/Tycho%20+%20Maven/ Maven and Tycho].
+
  
===Maven===
+
=== What's provided? ===
  
Maven 3.0 drives the builds. Projects are expected to provide standard Maven 3.0 POM files for their builds. The builds should be built in such a way that they can be run on the local workstation, or on the Eclipse build server. Note that builds can only be signed on the Eclipse build server.
+
Each Eclipse Project has access to one Jenkins instance (JIPP), including the following:
 +
 
 +
* (1) Jenkins instance, with (1) Resources Pack (see below)
 +
** Membership-sponsored projects may allocate more resources (see below)
 +
* Digital signing Service: Java JAR, Java Cryptography Extensions, Windows Portable Executable with Microsoft Authenticode,  macOS application bundles.
 +
* Packaging service: Apple Disk Image (.dmg), Linux Flatpak
 +
* Disk space: Ephemeral for builds, permanent for release builds.
 +
* (1) 3.75G/30G Linux Virtual Server (if needed) (courtesy of Microsoft Azure)
 +
* Access to worldwide download mirrors
 +
 
 +
=== Additional Resource Packs ===
 +
 
 +
Each Eclipse Project has access to one Resources Pack for building. For some projects, that may not be enough. Projects sponsored by [https://www.eclipse.org/membership/exploreMembership.php Eclipse Membership] (via Project Lead) have additional Packs, based on [https://www.eclipse.org/membership/become_a_member/membershipTypes.php membership level]. These Packs can be allocated to projects.
 +
 
 +
* Some resources are only available to [https://www.eclipse.org/membership/become_a_member/membershipTypes.php Enterprise and Strategic members].
 +
* Enterprise and Strategic members can engage with the Foundation to acquire additional Packs.
 +
 
 +
[[File:CBI Resource Packs.png|center]]
 +
 
 +
==== Resource Packs Included in Membership ====
 +
 
 +
Eclipse Foundation Member Organizations have access to additional Resource Packs, based on their [https://www.eclipse.org/membership/become_a_member/membershipTypes.php membership level].
 +
 
 +
[[File:CBI Resource Pack Assignments.png]]
 +
 
 +
 
 +
==== Assigning Resource Packs to a Project ====
 +
 
 +
Resource Packs are assigned by Eclipse Members to Eclipse Projects they sponsor (Members have a Project Lead on the Project). Packs are assigned as a whole to a single project (i.e., can’t split Packs across multiple projects). A member can assign several packs to a single project.
 +
 
 +
To assign a pack to a project, please [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins&short_desc=Assign%20Resource%20Pack%20to%20Project file a Bug].
 +
 
 +
== CI Environment (Third-party) ==
 +
 
 +
If you host your Eclipse project git repository at Github, we also support some third-party services:
 +
* ''TravisCI''. [https://travis-ci.com/ TravisCI] is free to use for open source projects, but you should be aware that it limits to 5 the number of executors per organizations. It means that only 5 builds can run concurrently in any organization. If your repositories are part of the [https://github.com/eclipse/ Eclipse] organization, you may face long delays between the trigger and the actually start of the job. If it's your case, and would like to stick to TravisCI for building, we suggest you ask to move to a [https://wiki.eclipse.org/GitHub#Project_dedicated_organization dedicated organization].
 +
 
 +
 
 +
We do '''not''' support
 +
* ''CircleCI''. https://circleci.com/
 +
 
 +
If an environment is not listed in the unsupported list, you can ask whether it can be supported by [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins opening a bug].
 +
 
 +
{{important|Third party build services (e.g., Travis) limitations|The Eclipse Foundation's IT Team is responsible for the credentials (OSSRH, SSH and GPG keys...) it creates for projects. Keeping track of where credentials are disseminated is crucial for security reasons. As long as we have no automation in place to set/update/remove credentials on 3rd party build services, we don't put any credential on such services. For instance, it means that you can't use them to publish to Maven Central, or to upload build results to download.eclipse.org.}}
 +
 
 +
==Maven==
 +
 
 +
Maven 3.x drives the builds. Projects are expected to provide standard Maven 3.x POM files for their builds. The builds should be built in such a way that they can be run on the local workstation, or on the Eclipse build server. Note that builds can only be signed on the Eclipse build server.
  
 
* [[Maven/Parent POM|Parent Maven POM]] for Eclipse projects;  
 
* [[Maven/Parent POM|Parent Maven POM]] for Eclipse projects;  
Line 61: Line 100:
 
* [[Maven|Maven repository support at Eclipse]].
 
* [[Maven|Maven repository support at Eclipse]].
  
===Tycho===
+
==Tycho==
  
 
Tycho is focused on a Maven-centric, manifest-first approach to building Eclipse plug-ins, features, update sites, RCP applications and OSGi bundles.
 
Tycho is focused on a Maven-centric, manifest-first approach to building Eclipse plug-ins, features, update sites, RCP applications and OSGi bundles.
Line 72: Line 111:
 
* [[Tycho/Packaging_Types|Packaging Types]]
 
* [[Tycho/Packaging_Types|Packaging Types]]
  
===Signing tool===
+
== p2 Repo checks ==
 +
 
 +
It's highly recommended that any Eclipse.org project runs frequently, and maybe even systematically, the [[CBI/p2repoAnalyzers/Repo Reports|p2 repo analyzer]] to make sure it conforms to some requirements of being a nice citizen in the Eclipse.org world.
 +
 
 +
==Nexus==
 +
 
 +
See [[Services/Nexus]].
 +
 
 +
==Signing tool==
 +
 
 +
* [http://git.eclipse.org/c/cbi/org.eclipse.cbi.git/tree/maven-plugins/README.md Maven plugins for signing artifacts]
 +
* [[IT_Infrastructure_Doc#Sign_my_plugins.2FZIP_files.3F|On demand signing tool]]
 +
 
 +
= Deliverables =
 +
 
 +
Additionally to recommendation and infrastructure, the CBI also produces pieces of software that are meant to be commonly used by all Eclipse.org projects.
 +
 
 +
== CBI license bundle ==
 +
 
 +
We offer a P2 repository containing the org.eclipse.license bundle which is located at:
 +
 
 +
    http://download.eclipse.org/cbi/updates/license/
 +
 
 +
This URL is a composite P2 repo containing the license bundle.
 +
 
 +
If you are using Tycho you can add the p2 repo to the <repositories> section of your pom.xml file. Something similar to this:
 +
 
 +
<source lang="xml">
 +
    <repository>
 +
      <id>license-feature</id>
 +
      <url>http://download.eclipse.org/cbi/updates/license/</url>
 +
      <layout>p2</layout>
 +
    </repository>
 +
</source>
 +
 
 +
In any particular feature which you need the license you can use the usual feature.xml section:
 +
 
 +
<source lang="xml">
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<feature
 +
      id="org.eclipse.help"
 +
      label="%featureName"
 +
      version="2.0.0.qualifier"
 +
      provider-name="%providerName"
 +
      plugin="org.eclipse.help.base"
 +
      license-feature="org.eclipse.license"
 +
      license-feature-version="1.0.0.qualifier"/>
 +
....
 +
</source>
 +
 
 +
==Signing tool==
 +
 
 +
* [https://git.eclipse.org/c/cbi/org.eclipse.cbi.git/tree/maven-plugins/README.md Maven plugins for signing artifacts]
 +
* [[IT_Infrastructure_Doc#Sign_my_plugins.2FZIP_files.3F|On demand signing tool]]
 +
 
 +
== p2 repo checks ==
 +
 
 +
A set of "tests" which create reports or can be ran as unit tests that check to correctness of p2 repositories. That is partially just "correctness" in general (such as, that jars are signed, etc.) but more so that repositories conform to the requirements of the Eclipse Simultaneous release (such as, that jars have correct "Provider names", licenses, etc.). For more information, see See [[CBI/p2repoAnalyzers/Repo Reports]].
  
[http://wiki.eclipse.org/IT_Infrastructure_Doc#Sign_my_plugins.2FZIP_files.3F On demand signing tool]
+
== p2 repo aggregator ==
  
==Related Topics and Links==
+
A tool to combine several p2 repositories. Among other things, it makes sure they all have consistent constraints (that is, can be "installed together") unlike a raw p2 mirror task. For more information see [[CBI/aggregator]].
* [http://wiki.eclipse.org/EclipseLTS Long Term Support]
+
* [http://wiki.eclipse.org/Build_Technologies List of Build Technologies]
+
  
==FAQ==
+
= Resources =
 +
* [https://dev.eclipse.org/mailman/listinfo/cbi-dev cbi-dev]
 +
* See your [[CBI/FAQ|Frequently Asked Question list]]
  
* See your [http://wiki.eclipse.org/CBI/FAQ Frequently Asked Question list]
 
  
==Meeting Minutes==
+
[[Category:CBI]] [[Category:Releng]] [[Category:Jenkins]]
* [http://wiki.eclipse.org/CBI/Jan10_2012 January 10, 2012]
+
* [http://wiki.eclipse.org/CBI/Jan24_2012 January 24, 2012]
+
* [http://wiki.eclipse.org/CBI/Feb7_2012 February 7, 2012]
+
* [http://wiki.eclipse.org/CBI/March6_2012 March 6, 2012]
+
* [http://wiki.eclipse.org/CBI/Mar20_2012 March 20, 2012]
+
* [http://wiki.eclipse.org/CBI/Code_Sprint_April_11_2012 Code Sprint #1, April 11, 2012] - in Ottawa, Canada
+
* [http://wiki.eclipse.org/CBI/Apr17_2012 April 17, 2012]
+
* [http://wiki.eclipse.org/CBI/May1_2012 May 1, 2012]
+
* [http://wiki.eclipse.org/CBI/May15_2012 May 15, 2012]
+
* [http://wiki.eclipse.org/CBI/May29_2012 May 29, 2012]
+
* [http://wiki.eclipse.org/CBI/June12_2012 June 12, 2012]
+
* [http://wiki.eclipse.org/CBI/June26_2012 June 26, 2012]
+
* [http://wiki.eclipse.org/CBI/July25_2012 July 25, 2012]
+

Revision as of 15:02, 7 November 2018

The Eclipse Common Build Infrastructure (CBI) is an initiative combining infrastructure, technologies and practices for building, testing and delivering Eclipse software.

Goals

Primary goals of the CBI initiative are:

  • Make it really easy to contribute Eclipse projects
    • Make it really easy to copy & modify source
    • Make it really easy to build
    • Make it really easy to test
    • Make it really easy to post a change for review
    • Make it really easy to sign software

Secondary goals are:

  • Get all Eclipse projects building their software on Eclipse Foundation hardware.
  • Make it easy for people to build custom Eclipse distributions.

Asking for Help

Service Level Agreement (SLA)

Most CBI services are Tier 2 - Best Effort, which means they are expected to be available at all times, and rapid restoration can be expected in the event of an outage. Eclipse Strategic Members can contact Webmaster in certain cases of off-hours support.

Please see IT_SLA for more information on the Eclipse Foundation IT Services SLA.

Preferred Build Technologies

CI Environment (Jenkins)

We provide dedicated Jenkins instances to projects. See also Jenkins.

Jenkins is a continuous integration (CI) server used on Eclipse servers for Eclipse projects as part of the Common Build Infrastructure (CBI). Jenkins instances are maintained by the Eclipse Webmasters/Release Engineer.

NOTE: JIPP instances are being migrated from a standalone implementation to a Kubernetes cluster implementation.

Requesting a JIPP instance

Please file a bug against Eclipse Foundation > Community > CI-Jenkins to request your project's own instance. Please ensure your project lead can +1 the request. Please specify if you wish to grant write access to your download area and/or code repositories.

Important.png
Security issues
There may be security issues related to using the Gerrit plugin, more especially when allowing the CI system to write directly to the code repos and downloads area. If you request plugins other than those installed by default, webmasters may not be able to help troubleshoot any issues that you may encounter with your instance.


What's provided?

Each Eclipse Project has access to one Jenkins instance (JIPP), including the following:

  • (1) Jenkins instance, with (1) Resources Pack (see below)
    • Membership-sponsored projects may allocate more resources (see below)
  • Digital signing Service: Java JAR, Java Cryptography Extensions, Windows Portable Executable with Microsoft Authenticode, macOS application bundles.
  • Packaging service: Apple Disk Image (.dmg), Linux Flatpak
  • Disk space: Ephemeral for builds, permanent for release builds.
  • (1) 3.75G/30G Linux Virtual Server (if needed) (courtesy of Microsoft Azure)
  • Access to worldwide download mirrors

Additional Resource Packs

Each Eclipse Project has access to one Resources Pack for building. For some projects, that may not be enough. Projects sponsored by Eclipse Membership (via Project Lead) have additional Packs, based on membership level. These Packs can be allocated to projects.

  • Some resources are only available to Enterprise and Strategic members.
  • Enterprise and Strategic members can engage with the Foundation to acquire additional Packs.
CBI Resource Packs.png

Resource Packs Included in Membership

Eclipse Foundation Member Organizations have access to additional Resource Packs, based on their membership level.

CBI Resource Pack Assignments.png


Assigning Resource Packs to a Project

Resource Packs are assigned by Eclipse Members to Eclipse Projects they sponsor (Members have a Project Lead on the Project). Packs are assigned as a whole to a single project (i.e., can’t split Packs across multiple projects). A member can assign several packs to a single project.

To assign a pack to a project, please file a Bug.

CI Environment (Third-party)

If you host your Eclipse project git repository at Github, we also support some third-party services:

  • TravisCI. TravisCI is free to use for open source projects, but you should be aware that it limits to 5 the number of executors per organizations. It means that only 5 builds can run concurrently in any organization. If your repositories are part of the Eclipse organization, you may face long delays between the trigger and the actually start of the job. If it's your case, and would like to stick to TravisCI for building, we suggest you ask to move to a dedicated organization.


We do not support

If an environment is not listed in the unsupported list, you can ask whether it can be supported by opening a bug.

Important.png
Third party build services (e.g., Travis) limitations
The Eclipse Foundation's IT Team is responsible for the credentials (OSSRH, SSH and GPG keys...) it creates for projects. Keeping track of where credentials are disseminated is crucial for security reasons. As long as we have no automation in place to set/update/remove credentials on 3rd party build services, we don't put any credential on such services. For instance, it means that you can't use them to publish to Maven Central, or to upload build results to download.eclipse.org.


Maven

Maven 3.x drives the builds. Projects are expected to provide standard Maven 3.x POM files for their builds. The builds should be built in such a way that they can be run on the local workstation, or on the Eclipse build server. Note that builds can only be signed on the Eclipse build server.

Tycho

Tycho is focused on a Maven-centric, manifest-first approach to building Eclipse plug-ins, features, update sites, RCP applications and OSGi bundles.

Helpful links:

p2 Repo checks

It's highly recommended that any Eclipse.org project runs frequently, and maybe even systematically, the p2 repo analyzer to make sure it conforms to some requirements of being a nice citizen in the Eclipse.org world.

Nexus

See Services/Nexus.

Signing tool

Deliverables

Additionally to recommendation and infrastructure, the CBI also produces pieces of software that are meant to be commonly used by all Eclipse.org projects.

CBI license bundle

We offer a P2 repository containing the org.eclipse.license bundle which is located at:

   http://download.eclipse.org/cbi/updates/license/

This URL is a composite P2 repo containing the license bundle.

If you are using Tycho you can add the p2 repo to the <repositories> section of your pom.xml file. Something similar to this:

    <repository>
      <id>license-feature</id>
      <url>http://download.eclipse.org/cbi/updates/license/</url>
      <layout>p2</layout>
    </repository>

In any particular feature which you need the license you can use the usual feature.xml section:

<?xml version="1.0" encoding="UTF-8"?>
<feature
      id="org.eclipse.help"
      label="%featureName"
      version="2.0.0.qualifier"
      provider-name="%providerName"
      plugin="org.eclipse.help.base"
      license-feature="org.eclipse.license"
      license-feature-version="1.0.0.qualifier"/> 
....

Signing tool

p2 repo checks

A set of "tests" which create reports or can be ran as unit tests that check to correctness of p2 repositories. That is partially just "correctness" in general (such as, that jars are signed, etc.) but more so that repositories conform to the requirements of the Eclipse Simultaneous release (such as, that jars have correct "Provider names", licenses, etc.). For more information, see See CBI/p2repoAnalyzers/Repo Reports.

p2 repo aggregator

A tool to combine several p2 repositories. Among other things, it makes sure they all have consistent constraints (that is, can be "installed together") unlike a raw p2 mirror task. For more information see CBI/aggregator.

Resources

Back to the top