Skip to main content
Jump to: navigation, search

Difference between revisions of "CBI"

Line 25: Line 25:
 
* Get all Eclipse projects building their software on Eclipse Foundation hardware.
 
* Get all Eclipse projects building their software on Eclipse Foundation hardware.
 
* Make it easy for people to build custom Eclipse distributions.
 
* Make it easy for people to build custom Eclipse distributions.
 
==Who is using it?==
 
 
There's a [https://ci.eclipse.org/ list of projects] building with CBI available.
 
 
==Eclipse CBI==
 
Projects hosted at Eclipse.org can request one hosted instance of CBI. Details are provided on the [[CBI/Product]] page.
 
  
 
==Preferred Build Technologies==
 
==Preferred Build Technologies==
Line 127: Line 120:
  
 
==Resources==
 
==Resources==
=== Mailing-list ===
+
* [https://dev.eclipse.org/mailman/listinfo/cbi-dev cbi-dev]
[https://dev.eclipse.org/mailman/listinfo/cbi-dev cbi-dev]
+
* See your [[CBI/FAQ|Frequently Asked Question list]]
  
=== FAQ ===
+
=Eclipse CBI (JIPP)=
 +
Projects hosted at Eclipse.org can request one hosted instance of CBI. Details are provided on the [[Jenkins]] page. There's a [https://ci.eclipse.org/ list of projects] currently building with CBI available.
  
* See your [[CBI/FAQ|Frequently Asked Question list]]
+
== Tools (and locations) ==
 +
Build tools like JDK, Maven, Ant and Gradle are already configured in every Jenkins instance.
 +
 
 +
* JDK
 +
** jdk10-latest (/shared/common/java/oracle/jdk-10_x64-latest)
 +
** jdk9-latest (/shared/common/jdk-9_x64-latest)
 +
** jdk1.8.0-latest (/shared/common/jdk1.8.0_x64-latest)
 +
** jdk1.7.0-latest (/shared/common/jdk1.7.0-latest)
 +
** jdk1.6.0-latest (/shared/common/jdk1.6.0-latest)
 +
** jdk1.5.0-latest (/shared/common/jdk1.5.0-latest)
 +
 
 +
* Maven
 +
** apache-maven-latest (/shared/common/apache-maven-latest)
 +
** apache-maven-3.0.5 (/shared/common/apache-maven-3.0.5)
 +
 
 +
* Ant
 +
** apache-ant-1.9.6 (/shared/common/apache-ant-1.9.6)
 +
 
 +
* Gradle
 +
** gradle-latest (/shared/common/gradle-latest)
 +
** gradle-3.1 (/shared/common/gradle-3.1)
 +
 
 +
More generally, all tools listed on http://build.eclipse.org/common/ are available from '''/shared/common/'''.
 +
 
 +
If you need tools that are not general purpose installed, project members can install them in your project's home directory, for example ~/buildtools. [https://dev.eclipse.org/mhonarc/lists/cbi-dev/msg01869.html See email on cbi-dev]
 +
 
 +
== Default plugins ==
 +
 
 +
The following plugins are installed by default. Additional plugins can be installed on request.
 +
 
 +
<div style="column-count:4;-moz-column-count:4;-webkit-column-count:4">
 +
* ace-editor
 +
* analysis-core
 +
* ant
 +
* antisamy-markup-formatter
 +
* authentication-tokens
 +
* bouncycastle-api
 +
* branch-api
 +
* build-timeout
 +
* cloudbees-folder
 +
* conditional-buildstep
 +
* credentials
 +
* credentials-binding
 +
* dashboard-view
 +
* disk-usage
 +
* display-url-api
 +
* docker-commons
 +
* docker-workflow
 +
* durable-task
 +
* external-monitor-job
 +
* extra-columns
 +
* find-bugs
 +
* gerrit-trigger
 +
* git
 +
* git-client
 +
* git-parameter
 +
* git-server
 +
* gradle
 +
* greenballs
 +
* handlebars
 +
* icon-shim
 +
* javadoc
 +
* jobConfigHistory
 +
* jquery
 +
* jquery-detached
 +
* junit
 +
* ldap
 +
* mailer
 +
* matrix-auth
 +
* matrix-project
 +
* maven-plugin
 +
* momentjs
 +
* pam-auth
 +
* parameterized-trigger
 +
* pipeline-build-step
 +
* pipeline-graph-analysis
 +
* pipeline-input-step
 +
* pipeline-milestone-step
 +
* pipeline-model-api
 +
* pipeline-model-declarative-agent
 +
* pipeline-model-definition
 +
* pipeline-model-extensions
 +
* pipeline-rest-api
 +
* pipeline-stage-step
 +
* pipeline-stage-tags-metadata
 +
* pipeline-stage-view
 +
* plain-credentials
 +
* promoted-builds
 +
* rebuild
 +
* resource-disposer
 +
* scm-api
 +
* script-security
 +
* sonar
 +
* ssh-credentials
 +
* ssh-slaves
 +
* structs
 +
* timestamper
 +
* token-macro
 +
* windows-slaves
 +
* workflow-aggregator
 +
* workflow-api
 +
* workflow-basic-steps
 +
* workflow-cps
 +
* workflow-cps-global-lib
 +
* workflow-durable-task-step
 +
* workflow-job
 +
* workflow-multibranch
 +
* workflow-scm-step
 +
* workflow-step-api
 +
* workflow-support
 +
* ws-cleanup
 +
* xvnc
 +
</div>
 +
 
 +
== Setup for specific plugins ==
 +
 
 +
=== GitHub Pull Request Builder Plugin ===
 +
 
 +
The [https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin GitHub Pull Request Builder Plugin] (GHPRB) allows to build/test pull requests and provide immediate feedback in the pull request on GitHub.
 +
 
 +
'''To set this up, please open a Bugzilla issue against the CI-Jenkins component (Product: Community) and request this feature.'''
 +
 
 +
Here are some details about what happens during the setup process:
 +
* The GHPRB plugin is installed in the JIPP.
 +
* Webmaster creates a GitHub bot user and adds it to the respective team on GitHub.
 +
* The credentials of the GitHub bot user are added to the JIPP (with user name and password, because SSH keys are not recommended/supported by the plugin).
 +
* The GHPRB plugin's main config is set up.
 +
 
 +
Once the ticket is resolved you should be able to configure and use the GHPRB plugin in your jobs.
 +
 
 +
Instructions how to set up GHPRB plugin in jobs can be found here:
 +
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md
 +
 
 +
'''Please note: ''' Currently we don't recommend to use the 'Use github hooks for build triggering' option. Instead, with this option turned off, Jenkins is polling GitHub instead. Which should work just fine in most cases.
 +
 
 +
More info can be found in the GitHub readme:
 +
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md
 +
 
 +
=== Jenkins Pipeline (aka configuration in code) ===
 +
 
 +
An example how Eclipse plugins can be build with Tycho using a Jenkins pipeline can be found here (Thanks to Mickael Istria!):<br/>
 +
https://github.com/eclipse/aCute/blob/master/Jenkinsfile
 +
 
 +
More info about Jenkins Pipeline can be found here:<br/>
 +
https://jenkins.io/doc/book/pipeline/<br/>
 +
https://jenkins.io/doc/book/pipeline/shared-libraries/
 +
 
 +
=== Gerrit Trigger Plugin ===
 +
 
 +
You may use the [https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger Jenkins Gerrit Trigger Plugin] in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.
 +
 
 +
[[Image:Jgit.gerrit-reviewer.png|450px]]
 +
 
 +
[[Image:Jgit.gerrit-vote.png|450px]]
 +
 
 +
Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.
 +
 
 +
====General configuration settings====
 +
 
 +
# Check ''This project is parameterized''. Click the ''Add'' button and select ''String Parameter''. Set the parameter ''Name'' to '''GERRIT_REFSPEC''' and ''Default Value'' to '''refs/heads/master'''.
 +
 
 +
[[Image:Gerrit-refspec-param.png|600px]]
 +
 
 +
====Configuration of Source Code Management====
 +
 
 +
# Under ''Source Code Management'' select Git.
 +
 
 +
# Under ''Repositories'', click on ''Advanced'' and change the '''Refspec''' to '''${GERRIT_REFSPEC}'''.
 +
 
 +
# Under ''Additional Behaviours'', add '''Strategy for choosing what to build''' and select '''Gerrit Trigger''' as a strategy.
 +
 
 +
[[Image:Jgit.gerrit-git-config.png|600px]]
 +
 
 +
Note that the section '''Branches to build''' won't be used and may be deleted.
 +
 
 +
====Configuration of Build Triggers ====
 +
 
 +
# Under ''Build Triggers'', select '''Gerrit event'''.
 +
 
 +
[[Image:Jgit.gerrit-gerrit-config.png|600px]]
 +
 
 +
# Under ''Trigger on'', click on ''Add'' and select at least '''Patchset Created'''. This will configure the job to run on each new patchset. You can also add additional trigger, like '''Comment Added Contains Regular Expression'''. In the example below, a build will be triggered for the latest patch set if the comment is exactly '''CI Bot, run a build please'''.
 +
 
 +
[[Image:gerrit-trigger-events.png|600px]]
 +
 
 +
# Finally, configure at least one ''Gerrit Project''. The pattern is the name of project (i.e. if your repository is <code>git.eclipse.org/<xx>/<yy>.git</code>, then fill the pattern <code>xx/yy</code>). The ''Branches'' section is the list of branch to listen for events as configured above. Generally, you want one, named '''master''' to build patches submitted for the master branche, or <code>**</code> to build patches submitted to each and every branches. Set the type to '''Path'''.
 +
 
 +
[[Image:gerrit-trigger-project.png|600px]]
 +
 
 +
====Configuration of the build action====
 +
 
 +
Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".
  
===Bugs===
 
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap;product=CBI;version=1.0;list_id=2249872 CBI 1.0]
 
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap;product=CBI;version=2.0;list_id=2249872 CBI 2.0]
 
* [https://bugs.eclipse.org/bugs/buglist.cgi?action=wrap&product=CBI&list_id=38248 List of All Bugs] (Product = CBI)
 
  
 
===Tutorials, News, and other resources===
 
===Tutorials, News, and other resources===
Line 149: Line 330:
  
  
==Meetings==
+
[[Category:CBI]] [[Category:Releng]] [[Category:Jenkins]]
 
+
=== Next Meeting ===
+
See the [http://wiki.eclipse.org/CBI/Conference conference bridge details]. Contact andrew dot ross at eclipse dot org if you would like to be added to the Calendar reminder. The dates the upcoming calls are as follows:
+
* November 15th, 9am EST
+
 
+
===Meeting Minutes===
+
* [http://wiki.eclipse.org/CBI/Jan10_2012 January 10, 2012]
+
* [http://wiki.eclipse.org/CBI/Jan24_2012 January 24, 2012]
+
* [http://wiki.eclipse.org/CBI/Feb7_2012 February 7, 2012]
+
* [http://wiki.eclipse.org/CBI/March6_2012 March 6, 2012]
+
* [http://wiki.eclipse.org/CBI/Mar20_2012 March 20, 2012]
+
* [http://wiki.eclipse.org/CBI/Code_Sprint_April_11_2012 Code Sprint #1, April 11, 2012] - in Ottawa, Canada
+
* [http://wiki.eclipse.org/CBI/Apr17_2012 April 17, 2012]
+
* [http://wiki.eclipse.org/CBI/May1_2012 May 1, 2012]
+
* [http://wiki.eclipse.org/CBI/May15_2012 May 15, 2012]
+
* [http://wiki.eclipse.org/CBI/May29_2012 May 29, 2012]
+
* [http://wiki.eclipse.org/CBI/June12_2012 June 12, 2012]
+
* [http://wiki.eclipse.org/CBI/June26_2012 June 26, 2012]
+
* [http://wiki.eclipse.org/CBI/July25_2012 July 25, 2012]
+
* [http://wiki.eclipse.org/CBI/November15_2012 November 15, 2012]
+
* [http://wiki.eclipse.org/CBI/January8_2013 January 8, 2013]
+
 
+
[[Category:CBI]]
+

Revision as of 13:50, 13 September 2018

The Eclipse Common Build Infrastructure (CBI) is an initiative combining infrastructure, technologies and practices for building Eclipse Software.

What is CBI

The core of CBI today is Maven with the Tycho plugins. The Tycho plugins teach Maven how to build (and consume) Eclipse plugins and OSGi bundles. This enables building Eclipse projects with "maven clean install" just as one would build other Maven projects.

Common services such as the Jar signing facility, MacOS signing facility, and Windows signing facility are also included with CBI. Other tools and services may be included in the future as the need arises.

Over time mature templates and common pom.xml files will be provided that set common values finely honed with experience.

One might go so far as to include Git, Jenkins, the build slaves, and Nexus (aka. the artifact repository & server side of Maven) as part of CBI since they are also common and crucial to builds.

Gerrit, Bugzilla, and the Downloads site are closely related. Some might consider them part of CBI as well.

Initiative Goals

Primary goals are:

  • Make it really easy to contribute Eclipse projects
    • Make it really easy to copy & modify source
    • Make it really easy to build
    • Make it really easy to test
    • Make it really easy to post a change for review
    • Make it really easy to sign software

Secondary goals are:

  • Get all Eclipse projects building their software on Eclipse Foundation hardware.
  • Make it easy for people to build custom Eclipse distributions.

Preferred Build Technologies

Jenkins

Maven

Maven 3.0 drives the builds. Projects are expected to provide standard Maven 3.0 POM files for their builds. The builds should be built in such a way that they can be run on the local workstation, or on the Eclipse build server. Note that builds can only be signed on the Eclipse build server.

Tycho

Tycho is focused on a Maven-centric, manifest-first approach to building Eclipse plug-ins, features, update sites, RCP applications and OSGi bundles.

Helpful links:

p2 Repo checks

It's highly recommended that any Eclipse.org project runs frequently, and maybe even systematically, the p2 repo analyzer to make sure it conforms to some requirements of being a nice citizen in the Eclipse.org world.

Nexus

Services/Nexus

Signing tool

Deliverables

Additionally to recommendation and infrastructure, the CBI also produces pieces of software that are meant to be commonly used by all Eclipse.org projects.

CBI License bundle

We offer a P2 repository containing the org.eclipse.license bundle which is located at:

   http://download.eclipse.org/cbi/updates/license/

This URL is a composite P2 repo containing the license bundle.


If you are using Tycho you can add the p2 repo to the <repositories> section of your pom.xml file. Something similar to this:

    <repository>
      <id>license-feature</id>
      <url>http://download.eclipse.org/cbi/updates/license/</url>
      <layout>p2</layout>
    </repository>

In any particular feature which you need the license you can use the usual feature.xml section:

<?xml version="1.0" encoding="UTF-8"?>
<feature
      id="org.eclipse.help"
      label="%featureName"
      version="2.0.0.qualifier"
      provider-name="%providerName"
      plugin="org.eclipse.help.base"
      license-feature="org.eclipse.license"
      license-feature-version="1.0.0.qualifier"/> 
....

Signing tool

p2 repo checks

A set of "tests" which create reports or can be ran as unit tests that check to correctness of p2 repositories. That is partially just "correctness" in general (such as, that jars are signed, etc.) but more so that repositories conform to the requirements of the Eclipse Simultaneous release (such as, that jars have correct "Provider names", licenses, etc.). For more information, see See CBI/p2repoAnalyzers/Repo Reports.

p2 repo aggregator

A tool to combine several p2 repositories. Among other things, it makes sure they all have consistent constraints (that is, can be "installed together") unlike a raw p2 mirror task. For more information see CBI/aggregator/manual.

Related Topics and Links

Resources

Eclipse CBI (JIPP)

Projects hosted at Eclipse.org can request one hosted instance of CBI. Details are provided on the Jenkins page. There's a list of projects currently building with CBI available.

Tools (and locations)

Build tools like JDK, Maven, Ant and Gradle are already configured in every Jenkins instance.

  • JDK
    • jdk10-latest (/shared/common/java/oracle/jdk-10_x64-latest)
    • jdk9-latest (/shared/common/jdk-9_x64-latest)
    • jdk1.8.0-latest (/shared/common/jdk1.8.0_x64-latest)
    • jdk1.7.0-latest (/shared/common/jdk1.7.0-latest)
    • jdk1.6.0-latest (/shared/common/jdk1.6.0-latest)
    • jdk1.5.0-latest (/shared/common/jdk1.5.0-latest)
  • Maven
    • apache-maven-latest (/shared/common/apache-maven-latest)
    • apache-maven-3.0.5 (/shared/common/apache-maven-3.0.5)
  • Ant
    • apache-ant-1.9.6 (/shared/common/apache-ant-1.9.6)
  • Gradle
    • gradle-latest (/shared/common/gradle-latest)
    • gradle-3.1 (/shared/common/gradle-3.1)

More generally, all tools listed on http://build.eclipse.org/common/ are available from /shared/common/.

If you need tools that are not general purpose installed, project members can install them in your project's home directory, for example ~/buildtools. See email on cbi-dev

Default plugins

The following plugins are installed by default. Additional plugins can be installed on request.

  • ace-editor
  • analysis-core
  • ant
  • antisamy-markup-formatter
  • authentication-tokens
  • bouncycastle-api
  • branch-api
  • build-timeout
  • cloudbees-folder
  • conditional-buildstep
  • credentials
  • credentials-binding
  • dashboard-view
  • disk-usage
  • display-url-api
  • docker-commons
  • docker-workflow
  • durable-task
  • external-monitor-job
  • extra-columns
  • find-bugs
  • gerrit-trigger
  • git
  • git-client
  • git-parameter
  • git-server
  • gradle
  • greenballs
  • handlebars
  • icon-shim
  • javadoc
  • jobConfigHistory
  • jquery
  • jquery-detached
  • junit
  • ldap
  • mailer
  • matrix-auth
  • matrix-project
  • maven-plugin
  • momentjs
  • pam-auth
  • parameterized-trigger
  • pipeline-build-step
  • pipeline-graph-analysis
  • pipeline-input-step
  • pipeline-milestone-step
  • pipeline-model-api
  • pipeline-model-declarative-agent
  • pipeline-model-definition
  • pipeline-model-extensions
  • pipeline-rest-api
  • pipeline-stage-step
  • pipeline-stage-tags-metadata
  • pipeline-stage-view
  • plain-credentials
  • promoted-builds
  • rebuild
  • resource-disposer
  • scm-api
  • script-security
  • sonar
  • ssh-credentials
  • ssh-slaves
  • structs
  • timestamper
  • token-macro
  • windows-slaves
  • workflow-aggregator
  • workflow-api
  • workflow-basic-steps
  • workflow-cps
  • workflow-cps-global-lib
  • workflow-durable-task-step
  • workflow-job
  • workflow-multibranch
  • workflow-scm-step
  • workflow-step-api
  • workflow-support
  • ws-cleanup
  • xvnc

Setup for specific plugins

GitHub Pull Request Builder Plugin

The GitHub Pull Request Builder Plugin (GHPRB) allows to build/test pull requests and provide immediate feedback in the pull request on GitHub.

To set this up, please open a Bugzilla issue against the CI-Jenkins component (Product: Community) and request this feature.

Here are some details about what happens during the setup process:

  • The GHPRB plugin is installed in the JIPP.
  • Webmaster creates a GitHub bot user and adds it to the respective team on GitHub.
  • The credentials of the GitHub bot user are added to the JIPP (with user name and password, because SSH keys are not recommended/supported by the plugin).
  • The GHPRB plugin's main config is set up.

Once the ticket is resolved you should be able to configure and use the GHPRB plugin in your jobs.

Instructions how to set up GHPRB plugin in jobs can be found here: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Please note: Currently we don't recommend to use the 'Use github hooks for build triggering' option. Instead, with this option turned off, Jenkins is polling GitHub instead. Which should work just fine in most cases.

More info can be found in the GitHub readme: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Jenkins Pipeline (aka configuration in code)

An example how Eclipse plugins can be build with Tycho using a Jenkins pipeline can be found here (Thanks to Mickael Istria!):
https://github.com/eclipse/aCute/blob/master/Jenkinsfile

More info about Jenkins Pipeline can be found here:
https://jenkins.io/doc/book/pipeline/
https://jenkins.io/doc/book/pipeline/shared-libraries/

Gerrit Trigger Plugin

You may use the Jenkins Gerrit Trigger Plugin in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.

Jgit.gerrit-reviewer.png

Jgit.gerrit-vote.png

Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.

General configuration settings

  1. Check This project is parameterized. Click the Add button and select String Parameter. Set the parameter Name to GERRIT_REFSPEC and Default Value to refs/heads/master.

Gerrit-refspec-param.png

Configuration of Source Code Management

  1. Under Source Code Management select Git.
  1. Under Repositories, click on Advanced and change the Refspec to ${GERRIT_REFSPEC}.
  1. Under Additional Behaviours, add Strategy for choosing what to build and select Gerrit Trigger as a strategy.

Jgit.gerrit-git-config.png

Note that the section Branches to build won't be used and may be deleted.

Configuration of Build Triggers

  1. Under Build Triggers, select Gerrit event.

Jgit.gerrit-gerrit-config.png

  1. Under Trigger on, click on Add and select at least Patchset Created. This will configure the job to run on each new patchset. You can also add additional trigger, like Comment Added Contains Regular Expression. In the example below, a build will be triggered for the latest patch set if the comment is exactly CI Bot, run a build please.

Gerrit-trigger-events.png

  1. Finally, configure at least one Gerrit Project. The pattern is the name of project (i.e. if your repository is git.eclipse.org/<xx>/<yy>.git, then fill the pattern xx/yy). The Branches section is the list of branch to listen for events as configured above. Generally, you want one, named master to build patches submitted for the master branche, or ** to build patches submitted to each and every branches. Set the type to Path.

Gerrit-trigger-project.png

Configuration of the build action

Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".


Tutorials, News, and other resources

Back to the top