Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Jenkins"

Line 1: Line 1:
−
[[File:Jenkins_logo.png|right|100px]]
+
[[File:Jenkins_logo.png|right|100ox]]
−
Jenkins is a continuous integration (CI) server used on Eclipse servers for Eclipse projects as part of the [[CBI|Common Build Infrastructure (CBI)]]. Jenkins instances are maintained by the Eclipse Webmasters/Release Engineer.
+
 
 +
Jenkins is a continuous integration (CI) server. It is in use on Eclipse servers for Eclipse projects as part of the [[CBI|Common Build Infrastructure (CBI)]]. This page is about the hosted service at Eclipse.org. For more information on the project itself, or to download Jenkins, please see the [https://jenkins.io Jenkins project] page.
 +
 
 +
Since Hudson is not maintained anymore, Jenkins is the replacement for it. They share a common ancestry, but have diverged in the past. Eventually all Hudson Instances Per Project (HIPP) will be completely replaced by Jenkins Instances Per Project (JIPP). The migration will be slow in the beginning, to minimize risks and impact on the running build infrastructure (see [https://wiki.eclipse.org/Jenkins#HIPP_to_JIPP_migration_.28HIPP2JIPP.29 HIPP2JIPP migration])
 +
 
 +
= General Information  =
 +
 
 +
Jenkins instances are maintained by the Eclipse Webmasters/Release Engineer.
  
 
* List of Jenkins Instances Per Project (JIPP):
 
* List of Jenkins Instances Per Project (JIPP):
−
** https://ci.eclipse.org/ (standalone)
+
** https://ci.eclipse.org/
−
** https://jenkins.eclipse.org/ (Kubernetes cluster)
+
** https://ci.polarsys.org/
  
−
NOTE: JIPP instances are being migrated from a standalone implementation to a Kubernetes cluster implementation.
+
== Asking for Help ==
  
 +
* Need help actually building your code: ask your project mentors, or ask on the Common Build mailing list (cbi-dev). There are no dumb questions.
 +
* Subscribe to cbi-dev here: https://dev.eclipse.org/mailman/listinfo/cbi-dev
  
−
= Jenkins @ Eclipse (JIPP) =
+
== Requesting a JIPP instance for CBI ==
−
== What's provided? ==
+
  
−
Each Eclipse Project has access to one Jenkins instance (JIPP), including the following:
+
Please see [[CBI]]
  
−
* (1) Jenkins instance, with (1) resources Base Pack (see below)
+
= Jenkins configuration and tools =
−
** Membership-sponsored projects may allocate more resources (see below)
+
−
* Digital signing Service: Java JAR, Java Cryptography Extensions, Windows Portable Executable with Microsoft Authenticode, macOS application bundles.
+
−
* Packaging service: Apple Disk Image (.dmg), Linux Flatpak
+
−
* Disk space: Ephemeral for builds, permanent for release builds.
+
−
* Access to worldwide download mirrors
+
  
−
== Additional Resource Packs ==
+
Check [[CI best practices]] for general recommendations how to setup Jenkins.
  
−
Each Eclipse Project has access to one Base pack for building. For some projects, that may not be enough. Projects sponsored by [https://www.eclipse.org/membership/exploreMembership.php Eclipse Membership] (via Project Lead) have additional Packs, based on [https://www.eclipse.org/membership/become_a_member/membershipTypes.php membership level]. These packs can be allocated to projects.
+
== Tools (and locations) ==
 +
Build tools like JDK, Maven, Ant and Gradle are already configured in every Jenkins instance.  
  
−
* Some Packs are only available to [https://www.eclipse.org/membership/become_a_member/membershipTypes.php Enterprise and Strategic members].
+
* JDK
−
* Enterprise and Strategic members can engage with the Foundation to acquire additional Packs.
+
** jdk10-latest (/shared/common/java/oracle/jdk-10_x64-latest)
 +
** jdk9-latest (/shared/common/jdk-9_x64-latest)
 +
** jdk1.8.0-latest (/shared/common/jdk1.8.0_x64-latest)
 +
** jdk1.7.0-latest (/shared/common/jdk1.7.0-latest)
 +
** jdk1.6.0-latest (/shared/common/jdk1.6.0-latest)
 +
** jdk1.5.0-latest (/shared/common/jdk1.5.0-latest)
  
−
[[File:CBI Resource Packs.png]]
+
* Maven
 +
** apache-maven-latest (/shared/common/apache-maven-latest)
 +
** apache-maven-3.0.5 (/shared/common/apache-maven-3.0.5)
  
−
=== Resource Packs Included in Membership ===
+
* Ant
 +
** apache-ant-1.9.6 (/shared/common/apache-ant-1.9.6)
  
−
Eclipse Foundation Member Organizations have access to Resource Packs above the Base Pack that is allotted to each Project, based on their [https://www.eclipse.org/membership/become_a_member/membershipTypes.php membership level].
+
* Gradle
 +
** gradle-latest (/shared/common/gradle-latest)
 +
** gradle-3.1 (/shared/common/gradle-3.1)
  
−
[[File:CBI Resource Pack Assignments.png]]
+
More generally, all tools listed on http://build.eclipse.org/common/ are available from '''/shared/common/'''.
  
 +
If you need tools that are not general purpose installed, project members can install them in your project's home directory, for example ~/buildtools. [https://dev.eclipse.org/mhonarc/lists/cbi-dev/msg01869.html See email on cbi-dev]
  
−
=== Assigning Resource Packs to a Project ===
+
== Default plugins ==
  
−
Resource Packs are assigned by Eclipse Members to Eclipse Projects they sponsor (Members have a Project Lead on the Project). Packs are assigned as a whole to a single project (i.e., can’t split Packs across multiple projects). A <ember can assign several packs to a single project.
+
The following plugins are installed by default. Additional plugins can be installed on request.
  
−
To assign a pack to a project, please [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins&short_desc=Assign%20Resource%20Pack%20to%20Project file a Bug].
+
<div style="column-count:4;-moz-column-count:4;-webkit-column-count:4">
 +
* ace-editor
 +
* analysis-core
 +
* ant
 +
* antisamy-markup-formatter
 +
* authentication-tokens
 +
* bouncycastle-api
 +
* branch-api
 +
* build-timeout
 +
* cloudbees-folder
 +
* conditional-buildstep
 +
* credentials
 +
* credentials-binding
 +
* dashboard-view
 +
* disk-usage
 +
* display-url-api
 +
* docker-commons
 +
* docker-workflow
 +
* durable-task
 +
* external-monitor-job
 +
* extra-columns
 +
* find-bugs
 +
* gerrit-trigger
 +
* git
 +
* git-client
 +
* git-parameter
 +
* git-server
 +
* gradle
 +
* greenballs
 +
* handlebars
 +
* icon-shim
 +
* javadoc
 +
* jobConfigHistory
 +
* jquery
 +
* jquery-detached
 +
* junit
 +
* ldap
 +
* mailer
 +
* matrix-auth
 +
* matrix-project
 +
* maven-plugin
 +
* momentjs
 +
* pam-auth
 +
* parameterized-trigger
 +
* pipeline-build-step
 +
* pipeline-graph-analysis
 +
* pipeline-input-step
 +
* pipeline-milestone-step
 +
* pipeline-model-api
 +
* pipeline-model-declarative-agent
 +
* pipeline-model-definition
 +
* pipeline-model-extensions
 +
* pipeline-rest-api
 +
* pipeline-stage-step
 +
* pipeline-stage-tags-metadata
 +
* pipeline-stage-view
 +
* plain-credentials
 +
* promoted-builds
 +
* rebuild
 +
* resource-disposer
 +
* scm-api
 +
* script-security
 +
* sonar
 +
* ssh-credentials
 +
* ssh-slaves
 +
* structs
 +
* timestamper
 +
* token-macro
 +
* windows-slaves
 +
* workflow-aggregator
 +
* workflow-api
 +
* workflow-basic-steps
 +
* workflow-cps
 +
* workflow-cps-global-lib
 +
* workflow-durable-task-step
 +
* workflow-job
 +
* workflow-multibranch
 +
* workflow-scm-step
 +
* workflow-step-api
 +
* workflow-support
 +
* ws-cleanup
 +
* xvnc
 +
</div>
  
−
== Asking for Help ==
+
== Setup for specific plugins ==
  
−
* Need help actually building your code: ask your project mentors, or ask on the Common Build mailing list (cbi-dev). There are no dumb questions.
+
=== GitHub Pull Request Builder Plugin ===
−
* Subscribe to cbi-dev here: https://dev.eclipse.org/mailman/listinfo/cbi-dev
+
  
−
== Requesting a JIPP instance ==
+
The [https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin GitHub Pull Request Builder Plugin] (GHPRB) allows to build/test pull requests and provide immediate feedback in the pull request on GitHub.
  
−
Please [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Community&component=CI-Jenkins&short_desc=JIPP%20Request file a bug] against Eclipse Foundation > Community > CI-Jenkins to request your project's own instance. Please ensure your project lead can +1 the request. Please specify if you wish to grant write access to your download or code repositories.
+
'''To set this up, please open a Bugzilla issue against the CI-Jenkins component (Product: Community) and request this feature.'''
  
−
{{Note|About write access| If your git repo is handled by Gerrit, granting write access to your code repositories is a different procedure, so you must ask specifically for it. If you don't use Gerrit, then granting write access to your download area automatically grants write access to your code repositories and vice-versa.}}
+
Here are some details about what happens during the setup process:
 +
* The GHPRB plugin is installed in the JIPP.
 +
* Webmaster creates a GitHub bot user and adds it to the respective team on GitHub.
 +
* The credentials of the GitHub bot user are added to the JIPP (with user name and password, because SSH keys are not recommended/supported by the plugin).
 +
* The GHPRB plugin's main config is set up.
  
−
{{important|Security issues| There may be security issues related to using the Gerrit plugin and there may be security issues related to allowing the CI system to write directly to your code repos and downloads area. If you request plugins other than those available on the Shared instance, webmaster may not be able to help troubleshoot any issues that you may encounter with your instance.}}
+
Once the ticket is resolved you should be able to configure and use the GHPRB plugin in your jobs.
  
−
= Jenkins configuration and tools  =
+
Instructions how to set up GHPRB plugin in jobs can be found here:
 +
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md
 +
 
 +
'''Please note: ''' Currently we don't recommend to use the 'Use github hooks for build triggering' option. Instead, with this option turned off, Jenkins is polling GitHub instead. Which should work just fine in most cases.
 +
 
 +
More info can be found in the GitHub readme:
 +
https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md
 +
 
 +
=== Jenkins Pipeline (aka configuration in code) ===
 +
 
 +
An example how Eclipse plugins can be build with Tycho using a Jenkins pipeline can be found here (Thanks to Mickael Istria!):<br/>
 +
https://github.com/eclipse/aCute/blob/master/Jenkinsfile
 +
 
 +
More info about Jenkins Pipeline can be found here:<br/>
 +
https://jenkins.io/doc/book/pipeline/<br/>
 +
https://jenkins.io/doc/book/pipeline/shared-libraries/
 +
 
 +
=== Gerrit Trigger Plugin ===
 +
 
 +
You may use the [https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger Jenkins Gerrit Trigger Plugin] in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.
 +
 
 +
[[Image:Jgit.gerrit-reviewer.png|450px]]
 +
 
 +
[[Image:Jgit.gerrit-vote.png|450px]]
 +
 
 +
Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.
 +
 
 +
====General configuration settings====
 +
 
 +
# Check ''This project is parameterized''. Click the ''Add'' button and select ''String Parameter''. Set the parameter ''Name'' to '''GERRIT_REFSPEC''' and ''Default Value'' to '''refs/heads/master'''.
 +
 
 +
[[Image:Gerrit-refspec-param.png|600px]]
 +
 
 +
====Configuration of Source Code Management====
 +
 
 +
# Under ''Source Code Management'' select Git.
 +
 
 +
# Under ''Repositories'', click on ''Advanced'' and change the '''Refspec''' to '''${GERRIT_REFSPEC}'''.
 +
 
 +
# Under ''Additional Behaviours'', add '''Strategy for choosing what to build''' and select '''Gerrit Trigger''' as a strategy.
 +
 
 +
[[Image:Jgit.gerrit-git-config.png|600px]]
 +
 
 +
Note that the section '''Branches to build''' won't be used and may be deleted.
 +
 
 +
====Configuration of Build Triggers ====
 +
 
 +
# Under ''Build Triggers'', select '''Gerrit event'''.
 +
 
 +
[[Image:Jgit.gerrit-gerrit-config.png|600px]]
 +
 
 +
# Under ''Trigger on'', click on ''Add'' and select at least '''Patchset Created'''. This will configure the job to run on each new patchset. You can also add additional trigger, like '''Comment Added Contains Regular Expression'''. In the example below, a build will be triggered for the latest patch set if the comment is exactly '''CI Bot, run a build please'''.
 +
 
 +
[[Image:gerrit-trigger-events.png|600px]]
 +
 
 +
# Finally, configure at least one ''Gerrit Project''. The pattern is the name of project (i.e. if your repository is <code>git.eclipse.org/<xx>/<yy>.git</code>, then fill the pattern <code>xx/yy</code>). The ''Branches'' section is the list of branch to listen for events as configured above. Generally, you want one, named '''master''' to build patches submitted for the master branche, or <code>**</code> to build patches submitted to each and every branches. Set the type to '''Path'''.
 +
 
 +
[[Image:gerrit-trigger-project.png|600px]]
 +
 
 +
====Configuration of the build action====
 +
 
 +
Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".
 +
 
 +
= Differences between Hudson and Jenkins =
 +
 
 +
{| class="wikitable"
 +
|
 +
!|Hudson
 +
!|Jenkins
 +
|-
 +
|Job templates "cascading"
 +
|built-in
 +
|only rudimentary with plugins
 +
|-
 +
|Maven3 configuration
 +
|Yes
 +
|No
 +
|-
 +
|[https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin GitHub pull request builder plugin]
 +
|No
 +
|Yes
 +
|}
 +
 
 +
= HIPP to JIPP migration (HIPP2JIPP) =
 +
 
 +
'''As of March 1st, 2018 all Hudson masters have been migrated to Jenkins.''' See https://ci.eclipse.org
  
−
* Check [[CI best practices]] for general recommendations how to setup Jenkins.
+
* The following tool was used to convert the configuration XMLs to a format that Jenkins understands: https://github.com/eclipse/hipp2jipp
−
* Check the Eclipse [[CBI]] project for tools, configurations and other information.
+
* Some know issues are listed here: https://github.com/eclipse/hipp2jipp#known-issues
 +
* Also some custom scripts (specific to the Eclipse Foundation's build infrastructure) have been used.
 +
* In most cases the migration worked straight-forward and configurations were converted without any problems. Backups were created and can be made available on request if configurations are lost.
  
−
[[Category:CBI]] [[Category:Releng]] [[Category:Jenkins]]
+
[[Category:Releng]] [[Category:Jenkins]]

Revision as of 15:40, 13 September 2018

100ox

Jenkins is a continuous integration (CI) server. It is in use on Eclipse servers for Eclipse projects as part of the Common Build Infrastructure (CBI). This page is about the hosted service at Eclipse.org. For more information on the project itself, or to download Jenkins, please see the Jenkins project page.

Since Hudson is not maintained anymore, Jenkins is the replacement for it. They share a common ancestry, but have diverged in the past. Eventually all Hudson Instances Per Project (HIPP) will be completely replaced by Jenkins Instances Per Project (JIPP). The migration will be slow in the beginning, to minimize risks and impact on the running build infrastructure (see HIPP2JIPP migration)

General Information

Jenkins instances are maintained by the Eclipse Webmasters/Release Engineer.

Asking for Help

Requesting a JIPP instance for CBI

Please see CBI

Jenkins configuration and tools

Check CI best practices for general recommendations how to setup Jenkins.

Tools (and locations)

Build tools like JDK, Maven, Ant and Gradle are already configured in every Jenkins instance.

  • JDK
    • jdk10-latest (/shared/common/java/oracle/jdk-10_x64-latest)
    • jdk9-latest (/shared/common/jdk-9_x64-latest)
    • jdk1.8.0-latest (/shared/common/jdk1.8.0_x64-latest)
    • jdk1.7.0-latest (/shared/common/jdk1.7.0-latest)
    • jdk1.6.0-latest (/shared/common/jdk1.6.0-latest)
    • jdk1.5.0-latest (/shared/common/jdk1.5.0-latest)
  • Maven
    • apache-maven-latest (/shared/common/apache-maven-latest)
    • apache-maven-3.0.5 (/shared/common/apache-maven-3.0.5)
  • Ant
    • apache-ant-1.9.6 (/shared/common/apache-ant-1.9.6)
  • Gradle
    • gradle-latest (/shared/common/gradle-latest)
    • gradle-3.1 (/shared/common/gradle-3.1)

More generally, all tools listed on http://build.eclipse.org/common/ are available from /shared/common/.

If you need tools that are not general purpose installed, project members can install them in your project's home directory, for example ~/buildtools. See email on cbi-dev

Default plugins

The following plugins are installed by default. Additional plugins can be installed on request.

  • ace-editor
  • analysis-core
  • ant
  • antisamy-markup-formatter
  • authentication-tokens
  • bouncycastle-api
  • branch-api
  • build-timeout
  • cloudbees-folder
  • conditional-buildstep
  • credentials
  • credentials-binding
  • dashboard-view
  • disk-usage
  • display-url-api
  • docker-commons
  • docker-workflow
  • durable-task
  • external-monitor-job
  • extra-columns
  • find-bugs
  • gerrit-trigger
  • git
  • git-client
  • git-parameter
  • git-server
  • gradle
  • greenballs
  • handlebars
  • icon-shim
  • javadoc
  • jobConfigHistory
  • jquery
  • jquery-detached
  • junit
  • ldap
  • mailer
  • matrix-auth
  • matrix-project
  • maven-plugin
  • momentjs
  • pam-auth
  • parameterized-trigger
  • pipeline-build-step
  • pipeline-graph-analysis
  • pipeline-input-step
  • pipeline-milestone-step
  • pipeline-model-api
  • pipeline-model-declarative-agent
  • pipeline-model-definition
  • pipeline-model-extensions
  • pipeline-rest-api
  • pipeline-stage-step
  • pipeline-stage-tags-metadata
  • pipeline-stage-view
  • plain-credentials
  • promoted-builds
  • rebuild
  • resource-disposer
  • scm-api
  • script-security
  • sonar
  • ssh-credentials
  • ssh-slaves
  • structs
  • timestamper
  • token-macro
  • windows-slaves
  • workflow-aggregator
  • workflow-api
  • workflow-basic-steps
  • workflow-cps
  • workflow-cps-global-lib
  • workflow-durable-task-step
  • workflow-job
  • workflow-multibranch
  • workflow-scm-step
  • workflow-step-api
  • workflow-support
  • ws-cleanup
  • xvnc

Setup for specific plugins

GitHub Pull Request Builder Plugin

The GitHub Pull Request Builder Plugin (GHPRB) allows to build/test pull requests and provide immediate feedback in the pull request on GitHub.

To set this up, please open a Bugzilla issue against the CI-Jenkins component (Product: Community) and request this feature.

Here are some details about what happens during the setup process:

  • The GHPRB plugin is installed in the JIPP.
  • Webmaster creates a GitHub bot user and adds it to the respective team on GitHub.
  • The credentials of the GitHub bot user are added to the JIPP (with user name and password, because SSH keys are not recommended/supported by the plugin).
  • The GHPRB plugin's main config is set up.

Once the ticket is resolved you should be able to configure and use the GHPRB plugin in your jobs.

Instructions how to set up GHPRB plugin in jobs can be found here: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Please note: Currently we don't recommend to use the 'Use github hooks for build triggering' option. Instead, with this option turned off, Jenkins is polling GitHub instead. Which should work just fine in most cases.

More info can be found in the GitHub readme: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Jenkins Pipeline (aka configuration in code)

An example how Eclipse plugins can be build with Tycho using a Jenkins pipeline can be found here (Thanks to Mickael Istria!):
https://github.com/eclipse/aCute/blob/master/Jenkinsfile

More info about Jenkins Pipeline can be found here:
https://jenkins.io/doc/book/pipeline/
https://jenkins.io/doc/book/pipeline/shared-libraries/

Gerrit Trigger Plugin

You may use the Jenkins Gerrit Trigger Plugin in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.

Jgit.gerrit-reviewer.png

Jgit.gerrit-vote.png

Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.

General configuration settings

  1. Check This project is parameterized. Click the Add button and select String Parameter. Set the parameter Name to GERRIT_REFSPEC and Default Value to refs/heads/master.

Gerrit-refspec-param.png

Configuration of Source Code Management

  1. Under Source Code Management select Git.
  1. Under Repositories, click on Advanced and change the Refspec to ${GERRIT_REFSPEC}.
  1. Under Additional Behaviours, add Strategy for choosing what to build and select Gerrit Trigger as a strategy.

Jgit.gerrit-git-config.png

Note that the section Branches to build won't be used and may be deleted.

Configuration of Build Triggers

  1. Under Build Triggers, select Gerrit event.

Jgit.gerrit-gerrit-config.png

  1. Under Trigger on, click on Add and select at least Patchset Created. This will configure the job to run on each new patchset. You can also add additional trigger, like Comment Added Contains Regular Expression. In the example below, a build will be triggered for the latest patch set if the comment is exactly CI Bot, run a build please.

Gerrit-trigger-events.png

  1. Finally, configure at least one Gerrit Project. The pattern is the name of project (i.e. if your repository is git.eclipse.org/<xx>/<yy>.git, then fill the pattern xx/yy). The Branches section is the list of branch to listen for events as configured above. Generally, you want one, named master to build patches submitted for the master branche, or ** to build patches submitted to each and every branches. Set the type to Path.

Gerrit-trigger-project.png

Configuration of the build action

Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".

Differences between Hudson and Jenkins

Hudson Jenkins
Job templates "cascading" built-in only rudimentary with plugins
Maven3 configuration Yes No
GitHub pull request builder plugin No Yes

HIPP to JIPP migration (HIPP2JIPP)

As of March 1st, 2018 all Hudson masters have been migrated to Jenkins. See https://ci.eclipse.org

  • The following tool was used to convert the configuration XMLs to a format that Jenkins understands: https://github.com/eclipse/hipp2jipp
  • Some know issues are listed here: https://github.com/eclipse/hipp2jipp#known-issues
  • Also some custom scripts (specific to the Eclipse Foundation's build infrastructure) have been used.
  • In most cases the migration worked straight-forward and configurations were converted without any problems. Backups were created and can be made available on request if configurations are lost.
Retrieved from "https://wiki.eclipse.org/index.php?title=Jenkins&oldid=428146"

Back to the top