Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Jenkins"

m (Fix typos)
(FAQ)
Line 225: Line 225:
  
 
== FAQ ==
 
== FAQ ==
 +
 +
{{important|FAQ|The following FAQ only applies to Eclipse projects running their builds on the '''new infrastructure based on CloudBees Core'''. It does not apply to projects running on the old infrastructure.}}
  
 
=== How do I run my build in a custom container? ===
 
=== How do I run my build in a custom container? ===

Revision as of 05:25, 7 November 2018

Jenkins logo.png

Jenkins is a continuous integration (CI) server. It is in use on Eclipse servers for Eclipse projects as part of the Common Build Infrastructure (CBI). This page is about the hosted service at Eclipse.org. For more information on the project itself, or to download Jenkins, please see the Jenkins project page.

Since Hudson is not maintained anymore, Jenkins is the replacement for it. They share a common ancestry, but have diverged in the past. Eventually all Hudson Instances Per Project (HIPP) will be completely replaced by Jenkins Instances Per Project (JIPP). The migration will be slow in the beginning, to minimize risks and impact on the running build infrastructure (see HIPP2JIPP migration)

Contents

General Information

Jenkins instances are maintained by the Eclipse Webmasters/Release Engineers.

Asking for Help

Requesting a JIPP instance for CBI

Please see CBI

Jenkins configuration and tools (clustered infra)

Tools (and locations on the default JNLP agent container)

Apache Maven

  • apache-maven-latest /opt/tools/apache-maven/latest (= apache-maven-3.5.4)
  • apache-maven-3.5.4 /opt/tools/apache-maven/3.5.4
  • apache-maven-3.3.9 /opt/tools/apache-maven/3.3.0
  • apache-maven-3.2.5 /opt/tools/apache-maven/3.2.5

JDK

OpenJDK

These binaries come from http://jdk.java.net. These are production-ready open-source builds of the Java Development Kit, an implementation of the Java SE Platform under the GNU General Public License, version 2, with the Classpath Exception. See the differences between these binaries and Oracle's one for version 11 onward on Oracle's director of product management blog post.

  • openjdk-latest /opt/tools/java/openjdk/latest(= openjdk-jdk11-latest)
  • openjdk-jdk11-latest /opt/tools/java/openjdk/jdk-11/latest = 11 GA
  • openjdk-jdk10-latest /opt/tools/java/openjdk/jdk-10/latest = 10.0.2
  • openjdk-jdk9-latest /opt/tools/java/openjdk/jdk-9/latest = 9.0.4

AdoptOpenJDK

These binaries come from https://adoptopenjdk.net. These OpenJDK binaries a built from a fully open source set of build scripts and infrastructure.

With HotSpot
  • adoptopenjdk-hotspot-latest /opt/tools/java/adoptopenjdk/hotspot-latest(= adoptopenjdk-hotspot-jdk11-latest)
  • adoptopenjdk-hotspot-jdk11-latest /opt/tools/java/adoptopenjdk/hotspot-jdk-11/latest = 11 GA
  • adoptopenjdk-hotspot-jdk10-latest /opt/tools/java/adoptopenjdk/hotspot-jdk-10/latest = 10.0.2
  • adoptopenjdk-hotspot-jdk9-latest /opt/tools/java/adoptopenjdk/hotspot-jdk-9/latest = 9.0.4
  • adoptopenjdk-hotspot-jdk8-latest /opt/tools/java/adoptopenjdk/hotspot-jdk-8/latest = 1.8.0u171
With OpenJ9

These binaries replace the traditional HotSpot implementation of the Java Virtual Machine implementation with Eclipse OpenJ9. Eclipse OpenJ9 is a high performance, scalable, Java virtual machine implementation that is fully compliant with the Java Virtual Machine Specification.

  • adoptopenjdk-openj9-latest /opt/tools/java/adoptopenjdk/openj9-latest(= adoptopenjdk-openj9-jdk11-latest)
  • adoptopenjdk-openj9-jdk11-latest /opt/tools/java/adoptopenjdk/openj9-jdk-11/latest = 11 GA
  • adoptopenjdk-openj9-jdk10-latest /opt/tools/java/adoptopenjdk/openj9-jdk-10/latest = 10.0.2
  • adoptopenjdk-openj9-jdk9-latest /opt/tools/java/adoptopenjdk/openj9-jdk-9/latest = 9.0.4
  • adoptopenjdk-openj9-jdk8-latest /opt/tools/java/adoptopenjdk/openj9-jdk-8/latest = 1.8.0u181

Oracle

These binaries come from the Oracle Technology Network. Note that the new Oracle Technology Network (OTN) License Agreement for Oracle Java SE is substantially different from the licenses under which previous versions of the JDK were offered. Previous versions of Oracle JDK were release under the previous Oracle Binary Code License (BCL) for Java SE. As such, starting with JDK 11, the Eclipse Foundation won't provide any version of the JDK from Oracle licensed under the -commercial- OTN terms. Previous versions listed below, will stay available as is. See the cosmetic and packaging differences between Oracle's OpenJDK Builds (GPL+CE) — simply named OpenJDK above — and Oracle JDK (OTN) on Oracle Director of Product Management's blog post.

  • oracle-jdk-latest /opt/tools/java/oracle/latest (= oracle-jdk10-latest)
  • oracle-jdk10-latest /opt/tools/java/oracle/jdk-10/latest = 10.0.2
  • oracle-jdk9-latest /opt/tools/java/oracle/jdk-9/latest = 9.0.4
  • oracle-jdk8-latest /opt/tools/java/oracle/jdk-8/latest = 1.8.0u181
  • oracle-jdk7-latest /opt/tools/java/oracle/jdk-7/latest = 1.7.0u80
  • oracle-jdk6-latest /opt/tools/java/oracle/jdk-6/latest = 1.6.0u45
  • oracle-jdk5-latest /opt/tools/java/oracle/jdk-5/latest = 1.5.0u22

Ant

  • apache-ant-latest (1.10.5, automatically installed from Apache server)

Default plugins

  • ace-editor
  • analysis-core
  • ant
  • antisamy-markup-formatter
  • apache-httpcomponents-client-4-api
  • async-http-client
  • authentication-tokens
  • aws-credentials
  • aws-java-sdk
  • blueocean-commons
  • bouncycastle-api
  • branch-api
  • build-timeout
  • cloudbees-assurance
  • cloudbees-blueocean-default-theme
  • cloudbees-folder
  • cloudbees-folders-plus
  • cloudbees-groovy-view
  • cloudbees-jsync-archiver
  • cloudbees-license
  • cloudbees-monitoring
  • cloudbees-nodes-plus
  • cloudbees-ssh-slaves
  • cloudbees-support
  • cloudbees-template
  • cloudbees-uc-data-api
  • cloudbees-view-creation-filter
  • cloudbees-workflow-template
  • cloudbees-workflow-ui
  • command-launcher
  • conditional-buildstep
  • config-file-provider
  • credentials
  • credentials-binding
  • dashboard-view
  • disk-usage
  • display-url-api
  • docker-commons
  • docker-workflow
  • durable-task
  • email-ext
  • envinject
  • envinject-api
  • extended-read-permission
  • external-monitor-job
  • extra-columns
  • findbugs
  • form-element-path
  • gerrit-trigger
  • ghprb
  • git
  • git-client
  • git-parameter
  • git-server
  • github
  • github-api
  • github-branch-source
  • gradle
  • greenballs
  • handlebars
  • icon-shim
  • infradna-backup
  • jackson2-api
  • javadoc
  • jdk-tool
  • jobConfigHistory
  • jquery
  • jquery-detached
  • jsch
  • junit
  • kube-agent-management
  • kubernetes
  • kubernetes-credentials
  • ldap
  • mailer
  • mapdb-api
  • matrix-auth
  • matrix-project
  • maven-plugin
  • metrics
  • momentjs
  • nectar-license
  • nectar-rbac
  • node-iterator-api
  • operations-center-agent
  • operations-center-client
  • operations-center-cloud
  • operations-center-context
  • pam-auth
  • parameterized-trigger
  • pipeline-build-step
  • pipeline-graph-analysis
  • pipeline-input-step
  • pipeline-milestone-step
  • pipeline-model-api
  • pipeline-model-declarative-agent
  • pipeline-model-definition
  • pipeline-model-extensions
  • pipeline-rest-api
  • pipeline-stage-step
  • pipeline-stage-tags-metadata
  • pipeline-stage-view
  • plain-credentials
  • promoted-builds
  • rebuild
  • resource-disposer
  • run-condition
  • scm-api
  • script-security
  • ssh-agent
  • ssh-credentials
  • ssh-slaves
  • structs
  • support-core
  • timestamper
  • token-macro
  • unique-id
  • variant
  • wikitext
  • windows-slaves
  • workflow-aggregator
  • workflow-api
  • workflow-basic-steps
  • workflow-cps
  • workflow-cps-checkpoint
  • workflow-cps-global-lib
  • workflow-durable-task-step
  • workflow-job
  • workflow-multibranch
  • workflow-scm-step
  • workflow-step-api
  • workflow-support
  • ws-cleanup
  • xvnc


FAQ

Important.png
FAQ
The following FAQ only applies to Eclipse projects running their builds on the new infrastructure based on CloudBees Core. It does not apply to projects running on the old infrastructure.


How do I run my build in a custom container?

You need to use Jenkins pipeline to do so. Then you can specify a Kubernetes pod template. See an example below.

pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    command:
    - cat
    tty: true
  - name: php
    image: php:7.2.10-alpine
    command:
    - cat
    tty: true
  - name: hugo
    image: eclipsecbi/hugo:0.42.1
    command:
    - cat
    tty: true
"""
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('maven') {
          sh 'mvn -version'
        }
        container('php') {
          sh 'php -version'
        }
        container('hugo') {
          sh 'hugo -version'
        }
      }
    }
  }
}

See the Kubernetes Jenkins plugin for more documentation.

How do I deploy artifacts to download.eclipse.org?

You cannot just cp stuff to a folder. You need to do that with ssh. The first thing to do is to request write access to the projects storage service for your Jenkins instance. This service provide access to the Eclipse Foundation file servers storage:

  • /home/httpd/data/download.eclipse.org
  • /home/httpd/data/archive.eclipse.org
  • /home/httpd/data/download.polarsys.org
  • /home/httpd/data/download.locationtech.org

Then, we will configure your Jenkins instance with SSH credentials. Depending on how you run your build, the way you will use them are different. See the different cases below.

Freestyle job

You need to activate the SSH Agent plugin in your job configuration and select the proper credentials genie.projectname (ssh://projects-storage.eclipse.org).

Project-storage-ssh-agent.png

Then you use ssh, scp and sftp commands to deploy artifacts to the server, e.g.,

scp target/my_artifact.jar genie.<projectname>@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/<projectname>/
ssh genie.<projectname>@projects-storage.eclipse.org ls -al /home/data/httpd/download.eclipse.org/<projectname</

Pipeline job without custom pod template

pipeline {
  agent any
 
  stages {
    stage('stage 1') {
      ...
    }
    stage('Deploy') {
      steps {
        sshagent ( ['project-storage.eclipse.org-bot-ssh']) {
          sh '''
            ssh genie.projectname@projects-storage.eclipse.org rm -rf /home/data/httpd/download.eclipse.org/projectname/snapshots
            ssh genie.projectname@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/projectname/snapshots
            scp -r repository/target/repository/* genie.projectname@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/projectname/snapshots
          '''
        }
      }
    }
  }
}

Pipeline job with custom pod template

pipeline {
  agent {
    kubernetes {
      label 'my-pod'
      yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    command:
    - cat
    tty: true
  - name: ssh-client
    image: 'eclipsecbi/ssh-client:1.0'
    command:
    - cat
    tty: true
    volumeMounts:
    - mountPath: /home/jenkins/.ssh
      name: volume-known-hosts
  volumes:
  - configMap:
      name: known-hosts
    name: volume-known-hosts
'''
    }
  }
  stages {
    stage('Stage 1') {
      ...
    }
    stage('Deploy') {
      steps {
        container('ssh-client') {
          sshagent ( ['project-storage.eclipse.org-bot-ssh']) {
            sh '''
              ssh genie.projectname@projects-storage.eclipse.org rm -rf /home/data/httpd/download.eclipse.org/projectname/snapshots
              ssh genie.projectname@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/projectname/snapshots
              scp -r repository/target/repository/* genie.projectname@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/projectname/snapshots
            '''
          }
        }
      }
    }
  }
}

My build fails with "No user exists for uid 1000100000", what's the issue?

First, you need to know that we run containers using an arbitrarily assigned user ID (1000100000) in our OpenShift cluster. This is for security reasons.

Unfortunately, most of images you can find on DockerHub (including official images) do not support running as an arbitrary user. Actually, most of them expect to run as root, which is definitely a bad practice. See also question below about how to run a container as root.

Moreover, some programs like ssh search for a mapping between the user ID (1000100000) and a user name on the system (here a container). It's very rare that any container anticipate this need and actually created a user with ID=1000100000. To avoid this error, you need to customize the image. OpenShift publishes guidelines with best practices about how to create Docker images. More specifically, see the section about how to support running with arbitrary user ID.

If you want to see in practice, have a look at some images we've defined to run in the cluster on this GitHub repository.

How can I run my build in a container with root privileges?

Unfortunately, for security reasons, you cannot do that. We run an infrastructure open to the internet, which potentially runs stuff from non-trusted code (e.g., PR) so we need to follow a strict policy to protect the common good.

More specifically, we run containers using an arbitrarily assigned user ID (1000100000) in our OpenShift cluster. The security context constraints we use for running projects' containers is "restricted". You cannot change this level from your podTemplate.

Unfortunately, most of images you can find on DockerHub (including official images) do not support running as an arbitrary user. Actually, most of them expect to run as root, which is definitely a bad practice.

OpenShift publishes guidelines with best practices about how to create Docker images. More specifically, see the section about how to support running with arbitrary user ID.

I want to build a custom Docker image (with docker build), but it does not work. What should I do?

You cannot currently build images on the cluster (i.e. docker build does not work). We plan to address this shortcoming shortly.

My build fails with "Host key verification failed", what should I do?

As long as you stay in the default jnlp docker image (i.e. use a Freestyle Job or a Pipeline job without custom pod template), you'll benefit from our existing configuration where we mount a known_hosts file in the ~/.ssh folder of all containers.

If you define a custom pod template, you need to add some configuration to mount this config map in your containers. The only thing that you have to know is the config map name known-hosts and mount it at the proper location /home/jenkins/.ssh.

pipeline {
  agent {
    kubernetes {
      label 'my-agent-pod'
      yaml """
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: maven
    image: maven:alpine
    command:
    - cat
    tty: true
    volumeMounts:
    - mountPath: /home/jenkins/.ssh
      name: volume-known-hosts
  volumes:
  - configMap:
      name: known-hosts
    name: volume-known-hosts
"""
    }
  }
  stages {
    ...
  }
}

Currently, the known_hosts file we provide as the key for the following sites:

  • git.eclipse.org:22
  • git.eclipse.org:29418
  • build.eclipse.org
  • github.com

If you need any other site to be added, feel free to open a request.

How do I use the local Nexus server as proxy for Maven Central (artifact caching)?

Every JIPP on the new infra already has a Maven settings file set up that specifies our local Nexus instance as cache for Maven Central. Unfortunately in the new infra we have not found a way to make this the default Maven settings file for all Maven builds (yet). That's why it needs to be specified manually.

The name of the settings file has the format settings-<projectshortname>.xml, e.g. settings-metro.xml for the Eclipse Metro project.

Freestyle job

In a freestyle job the settings file needs to be injected. It can either be used with a variable (e.g. MVNSETTINGS), which you can reference later with mvn -s $MVNSETTINGS clean verify [...] in a shell build step.

ProvideConfigFiles variable .png

Or you can set the target to /home/jenkins/.m2/settings.xml. The settings.xml will then be picked up automatically by Maven builds.

ProvideConfigFiles target.png

Pipeline job

In a pipeline job you have to specify the fileId of the Maven settings file as an UUID. You can find out the correct fileId by using the pipeline syntax generator and choosing "configFileProvider: Provide Configuration files" as the sample step.

[...]
configFileProvider([configFile(fileId: '5f77ec66-dc5e-4d29-999f-311501789ba0', variable: 'MVN_SETTINGS')]) {
  sh "${mvnHome}/bin/mvn -s $MVN_SETTINGS clean verify"
}
[...]

Jenkins configuration and tools (bare-metal infra)

Check CI best practices for general recommendations how to setup Jenkins.

Tools (and locations)

Build tools like JDK, Maven, Ant and Gradle are already configured in every Jenkins instance.

  • JDK

Tools labeled jdk are Oracle JDK licensed under the Oracle Binary Code License (BCL) for Java SE. Tools labeled openjdk are Oracle builds of OpenJDK under the GPL+CE license. Starting with JDK 11, the Eclipse Foundation won't provide any version of the JDK from Oracle licensed under the -commercial- Oracle Technology Network (OTN) terms. See OpenJDK and Oracle JDK sections above to learn more.

    • openjdk-jdk12-ea-latest (/shared/common/java/openjdk/jdk-12_x64-latest)
    • openjdk-jdk11-latest (/shared/common/java/openjdk/jdk-11_x64-latest)
    • jdk10-latest (/shared/common/java/oracle/jdk-10_x64-latest)
    • jdk9-latest (/shared/common/jdk-9_x64-latest)
    • jdk1.8.0-latest (/shared/common/jdk1.8.0_x64-latest)
    • jdk1.7.0-latest (/shared/common/jdk1.7.0-latest)
    • jdk1.6.0-latest (/shared/common/jdk1.6.0-latest)
    • jdk1.5.0-latest (/shared/common/jdk1.5.0-latest)
  • Maven
    • apache-maven-latest (/shared/common/apache-maven-latest)
    • apache-maven-3.0.5 (/shared/common/apache-maven-3.0.5)
  • Ant
    • apache-ant-1.9.6 (/shared/common/apache-ant-1.9.6)
  • Gradle
    • gradle-latest (/shared/common/gradle-latest)
    • gradle-3.1 (/shared/common/gradle-3.1)

More generally, all tools listed on http://build.eclipse.org/common/ are available from /shared/common/.

If you need tools that are not general purpose installed, project members can install them in your project's home directory, for example ~/buildtools. See email on cbi-dev

Default plugins

The following plugins are installed by default. Additional plugins can be installed on request.

  • ace-editor
  • analysis-core
  • ant
  • antisamy-markup-formatter
  • authentication-tokens
  • bouncycastle-api
  • branch-api
  • build-timeout
  • cloudbees-folder
  • conditional-buildstep
  • credentials
  • credentials-binding
  • dashboard-view
  • disk-usage
  • display-url-api
  • docker-commons
  • docker-workflow
  • durable-task
  • external-monitor-job
  • extra-columns
  • find-bugs
  • gerrit-trigger
  • git
  • git-client
  • git-parameter
  • git-server
  • gradle
  • greenballs
  • handlebars
  • icon-shim
  • javadoc
  • jobConfigHistory
  • jquery
  • jquery-detached
  • junit
  • ldap
  • mailer
  • matrix-auth
  • matrix-project
  • maven-plugin
  • momentjs
  • pam-auth
  • parameterized-trigger
  • pipeline-build-step
  • pipeline-graph-analysis
  • pipeline-input-step
  • pipeline-milestone-step
  • pipeline-model-api
  • pipeline-model-declarative-agent
  • pipeline-model-definition
  • pipeline-model-extensions
  • pipeline-rest-api
  • pipeline-stage-step
  • pipeline-stage-tags-metadata
  • pipeline-stage-view
  • plain-credentials
  • promoted-builds
  • rebuild
  • resource-disposer
  • scm-api
  • script-security
  • sonar
  • ssh-credentials
  • ssh-slaves
  • structs
  • timestamper
  • token-macro
  • windows-slaves
  • workflow-aggregator
  • workflow-api
  • workflow-basic-steps
  • workflow-cps
  • workflow-cps-global-lib
  • workflow-durable-task-step
  • workflow-job
  • workflow-multibranch
  • workflow-scm-step
  • workflow-step-api
  • workflow-support
  • ws-cleanup
  • xvnc

Setup for specific plugins

GitHub Pull Request Builder Plugin

The GitHub Pull Request Builder Plugin (GHPRB) allows to build/test pull requests and provide immediate feedback in the pull request on GitHub.

To set this up, please open a Bugzilla issue against the CI-Jenkins component (Product: Community) and request this feature.

Here are some details about what happens during the setup process:

  • The GHPRB plugin is installed in the JIPP.
  • Webmaster creates a GitHub bot user and adds it to the respective team on GitHub.
  • The credentials of the GitHub bot user are added to the JIPP (with user name and password, because SSH keys are not recommended/supported by the plugin).
  • The GHPRB plugin's main config is set up.

Once the ticket is resolved you should be able to configure and use the GHPRB plugin in your jobs.

Instructions how to set up GHPRB plugin in jobs can be found here: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Please note: Please note that the 'Use github hooks for build triggering' option has to be disabled, since it requires admin permissions for the GitHub bot user, which we don't allow. With this option turned off, Jenkins is polling GitHub instead. Which should work just fine in most cases.

More info can be found in the GitHub readme: https://github.com/jenkinsci/ghprb-plugin/blob/master/README.md

Jenkins Pipeline (aka configuration in code)

An example how Eclipse plugins can be build with Tycho using a Jenkins pipeline can be found here (Thanks to Mickael Istria!):
https://github.com/eclipse/aCute/blob/master/Jenkinsfile

More info about Jenkins Pipeline can be found here:
https://jenkins.io/doc/book/pipeline/
https://jenkins.io/doc/book/pipeline/shared-libraries/

Gerrit Trigger Plugin

You may use the Jenkins Gerrit Trigger Plugin in order to run a Jenkins job to verify each new patchset uploaded to Gerrit for code review. Jenkins (named "CI Bot") will then also vote on these changes using the "Verify" voting category.

Jgit.gerrit-reviewer.png

Jgit.gerrit-vote.png

Below, the configuration sections for the Git plugin and the Gerrit trigger plugin of the verification job used by the EGit project may serve as an example.

General configuration settings

  1. Check This project is parameterized. Click the Add button and select String Parameter. Set the parameter Name to GERRIT_REFSPEC and Default Value to refs/heads/master.

Gerrit-refspec-param.png

Configuration of Source Code Management

  1. Under Source Code Management select Git.
  1. Under Repositories, click on Advanced and change the Refspec to ${GERRIT_REFSPEC}.
  1. Under Additional Behaviours, add Strategy for choosing what to build and select Gerrit Trigger as a strategy.

Jgit.gerrit-git-config.png

Note that the section Branches to build won't be used and may be deleted.

Configuration of Build Triggers

  1. Under Build Triggers, select Gerrit event.

Jgit.gerrit-gerrit-config.png

  1. Under Trigger on, click on Add and select at least Patchset Created. This will configure the job to run on each new patchset. You can also add additional trigger, like Comment Added Contains Regular Expression. In the example below, a build will be triggered for the latest patch set if the comment is exactly CI Bot, run a build please.

Gerrit-trigger-events.png

  1. Finally, configure at least one Gerrit Project. The pattern is the name of project (i.e. if your repository is git.eclipse.org/<xx>/<yy>.git, then fill the pattern xx/yy). The Branches section is the list of branch to listen for events as configured above. Generally, you want one, named master to build patches submitted for the master branche, or ** to build patches submitted to each and every branches. Set the type to Path.

Gerrit-trigger-project.png

Configuration of the build action

Under "Build" click the "Add a build step" button, and select the appropriate action. The actual action depends on what you want Hudson to do. A typical example, for projects build with Maven is to select "Invoke Maven 3" and set "Maven 3" to "apache-maven-latest" and "Goals" to "clean verify".

Howto

Build my project's website with Jenkins?

The preferred static website generator for build Eclipse project websites is Hugo. You should first put your Hugo sources in a dedicated Git repository, either at GitHub if your source code is already hosted at GitHub or at git.eclipse.org. If you don't have such a repository already, feel free to open a request, the Eclipse IT team will create one for you.

Note that each and every Eclipse project automatically gets a Git repository with git.eclipse.org/www.eclipse.org/<project_name> (see this repository index for complete list). This is not where you want to push your Hugo sources. This repository contains the webpages that are automatically and regularly pulled and published on the www.eclipse.org HTTP server. All the content from the master branch will eventually be available at the URL https://www.eclipse.org/<project_name/>.

Once your Hugo sources are in the proper repository, create a file named Jenkinsfile at the root of the repository with the following content (don't forget to specify the proper value for PROJECT_NAME and PROJECT_BOT_NAME environment variable):

pipeline {
 
  agent {
    kubernetes {
      label 'hugo-agent'
      yaml """
apiVersion: v1
metadata:
  labels:
    run: hugo
  name: hugo-pod
spec:
  containers:
    - name: hugo
      image: eclipsecbi/hugo:0.42.1
      command:
      - cat
      tty: true
"""
    }
  }
 
  environment {
    PROJECT_NAME = "<project_name>" // must be all lowercase.
    PROJECT_BOT_NAME = "<Project_name> Bot" // Capitalize the name
 
  }
 
  options {
    buildDiscarder(logRotator(numToKeepStr: '5'))
    checkoutToSubdirectory('hugo')
  }
 
  stages {
    stage('Checkout www repo') {
      steps {
        dir('www') {
            git branch: '$env.BRANCH_NAME',
                url: 'ssh://genie.${PROJECT_NAME}@git.eclipse.org:29418/www.eclipse.org/${PROJECT_NAME}.git',
                credentialsId: 'git.eclipse.org-bot-ssh'
        }
      }
    }
    stage('Build website with Hugo') {
      steps {
        container('hugo') {
            dir('hugo') {
                sh 'hugo -b https://www.eclipse.org/${PROJECT_NAME}/'
            }
        }
      }
    }
    stage('Push to $env.BRANCH_NAME branch') {
      steps {
        sh 'cp -Rvf hugo/public/* www/'
        dir('www') {
            sshagent(['git.eclipse.org-bot-ssh']) {
                sh '''
                git add -A
                if ! git diff --cached --exit-code; then
                  echo "Changes have been detected, publishing to repo 'www.eclipse.org/${PROJECT_NAME}'"
                  git config --global user.email "${PROJECT_NAME}-bot@eclipse.org"
                  git config --global user.name "${PROJECT_BOT_NAME}"
                  git commit -m "Website build ${JOB_NAME}-${BUILD_NUMBER}"
                  git log --graph --abbrev-commit --date=relative -n 5
                  git push origin HEAD:$env.BRANCH_NAME
                else
                  echo "No change have been detected since last build, nothing to publish"
                fi
                '''
            }
        }
      }
    }
  }
}


Finally, you can create a multibranch pipeline job on your project's Jenkins instance. It will automatically be triggered on every new push to your Hugo source repository, build the website and push it to the git.eclipse.org/www.eclipse.org/<project_name>.git repository. As mentioned above, the Eclipse Foundation website's infrastructure will eventually pull the content of the latter and your website will be published and available on http://www.eclipse.org/<project_name>.

If you don't have a Jenkins instance already, ask for one. If you need assistance with the process, open a ticket.

HIPP to JIPP migration (HIPP2JIPP)

As of March 1st, 2018 all Hudson masters have been migrated to Jenkins. See https://ci.eclipse.org

  • The following tool was used to convert the configuration XMLs to a format that Jenkins understands: https://github.com/eclipse/hipp2jipp
  • Some know issues are listed here: https://github.com/eclipse/hipp2jipp#known-issues
  • Also some custom scripts (specific to the Eclipse Foundation's build infrastructure) have been used.
  • In most cases the migration worked straight-forward and configurations were converted without any problems. Backups were created and can be made available on request if configurations are lost.

Back to the top