Tycho:How to deploy to a Maven repository

The typical use case for Tycho builds is to create a p2 Update Site that gets published. In case a project also contains some general artifacts that should be published to a Maven repository like Maven Central via OSSRH, several additional steps are necessary.

Organizational

For deploying to OSSRH it is required that the project has its own Jenkins instance. This is necessary because of the key management.

The following information is provided in the Jenkins Eclipse Wiki.

Deploying artifacts to OSSRH (OSS Repository Hosting provided by Sonatype) requires an account at OSSRH. It is also required to sign all artifacts with GPG. The Eclipse IT team will set this up for the project. For this you need to file a bug via the Helpdesk

The Eclipse IT team will then create a ticket at https://issues.sonatype.org to get a project created so the artifacts can be published to https://oss.sonatype.org

Once the project is setup at https://oss.sonatype.org, the Eclipse IT team will prepare the Jenkins instance of the project. This means the default settings.xml on the server will contain the necessary information for the server, the credentials and the secret file needed for GPG signing.

You also need to sign up for an account at https://issues.sonatype.org/ This is necessary to comment on the created ticket and to release the artifacts once they are deployed to https://oss.sonatype.org

For the process you should comment on the created OOSRH ticket that you need permission on the created project for deployment and let the Eclipse webmaster add a comment to verify this request.

Maven Configuration

The Maven configuration needs to be extended in order to be able to publish to OSSRH. As not every build should be directly promoted, it is a good practice to have that configuration in a separate profile.

In that profile you need to add the distributionManagement section and configure the maven-gpg-plugin. The distributionManagement is necessary to configure where to deploy the artifacts. The id 'ossrh' is used in the default settings.xml file in the Jenkins, so it has to be used this way. The maven-gpg-plugin is needed to sign the artifacts with the secret provided by Sonatype.

Note
With GPG versions > 2.1 the --pinentry-mode loopback needs to be added as gpg argument. If your Jenkins instance is using the migration template, which is needed for example to run UI tests, GPG 2.0.22 is used, which does not know this gpg argument.

The following snippet adds the profile maven-publish to a Jenkins job that is running on centos-7 / migration

<profile>
  <id>maven-publish</id>
 
  <distributionManagement>
    <snapshotRepository>
      <id>ossrh</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
    <repository>
      <id>ossrh</id>
        <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
      </repository>
  </distributionManagement>
 
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-gpg-plugin</artifactId>
        <version>1.6</version>
        <executions>
          <execution>
            <id>sign-artifacts</id>
            <phase>verify</phase>
            <goals>
              <goal>sign</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</profile>

The following snippet adds the profile maven-publish to a Jenkins job that is running on newer templates

<profile>
  <id>maven-publish</id>
 
  <distributionManagement>
    <snapshotRepository>
      <id>ossrh</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
    </snapshotRepository>
    <repository>
      <id>ossrh</id>
      <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
    </repository>
  </distributionManagement>
 
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-gpg-plugin</artifactId>
        <version>1.6</version>
        <executions>
          <execution>
            <id>sign-artifacts</id>
            <phase>verify</phase>
            <goals>
              <goal>sign</goal>
            </goals>
            <configuration>
              <gpgArguments>
                <arg>--pinentry-mode</arg>
                <arg>loopback</arg>
              </gpgArguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</profile>

You also need to add additional meta information sections like name, description, url, scm, licenses and developers to provide more information about the project in Maven Central. This is required to ensure a minimum level of quality of the components available in the Central Repository. Further information can be found here.

In a typical PDE project layout you will deploy a lot more artifacts than necessary or even intended. This is because a PDE project layout typically has:

• plugins
• features
• test plugins/fragments
• releng projects (target platform, update site)
• a parent pom
• in a structured environment also connector poms for the sub-folders

Outside the Eclipse universe only the plugins/bundles are interesting for other developers. Features, update sites and target platform are solely used with p2 and typically do not need to be published on Maven Central. Test plugins/fragments should never be published, as well as connector poms. The parent pom on the other hand is needed to resolve the bundle artifacts that point to the parent pom.

To keep a deployment clean you can configure the maven-deploy-plugin and set the skip configuration parameter to true on projects that should not be promoted.

As the parent pom also needs to be published, it is not possible to skip on parent level and only enable the deployment only for the plugins that should be deployed. Instead you can define Maven profiles that get activated for example if a file exists and then sets the property maven.deploy.skip.

In the pluginManagement of the parent pom.xml add the following plugin configuration to configure the version of the maven-deploy-plugin:

<plugin> 
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-deploy-plugin</artifactId>
  <version>2.8.2</version>
</plugin>

If you have added pom.xml files to the plugin projects to specify the dependencies in Maven style, you could then simply add the following profile to the profiles section, that skips all projects that do not have a pom.xml file:

<profile>
  <id>skip-deploy</id>
  <activation>
    <file>
      <missing>pom.xml</missing>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>

This of course means that you have to skip the deployment of projects that contain a pom.xml file, but should not be published (e.g. product builds that contain a pom.xml for customization). This can be done by adding the following configuration to that pom.xml:

<build>
  <plugins>
    <!-- do not publish this artifact to Maven repositories -->
    <plugin>
      <artifactId>maven-deploy-plugin</artifactId>
        <configuration>
          <skip>true</skip>
        </configuration>
    </plugin>
  </plugins>
</build>

If you want to stick to a pomless build without any additional pom.xml files in your project, you need to have several profiles with different activation patterns, like the following snippet for example:

<!-- do not deploy connector pom.xml files -->
<profile>
  <id>skip-deploy-connector-pom</id>
  <activation>
    <file>
      <exists>.polyglot.pom.tycho</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>
 
<!-- do not deploy features -->
<profile>
  <id>skip-deploy-eclipse-feature</id>
  <activation>
    <file>
      <exists>feature.xml</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>
 
<!-- do not deploy update sites -->
<profile>
  <id>skip-deploy-eclipse-updatesite</id>
  <activation>
    <file>
      <exists>category.xml</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>
 
<!-- do not deploy test bundles -->
<profile>
  <id>skip-deploy-eclipse-test</id>
  <activation>
    <file>
      <exists>no_deploy.txt</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>
 
<!-- do not deploy target definitions -->
<profile>
  <id>skip-deploy-eclipse-target</id>
  <activation>
    <file>
      <exists>[project_file].target</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>
 
<!-- do not deploy products -->
<profile>
  <id>skip-deploy-eclipse-product</id>
  <activation>
    <file>
      <exists>[project_file].product</exists>
    </file>
  </activation>
  <properties>
    <maven.deploy.skip>true</maven.deploy.skip>
  </properties>
</profile>

Note
For tests you could also use the pattern target/surefire.properties which works locally, but fails on Jenkins as the file doesn't seem to exist at the time the check is performed. In such a case you need some marker file to be able identify test plugin projects that should not be deployed, e.g. an empty file named no_deploy.txt

Note
For products and target definitions there is no generic pattern for the filename. Wildcards seem to be not supported, so you have to use the concrete filename of your project.

To verify which profiles are activated, you can use the following command:

mvn help:active-profiles

Jenkins configuration

To be able to deploy artifacts to OSSRH, the build needs to be executed on the dedicated JIPP instance. This is because the artifacts need to be signed with GPG, and the keys will be available on your JIPP instance once the Eclipse webmaster has received it from Sonatype and configured the JIPP instance.

You can find the detailed information in How can artifacts be deployed to OSSRH / Maven Central?. The following will only give some hints on the configuration in Jenkins:

• Create a dedicated release build configuration (it is a good practice to separate the release build job from other jobs like Gerrit triggers or SNAPSHOT builds)
• Ensure to checkout the release branch (it is a good practice to have release branches separated from master or develop branches, but if master is the branch you release, than this is your release branch)
• Configure the secrets file as described here
• Add the shell script for GPG signing as build step BEFORE the Maven execution step
• Add the Maven build step mvn clean deploy -DskipTests=true -Pmaven-publish
  • The goal deploy is the last phase in the Maven build lifecycle and should only be used in the build environment as it copies the final package to the remote repository. Therefore it is discussable if a separate profile is really needed, but a clean separation via profile can be seen as second guard.
  • The test execution is skipped via -DskipTests=true, as usually the tests should be green BEFORE even considering a deployment. Therefore the execution of tests should not be necessary here to safe processing time.
  • Via -Pmaven-publish the profile is activated.

If you use the release build job for Maven deployment AND Eclipse update site deployment, you should also execute the eclipse-jarsigner-plugin (assuming you already have a dedicated profile for that job named sign

mvn clean deploy -DskipTests=true -Psign,maven-publish

Note
For testing the deployment without directly publishing, you can set the system property altDeploymentRepository to point to a folder in your local environment.

mvn clean deploy -DaltDeploymentRepository=local-repo::default::file:../../repo_test/releases

Release Process

The following section describes a typical release process. It is not mandatory to follow it exactly, but it should give hints on the required steps.

• Create a dedicated release branch releases/VERSION, e.g. releases/1.0.0

Note
Some projects use their master branch for the release or have another branch scheme for releases. So releases/VERSION is just a common pattern that can be used.

• Remove -SNAPSHOT and .qualifier from any version numbers in the release branch

Note
This is necessary for Maven to push the build to the proper Maven Central release staging area. Be careful not to change anything in source code. This should mainly affect:

• category.xml
• pom.xml
• feature.xml
• MANIFEST.MF

Using the tycho-versions-plugin this step should be simple:

mvn org.eclipse.tycho:tycho-versions-plugin:set-version -DnewVersion=VERSION

• Push the changes to the release branch
• Trigger the build on Jenkins

Once the build finishes successfully, you have deployed the artifacts to the OSSRH staging area. According to the documentation you now need to release the deployment to Maven Central.

• Login to https://oss.sonatype.org/
  • In the left menu check for Build Promotion - Staging Repositories
  • Select your repository and validate if it meets your expectations
  • If everything is correct Close the repository, otherwise Drop it and fix any issues
  • Once it is closed, press the Release button to trigger the release to the Central repository
  • The sync needs to be enabled manually, so you need to add a comment on the OSSRH ticket that everything worked, so they can enable the sync

• Finally update the version in the master/develop branch

mvn org.eclipse.tycho:tycho-versions-plugin:set-version -DnewVersion=NEW_VERSION_NUMBER.qualifier

The manual release process by logging into https://oss.sonatype.org/ can be automized by using the Nexus Staging Plugin as described here. You need to add the following plugin configuration to the plugins section of your parent pom.xml. If you have introduced a dedicated profile for the deployment, it is a good place to put the configuration in that profile.

<plugin>
  <groupId>org.sonatype.plugins</groupId>
  <artifactId>nexus-staging-maven-plugin</artifactId>
  <version>1.6.7</version>
  <extensions>true</extensions>
  <configuration>
     <serverId>ossrh</serverId>
     <nexusUrl>https://oss.sonatype.org/</nexusUrl>
     <autoReleaseAfterClose>true</autoReleaseAfterClose>
  </configuration>
</plugin>