Revision as of 14:54, 21 February 2019

Background and future
This page summarizes the result of bug 484004 and bug 510072.
Follow-up will happen in bugs like bug 497982 and bug 406557

Preparing a Repository

Unless stated otherwise the following steps are implemented using bash scripts.

Sources

The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central

• script: CBIaggregator.sh
• aggregation model: SDK4Mvn.aggr

Build Job

This script drives the hudson job https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/CBIaggregator

Preparing the tools

• unpack Eclipse SDK for use as director application (can we use a smaller installation?)
• install the cbiAggregator

Run the aggregator

• the aggregation model basically pulls everything from a given release repo
• configure for all operating systems
• specifies these properties: packedStrategy="UNPACK_AS_SIBLING" mavenResult="true" versionFormat="MavenRelease"
• define maven mappings to map all 3rd party artifacts to their original GAV, as of SDK Neon.2 these are:
  • com/jcraft/jsch/0.1.53
  • com/ibm/icu/icu4j/56.1.0
  • commons-jxpath/commons-jxpath/1.3.0
  • javax/inject/javax.inject/1.0.0
  • junit/junit/4.12.0
  • org/apache/xmlgraphics/batik-css/1.7.0
  • org/apache/lucene/lucene-analyzers/3.5.0
  • org/apache/lucene/lucene-core/3.5.0
  • org/apache/felix/org.apache.felix.gogo.runtime/0.10.0
  • org/apache/felix/org.apache.felix.gogo.shell/0.10.0
  • org/apache/ant/ant/1.9.6
  • org/tukaani/xz/1.3.0
  • org/ow2/sat4j/org.ow2.sat4j.core/2.3.5
  • org/ow2/sat4j/org.ow2.sat4j.pb/2.3.5
  • org/ow2/asm/asm-tree/5.0.1
  • org/ow2/asm/asm/5.0.1
• Since Neon.3 some of the above specify Version Pattern/Template in order to cut of undesired ".0" parts, so junit, e.g., is now 4.12, not 4.12.0

TODO
check the above list against eclipse-sdk-prereqs.target

Remove undesired content

• p2-only content: p2.index p2.packed content.jar artifacts.jar
• all features (irrelevant in plain maven use): *feature.group and *feature.jar
• test plug-ins: org/eclipse/*/*test*
• tooling* (what is this?)
• directories containing a pom but no corresponding jar file
• "garbage", i.e., 3rd party artifacts that after previous clean-up are no longer referenced from any org/eclipse/{platform,jdt,pde} pom files
• directories that are empty after previous clean-up

Technically, some of the clean-up is not necessary, but it helps to give a good picture of what will be published.

Enrich POMs

This step is implemented as a small Java tool.

Sources

The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central, see src/ and enrichPoms.jardesc

Build Job

The tool is built on the fly from CBIaggregator.sh:264 (during hudson job CBIaggregator)

Add some required information to the generated poms:

• dynamic content (retrieved mostly from MANIFEST.MF):
  • name (typically matches what the aggregator filled in as description)
  • url
  • scm connection, tag and url
• semi dynamic content
  • developers (start from the git url, and match it against known project names to link to the corresponding "Who's Involved" page of PMI)
• static content
  • copyright
  • license
  • organization
  • issue management
• hard wired corrections
  • scm connection for emf & ecf
  • fix name of org.eclipse.core.resources.win32.x86* fragments
• fix checksums (md5 & sha1 after pom update -- not needed for publishing)

Add Javadoc stubs

Use a heuristic mapping from bundle name to name of the doc bundle holding corresponding javadoc, one of:

• org.eclipse.jdt.doc.isv
• org.eclipse.pde.doc.user
• org.eclipse.platform.doc.isv

Then

• create a README.txt referring to the doc bundle
• jar the README.txt
• copy the jar to the locations so that it will be picked up by maven

Create missing sources artifacts

Workaround for sources artifacts missing for 8 artifacts (bug 510976).

• org.eclipse.core.net.win32.x86_64
• org.eclipse.core.net.win32.x86
• org.eclipse.core.net.linux.x86_64
• org.eclipse.core.net.linux.x86
• org.eclipse.tips.ui
• org.eclipse.tips.json
• org.eclipse.tips.ide
• org.eclipse.tips.core

Create sources jars:

• check out sources from git
• respect content of build.properties
• manually jar it up and put it into the location expected by maven

Sign and Upload to OSSRH

hudson jobs

• https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/PublishPlatformToMaven.Neon
• https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/PublishJDTtoMaven.Neon
• https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/PublishPDEToMaven.Neon/

Steps:

• copy relevant section of prepared repo into current hudson workspace
• for each *.pom, *.jar, *-sources.jar and *-javadoc.jar
  • invoke maven (cli)
    • pass settings configured for GPG signing and OSSRH account (-s).
    • use a generic pom (-f) (configure maven-gpg-plugin and nexus-staging-maven-plugin (is the latter needed here?))

    see https://bugs.eclipse.org/484004#c70

    • call goal gpg:sign-and-deploy-file
    • passing these properties (-D): url, repositoryId, file, pomFile, classifier

Caveat:

If gpg:sign-and-deploy-file doesn't work for some reason, it's tempting to try manual upload of artifacts instead. This, however, doesn't seem to scale, i.e., it's OK for one artifact, but not for a significant number of artifacts. See how I struggled : OSSRH-27091 comment 389077 (look for item (2)).

Validate and Release

• Login to OSSRH nexus (personal account must have been assigned permissions for the targeted staging area, simply request this in the Jira ticket).
• Identify and select the staging repository created by the previous step (name is group-without-dots-dash-number, where number starts at 1000, e.g., orgeclipseplatform-1005)
• Click close, and wait for validation results to appear under "Activity"
• On success click release, otherwise click drop, fix errors and try again