Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "Platform-releng/Publish To Maven Central"

(Link to OSSRH nexus)
(the sources have migrated to releng git)
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{tip|Background and future|This page summarizes the result of {{FixedBug|484004}}.<br/>Follow-up will happen in bugs like {{Bug|510072}} and {{Bug|497982}}}}
+
{{tip|Background and future|This page summarizes the result of {{FixedBug|484004}} and  {{FixedBug|510072}}.<br/>Follow-up will happen in bugs like {{Bug|497982}} and {{Bug|406557}}}}
  
 
=Preparing a Repository=
 
=Preparing a Repository=
 
Unless stated otherwise the following steps are implemented using bash scripts.
 
Unless stated otherwise the following steps are implemented using bash scripts.
;Sources:During development the sources are hosted in http://git.eclipse.org/c/objectteams/org.eclipse.objectteams.git/tree/releng/EnrichPoms/
+
;Sources:The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central
:* script: [http://git.eclipse.org/c/objectteams/org.eclipse.objectteams.git/tree/releng/EnrichPoms/CBIaggregator.sh CBIaggregator.sh]
+
:* script: [https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central/CBIaggregator.sh CBIaggregator.sh]
:* aggregation model: [http://git.eclipse.org/c/objectteams/org.eclipse.objectteams.git/tree/releng/EnrichPoms/SDK4Mvn.aggr SDK4Mvn.aggr]
+
:* aggregation model: [https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central/SDK4Mvn.aggr SDK4Mvn.aggr]
 
;Build Job:This script drives the hudson job https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/CBIaggregator
 
;Build Job:This script drives the hudson job https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/CBIaggregator
  
Line 49: Line 49:
 
=Enrich POMs=
 
=Enrich POMs=
 
''This step is implemented as a small Java tool''.
 
''This step is implemented as a small Java tool''.
;Sources:During development the sources are hosted in http://git.eclipse.org/c/objectteams/org.eclipse.objectteams.git/tree/releng/EnrichPoms/
+
;Sources:The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central, see <code>src/</code> and <code>enrichPoms.jardesc</code>
;Build Job:The tool is built via hudson job https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/pomEnricher
+
;Build Job:The tool is built on the fly from [https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central/CBIaggregator.sh#n264 CBIaggregator.sh:264] (during hudson job [https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/CBIaggregator CBIaggregator])
  
  
Line 82: Line 82:
  
 
=Create missing sources artifacts=
 
=Create missing sources artifacts=
Workaround for sources artifacts missing for 8 artifacts.
+
Workaround for sources artifacts missing for 8 artifacts ({{bug|510976}}).
 
* org.eclipse.core.net.win32.x86_64
 
* org.eclipse.core.net.win32.x86_64
 
* org.eclipse.core.net.win32.x86
 
* org.eclipse.core.net.win32.x86
 
* org.eclipse.core.net.linux.x86_64
 
* org.eclipse.core.net.linux.x86_64
 
* org.eclipse.core.net.linux.x86
 
* org.eclipse.core.net.linux.x86
* org.eclipse.swt.tools
+
* org.eclipse.tips.ui
* org.eclipse.releng.tools
+
* org.eclipse.tips.json
* org.eclipse.update.core
+
* org.eclipse.tips.ide
* org.eclipse.osgi.compatibility.plugins
+
* org.eclipse.tips.core
  
 
Create sources jars:
 
Create sources jars:

Revision as of 14:54, 21 February 2019

Idea.png
Background and future
This page summarizes the result of bug 484004 and bug 510072.
Follow-up will happen in bugs like bug 497982 and bug 406557


Preparing a Repository

Unless stated otherwise the following steps are implemented using bash scripts.

Sources
The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central
Build Job
This script drives the hudson job https://hudson.eclipse.org/releng/view/Publish%20to%20Maven%20Central/job/CBIaggregator

Preparing the tools

  • unpack Eclipse SDK for use as director application (can we use a smaller installation?)
  • install the cbiAggregator

Run the aggregator

  • the aggregation model basically pulls everything from a given release repo
  • configure for all operating systems
  • specifies these properties: packedStrategy="UNPACK_AS_SIBLING" mavenResult="true" versionFormat="MavenRelease"
  • define maven mappings to map all 3rd party artifacts to their original GAV, as of SDK Neon.2 these are:
    • com/jcraft/jsch/0.1.53
    • com/ibm/icu/icu4j/56.1.0
    • commons-jxpath/commons-jxpath/1.3.0
    • javax/inject/javax.inject/1.0.0
    • junit/junit/4.12.0
    • org/apache/xmlgraphics/batik-css/1.7.0
    • org/apache/lucene/lucene-analyzers/3.5.0
    • org/apache/lucene/lucene-core/3.5.0
    • org/apache/felix/org.apache.felix.gogo.runtime/0.10.0
    • org/apache/felix/org.apache.felix.gogo.shell/0.10.0
    • org/apache/ant/ant/1.9.6
    • org/tukaani/xz/1.3.0
    • org/ow2/sat4j/org.ow2.sat4j.core/2.3.5
    • org/ow2/sat4j/org.ow2.sat4j.pb/2.3.5
    • org/ow2/asm/asm-tree/5.0.1
    • org/ow2/asm/asm/5.0.1
  • Since Neon.3 some of the above specify Version Pattern/Template in order to cut of undesired ".0" parts, so junit, e.g., is now 4.12, not 4.12.0
Important.png
TODO
check the above list against eclipse-sdk-prereqs.target


Remove undesired content

  • p2-only content: p2.index p2.packed content.jar artifacts.jar
  • all features (irrelevant in plain maven use): *feature.group and *feature.jar
  • test plug-ins: org/eclipse/*/*test*
  • tooling* (what is this?)
  • directories containing a pom but no corresponding jar file
  • "garbage", i.e., 3rd party artifacts that after previous clean-up are no longer referenced from any org/eclipse/{platform,jdt,pde} pom files
  • directories that are empty after previous clean-up

Technically, some of the clean-up is not necessary, but it helps to give a good picture of what will be published.

Enrich POMs

This step is implemented as a small Java tool.

Sources
The sources are hosted in https://git.eclipse.org/c/platform/eclipse.platform.releng.git/tree/publish-to-maven-central, see src/ and enrichPoms.jardesc
Build Job
The tool is built on the fly from CBIaggregator.sh:264 (during hudson job CBIaggregator)


Add some required information to the generated poms:

  • dynamic content (retrieved mostly from MANIFEST.MF):
    • name (typically matches what the aggregator filled in as description)
    • url
    • scm connection, tag and url
  • semi dynamic content
    • developers (start from the git url, and match it against known project names to link to the corresponding "Who's Involved" page of PMI)
  • static content
    • copyright
    • license
    • organization
    • issue management
  • hard wired corrections
    • scm connection for emf & ecf
    • fix name of org.eclipse.core.resources.win32.x86* fragments
  • fix checksums (md5 & sha1 after pom update -- not needed for publishing)

Add Javadoc stubs

Use a heuristic mapping from bundle name to name of the doc bundle holding corresponding javadoc, one of:

  • org.eclipse.jdt.doc.isv
  • org.eclipse.pde.doc.user
  • org.eclipse.platform.doc.isv

Then

  • create a README.txt referring to the doc bundle
  • jar the README.txt
  • copy the jar to the locations so that it will be picked up by maven

Create missing sources artifacts

Workaround for sources artifacts missing for 8 artifacts (bug 510976).

  • org.eclipse.core.net.win32.x86_64
  • org.eclipse.core.net.win32.x86
  • org.eclipse.core.net.linux.x86_64
  • org.eclipse.core.net.linux.x86
  • org.eclipse.tips.ui
  • org.eclipse.tips.json
  • org.eclipse.tips.ide
  • org.eclipse.tips.core

Create sources jars:

  • check out sources from git
  • respect content of build.properties
  • manually jar it up and put it into the location expected by maven

Sign and Upload to OSSRH

hudson jobs

Steps:

  • copy relevant section of prepared repo into current hudson workspace
  • for each *.pom, *.jar, *-sources.jar and *-javadoc.jar
    • invoke maven (cli)
      • pass settings configured for GPG signing and OSSRH account (-s).
      • use a generic pom (-f) (configure maven-gpg-plugin and nexus-staging-maven-plugin (is the latter needed here?))
      see https://bugs.eclipse.org/484004#c70
      • call goal gpg:sign-and-deploy-file
      • passing these properties (-D): url, repositoryId, file, pomFile, classifier

Caveat:

If gpg:sign-and-deploy-file doesn't work for some reason, it's tempting to try manual upload of artifacts instead. This, however, doesn't seem to scale, i.e., it's OK for one artifact, but not for a significant number of artifacts. See how I struggled : OSSRH-27091 comment 389077 (look for item (2)).

Validate and Release

  • Login to OSSRH nexus (personal account must have been assigned permissions for the targeted staging area, simply request this in the Jira ticket).
  • Identify and select the staging repository created by the previous step (name is group-without-dots-dash-number, where number starts at 1000, e.g., orgeclipseplatform-1005)
  • Click close, and wait for validation results to appear under "Activity"
  • On success click release, otherwise click drop, fix errors and try again

Back to the top