Difference between revisions of "JakartaEE New Infra Release Job"

Revision as of 10:11, 9 October 2018

Following Jenkins job allows to release project component from any branch of specific GitHub repository. Features:

automatic version numbers generation
existing version overwrite
dry run (without GitHub repository modification and OSS nexus upload)

Job setup must also follow this guide

Job parameters

This project is parameterized: checked

Name	Type	Default	Description
RELEASE_VERSION	String		Version to release. Default value is from POM snapshot.
NEXT_VERSION	String		Next snapshot version to set (e.g. 1.2.3-SNAPSHOT). Default value is RELEASE_VERSION with last component incremented by 1.
BRANCH	String	master	Branch to release. Default value is master.
DRY_RUN	Boolean	false	Do not publish artifacts to OSSRH and code changes to GitHub.
OVERWRITE	Boolean	false	Overwrite existing version in git and OSSRH staging repositories.

Shell script

Job Environment

Job is being executed by bash. Specific settings for Java build tools in new Jenkins environment:

#!/bin/bash -ex

TOOLS_PREFIX='/opt/tools'
JAVA_PREFIX="${TOOLS_PREFIX}/java/oracle"
MVN_HOME="${TOOLS_PREFIX}/apache-maven/latest"
JAVA_HOME="${JAVA_PREFIX}/jdk-9/latest"
PATH="${MVN_HOME}/bin:${JAVA_HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

Input Parameters Processing

RELEASE_VERSION and NEXT_VERSION values can be empty. Following code replaces empty values with proper values:

RELEASE_VERSION: version number from current snapshot version string
NEXT_VERSION: RELEASE_VERSION with latest version component incremented by 1 and -SNAPSHOT suffix

Release tag name is also prepared.

# Maven plugins
VERSIONS_PLUGIN='org.codehaus.mojo:versions-maven-plugin:2.5'
HELP_PLUGIN='org.apache.maven.plugins:maven-help-plugin:2.1.1'

# Top level pom.xml is  in root directory
# Check whether top level pom.xml contains SNAPSHOT version
if ! grep '<version>' pom.xml | grep 'SNAPSHOT' ; then
  echo '-[ Missing SNAPSHOT version in POM! ]-------------------------------------------'
  exit 1
fi

# Compute release versions
SNAPSHOT_VERSION=`mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.version 2> /dev/null | grep -E '^[0-9]+(\.[0-9]+)+-SNAPSHOT$'`

if [ -z "${RELEASE_VERSION}" ]; then
  if [ -z ${SNAPSHOT_VERSION} ]; then
    echo '-[ Missing required snapshot version number! ]----------------------------------'
  fi
  RELEASE_VERSION=`echo ${SNAPSHOT_VERSION} | sed -e 's/-SNAPSHOT//'`
fi

# Bash specific code
if [ -z "${NEXT_VERSION}" ]; then
  NEXT_VERSION=`echo ${RELEASE_VERSION} | sed -e 's/\([0-9][0-9]*\.[0-9][0-9]*\).*/\1/'`
  set -f
  NEXT_COMPONENTS=(${RELEASE_VERSION//\./ })
  LAST_INDEX=$((${#NEXT_COMPONENTS[@]} - 1))
  NEXT_COMPONENTS[${LAST_INDEX}]=$((${NEXT_COMPONENTS[${LAST_INDEX}]} + 1))
  NEXT_VERSION=`echo ${NEXT_COMPONENTS[@]} | tr ' ' '.'`'-SNAPSHOT'
fi

RELEASE_TAG="${RELEASE_VERSION}-RELEASE"

echo "Current version: ${SNAPSHOT_VERSION}"
echo "Release version: ${RELEASE_VERSION}"
echo "Next version:    ${NEXT_VERSION}"
echo "Release tag:     ${RELEASE_TAG}"

if [ -z "${SNAPSHOT_VERSION}" -o -z "${RELEASE_VERSION}" -o -z "${NEXT_VERSION}" ]; then
  echo '-[ Missing required version numbers! ]------------------------------------------'
  exit 1
fi

Git and Build Environment Preparation

If OVERWRITE argument is active (true), remote release branch and tag names are deleted. If DRY_RUN argument is active (true), deploy:deploy argument won't be passed to maven. Old local release branch and tag are always deleted if exist and local release branch is initialized to store all code changes related to this release. GitHub bot name and e-mail are set in git configuration.

if [ ${DRY_RUN} = 'true' ]; then
  echo '-[ Dry run turned on ]----------------------------------------------------------'
  MVN_DEPLOY_ARGS=''
  echo '-[ Skipping GitHub branch and tag checks ]--------------------------------------'
else
  MVN_DEPLOY_ARGS='deploy:deploy'
  GIT_ORIGIN=`git remote`
  echo '-[ Prepare branch ]-------------------------------------------------------------'
  if [[ -n `git branch -r | grep "${GIT_ORIGIN}/${RELEASE_VERSION}"` ]]; then
    if [ "${OVERWRITE}" = 'true' ]; then
      echo "${GIT_ORIGIN}/${RELEASE_VERSION} branch already exists, deleting"
      git push --delete origin "${RELEASE_VERSION}" && true
    else
      echo "Error: ${GIT_ORIGIN}/${RELEASE_VERSION} branch already exists"
      exit 1
    fi
  fi
  echo '-[ Release tag cleanup ]--------------------------------------------------------'
  if [[ -n `git ls-remote --tags ${GIT_ORIGIN} | grep "${RELEASE_TAG}"` ]]; then
    if [ "${OVERWRITE}" = 'true' ]; then
      echo "${RELEASE_TAG} tag already exists, deleting"
      git push --delete origin "${RELEASE_TAG}" && true
    else
      echo "Error: ${RELEASE_TAG} tag already exists"
      exit 1
    fi
  fi
fi

# Always delete local branch if exists
git branch --delete "${RELEASE_VERSION}" && true
git checkout -b ${RELEASE_VERSION}
# Always delete local tag if exists
git tag --delete "${RELEASE_TAG}" && true

# Setup jaxb-impl-bot account information
git config --global user.email "jaxb-impl-bot@eclipse.org"
git config --global user.name "Eclipse JAXBimpl Bot"

Workaround: GPG initialization

GPG setup is missing in Docker image and GPG keys are passed in KEYRING environment variables. Content of this variable must be processed and stored into GPG key store.

# Workaround: GPG initialization
gpg --batch --import ${KEYRING}
for fpr in $(gpg --list-keys --with-colons  | awk -F: '/fpr:/ {print $10}' | sort -u);
do
  echo -e "5\ny\n" |  gpg --batch --command-fd 0 --expert --edit-key $fpr trust;
done

Set Release Version

Project Group ID and Artifact ID values are retrieved from maven project (pom.xml). Those values are used in git commit messages. Release version is set in maven project (pom.xml). Updated project is stored in git release branch with corresponding message.

# Project identifiers
ARTIFACT_ID=$(mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.artifactId | grep -Ev '(^\[)')
GROUP_ID=$(mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.groupId | grep -Ev '(^\[)')

echo '-[ Set release version ]--------------------------------------------------------'
# Set release version
mvn -U -C \
    -DnewVersion="${RELEASE_VERSION}" \
    -DgenerateBackupPoms=false \
    clean ${VERSIONS_PLUGIN}:set

echo '-[ Commit modified pom.xml files ]----------------------------------------------'
POM_FILES=`git status | grep -E 'modified:.*pom\.xml' | sed -e 's/[[:space:]][[:space:]]*modified:[[:space:]][[:space:]]*//'`
git add ${POM_FILES} && \
git commit -m "Prepare release ${GROUP_ID}:${ARTIFACT_ID}:${RELEASE_VERSION}"

Build and Deploy Artifacts

All artefacts are built and deployed to the staging repository. Code with release versions is tagged with release tag. This maven task requires specific settings.xml file with GPG setup. Profile oss-release is defined in parent POM. It contains setup for packaging sources and javadoc and specific setup for GPG signing plugin (--pinentry-mode loopback). Minimal version of parent POM is 1.0.2. Project's pom.xml must define versions for the following plugins:

maven-enforcer-plugin
maven-source-plugin
maven-javadoc-plugin
maven-gpg-plugin

echo '-[ Deploy artifacts to staging repository ]-------------------------------------'
mvn -U -C -s /home/jenkins/.m2/settings-jaxws.xml \
    -DskipTests -Ddoclint=none -Poss-release \
    clean package source:jar javadoc:jar gpg:sign install:install ${MVN_DEPLOY_ARGS}

echo '-[ Tag release ]----------------------------------------------------------------'
git tag "${RELEASE_TAG}" -m "Release ${GROUP_ID}:${ARTIFACT_ID}:${RELEASE_VERSION}"

Set Next Development Cycle Version

Next development cycle SNAPSHOT version is set in maven project (pom.xml). Updated project is stored in git release branch with corresponding message.

echo '-[ Set next snapshot version ]--------------------------------------------------'
mvn -U -C \
    -DnewVersion="${NEXT_VERSION}" \
    -DgenerateBackupPoms=false \
    clean ${VERSIONS_PLUGIN}:set

echo '-[ Commit modified pom.xml files ]----------------------------------------------'
POM_FILES=`git status | grep -E 'modified:.*pom\.xml' | sed -e 's/[[:space:]][[:space:]]*modified:[[:space:]][[:space:]]*//'`
git add ${POM_FILES} && \
git commit -m "Prepare next development cycle for ${NEXT_VERSION}"

Update Git Repository

If DRY_RUN argument is not active (false), release branch and tag will be pushed to GitHub.

if [ ${DRY_RUN} = 'true' ]; then
  echo '-[ Skipping GitHub update ]-----------------------------------------------------'
else
  echo '-[ Push branch and tag to GitHub ]----------------------------------------------'
  git push origin "${RELEASE_VERSION}"
  git push origin "${RELEASE_TAG}"
fi

Project pom.xml in a Subdirectory

Previous script will work only when pom.xml is in top level directory (WORKSPACE) of the Jenkins job. Here is script modified for top level pom.xml in a subdirectory:

#!/bin/bash -ex

TOOLS_PREFIX='/opt/tools'
JAVA_PREFIX="${TOOLS_PREFIX}/java/oracle"
MVN_HOME="${TOOLS_PREFIX}/apache-maven/latest"
JAVA_HOME="${JAVA_PREFIX}/jdk-9/latest"
PATH="${MVN_HOME}/bin:${JAVA_HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# Maven plugins
VERSIONS_PLUGIN='org.codehaus.mojo:versions-maven-plugin:2.5'
HELP_PLUGIN='org.apache.maven.plugins:maven-help-plugin:2.1.1'

# Directory with project top level pom.xml
BUILD_DIR="${WORKSPACE}/api"

cd ${BUILD_DIR}
# Check whether top level pom.xml contains SNAPSHOT version
if ! grep '<version>' pom.xml | grep 'SNAPSHOT' ; then
  echo '-[ Missing SNAPSHOT version in POM! ]-------------------------------------------'
  exit 1
fi

# Compute release versions
SNAPSHOT_VERSION=`mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.version 2> /dev/null | grep -E '^[0-9]+(\.[0-9]+)+-SNAPSHOT$'`

if [ -z "${RELEASE_VERSION}" ]; then
  if [ -z ${SNAPSHOT_VERSION} ]; then
    echo '-[ Missing required snapshot version number! ]----------------------------------'
  fi
  RELEASE_VERSION=`echo ${SNAPSHOT_VERSION} | sed -e 's/-SNAPSHOT//'`
fi

# Bash specific code
if [ -z "${NEXT_VERSION}" ]; then
  NEXT_VERSION=`echo ${RELEASE_VERSION} | sed -e 's/\([0-9][0-9]*\.[0-9][0-9]*\).*/\1/'`
  set -f
  NEXT_COMPONENTS=(${RELEASE_VERSION//\./ })
  LAST_INDEX=$((${#NEXT_COMPONENTS[@]} - 1))
  NEXT_COMPONENTS[${LAST_INDEX}]=$((${NEXT_COMPONENTS[${LAST_INDEX}]} + 1))
  NEXT_VERSION=`echo ${NEXT_COMPONENTS[@]} | tr ' ' '.'`'-SNAPSHOT'
fi

RELEASE_TAG="${RELEASE_VERSION}-RELEASE"

echo "Current version: ${SNAPSHOT_VERSION}"
echo "Release version: ${RELEASE_VERSION}"
echo "Next version:    ${NEXT_VERSION}"
echo "Release tag:     ${RELEASE_TAG}"

if [ -z "${SNAPSHOT_VERSION}" -o -z "${RELEASE_VERSION}" -o -z "${NEXT_VERSION}" ]; then
  echo '-[ Missing required version numbers! ]------------------------------------------'
  exit 1
fi

if [ ${DRY_RUN} = 'true' ]; then
  echo '-[ Dry run turned on ]----------------------------------------------------------'
  MVN_DEPLOY_ARGS=''
  echo '-[ Skipping GitHub branch and tag checks ]--------------------------------------'
else
  MVN_DEPLOY_ARGS='deploy:deploy'
  GIT_ORIGIN=`git remote`
  echo '-[ Prepare branch ]-------------------------------------------------------------'
  if [[ -n `git branch -r | grep "${GIT_ORIGIN}/${RELEASE_VERSION}"` ]]; then
    if [ "${OVERWRITE}" = 'true' ]; then
      echo "${GIT_ORIGIN}/${RELEASE_VERSION} branch already exists, deleting"
      git push --delete origin "${RELEASE_VERSION}" && true
    else
      echo "Error: ${GIT_ORIGIN}/${RELEASE_VERSION} branch already exists"
      exit 1
    fi
  fi
  echo '-[ Release tag cleanup ]--------------------------------------------------------'
  if [[ -n `git ls-remote --tags ${GIT_ORIGIN} | grep "${RELEASE_TAG}"` ]]; then
    if [ "${OVERWRITE}" = 'true' ]; then
      echo "${RELEASE_TAG} tag already exists, deleting"
      git push --delete origin "${RELEASE_TAG}" && true
    else
      echo "Error: ${RELEASE_TAG} tag already exists"
      exit 1
    fi
  fi
fi

cd ${WORKSPACE}
# Always delete local branch if exists
git branch --delete "${RELEASE_VERSION}" && true
git checkout -b ${RELEASE_VERSION}
# Always delete local tag if exists
git tag --delete "${RELEASE_TAG}" && true

# Setup jaxb-impl-bot account information
git config --global user.email "jaxb-impl-bot@eclipse.org"
git config --global user.name "Eclipse JAXBimpl Bot"
# Workaround: GPG initialization
gpg --batch --import ${KEYRING}
for fpr in $(gpg --list-keys --with-colons  | awk -F: '/fpr:/ {print $10}' | sort -u);
do
  echo -e "5\ny\n" |  gpg --batch --command-fd 0 --expert --edit-key $fpr trust;
done

cd ${BUILD_DIR}
# Project identifiers
ARTIFACT_ID=$(mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.artifactId | grep -Ev '(^\[)')
GROUP_ID=$(mvn -B ${HELP_PLUGIN}:evaluate -Dexpression=project.groupId | grep -Ev '(^\[)')

echo '-[ Set release version ]--------------------------------------------------------'
# Set release version
mvn -U -C \
    -DnewVersion="${RELEASE_VERSION}" \
    -DgenerateBackupPoms=false \
    clean ${VERSIONS_PLUGIN}:set

cd ${WORKSPACE}
echo '-[ Commit modified pom.xml files ]----------------------------------------------'
POM_FILES=`git status | grep -E 'modified:.*pom\.xml' | sed -e 's/[[:space:]][[:space:]]*modified:[[:space:]][[:space:]]*//'`
git add ${POM_FILES} && \
git commit -m "Prepare release ${GROUP_ID}:${ARTIFACT_ID}:${RELEASE_VERSION}"

cd ${BUILD_DIR}
echo '-[ Deploy artifacts to staging repository ]-------------------------------------'
mvn -U -C -s /home/jenkins/.m2/settings-jaxws.xml \
    -DskipTests -Ddoclint=none -Poss-release \
    clean package source:jar javadoc:jar gpg:sign install:install ${MVN_DEPLOY_ARGS}

cd ${WORKSPACE}
echo '-[ Tag release ]----------------------------------------------------------------'
git tag "${RELEASE_TAG}" -m "Release ${GROUP_ID}:${ARTIFACT_ID}:${RELEASE_VERSION}"

cd ${BUILD_DIR}
echo '-[ Set next snapshot version ]--------------------------------------------------'
mvn -U -C \
    -DnewVersion="${NEXT_VERSION}" \
    -DgenerateBackupPoms=false \
    clean ${VERSIONS_PLUGIN}:set

cd ${WORKSPACE}
echo '-[ Commit modified pom.xml files ]----------------------------------------------'
POM_FILES=`git status | grep -E 'modified:.*pom\.xml' | sed -e 's/[[:space:]][[:space:]]*modified:[[:space:]][[:space:]]*//'`
git add ${POM_FILES} && \
git commit -m "Prepare next development cycle for ${NEXT_VERSION}"

if [ ${DRY_RUN} = 'true' ]; then
  echo '-[ Skipping GitHub update ]-----------------------------------------------------'
else
  echo '-[ Push branch and tag to GitHub ]----------------------------------------------'
  git push origin "${RELEASE_VERSION}"
  git push origin "${RELEASE_TAG}"
fi

Screenshots

@@ Line 394: / Line 394: @@
 </pre>
-== Screenshots ==
+== [https://wiki.eclipse.org/JakartaEE_New_Infra_Release_Job_Screenshots Screenshots] ==
-[[File:Jenkins1-header.png]]
-[[File:Jenkins2-arguments1.png]]
-[[File:Jenkins3-arguments2.png]]
-[[File:Jenkins4-repository.png]]
-[[File:Jenkins5-bindings1.png]]
-[[File:Jenkins6-bindings2.png]]
-[[File:Jenkins7-script.png]]

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Difference between revisions of "JakartaEE New Infra Release Job"

Revision as of 10:11, 9 October 2018

Contents

Job parameters

Shell script

Job Environment

Input Parameters Processing

Git and Build Environment Preparation

Workaround: GPG initialization

Set Release Version

Build and Deploy Artifacts

Set Next Development Cycle Version

Update Git Repository

Project pom.xml in a Subdirectory

Screenshots

Breadcrumbs

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Difference between revisions of "JakartaEE New Infra Release Job"

Revision as of 10:11, 9 October 2018

Contents

Job parameters

Shell script

Job Environment

Input Parameters Processing

Git and Build Environment Preparation

Workaround: GPG initialization

Set Release Version

Build and Deploy Artifacts

Set Next Development Cycle Version

Update Git Repository

Project pom.xml in a Subdirectory

Screenshots