TCK:Build From Jakarta EE TCK Repo And Run

Introduction

This wiki describes how to run TCK for any technology from the JakartaEE-TCK GitHub repository. This is an Unofficial TCK, unofficial TCK run, no compatibility claim can be made based on the results. However, this description allows for setting the TCK up and running, so that tests can be added, or modified and debugged.

There is a work in progress on the official TCK run.

Most technologies contain a brief Test Coverage Document, a TCD file available at https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/internal/docs/${technology_name}

Notions & Notations

The following are the acronyms that are possible to meet when working with Jakarta EE TCK:

• CTS - Compatibility Test Suite, aka Jakarta EE TCK, basically a superset of all TCKs + some tests that for an interaction of two or more technologies in full Jakarta EE platform. There can be an exception, some technologies contain tests that are only in standalone TCK, most of the technologies contain tests that are only part of full Jakarta EE TCK and are not part of standalone TCK. Formerly Java EE CTS was a trademark, the word CTS seems to be obsolete in Jakarta EE environment in favor of Jakarta EE TCK.
• JCP - An organization defining the formal process of creating a JSR, a process that is obsolete in Jakarta EE
• JSR - A technology with a given number and process of being created, maintained and tested by TCK. Since JSR is bound to JCP process, the JSR term seems to be obsolete in Jakarta EE.
• RI - Reference implementation. JCP process required to have a reference implementation of any Specification as a proof of concept. The reference implementation for whole Java EE was the Glassfish Application Server. The term reference implementation is obsolete in Jakarta EE.
• Specification - Technology Specification - a combination of API and a written document. Some technology can have only the API. TCK should contain assertions from both the API Javadoc and from the written document. In Java EE world, both were part of JSR, and it is accessible from JCP home page.
• Standalone TCK - TCK that contains tests mandatory for a given technology, i.e. it does not contain tests for behavior the technology Specification describes for full Java EE/Jakarta EE environment. Some technology does not have standalone TCK (e.g. EJB)
• TCK - Technology Compatibility Kit, a set of tests, assertions, and documentation for a single technology, or in case of Jakarta EE TCK, for the whole umbrella of technologies.
• technology - A single technology formerly defined by a JSR. It can be Servlet, JAX-B, EJB, CDI, ... The list of technologies in Jakarta EE TCK can be seen for instance in the install folder, where there is a subfolder for each technology that has a standalone TCK in Jakarta EE TCK workspace. Sometimes, a technology TCK is called deliverable. In Jakarta EE, they can be called API/Specification projects.
• vehicle - Java EE Specification defined 4 Application Components, Appclient, Applet, Servlet, and EJB. Jakarta EE TCK runs the tests in these components (but Applet), but it is more fine grained, based on what possibilities the technology Specification defines. The set of vehicles the TCK runs at is defined in vehicle.properties file. Note that this one is for whole Jakarta EE TCK, each standalone TCK has a separate vehicle.properties.
• VI - Vendors implementation. Any implementation of a technology created by vendors. Sometimes to test proper VI behavior, part of a test needs to be run by RI, part of VI. For the case of RI, VI=RI.

Needed Steps

Jakarta EE TCK is an ANT based technology. There is a configuration file which contains classpath, which needs to be set for both API and implementation is tested. Jakarta EE TCK comes with predefined configuration scripts for Glassfish Application Server (few technologies has a predefined configuration for Apache Tomcat as well). This is why the following text contains a description of the building and running the TCK against Glassfish. The description is made with Eclipse Jenkins in mind, some Jenkins related properties such as ${WORKSPACE} are used.

• Eclipse Glassfish with integrated technology implementation
• Properly set configuration file against the Glassfish
• TCK being built against the Glassfish
• TCK being run against the Glassfish

Building a Glassfish with latest technology implementation

Currently, mostly every API and implementation has a Jenkins build job on Eclipse Jenkins infrastructure. The important part is to have the built artifacts available so that it is possible to use them when building an Eclipse Glassfish. They can be stored in some nexus, or FTP (download.eclipse.org) or possibly just Jenkins artifacts

Glassfish can be built

• from Glassfish git repository - Update the versions in the dependency files and build it - glassfish should grab versions from the (local) maven repository
• from an existing binary distribution - Sometimes it is just good enough to replace existing binaries in Glassfish modules folder with a new version. Sometimes it is more complicated, for instance when glassfish internals depends on the tested technology and new versions have incompatible changes.

Configuring the ts.jte configuration file

ts.jte file is part of Jakarta EE TCK GitHub repository. For each technology, it can be found in /install/${technology}/bin/ts.jte. In case of Jakarta EE TCK bundle, ${technology} is j2ee. For some technology, it is mostly prefilled, some technology keeps the file unfilled. Ts.jte fle is a property file in a form [property_name]=[property_value]. The common important properties are:

• ${technlogy}.classpath or ${technology}_impl.classpath - it contains the implementation jars needed for the TCK to be build & run. The property is referenced from command.testExecute property, where the proper name is available.
• web.home, sometimes javaee.home, refers to a folder where Glassfish is to be deployed, e.g. ${WORKSPACE}/glassfish5
• impl.vi=glassfish. For some technology (JAX-WS, SAAJ,...) some support for impl.vi=tomcat is available as well.
• impl.vi.deploy.dir - Folder where the wars/ears that are part of TCKs are deployed. For glassfish, impl.vi.deploy.dir=path_to_glassfish5/glassfish/domains/domain1/autodeploy
• harness.log.traceflag. Set to true for extra debugging information
• work.dir and report.dir. Folders where results of TCK run is stored.

Building a standalone TCK

There are standalone TCK bundles available. If the goal is not to modify the TCK, this step can be skipped, then.

This paragraph describes building the TCK from Jakarta EE TCK repository. For test modifications, debugging messages and creation of new tests, this repository can be forked. Jakarta EE TCK is ant based technology. Java EE CTS contained a version of ant, Jakarta EE TCK does not.

The first step is to set the shell environment of the Eclipse Jenkins job, location of Java, Maven in provided filesystem. A used solution is to set:

TOOLS_PREFIX='/opt/tools'
JAVA_PREFIX="${TOOLS_PREFIX}/java/oracle"
export JAVA_HOME="${JAVA_PREFIX}/jdk-8/latest"
PATH="${MVN_HOME}/bin:${JAVA_HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

The second step it is downloading ant. It used to be stored in a root of Jakarta EE TCK workspace, under tools/ant folder. So set the shell environment and download it to the expected folder:

ANT_VERSION=1.9.13
ANT_HOME="${WORKSPACE}/${TCK_FOLDER}/tools/ant"
mkdir download && cd download
PATH="${ANT_HOME}/bin:${PATH}"
export ANT_OPTS=-Djavax.xml.accessExternalSchema=all
wget -q http://mirror.koddos.net/apache/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz -O ant.tar.gz
tar xfz ant.tar.gz
cp -a ${WORKSPACE}/download/apache-ant-${ANT_VERSION}/. ${WORKSPACE}/${TCK_FOLDER}/tools/ant/

Set the TCK respective environment settings, TS_HOME is the root of the checked out Jakarta EE repository, deliverabledir is a folder name in install, src folders (as well as classes, and dist created during build time), javaee_home is the Glassfish folder:

export TS_HOME=${WORKSPACE}/${technology}tck
export deliverabledir=${technology}
export javaee_home=${WORKSPACE}/glassfish5

Update ts.jte with an updated ts.jte in the TCK, for instance, if downloaded to ${WORKSPACE}/download:

yes | cp -vr ${WORKSPACE}/download/ts.jte ${WORKSPACE}/${TS_HOME}/install/jaxrs/bin/ts.jte

Bring over the Eclipse Glassfish with updated technology implementation bits Build the TCK:

cd ${TS_HOME}/install/${deliverabledir}/bin
ant build.all

Pack the TCK bundle:

cd ${TS_HOME}/release/tools/
ant <target>

where target is ${technology} for tck tests only or j2ee-{technology} for all tests of the technology that were part of the CTS, i.e. contains tests that tests interoperability of two technologies, such as JSON-B and CDI

Running the standalone TCK

Suppose there is Glassfish, ts.jte and pre-built TCK in ${WORKSPACE}/download folder. Set the shell environment:

TOOLS_PREFIX='/opt/tools'
JAVA_PREFIX="${TOOLS_PREFIX}/java/oracle"
export JAVA_HOME="${JAVA_PREFIX}/jdk-8/latest"
PATH="${MVN_HOME}/bin:${JAVA_HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

Bring over ANT:

ANT_HOME="${WORKSPACE}/${technology}tck/tools/ant"
PATH="${ANT_HOME}/bin:${PATH}"
export ANT_OPTS=-Djavax.xml.accessExternalSchema=all

#get ANT
cd ${WORKSPACE}/download
wget -q http://mirror.koddos.net/apache/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz -O ant.tar.gz
tar xfz ant.tar.gz

Unzip Glassfish, TCK, ANT:

cd ${WORKSPACE}
unzip -q ${WORKSPACE}/download/glassfish.zip
unzip -q ${WORKSPACE}/download/${technology}tck-${version}_latest.zip
mkdir -p ${WORKSPACE}/${technology}tck/tools/ant && cp -a ${WORKSPACE}/download/apache-ant-${ANT_VERSION}/. ${WORKSPACE}/${technology}tck/tools/ant/

Copy ts.jte over (note that the location differs from the location needed for TCK build):

yes | cp -rfv ${WORKSPACE}/download/ts.jte ${WORKSPACE}/${technology}tck/bin/ts.jte

Set TCK respective environment properties:

export TS_HOME=${WORKSPACE}/${technology}tck
export deliverabledir=${technology}
export javaee_home=${WORKSPACE}/glassfish5

Configure and start Glassfish:

cd ${TS_HOME}/bin
ant config.vi

Deploy the prebuild application WARs, EARS,...

ant deploy.all

And run the TCK:

ant run.all

Note that some technologies may require some additional steps before running the tests. For instance, JPA TCK needs to start a database.

Jenkins Pipelines

The following pipelines examples for JSON-B can be copy & pasted (and slightly modified) into an Eclipse Jenkins instance:

• Configuring the ts.jte

 - Update env.deliverabledir for the proper technology (see a list of technologies)
 - Update env.JARS_ON_CP variable to contain proper jars to be put on the TCK classpath

• Building the Glassfish with a new API

 - Update API_BUILD_JOB variable. It contains a link to a job that archives the built API. jenkins://./ refers to the same Jenkins instance where this pipeline job is to be used. The name of the job containing the API jar is to be appended.
 - Alternatively, update env.GF_URL to contain a link to a Glassfish that is to be used for the API to be integrated there.

• Running the TCK tests

 - Update GF_BUILD_JOB variable to contain a link to the pipeline job that integrates the API to the Glassfish
 - Update TS_JTE_BUILD_JOB variable to contain a link to the pipeline job that configures the ts.jte file
 - Update TCK_BUNDLE_URL variable to contain a link to the TCK bundle to be used. The bundles are available at Eclipse Downloads site.
 - Update API_JAR_NAME variable to contain the API jar name. This jar is then archive to have a reference the API the TCK is run against. The run result is archived, too.
 - Update env.deliverabledir variable to contain the proper technology name

Usual Configuration Errors

• java.io.FileNotFoundException: <ts.home>/bin/xml/impl/${impl.vi}/config.vi.xml (No such file or directory) - need to setup impl.vi=glassfish in ts.jte file.
• Unknown target config.vi - some TCKs do not configure the glassfish, for instance JSON-P. Do not call ant config.vi, then.

Caveats

• Jakarta EE TCK is currently working with JDK 8 only. Make sure the TCK is run with a proper JDK version.
• Exception on JDK 8: Failed to read schema document 'xjc.xsd', because 'file' access is not allowed due to the restriction set by the accessExternalSchema property. To prevent this, use: export ANT_OPTS=-Djavax.xml.accessExternalSchema=all
• jakarta in the API jar names. When building TCK, the API in the Glassfish may contain javax prefixed jars. Make sure the ts.jte contains a path to the proper jar. When running the TCK, the API should be jakarta prefixed, make sure ts.jte refers to the new API.

References

Brief Jakarta EE TCK build/run description at GitHub Wiki