Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Teneo/Teneo Build Setup"
(82 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
__TOC__ | __TOC__ | ||
+ | |||
+ | '''Teneo has been moved from CVS to GIT in November 2011, the build setup has changed because of this. This page maybe partially outdated. Please check [http://martintaal.wordpress.com/2011/11/01/moving-texo-and-teneo-over-to-git/ this] blog for more information.''' | ||
This page describes the Teneo build setup. Teneo uses [[Buckminster|Buckminster]] and [[Hudson|Hudson]] for its continuous build infrastructure. | This page describes the Teneo build setup. Teneo uses [[Buckminster|Buckminster]] and [[Hudson|Hudson]] for its continuous build infrastructure. | ||
− | The | + | The Teneo Hudson job, which uses the build setup as it is described in this wiki page, is available [https://build.eclipse.org/hudson/job/emf-teneo-nightly here]. |
− | + | ||
− | + | To understand the content of this wiki page it is necessary to understand the main Buckminster concepts. These are described in detail in the [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/doc/BuckyBook.pdf Bucky Book]. Some Buckminster concepts can seem fairly abstract initially, the 'Understanding Buckminster' section below may help here. | |
== Try it == | == Try it == | ||
− | + | It is quite easy to do a Teneo build. To download Teneo and do a build (building, testing, generating p2 site) you need to do the following steps: | |
<source lang="bash"> | <source lang="bash"> | ||
− | cvs -d dev.eclipse.org:/cvsroot/modeling co -d org.eclipse.emf.teneo.releng.buckminster \ | + | cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling co -d org.eclipse.emf.teneo.releng.buckminster \ |
org.eclipse.emf/org.eclipse.emf.teneo/releng/org.eclipse.emf.teneo.releng.buckminster | org.eclipse.emf/org.eclipse.emf.teneo/releng/org.eclipse.emf.teneo.releng.buckminster | ||
cd org.eclipse.emf.teneo.releng.buckminster | cd org.eclipse.emf.teneo.releng.buckminster | ||
ant | ant | ||
</source> | </source> | ||
− | + | The above assumes that ant is installed and that JAVA_HOME is set to Java 5.0 or higher. Note, the build downloads Buckminster using the Eclipse director, the director has sometimes trouble with certain java 6 versions. So if you get 'Unpack facility not configured' errors then check out the troubleshooting section below. | |
− | After about 20 minutes you will see a | + | After about 20 minutes you will see a build folder (next to the org.eclipse.emf.teneo.releng.buckminster folder) with a number of subfolders: |
− | * build: contains the targetplatform for building the main plugins and the build workspace itself | + | * build: contains the targetplatform (the tp subfolder) for building the main plugins and the build workspace itself (the workspace subfolder) |
− | * test: contains the targetplatform for testing, the workspace contains the workspace used for the test projects. | + | * test: contains the targetplatform for testing (in the tp subfolder), the workspace subfolder contains the workspace used for the test projects. |
+ | * output: contains temporary output from the build steps. | ||
* result: contains the generation result (p2 site) and the test results. | * result: contains the generation result (p2 site) and the test results. | ||
+ | * tools: contains buckminster and the p2 director applications. These are downloaded as part of the build. | ||
The build workspace (build/workspace) and test workspace (test/workspace) can be opened in Eclipse as workspaces. | The build workspace (build/workspace) and test workspace (test/workspace) can be opened in Eclipse as workspaces. | ||
− | == | + | == Understanding Buckminster == |
+ | Initially some Buckminster concepts can appear fairly abstract (for me at least) and it may not be directly clear why several xml files are needed to define the process. This section may help to make it a bit clearer. | ||
− | The | + | Main Buckminster functionality (as it is used for the Teneo build): |
− | # build the main deliverables: | + | * An important task performed by Buckminster is the creation of a workspace with target platform. The workspace is the same or similar to what you see in your IDE. The build and test runs are done using the created workspace/target platform. |
+ | * Buckminster uses the includes and dependency definitions in the MANIFEST.MF (for a plugin project) and the feature.xml (for a feature project) to download related projects or binaries. Additional dependencies are defined in a buckminster.cspex file. The build and buckminster files often only refer to so-called root (feature) projects. The includes and dependencies of the root project (often a feature project) then automatically drive the download of all other relevant projects and binaries into the workspace and target platform. | ||
+ | * Buckminster can search many types of locations for dependencies. The locations and their search order is defined in a configuration file (the *.rmap file). Development projects are typically downloaded from cvs while binary dependencies are often coming from update sites. | ||
+ | * When downloading dependencies, Buckminster determines which dependency to place in the workspace (as a development project) and which of the dependencies to place in the target platform. The choice where to place what is defined in the buckminster *.rmap and *.mspec file using an attribute: source="true", which signals a dependency to be a source (so should be placed in the workspace). | ||
+ | * Buckminster has several commands which can be executed against a workspace/target platform, for example run tests, build a p2.site etc. | ||
+ | |||
+ | |||
+ | A short description on the meaning of some files (i.e. how they are used in the Teneo build): | ||
+ | * rmap file: defines where/how to download dependencies and development projects, contains cvs locations, update site url's and specifies an order in which these locations are searched. Also identifies which sources are considered to contain sources (i.e. development projects) and which are used for binary dependencies. | ||
+ | * cquery file: defines which rmap file to use and defines the root (feature) project, the content (defined in the feature.xml) and the dependencies of the content are downloaded, also contains extra directives which are passed to Buckminster when resolving dependencies. | ||
+ | * mspec file: is the starting point. It specifies which cquery file to use and also how the plugins/development projects downloaded as part of resolving the cquery file should be treated, or materialized. Materialized means: where to place the downloaded plugins/development projects, in the workspace or in the target platform. | ||
+ | |||
+ | == Teneo Main builds steps (build, generate, test) == | ||
+ | |||
+ | The Teneo build process is executed through an ant script which calls Buckminster for the main build steps. The ant script goes through the following steps: | ||
+ | # download the p2 director and using the director, download buckminster. | ||
+ | # build the main deliverables (using buckminster): | ||
## download the development projects from CVS and use them to create a workspace with a target platform | ## download the development projects from CVS and use them to create a workspace with a target platform | ||
## download/retrieve the dependencies and set them in the targetplatform | ## download/retrieve the dependencies and set them in the targetplatform | ||
Line 35: | Line 55: | ||
## build the update site | ## build the update site | ||
## create downloadable zips | ## create downloadable zips | ||
− | # build the test projects: | + | # build the test projects (using buckminster): |
## download the development test projects from CVS and use them to create a workspace with a target platform | ## download the development test projects from CVS and use them to create a workspace with a target platform | ||
## download/retrieve the dependencies and set them in the targetplatform. This includes the artifacts generated in the previous step. | ## download/retrieve the dependencies and set them in the targetplatform. This includes the artifacts generated in the previous step. | ||
## build/compile the test plugins | ## build/compile the test plugins | ||
− | # run the test cases | + | # run the test cases and convert the test results to a format which can be read by hudson |
− | + | ||
+ | The ant script also take care of copying the output (p2 site and features/plugins) and creating zip files etc. | ||
+ | |||
+ | The following sections describe the necessary files for the build: the ant xml files and the buckminster files. | ||
== Structure of build projects/features == | == Structure of build projects/features == | ||
− | + | The build infrastructure for Teneo is centered around a releng project containing the ant build xml file and the main Buckminster files. A number of other feature projects are used to define which plugins are part of the build, part of the update site and which are part of the tests etc. | |
− | * '''org.eclipse.emf.teneo.releng.buckminster''' | + | |
− | * '''org.eclipse.emf.teneo.site-feature''': defines the update site and the main features it contains (Teneo Hibernate runtime and SDK and Teneo EclipseLink runtime and SDK). This project contains a buckminster. | + | The releng and feature projects can be found in this location in cvs: |
+ | * dev.eclipse.org | ||
+ | * /cvsroot/modeling | ||
+ | * org.eclipse.emf/org.eclipse.emf.teneo/releng and org.eclipse.emf/org.eclipse.emf.teneo/features | ||
+ | |||
+ | |||
+ | The '''org.eclipse.emf.teneo.releng.buckminster''' project contains the ant files, and the buckminster files for the build and test step. The content of this project and the buckminster setup is described in more detail below. This project is the most important one to look at when trying to understand Teneo's build setup. | ||
+ | |||
+ | The feature projects are used to structure the output of the build in the correct way, i.e. which features/plugins are made available in an update site, which features/plugins are involved in testing etc. | ||
+ | |||
+ | A summary of these feature projects: | ||
+ | * '''org.eclipse.emf.teneo.site-feature''': defines the update site and the main features it contains (Teneo Hibernate runtime and SDK and Teneo EclipseLink runtime and SDK). This project contains a buckminster.cspex file to define extra dependencies (this is described in more detail below). The included features: | ||
** '''org.eclipse.emf.teneo.hibernate-feature''': the runtime feature, this is a standard PDE feature project (the plugins are defined in the feature.xml). | ** '''org.eclipse.emf.teneo.hibernate-feature''': the runtime feature, this is a standard PDE feature project (the plugins are defined in the feature.xml). | ||
** '''org.eclipse.emf.teneo.hibernate.sdk-feature''': the SDK feature includes the runtime feature and source feature (see below for more information on building SDK features). | ** '''org.eclipse.emf.teneo.hibernate.sdk-feature''': the SDK feature includes the runtime feature and source feature (see below for more information on building SDK features). | ||
Line 54: | Line 88: | ||
* '''org.eclipse.emf.teneo.tests-feature''': contains the test plugins and their dependencies. The main test plugins contain test launch configurations (see below running test cases). | * '''org.eclipse.emf.teneo.tests-feature''': contains the test plugins and their dependencies. The main test plugins contain test launch configurations (see below running test cases). | ||
− | == | + | == Main build project: org.eclipse.emf.teneo.releng.buckminster == |
− | + | After downloading the main build project from cvs you can see that it consists of several files in the root and two subfolders: build and test. Here is a summary for the meaning of different files: | |
+ | * '''ant_common.xml''': contains macros which are used both in test and build phases, the main macros facilitate calling buckminster. This ant file is imported by other build files. | ||
+ | * '''buckminster.properties''': defines several properties related to how to sign plugins and where to download the director and buckminster applications. | ||
+ | * '''build.xml''': the main build file which drives the build and test phases. Its content is described in more detail below. | ||
− | |||
− | <source lang=" | + | The Teneo build consists of two steps: 1) build the core plugins and use these as part of the target platform when 2) building and running the test plugins. These two steps are visible as subfolders in the main build project, the build and test subfolders. |
+ | |||
+ | The build.xml file has 3 main targets: | ||
+ | * build and test: is the main target, builds the core plugins and runs the test cases | ||
+ | * promote: copies the created update site to another location where it is available for everyone, uses standard ant copy commands | ||
+ | * clean.all: cleans the tools and output directories, uses standard ant remove directory commands | ||
+ | |||
+ | |||
+ | The interesting one is the build and test command, this one is explained in more detail. | ||
+ | <source lang="xml"> | ||
+ | <target name="build.and.test" depends="init,install.buckminster"> | ||
+ | <echo message="Building" /> | ||
+ | <ant dir="${basedir}/build" antfile="teneo_build.xml" target="run" inheritRefs="true" /> | ||
+ | |||
+ | <echo message="Testing" /> | ||
+ | <ant dir="${basedir}/test" antfile="teneo_test.xml" target="run" inheritRefs="true" /> | ||
+ | </target> | ||
</source> | </source> | ||
+ | This target first creates the target directories (the init-target) and then downloads buckminster from an update site (the install.buckminster target). The next step is to call the teneo_build.xml and teneo_test.xml to resp. build the core plugins and test them. The created target platform and workspace for both steps can be found here: | ||
+ | * [https://build.eclipse.org/hudson/job/emf-teneo-nightly/ws/build/build/ build workspace/target platform] | ||
+ | * [https://build.eclipse.org/hudson/job/emf-teneo-nightly/ws/build/test/ test workspace/target platform] | ||
− | === | + | === Build folder content === |
− | === buckminster. | + | The build setup consists of 4 files: |
+ | * '''teneo.cquery''': this files defines the root feature project to retrieve from cvs, the content of the root feature project in the feature.xml (and an additional dependency definition file, a cspex file, see below) specifies which development projects to download. | ||
+ | * '''teneo.mspec''': this file basically specifies that dependencies which are source (source="true") should be materialized in the workspace, while the other dependencies are materialized in the target platform. | ||
+ | * '''teneo.rmap''': defines where and how development projects and dependencies can be downloaded. This file for example lists cvs locations and several update sites. | ||
+ | * '''teneo_build.xml''': the ant file which executes the build step by calling Buckminster to create the workspace and target platform, download the development projects and dependencies, build the plugins and create the update site with the resulting features/plugins. | ||
+ | |||
+ | |||
+ | The mspec file is the starting point for Buckminster. The teneo_build.xml calls Buckminster with this file passed as a parameter. Its content: | ||
+ | <source lang="xml"> | ||
+ | <mspec:mspec xmlns:mspec="http://www.eclipse.org/buckminster/MetaData-1.0" materializer="p2" name="teneo.mspec" | ||
+ | url="teneo.cquery" conflictResolution="UPDATE"> | ||
+ | <mspec:property key="target.os" value="*"/> | ||
+ | <mspec:property key="target.ws" value="*"/> | ||
+ | <mspec:property key="target.arch" value="*"/> | ||
+ | <mspec:mspecNode | ||
+ | filter="(buckminster.source=true)" | ||
+ | materializer="workspace"/> | ||
+ | </mspec:mspec> | ||
+ | </source> | ||
+ | This file specifies several things: | ||
+ | * which query file to use, the cquery file is explained below, it defines which root feature project to download as a starting point for the build. | ||
+ | * how to materialize: as a default materialize to the target platform (materializer="p2"), only if source is "true" materialize to the workspace | ||
+ | |||
+ | |||
+ | As noted above the mspec file defines which cquery file to use, the content of the cquery file: | ||
+ | <source lang="xml"> | ||
+ | <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="teneo.rmap"> | ||
+ | <cq:rootRequest name="org.eclipse.emf.teneo.site" componentType="eclipse.feature"/> | ||
+ | <cq:advisorNode namePattern="org\.eclipse\.emf\.teneo.*\.source" skipComponent="true"/> | ||
+ | </cq:componentQuery> | ||
+ | </source> | ||
+ | * The cquery file defines which rmap file to use, the rmap file specifies the download locations (update sites) and cvs locations to download dependencies. | ||
+ | * Buckminster needs a root (feature) project to use as a starting point. This is defined by the rootRequest tag. | ||
+ | * The advisorNode tag is used to tell Buckminster to ignore dependencies (defined in feature.xml files) which have a pattern starting with org.eclipse.emf.teneo and ending with source. This is needed because the SDK feature projects contain these dependencies. However, these *.source plugins/features do not exist in cvs but are created by Buckminster/Eclipse during the build. So they should be ignored when downloading dependencies. See [http://www.eclipse.org/forums/index.php?t=tree&th=162136& here] for a thread on the Buckminster newsgroup on this. | ||
+ | |||
+ | |||
+ | The rmap file describes where Buckminster can download the development projects and dependencies. For example the following snippet defines the cvs location for teneo plugins (note also the source="true" attribute which signals Buckminster to treat these dependencies as source and put them in the workspace (see the mspec file above)). | ||
+ | <source lang="xml"> | ||
+ | <searchPath name="teneo.plugins"> | ||
+ | <provider componentTypes="osgi.bundle" readerType="cvs" source="true"> | ||
+ | <uri format=":pserver:anonymous@dev.eclipse.org:/cvsroot/modeling,org.eclipse.emf/org.eclipse.emf.teneo/plugins/{0}"> | ||
+ | <bc:propertyRef key="buckminster.component" /> | ||
+ | </uri> | ||
+ | </provider> | ||
+ | </searchPath> | ||
+ | </source> | ||
+ | |||
+ | As a last part of this section, the main parts of the teneo_build.xml file are discussed. To look at it in detail check out the project from cvs. | ||
+ | |||
+ | The run target contains the main build steps. | ||
+ | |||
+ | The following sets the JRE in the workspace. This prevents warnings when building the plugins. | ||
+ | <source lang="xml"> | ||
+ | <echo message="Setting jre to ${env.JAVA_HOME}"/> | ||
+ | <buckminster command="installJRE" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="--location" /> | ||
+ | <arg value="${env.JAVA_HOME}" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Set the path to the target platform. Buckminster will download binary dependencies to the target platform path and set the workspace to use this target platform. | ||
+ | <source lang="xml"> | ||
+ | <echo message="Setting targetPlatformPath to ${targetPlatformPath}" /> | ||
+ | <buckminster command="setpref" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="targetPlatformPath=${targetPlatformPath}" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Now tell Buckminster to download everything. Note that the mspec file is used. It defines a root feature project which drives everything. | ||
+ | <source lang="xml"> | ||
+ | <echo message="Importing projects into workspace ${workspacePath}"/> | ||
+ | <echo message="Importing binaries into target platform ${targetPlatformPath}" /> | ||
+ | <buckminster command="import" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="${basedir}/teneo.mspec" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Now build the plugins: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Building" /> | ||
+ | <buckminster command="build" workspace="${workspacePath}"/> | ||
+ | </source> | ||
+ | |||
+ | And create the p2 site, note that the top feature used for the site is not included in the site (as a default, see an option a few lines further down). | ||
+ | <source lang="xml"> | ||
+ | <echo message="Creating p2 site" /> | ||
+ | <buckminster command="perform" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="org.eclipse.emf.teneo.site:eclipse.feature#site.p2" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Copy the p2 site to the result directory: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Copying p2 site to result" /> | ||
+ | <antcall target="copy.p2.site" inheritRefs="true" inheritAll="true"/> | ||
+ | </source> | ||
+ | |||
+ | And create some other p2 sites for other types of results (may not be relevant for other projects than Teneo). Note the usage of the site.include.top parameter to make sure that the feature project passes in the cmdargs are also included in the p2 site. | ||
+ | <source lang="xml"> | ||
+ | <echo message="Creating p2 sites for feature zip files" /> | ||
+ | <buckminster command="perform" workspace="${workspacePath}"> | ||
+ | <globargs> | ||
+ | <arg value="-Dsite.include.top=true"/> | ||
+ | </globargs> | ||
+ | <cmdargs> | ||
+ | <arg value="org.eclipse.emf.teneo.hibernate:eclipse.feature#site.p2" /> | ||
+ | <arg value="org.eclipse.emf.teneo.eclipselink:eclipse.feature#site.p2" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | This concludes the build step. The next step is testing using the output from the build process. | ||
+ | |||
+ | === Test folder content === | ||
+ | The test folder content is very similar to the build folder content: | ||
+ | * '''teneo.cquery''': this files defines the root feature project to retrieve from cvs, the content of the root feature project in the feature.xml (and an additional dependency definition file, a cspex file) specifies which development projects to download. | ||
+ | * '''teneo.mspec''': this file basically specifies that dependencies which are source (source="true") should be materialized in the workspace, while the other dependencies are materialized in the target platform. | ||
+ | * '''teneo.rmap''': defines where and how development projects and dependencies can be downloaded. This file for example lists cvs locations and several update sites. One of the update sites to download dependencies from is the update site created by the build step. | ||
+ | * '''teneo_test.xml''': the ant file which executes the test step by calling Buckminster to create the workspace (with test development projects) and target platform, download the development projects and dependencies, build the plugins and run the test cases. | ||
+ | |||
+ | |||
+ | The mspec and cquery file are very similar to the files in the build subfolder. The rmap file is slightly different from the build step rmap file. It has an additional entry which tells Buckminster to download dependencies on core Teneo plugins from the update site created in the build step. This is visible in this snippet: | ||
+ | <source lang="xml"> | ||
+ | <searchPath name="teneo.platform"> | ||
+ | <provider componentTypes="eclipse.feature,osgi.bundle" readerType="p2" source="false" mutable="false"> | ||
+ | <uri format="file:{0}"> | ||
+ | <bc:propertyRef key="buildSitePath" /> | ||
+ | </uri> | ||
+ | </provider> | ||
+ | </searchPath> | ||
+ | </source> | ||
+ | |||
+ | The teneo_test.xml has a run target which performs the main steps. The main steps are described here. '''Note that the buckminster tag is defined by a macro in the ant_common.xml file''. | ||
+ | |||
+ | First set the JRE in the workspace, this prevents unwanted warnings in the output console: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Setting jre to ${env.JAVA_HOME}"/> | ||
+ | <buckminster command="installJRE" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="--location" /> | ||
+ | <arg value="${env.JAVA_HOME}" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Set the target platform path in the workspace and download dependencies and development projects: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Using workspace ${workspacePath}" /> | ||
+ | <echo message="Setting targetPlatformPath to ${targetPlatformPath}" /> | ||
+ | <buckminster command="setpref" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="targetPlatformPath=${targetPlatformPath}" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | |||
+ | <echo message="Importing projects into workspace ${workspacePath}" /> | ||
+ | <echo message="Importing binaries into target platform ${targetPlatformPath}" /> | ||
+ | <buckminster command="import" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="${basedir}/teneo.mspec" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Build the plugins: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Building" /> | ||
+ | <buckminster command="build" workspace="${workspacePath}" /> | ||
+ | </source> | ||
+ | |||
+ | Run the test cases. The emma command will run junit with code coverage checking. See a later section on code coverage setup. Instead of the emma command, the command 'junit' could also have been used (but then no code coverage report is created ofcourse). The junit test run itself consists of calling a launch configuration and defining the output of the test results. The launch configuration has been created manually and checked into cvs. | ||
+ | <source lang="xml"> | ||
+ | <echo message="Running tests Hibernate" /> | ||
+ | <buckminster command="emma" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="-l" /> | ||
+ | <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> | ||
+ | <arg value="-o" /> | ||
+ | <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> | ||
+ | <arg value="--stdout" /> | ||
+ | <arg value="${testResultsPath}/output/stdout.txt" /> | ||
+ | <arg value="--stderr" /> | ||
+ | <arg value="${testResultsPath}/output/stderr.txt" /> | ||
+ | <arg value="--xml" /> | ||
+ | <arg value="${testResultsPath}/output/hibernate-coverageReport.xml" /> | ||
+ | <arg value="--flatXML"/> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | The flatXML argument is used to convert the output from the tests to a result which can be read by Hudson. | ||
+ | |||
+ | == Defining dependencies: buckminster.cspex == | ||
+ | |||
+ | Buckminster uses the explicit plugin dependencies in the MANIFEST.MF to download related project and binary dependencies. '''Buckminster can however not directly/automatically resolve dependencies defined by the import-packages directive'''. This is logical as Buckminster can not identify the plugins to download, just on the basis of the package name. To solve this, it is possible to define additional dependencies using a so-called buckminster.cspex file. The Teneo build has a buckminster.cspex file in two feature projects: | ||
+ | * org.eclipse.emf.teneo.site-feature | ||
+ | * org.eclipse.emf.teneo.site-feature | ||
+ | |||
+ | |||
+ | These two feature projects are used in the rootRequest of the cquery files. | ||
+ | |||
+ | Note, the file containing the additional dependencies '''must be called buckminster.cspex'''. | ||
+ | |||
+ | The content of a cspex file: | ||
+ | <source lang="xml"> | ||
+ | <cspecExtension | ||
+ | xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
+ | xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" | ||
+ | xmlns="http://www.eclipse.org/buckminster/CSpec-1.0" componentType="eclipse.feature" version="1.0.0"> | ||
+ | <dependencies> | ||
+ | <dependency name="org.eclipse.equinox.executable" componentType="eclipse.feature"/> | ||
+ | <dependency name="org.eclipse.platform" componentType="eclipse.feature"/> | ||
+ | <dependency name="org.hibernate" componentType="eclipse.feature"/> | ||
+ | <dependency name="javax.persistence" componentType="eclipse.feature"/> | ||
+ | <dependency name="org.dom4j" componentType="eclipse.feature"/> | ||
+ | <dependency name="org.apache.commons.logging" componentType="eclipse.feature"/> | ||
+ | </dependencies> | ||
+ | </cspecExtension> | ||
+ | </source> | ||
+ | This cspex file defines dependencies on several eclipse features. The rmap file define where to find these features (i.e. which update site for example). | ||
== Testing == | == Testing == | ||
− | |||
− | == Links == | + | Testcases are run through Buckminster using a launch configuration (so Buckminster calls a launch configuration). Teneo has 2 launch configuration one to run tests for Hibernate and one for EclipseLink. The launch configuration is easy to create, after running your main test suite in Eclipse, a run configuration should be present in the window Run > Run Configurations. |
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.test.launch.png|center|700px]] | ||
+ | |||
+ | |||
+ | To store the run/launch configuration in a file: in the last tab of the configuration (Common), the save as location is set to a shared file in a project (org.eclipse.emf.teneo.hibernate.test in this case). The file is checked-in into cvs. This shared file is then used as the input to the test run through buckminster: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Running tests Hibernate" /> | ||
+ | <buckminster command="junit" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="-l" /> | ||
+ | <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> | ||
+ | <arg value="-o" /> | ||
+ | <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> | ||
+ | <arg value="--stdout" /> | ||
+ | <arg value="${testResultsPath}/output/stdout.txt" /> | ||
+ | <arg value="--stderr" /> | ||
+ | <arg value="${testResultsPath}/output/stderr.txt" /> | ||
+ | <arg value="--flatXML"/> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | The above command will tell Buckminster to run the launch configuration AllTests.launch located in the org.eclipse.emf.teneo.hibernate.test project (which was downloaded into the workspace). The output location is specified with the -o parameter. The xml in the output location is used by Hudson to create a test report. | ||
+ | |||
+ | Some links to the Teneo build workspace for the test results: | ||
+ | * [https://build.eclipse.org/hudson/job/emf-teneo-nightly/ws/build/result/test/ raw test output] | ||
+ | * [https://build.eclipse.org/hudson/job/emf-teneo-nightly/lastCompletedBuild/testReport/ test report of a successful build] | ||
+ | |||
+ | == SDK Feature (Including a source feature) == | ||
+ | The Teneo SDK feature contains both the runtime plugins as well as the source plugins. The source plugins are special because they not exist in cvs. The source plugins are created by Buckminster in the build process. The question encountered during setting up the Teneo build, [http://www.eclipse.org/forums/index.php?t=tree&th=162136& how to setup a SDK feature]. | ||
+ | |||
+ | The solution was actually quite simple. Teneo Hibernate has two feature projects: | ||
+ | * org.eclipse.emf.teneo.hibernate-feature: the feature.xml includes all the relevant core Teneo plugins | ||
+ | * org.eclipse.emf.teneo.hibernate.sdk-feature: its feature.xml includes the org.eclipse.emf.teneo.hibernate feature and the org.eclipse.emf.teneo.hibernate.source feature. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.sdk.feature.png|center]] | ||
+ | |||
+ | |||
+ | The org.eclipse.emf.teneo.hibernate.source feature does not exist (therefore the warning in the feature.xml above). It is created at build-time. To ensure that Buckminster won't try to resolve and download the org.eclipse.emf.teneo.hibernate.source feature (and fail because it can't find it) an advisory node is added to the cquery file: | ||
+ | <source lang="xml"> | ||
+ | <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="teneo.rmap"> | ||
+ | <cq:rootRequest name="org.eclipse.emf.teneo.site" componentType="eclipse.feature"/> | ||
+ | <cq:advisorNode namePattern="org\.eclipse\.emf\.teneo.*\.source" skipComponent="true"/> | ||
+ | </cq:componentQuery> | ||
+ | </source> | ||
+ | |||
+ | The above setup is enough to create a SDK feature which contains the runtime feature (and its plugins) and the source feature with the source plugins. The next step is to define the update site, this is explained next. | ||
+ | |||
+ | == Defining and creating an update site == | ||
+ | The Teneo build creates an update site. For an example see this location: | ||
+ | [https://build.eclipse.org/hudson/job/emf-teneo-nightly/lastSuccessfulBuild/artifact/build/result/p2site/ update site]. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.update.site.png|center|700px]] | ||
+ | |||
+ | |||
+ | The update site is defined through a feature project: the org.eclipse.emf.teneo.site-feature project. It has two important files: | ||
+ | * feature.xml: defines which features are part of the update site | ||
+ | * category.xml: defines how the features are categorized | ||
+ | |||
+ | |||
+ | The feature.xml contains 4 features: | ||
+ | <source lang="xml"> | ||
+ | <includes | ||
+ | id="org.eclipse.emf.teneo.eclipselink" | ||
+ | version="0.0.0"/> | ||
+ | |||
+ | <includes | ||
+ | id="org.eclipse.emf.teneo.eclipselink.sdk" | ||
+ | version="0.0.0"/> | ||
+ | |||
+ | <includes | ||
+ | id="org.eclipse.emf.teneo.hibernate" | ||
+ | version="0.0.0"/> | ||
+ | |||
+ | <includes | ||
+ | id="org.eclipse.emf.teneo.hibernate.sdk" | ||
+ | version="0.0.0"/> | ||
+ | </source> | ||
+ | |||
+ | The category.xml creates categories for these features: | ||
+ | <source lang="xml"> | ||
+ | <site> | ||
+ | <feature id="org.eclipse.emf.teneo.hibernate" version="0.0.0"> | ||
+ | <category name="org.eclipse.emf.teneo.hibernate"/> | ||
+ | </feature> | ||
+ | <feature id="org.eclipse.emf.teneo.hibernate.sdk" version="0.0.0"> | ||
+ | <category name="org.eclipse.emf.teneo.hibernate"/> | ||
+ | </feature> | ||
+ | <feature id="org.eclipse.emf.teneo.eclipselink" version="0.0.0"> | ||
+ | <category name="org.eclipse.emf.teneo.eclipselink"/> | ||
+ | </feature> | ||
+ | <feature id="org.eclipse.emf.teneo.eclipselink.sdk" version="0.0.0"> | ||
+ | <category name="org.eclipse.emf.teneo.eclipselink"/> | ||
+ | </feature> | ||
+ | <category-def name="org.eclipse.emf.teneo.hibernate" label="EMF Teneo Hibernate"/> | ||
+ | <category-def name="org.eclipse.emf.teneo.eclipselink" label="EMF Teneo EclipseLink"/> | ||
+ | </site> | ||
+ | </source> | ||
+ | |||
+ | To build the update site the build xml file contains the following snippet: | ||
+ | <source lang="xml"> | ||
+ | <echo message="Creating p2 site" /> | ||
+ | <buckminster command="perform" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="org.eclipse.emf.teneo.site:eclipse.feature#site.p2" /> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | Buckminster is called with the org.eclipse.emf.teneo.site feature (note feature id is used here, not the project name!), and the site.p2 call. As a default (is option controlled) the content of the passed in feature is used for the content of the site, so not the feature project (org.eclipse.emf.teneo.site) itself is not part of the update site. | ||
+ | |||
+ | == Code Coverage == | ||
+ | It is quite easy to enable code coverage analysis as part of the test run. This section explains how to set it up. | ||
+ | |||
+ | First make sure that the org.eclipse.buckminster.emma.headless.feature.feature.group feature is installed as part of the Buckminster install. See the install.buckminster target in Teneo's build.xml. | ||
+ | |||
+ | To enable code coverage analysis in Buckminster: | ||
+ | * use the 'emma' command instead of the 'junit' command when calling Buckminster | ||
+ | * pass an extra xml parameter with the output location of the code coverage analysis | ||
+ | <source lang="xml"> | ||
+ | <buckminster command="emma" workspace="${workspacePath}"> | ||
+ | <cmdargs> | ||
+ | <arg value="-l" /> | ||
+ | <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> | ||
+ | <arg value="-o" /> | ||
+ | <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> | ||
+ | <arg value="--stdout" /> | ||
+ | <arg value="${testResultsPath}/output/stdout.txt" /> | ||
+ | <arg value="--stderr" /> | ||
+ | <arg value="${testResultsPath}/output/stderr.txt" /> | ||
+ | <arg value="--xml" /> | ||
+ | <arg value="${testResultsPath}/output/hibernate-coverageReport.xml" /> | ||
+ | <arg value="--flatXML"/> | ||
+ | </cmdargs> | ||
+ | </buckminster> | ||
+ | </source> | ||
+ | |||
+ | Then in Hudson set the following configuration: | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.emma.png|center|700px]] | ||
+ | |||
+ | |||
+ | '''Note''': the Emma XML report value is not a pattern, it must be the actual file name of the xml output, i.e. the filename specified in the Buckminster call in the xml parameter. As only one file can be specified it is not possible to perform code coverage analysis on multiple separate Buckminster calls. | ||
+ | |||
+ | It is also possible to collect multiple buckminster emma commands into one code coverage file. The emma commands need to be executed together in one script file passed to Buckminster. This is explained in more detail in this [http://www.eclipse.org/forums/index.php?t=msg&&th=163600&goto=518062#msg_518062 thread] on the Buckminster newsgroup. | ||
+ | |||
+ | An important thing to note is that Emma does code coverage analysis on the development projects in the workspace. In case of Teneo this is actually less useful as the core plugins are not part of the workspace but of the target platform. This because with Teneo we want to test the actually binary core plugins and not the development projects. So to really make use of code coverage one should run the test cases with all the core plugins also as development projects in the workspace. This is what the CDO project does (see related links at the end of this wiki page). | ||
+ | |||
+ | == Signing and pack200== | ||
+ | Enabling signing and pack200 is quite straightforward with Buckminster. To enable this set the following properties: | ||
+ | * site.pack200 to true | ||
+ | * site.signing to true | ||
+ | * signing.type (Teneo has this set to eclipse.local) | ||
+ | * eclipse.staging.area (Teneo has this set to ${env.HUDSON_HOME}/jobs/${env.JOB_NAME}/signing) | ||
+ | |||
+ | The last two are set in the buckminster.properties file. The first two are set through Hudson and handled in the build.xml file like this: | ||
+ | <source lang="xml"> | ||
+ | <condition property="site.pack200" value="${env.SITE_PACK200}" else="false"> | ||
+ | <isset property="env.SITE_PACK200" /> | ||
+ | </condition> | ||
+ | |||
+ | <condition property="site.signing" value="${env.SITE_SIGNING}" else="false"> | ||
+ | <isset property="env.SITE_SIGNING" /> | ||
+ | </condition> | ||
+ | </source> | ||
+ | |||
+ | == Hudson Configuration == | ||
+ | |||
+ | The Teneo Hudson configuration can be found [https://build.eclipse.org/hudson/job/emf-teneo-nightly/configure here] (might be not accessible, depending on authorizations). This section will touch the main parts of the configuration and show the resulting build start page. | ||
+ | |||
+ | The Hudson setup starts with the definition of a number of parameters. These parameters are visible as input fields when a build is triggered manually, the default values are used when a cvs change triggers a build. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.build.hudson.parameterized.png|center|700px]] | ||
+ | |||
+ | |||
+ | The Teneo build also has parameters for signing and pack200. The parameters are available in ant as environment ant variables (e.g. env.SITE_PACK200). The BUILD_TARGET parameter is used to define which ant target to call. | ||
+ | |||
+ | The next configuration selects a specific jre: | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.jre.png|center|700px]] | ||
+ | |||
+ | |||
+ | The cvs location and polling frequency define two things: | ||
+ | * it tells Hudson to download everything from the cvs location to the workspace (see [https://build.eclipse.org/hudson/job/emf-teneo-nightly/ws/org.eclipse.emf/org.eclipse.emf.teneo/ here] for the workspace result) | ||
+ | * it tells Hudson which cvs locations to watch and how often to poll/check cvs for changes | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.cvs.png|center|700px]] | ||
+ | |||
+ | |||
+ | Tell Hudson not to let your build hang forever: | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.abort.png|center|700px]] | ||
+ | |||
+ | |||
+ | The following part is important, it tells Hudson what to execute (note the path to the ant file is made visible when you press the advanced button). Note: | ||
+ | * the usage of the BUILD_TARGET parameter, this parameter is defined in the beginning of the Hudson configuration (see above) | ||
+ | * the extra property which is needed to solve an issue with the directory (it can't work with all jvm's), see this [http://www.eclipse.org/forums/index.php?t=msg&th=164621&start=0& forum thread]. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.ant.png|center|700px]] | ||
+ | |||
+ | |||
+ | The next section defines where Hudson can find the results. The results are available as the last successfull build artifacts. The test result location corresponds to the output location of the test runs. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.result.png|center|700px]] | ||
+ | |||
+ | |||
+ | The last screenshot shows how the build start page looks like. Each of the parameters is visible as an input field. The shown values are the defaults for when the build is automatically triggered on a cvs change. | ||
+ | |||
+ | |||
+ | [[Image:org.eclipse.emf.teneo.hudson.build.start.png|center|700px]] | ||
+ | |||
+ | == Troubleshooting == | ||
+ | |||
+ | This section lists some issues and their solution (see also the links to newsgroup threads in a later section of this wiki page): | ||
+ | * Getting error: 'Unpack facility not configured' when buckminster is downloaded in the build step | ||
+ | The director (which downloads Buckminster) does not work fine with certain jre's, see this solution: http://www.eclipse.org/forums/index.php?t=msg&th=164621&start=0& if you download the Teneo build and try it out locally then you have to edit the following line in the build.xml file and set the jre/bin location specific for your case, like here: | ||
+ | <source lang="xml"> | ||
+ | <jvmarg value="-Dorg.eclipse.update.jarprocessor.pack200=/home/mtaal/myapps/java/jdk1.5.0_15/jre/bin" /> | ||
+ | </source> | ||
+ | |||
+ | * Solving the warning: [java] WARN: CVS file org.eclipse.emf/org.eclipse.emf.cdo/hibernate/org.eclipse.emf.cdo.tests.hibernate/src/org/eclipse/emf/cdo/tests/hibernate/external_model1_4.persistence.xml either contains invalid line endings on the server (CR/LF instead of just LF) or is a binary file that is not marked as -kb. | ||
+ | ** '''Solution''': this can be repaired in Eclipse through the menu: File > Convert Line Delimiters to. Convert all line delimiters to Unix. This menu option can be executed for one or more development projects. | ||
+ | |||
+ | * Solving the warning: [java] Warning: file /opt/users/hudsonbuild/.hudson/jobs/emf-teneo-nightly/workspace/build/build/workspace/plugins/org.eclipse.emf.teneo.annotations/META-INF/MANIFEST.MF,line 9: The JRE container on the classpath is not a perfect match to the 'J2SE-1.5' execution environment | ||
+ | ** '''Solution''': do the following: 1) select an explicit JRE/JDK in Hudson (see Hudson configuration), 2) set an explicit jre in the workspace (see examples above in the teneo_build.xml and teneo_test.xml), 3) for plugin projects showing the warning, right-mouse click on the plugin project and then 'PDE Tools > Update Classpath'. | ||
+ | |||
+ | * Out of memory error: | ||
+ | ** '''Solution''': see this bugzilla for a solution https://bugs.eclipse.org/bugs/show_bug.cgi?id=306102 | ||
+ | |||
+ | * Out of memory error when processing test results: this occurs at the end of the test run. The buckminster junit plugin converts the complete test output to an in-memory String before writing to disk. | ||
+ | ** '''Solution''': This can be solved by setting the output log level of testcases to a higher level (for example 'error' instead of 'debug'). | ||
+ | |||
+ | == Related Links == | ||
For the Teneo setup some scripts have been re-used from the Buckminster build itself. The Buckminster releng/build project can be found [http://dev.eclipse.org/viewsvn/index.cgi/trunk/org.eclipse.buckminster.releng/?root=Tools_BUCKMINSTER here] in svn. | For the Teneo setup some scripts have been re-used from the Buckminster build itself. The Buckminster releng/build project can be found [http://dev.eclipse.org/viewsvn/index.cgi/trunk/org.eclipse.buckminster.releng/?root=Tools_BUCKMINSTER here] in svn. | ||
− | + | During the Teneo build setup several newsgroup posts and bugzillas were used to discuss issues and solutions: | |
+ | * [http://www.eclipse.org/forums/index.php?t=tree&th=161641& Qualifier replacement not happening] | ||
+ | * [http://www.eclipse.org/forums/index.php?t=msg&th=162006&start=0& How to make a build fail on a compile error] | ||
+ | * [http://www.eclipse.org/forums/index.php?t=msg&th=162179&start=0& Creating separate zip files with feature and plugins] | ||
+ | * [http://www.eclipse.org/forums/index.php?t=tree&th=162762& How to prevent: JRE container on the classpath is not a perfect match to the 'J2SE-1.5'] | ||
+ | * [http://www.eclipse.org/forums/index.php?t=msg&goto=514848& What does promote do?] | ||
+ | * [http://www.eclipse.org/forums/index.php?t=msg&goto=510603& mspec to target platform and workspace] | ||
+ | * [https://bugs.eclipse.org/bugs/show_bug.cgi?id=301364 Request for Hudson jobs for EMFT Texo and EMF Teneo] | ||
+ | * [https://bugs.eclipse.org/bugs/show_bug.cgi?id=302153 How to create a SDK feature] | ||
+ | * [https://bugs.eclipse.org/bugs/show_bug.cgi?id=303867 Provide an option to flatten JUnit testReport XMLs] | ||
+ | |||
+ | The [[CDO|CDO]] project also uses buckminster for its build. Some links/information: | ||
+ | * [https://build.eclipse.org/hudson/job/emf-cdo-integration/ CDO Hudson job] | ||
+ | * The CDO Buckminster releng project can be found here: | ||
+ | dev.eclipse.org, /cvsroot/modeling, org.eclipse.emf/org.eclipse.emf.cdo/releng and then the org.eclipse.emf.cdo.releng project | ||
+ | |||
+ | == TODO == | ||
+ | * Update the modeling update site and download pages with output from the build. | ||
+ | * Hudson configuration: currently the promote and clean.all ant targets will report a fail in Hudson because Hudson expects test results as part of the build output, obviously these two targets will not create test results. Possibly these two targets have to be moved to a separate Hudson job. |
Latest revision as of 16:47, 1 November 2011
Contents
- 1 Try it
- 2 Understanding Buckminster
- 3 Teneo Main builds steps (build, generate, test)
- 4 Structure of build projects/features
- 5 Main build project: org.eclipse.emf.teneo.releng.buckminster
- 6 Defining dependencies: buckminster.cspex
- 7 Testing
- 8 SDK Feature (Including a source feature)
- 9 Defining and creating an update site
- 10 Code Coverage
- 11 Signing and pack200
- 12 Hudson Configuration
- 13 Troubleshooting
- 14 Related Links
- 15 TODO
Teneo has been moved from CVS to GIT in November 2011, the build setup has changed because of this. This page maybe partially outdated. Please check this blog for more information.
This page describes the Teneo build setup. Teneo uses Buckminster and Hudson for its continuous build infrastructure.
The Teneo Hudson job, which uses the build setup as it is described in this wiki page, is available here.
To understand the content of this wiki page it is necessary to understand the main Buckminster concepts. These are described in detail in the Bucky Book. Some Buckminster concepts can seem fairly abstract initially, the 'Understanding Buckminster' section below may help here.
Try it
It is quite easy to do a Teneo build. To download Teneo and do a build (building, testing, generating p2 site) you need to do the following steps:
cvs -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling co -d org.eclipse.emf.teneo.releng.buckminster \ org.eclipse.emf/org.eclipse.emf.teneo/releng/org.eclipse.emf.teneo.releng.buckminster cd org.eclipse.emf.teneo.releng.buckminster ant
The above assumes that ant is installed and that JAVA_HOME is set to Java 5.0 or higher. Note, the build downloads Buckminster using the Eclipse director, the director has sometimes trouble with certain java 6 versions. So if you get 'Unpack facility not configured' errors then check out the troubleshooting section below.
After about 20 minutes you will see a build folder (next to the org.eclipse.emf.teneo.releng.buckminster folder) with a number of subfolders:
- build: contains the targetplatform (the tp subfolder) for building the main plugins and the build workspace itself (the workspace subfolder)
- test: contains the targetplatform for testing (in the tp subfolder), the workspace subfolder contains the workspace used for the test projects.
- output: contains temporary output from the build steps.
- result: contains the generation result (p2 site) and the test results.
- tools: contains buckminster and the p2 director applications. These are downloaded as part of the build.
The build workspace (build/workspace) and test workspace (test/workspace) can be opened in Eclipse as workspaces.
Understanding Buckminster
Initially some Buckminster concepts can appear fairly abstract (for me at least) and it may not be directly clear why several xml files are needed to define the process. This section may help to make it a bit clearer.
Main Buckminster functionality (as it is used for the Teneo build):
- An important task performed by Buckminster is the creation of a workspace with target platform. The workspace is the same or similar to what you see in your IDE. The build and test runs are done using the created workspace/target platform.
- Buckminster uses the includes and dependency definitions in the MANIFEST.MF (for a plugin project) and the feature.xml (for a feature project) to download related projects or binaries. Additional dependencies are defined in a buckminster.cspex file. The build and buckminster files often only refer to so-called root (feature) projects. The includes and dependencies of the root project (often a feature project) then automatically drive the download of all other relevant projects and binaries into the workspace and target platform.
- Buckminster can search many types of locations for dependencies. The locations and their search order is defined in a configuration file (the *.rmap file). Development projects are typically downloaded from cvs while binary dependencies are often coming from update sites.
- When downloading dependencies, Buckminster determines which dependency to place in the workspace (as a development project) and which of the dependencies to place in the target platform. The choice where to place what is defined in the buckminster *.rmap and *.mspec file using an attribute: source="true", which signals a dependency to be a source (so should be placed in the workspace).
- Buckminster has several commands which can be executed against a workspace/target platform, for example run tests, build a p2.site etc.
A short description on the meaning of some files (i.e. how they are used in the Teneo build):
- rmap file: defines where/how to download dependencies and development projects, contains cvs locations, update site url's and specifies an order in which these locations are searched. Also identifies which sources are considered to contain sources (i.e. development projects) and which are used for binary dependencies.
- cquery file: defines which rmap file to use and defines the root (feature) project, the content (defined in the feature.xml) and the dependencies of the content are downloaded, also contains extra directives which are passed to Buckminster when resolving dependencies.
- mspec file: is the starting point. It specifies which cquery file to use and also how the plugins/development projects downloaded as part of resolving the cquery file should be treated, or materialized. Materialized means: where to place the downloaded plugins/development projects, in the workspace or in the target platform.
Teneo Main builds steps (build, generate, test)
The Teneo build process is executed through an ant script which calls Buckminster for the main build steps. The ant script goes through the following steps:
- download the p2 director and using the director, download buckminster.
- build the main deliverables (using buckminster):
- download the development projects from CVS and use them to create a workspace with a target platform
- download/retrieve the dependencies and set them in the targetplatform
- build the software (compile etc.)
- build the update site
- create downloadable zips
- build the test projects (using buckminster):
- download the development test projects from CVS and use them to create a workspace with a target platform
- download/retrieve the dependencies and set them in the targetplatform. This includes the artifacts generated in the previous step.
- build/compile the test plugins
- run the test cases and convert the test results to a format which can be read by hudson
The ant script also take care of copying the output (p2 site and features/plugins) and creating zip files etc.
The following sections describe the necessary files for the build: the ant xml files and the buckminster files.
Structure of build projects/features
The build infrastructure for Teneo is centered around a releng project containing the ant build xml file and the main Buckminster files. A number of other feature projects are used to define which plugins are part of the build, part of the update site and which are part of the tests etc.
The releng and feature projects can be found in this location in cvs:
- dev.eclipse.org
- /cvsroot/modeling
- org.eclipse.emf/org.eclipse.emf.teneo/releng and org.eclipse.emf/org.eclipse.emf.teneo/features
The org.eclipse.emf.teneo.releng.buckminster project contains the ant files, and the buckminster files for the build and test step. The content of this project and the buckminster setup is described in more detail below. This project is the most important one to look at when trying to understand Teneo's build setup.
The feature projects are used to structure the output of the build in the correct way, i.e. which features/plugins are made available in an update site, which features/plugins are involved in testing etc.
A summary of these feature projects:
- org.eclipse.emf.teneo.site-feature: defines the update site and the main features it contains (Teneo Hibernate runtime and SDK and Teneo EclipseLink runtime and SDK). This project contains a buckminster.cspex file to define extra dependencies (this is described in more detail below). The included features:
- org.eclipse.emf.teneo.hibernate-feature: the runtime feature, this is a standard PDE feature project (the plugins are defined in the feature.xml).
- org.eclipse.emf.teneo.hibernate.sdk-feature: the SDK feature includes the runtime feature and source feature (see below for more information on building SDK features).
- org.eclipse.emf.teneo.eclipselink-feature: the EclipseLink runtime feature, this is a standard PDE feature project (the plugins are defined in the feature.xml).
- org.eclipse.emf.teneo.eclipselink.sdk-feature: the SDK feature includes the runtime feature and source feature (see below for more information on building SDK features).
- org.eclipse.emf.teneo.tests-feature: contains the test plugins and their dependencies. The main test plugins contain test launch configurations (see below running test cases).
Main build project: org.eclipse.emf.teneo.releng.buckminster
After downloading the main build project from cvs you can see that it consists of several files in the root and two subfolders: build and test. Here is a summary for the meaning of different files:
- ant_common.xml: contains macros which are used both in test and build phases, the main macros facilitate calling buckminster. This ant file is imported by other build files.
- buckminster.properties: defines several properties related to how to sign plugins and where to download the director and buckminster applications.
- build.xml: the main build file which drives the build and test phases. Its content is described in more detail below.
The Teneo build consists of two steps: 1) build the core plugins and use these as part of the target platform when 2) building and running the test plugins. These two steps are visible as subfolders in the main build project, the build and test subfolders.
The build.xml file has 3 main targets:
- build and test: is the main target, builds the core plugins and runs the test cases
- promote: copies the created update site to another location where it is available for everyone, uses standard ant copy commands
- clean.all: cleans the tools and output directories, uses standard ant remove directory commands
The interesting one is the build and test command, this one is explained in more detail.
<target name="build.and.test" depends="init,install.buckminster"> <echo message="Building" /> <ant dir="${basedir}/build" antfile="teneo_build.xml" target="run" inheritRefs="true" /> <echo message="Testing" /> <ant dir="${basedir}/test" antfile="teneo_test.xml" target="run" inheritRefs="true" /> </target>
This target first creates the target directories (the init-target) and then downloads buckminster from an update site (the install.buckminster target). The next step is to call the teneo_build.xml and teneo_test.xml to resp. build the core plugins and test them. The created target platform and workspace for both steps can be found here:
Build folder content
The build setup consists of 4 files:
- teneo.cquery: this files defines the root feature project to retrieve from cvs, the content of the root feature project in the feature.xml (and an additional dependency definition file, a cspex file, see below) specifies which development projects to download.
- teneo.mspec: this file basically specifies that dependencies which are source (source="true") should be materialized in the workspace, while the other dependencies are materialized in the target platform.
- teneo.rmap: defines where and how development projects and dependencies can be downloaded. This file for example lists cvs locations and several update sites.
- teneo_build.xml: the ant file which executes the build step by calling Buckminster to create the workspace and target platform, download the development projects and dependencies, build the plugins and create the update site with the resulting features/plugins.
The mspec file is the starting point for Buckminster. The teneo_build.xml calls Buckminster with this file passed as a parameter. Its content:
<mspec:mspec xmlns:mspec="http://www.eclipse.org/buckminster/MetaData-1.0" materializer="p2" name="teneo.mspec" url="teneo.cquery" conflictResolution="UPDATE"> <mspec:property key="target.os" value="*"/> <mspec:property key="target.ws" value="*"/> <mspec:property key="target.arch" value="*"/> <mspec:mspecNode filter="(buckminster.source=true)" materializer="workspace"/> </mspec:mspec>
This file specifies several things:
- which query file to use, the cquery file is explained below, it defines which root feature project to download as a starting point for the build.
- how to materialize: as a default materialize to the target platform (materializer="p2"), only if source is "true" materialize to the workspace
As noted above the mspec file defines which cquery file to use, the content of the cquery file:
<cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="teneo.rmap"> <cq:rootRequest name="org.eclipse.emf.teneo.site" componentType="eclipse.feature"/> <cq:advisorNode namePattern="org\.eclipse\.emf\.teneo.*\.source" skipComponent="true"/> </cq:componentQuery>
- The cquery file defines which rmap file to use, the rmap file specifies the download locations (update sites) and cvs locations to download dependencies.
- Buckminster needs a root (feature) project to use as a starting point. This is defined by the rootRequest tag.
- The advisorNode tag is used to tell Buckminster to ignore dependencies (defined in feature.xml files) which have a pattern starting with org.eclipse.emf.teneo and ending with source. This is needed because the SDK feature projects contain these dependencies. However, these *.source plugins/features do not exist in cvs but are created by Buckminster/Eclipse during the build. So they should be ignored when downloading dependencies. See here for a thread on the Buckminster newsgroup on this.
The rmap file describes where Buckminster can download the development projects and dependencies. For example the following snippet defines the cvs location for teneo plugins (note also the source="true" attribute which signals Buckminster to treat these dependencies as source and put them in the workspace (see the mspec file above)).
<searchPath name="teneo.plugins"> <provider componentTypes="osgi.bundle" readerType="cvs" source="true"> <uri format=":pserver:anonymous@dev.eclipse.org:/cvsroot/modeling,org.eclipse.emf/org.eclipse.emf.teneo/plugins/{0}"> <bc:propertyRef key="buckminster.component" /> </uri> </provider> </searchPath>
As a last part of this section, the main parts of the teneo_build.xml file are discussed. To look at it in detail check out the project from cvs.
The run target contains the main build steps.
The following sets the JRE in the workspace. This prevents warnings when building the plugins.
<echo message="Setting jre to ${env.JAVA_HOME}"/> <buckminster command="installJRE" workspace="${workspacePath}"> <cmdargs> <arg value="--location" /> <arg value="${env.JAVA_HOME}" /> </cmdargs> </buckminster>
Set the path to the target platform. Buckminster will download binary dependencies to the target platform path and set the workspace to use this target platform.
<echo message="Setting targetPlatformPath to ${targetPlatformPath}" /> <buckminster command="setpref" workspace="${workspacePath}"> <cmdargs> <arg value="targetPlatformPath=${targetPlatformPath}" /> </cmdargs> </buckminster>
Now tell Buckminster to download everything. Note that the mspec file is used. It defines a root feature project which drives everything.
<echo message="Importing projects into workspace ${workspacePath}"/> <echo message="Importing binaries into target platform ${targetPlatformPath}" /> <buckminster command="import" workspace="${workspacePath}"> <cmdargs> <arg value="${basedir}/teneo.mspec" /> </cmdargs> </buckminster>
Now build the plugins:
<echo message="Building" /> <buckminster command="build" workspace="${workspacePath}"/>
And create the p2 site, note that the top feature used for the site is not included in the site (as a default, see an option a few lines further down).
<echo message="Creating p2 site" /> <buckminster command="perform" workspace="${workspacePath}"> <cmdargs> <arg value="org.eclipse.emf.teneo.site:eclipse.feature#site.p2" /> </cmdargs> </buckminster>
Copy the p2 site to the result directory:
<echo message="Copying p2 site to result" /> <antcall target="copy.p2.site" inheritRefs="true" inheritAll="true"/>
And create some other p2 sites for other types of results (may not be relevant for other projects than Teneo). Note the usage of the site.include.top parameter to make sure that the feature project passes in the cmdargs are also included in the p2 site.
<echo message="Creating p2 sites for feature zip files" /> <buckminster command="perform" workspace="${workspacePath}"> <globargs> <arg value="-Dsite.include.top=true"/> </globargs> <cmdargs> <arg value="org.eclipse.emf.teneo.hibernate:eclipse.feature#site.p2" /> <arg value="org.eclipse.emf.teneo.eclipselink:eclipse.feature#site.p2" /> </cmdargs> </buckminster>
This concludes the build step. The next step is testing using the output from the build process.
Test folder content
The test folder content is very similar to the build folder content:
- teneo.cquery: this files defines the root feature project to retrieve from cvs, the content of the root feature project in the feature.xml (and an additional dependency definition file, a cspex file) specifies which development projects to download.
- teneo.mspec: this file basically specifies that dependencies which are source (source="true") should be materialized in the workspace, while the other dependencies are materialized in the target platform.
- teneo.rmap: defines where and how development projects and dependencies can be downloaded. This file for example lists cvs locations and several update sites. One of the update sites to download dependencies from is the update site created by the build step.
- teneo_test.xml: the ant file which executes the test step by calling Buckminster to create the workspace (with test development projects) and target platform, download the development projects and dependencies, build the plugins and run the test cases.
The mspec and cquery file are very similar to the files in the build subfolder. The rmap file is slightly different from the build step rmap file. It has an additional entry which tells Buckminster to download dependencies on core Teneo plugins from the update site created in the build step. This is visible in this snippet:
<searchPath name="teneo.platform"> <provider componentTypes="eclipse.feature,osgi.bundle" readerType="p2" source="false" mutable="false"> <uri format="file:{0}"> <bc:propertyRef key="buildSitePath" /> </uri> </provider> </searchPath>
The teneo_test.xml has a run target which performs the main steps. The main steps are described here. 'Note that the buckminster tag is defined by a macro in the ant_common.xml file.
First set the JRE in the workspace, this prevents unwanted warnings in the output console:
<echo message="Setting jre to ${env.JAVA_HOME}"/> <buckminster command="installJRE" workspace="${workspacePath}"> <cmdargs> <arg value="--location" /> <arg value="${env.JAVA_HOME}" /> </cmdargs> </buckminster>
Set the target platform path in the workspace and download dependencies and development projects:
<echo message="Using workspace ${workspacePath}" /> <echo message="Setting targetPlatformPath to ${targetPlatformPath}" /> <buckminster command="setpref" workspace="${workspacePath}"> <cmdargs> <arg value="targetPlatformPath=${targetPlatformPath}" /> </cmdargs> </buckminster> <echo message="Importing projects into workspace ${workspacePath}" /> <echo message="Importing binaries into target platform ${targetPlatformPath}" /> <buckminster command="import" workspace="${workspacePath}"> <cmdargs> <arg value="${basedir}/teneo.mspec" /> </cmdargs> </buckminster>
Build the plugins:
<echo message="Building" /> <buckminster command="build" workspace="${workspacePath}" />
Run the test cases. The emma command will run junit with code coverage checking. See a later section on code coverage setup. Instead of the emma command, the command 'junit' could also have been used (but then no code coverage report is created ofcourse). The junit test run itself consists of calling a launch configuration and defining the output of the test results. The launch configuration has been created manually and checked into cvs.
<echo message="Running tests Hibernate" /> <buckminster command="emma" workspace="${workspacePath}"> <cmdargs> <arg value="-l" /> <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> <arg value="-o" /> <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> <arg value="--stdout" /> <arg value="${testResultsPath}/output/stdout.txt" /> <arg value="--stderr" /> <arg value="${testResultsPath}/output/stderr.txt" /> <arg value="--xml" /> <arg value="${testResultsPath}/output/hibernate-coverageReport.xml" /> <arg value="--flatXML"/> </cmdargs> </buckminster>
The flatXML argument is used to convert the output from the tests to a result which can be read by Hudson.
Defining dependencies: buckminster.cspex
Buckminster uses the explicit plugin dependencies in the MANIFEST.MF to download related project and binary dependencies. Buckminster can however not directly/automatically resolve dependencies defined by the import-packages directive. This is logical as Buckminster can not identify the plugins to download, just on the basis of the package name. To solve this, it is possible to define additional dependencies using a so-called buckminster.cspex file. The Teneo build has a buckminster.cspex file in two feature projects:
- org.eclipse.emf.teneo.site-feature
- org.eclipse.emf.teneo.site-feature
These two feature projects are used in the rootRequest of the cquery files.
Note, the file containing the additional dependencies must be called buckminster.cspex.
The content of a cspex file:
<cspecExtension xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:bc="http://www.eclipse.org/buckminster/Common-1.0" xmlns="http://www.eclipse.org/buckminster/CSpec-1.0" componentType="eclipse.feature" version="1.0.0"> <dependencies> <dependency name="org.eclipse.equinox.executable" componentType="eclipse.feature"/> <dependency name="org.eclipse.platform" componentType="eclipse.feature"/> <dependency name="org.hibernate" componentType="eclipse.feature"/> <dependency name="javax.persistence" componentType="eclipse.feature"/> <dependency name="org.dom4j" componentType="eclipse.feature"/> <dependency name="org.apache.commons.logging" componentType="eclipse.feature"/> </dependencies> </cspecExtension>
This cspex file defines dependencies on several eclipse features. The rmap file define where to find these features (i.e. which update site for example).
Testing
Testcases are run through Buckminster using a launch configuration (so Buckminster calls a launch configuration). Teneo has 2 launch configuration one to run tests for Hibernate and one for EclipseLink. The launch configuration is easy to create, after running your main test suite in Eclipse, a run configuration should be present in the window Run > Run Configurations.
To store the run/launch configuration in a file: in the last tab of the configuration (Common), the save as location is set to a shared file in a project (org.eclipse.emf.teneo.hibernate.test in this case). The file is checked-in into cvs. This shared file is then used as the input to the test run through buckminster:
<echo message="Running tests Hibernate" /> <buckminster command="junit" workspace="${workspacePath}"> <cmdargs> <arg value="-l" /> <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> <arg value="-o" /> <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> <arg value="--stdout" /> <arg value="${testResultsPath}/output/stdout.txt" /> <arg value="--stderr" /> <arg value="${testResultsPath}/output/stderr.txt" /> <arg value="--flatXML"/> </cmdargs> </buckminster>
The above command will tell Buckminster to run the launch configuration AllTests.launch located in the org.eclipse.emf.teneo.hibernate.test project (which was downloaded into the workspace). The output location is specified with the -o parameter. The xml in the output location is used by Hudson to create a test report.
Some links to the Teneo build workspace for the test results:
SDK Feature (Including a source feature)
The Teneo SDK feature contains both the runtime plugins as well as the source plugins. The source plugins are special because they not exist in cvs. The source plugins are created by Buckminster in the build process. The question encountered during setting up the Teneo build, how to setup a SDK feature.
The solution was actually quite simple. Teneo Hibernate has two feature projects:
- org.eclipse.emf.teneo.hibernate-feature: the feature.xml includes all the relevant core Teneo plugins
- org.eclipse.emf.teneo.hibernate.sdk-feature: its feature.xml includes the org.eclipse.emf.teneo.hibernate feature and the org.eclipse.emf.teneo.hibernate.source feature.
The org.eclipse.emf.teneo.hibernate.source feature does not exist (therefore the warning in the feature.xml above). It is created at build-time. To ensure that Buckminster won't try to resolve and download the org.eclipse.emf.teneo.hibernate.source feature (and fail because it can't find it) an advisory node is added to the cquery file:
<cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="teneo.rmap"> <cq:rootRequest name="org.eclipse.emf.teneo.site" componentType="eclipse.feature"/> <cq:advisorNode namePattern="org\.eclipse\.emf\.teneo.*\.source" skipComponent="true"/> </cq:componentQuery>
The above setup is enough to create a SDK feature which contains the runtime feature (and its plugins) and the source feature with the source plugins. The next step is to define the update site, this is explained next.
Defining and creating an update site
The Teneo build creates an update site. For an example see this location: update site.
The update site is defined through a feature project: the org.eclipse.emf.teneo.site-feature project. It has two important files:
- feature.xml: defines which features are part of the update site
- category.xml: defines how the features are categorized
The feature.xml contains 4 features:
<includes id="org.eclipse.emf.teneo.eclipselink" version="0.0.0"/> <includes id="org.eclipse.emf.teneo.eclipselink.sdk" version="0.0.0"/> <includes id="org.eclipse.emf.teneo.hibernate" version="0.0.0"/> <includes id="org.eclipse.emf.teneo.hibernate.sdk" version="0.0.0"/>
The category.xml creates categories for these features:
<site> <feature id="org.eclipse.emf.teneo.hibernate" version="0.0.0"> <category name="org.eclipse.emf.teneo.hibernate"/> </feature> <feature id="org.eclipse.emf.teneo.hibernate.sdk" version="0.0.0"> <category name="org.eclipse.emf.teneo.hibernate"/> </feature> <feature id="org.eclipse.emf.teneo.eclipselink" version="0.0.0"> <category name="org.eclipse.emf.teneo.eclipselink"/> </feature> <feature id="org.eclipse.emf.teneo.eclipselink.sdk" version="0.0.0"> <category name="org.eclipse.emf.teneo.eclipselink"/> </feature> <category-def name="org.eclipse.emf.teneo.hibernate" label="EMF Teneo Hibernate"/> <category-def name="org.eclipse.emf.teneo.eclipselink" label="EMF Teneo EclipseLink"/> </site>
To build the update site the build xml file contains the following snippet:
<echo message="Creating p2 site" /> <buckminster command="perform" workspace="${workspacePath}"> <cmdargs> <arg value="org.eclipse.emf.teneo.site:eclipse.feature#site.p2" /> </cmdargs> </buckminster>
Buckminster is called with the org.eclipse.emf.teneo.site feature (note feature id is used here, not the project name!), and the site.p2 call. As a default (is option controlled) the content of the passed in feature is used for the content of the site, so not the feature project (org.eclipse.emf.teneo.site) itself is not part of the update site.
Code Coverage
It is quite easy to enable code coverage analysis as part of the test run. This section explains how to set it up.
First make sure that the org.eclipse.buckminster.emma.headless.feature.feature.group feature is installed as part of the Buckminster install. See the install.buckminster target in Teneo's build.xml.
To enable code coverage analysis in Buckminster:
- use the 'emma' command instead of the 'junit' command when calling Buckminster
- pass an extra xml parameter with the output location of the code coverage analysis
<buckminster command="emma" workspace="${workspacePath}"> <cmdargs> <arg value="-l" /> <arg value="org.eclipse.emf.teneo.hibernate.test/AllTests.launch" /> <arg value="-o" /> <arg value="${testResultsPath}/output/test-results-hibernate.xml" /> <arg value="--stdout" /> <arg value="${testResultsPath}/output/stdout.txt" /> <arg value="--stderr" /> <arg value="${testResultsPath}/output/stderr.txt" /> <arg value="--xml" /> <arg value="${testResultsPath}/output/hibernate-coverageReport.xml" /> <arg value="--flatXML"/> </cmdargs> </buckminster>
Then in Hudson set the following configuration:
Note: the Emma XML report value is not a pattern, it must be the actual file name of the xml output, i.e. the filename specified in the Buckminster call in the xml parameter. As only one file can be specified it is not possible to perform code coverage analysis on multiple separate Buckminster calls.
It is also possible to collect multiple buckminster emma commands into one code coverage file. The emma commands need to be executed together in one script file passed to Buckminster. This is explained in more detail in this thread on the Buckminster newsgroup.
An important thing to note is that Emma does code coverage analysis on the development projects in the workspace. In case of Teneo this is actually less useful as the core plugins are not part of the workspace but of the target platform. This because with Teneo we want to test the actually binary core plugins and not the development projects. So to really make use of code coverage one should run the test cases with all the core plugins also as development projects in the workspace. This is what the CDO project does (see related links at the end of this wiki page).
Signing and pack200
Enabling signing and pack200 is quite straightforward with Buckminster. To enable this set the following properties:
- site.pack200 to true
- site.signing to true
- signing.type (Teneo has this set to eclipse.local)
- eclipse.staging.area (Teneo has this set to ${env.HUDSON_HOME}/jobs/${env.JOB_NAME}/signing)
The last two are set in the buckminster.properties file. The first two are set through Hudson and handled in the build.xml file like this:
<condition property="site.pack200" value="${env.SITE_PACK200}" else="false"> <isset property="env.SITE_PACK200" /> </condition> <condition property="site.signing" value="${env.SITE_SIGNING}" else="false"> <isset property="env.SITE_SIGNING" /> </condition>
Hudson Configuration
The Teneo Hudson configuration can be found here (might be not accessible, depending on authorizations). This section will touch the main parts of the configuration and show the resulting build start page.
The Hudson setup starts with the definition of a number of parameters. These parameters are visible as input fields when a build is triggered manually, the default values are used when a cvs change triggers a build.
The Teneo build also has parameters for signing and pack200. The parameters are available in ant as environment ant variables (e.g. env.SITE_PACK200). The BUILD_TARGET parameter is used to define which ant target to call.
The next configuration selects a specific jre:
The cvs location and polling frequency define two things:
- it tells Hudson to download everything from the cvs location to the workspace (see here for the workspace result)
- it tells Hudson which cvs locations to watch and how often to poll/check cvs for changes
Tell Hudson not to let your build hang forever:
The following part is important, it tells Hudson what to execute (note the path to the ant file is made visible when you press the advanced button). Note:
- the usage of the BUILD_TARGET parameter, this parameter is defined in the beginning of the Hudson configuration (see above)
- the extra property which is needed to solve an issue with the directory (it can't work with all jvm's), see this forum thread.
The next section defines where Hudson can find the results. The results are available as the last successfull build artifacts. The test result location corresponds to the output location of the test runs.
The last screenshot shows how the build start page looks like. Each of the parameters is visible as an input field. The shown values are the defaults for when the build is automatically triggered on a cvs change.
Troubleshooting
This section lists some issues and their solution (see also the links to newsgroup threads in a later section of this wiki page):
- Getting error: 'Unpack facility not configured' when buckminster is downloaded in the build step
The director (which downloads Buckminster) does not work fine with certain jre's, see this solution: http://www.eclipse.org/forums/index.php?t=msg&th=164621&start=0& if you download the Teneo build and try it out locally then you have to edit the following line in the build.xml file and set the jre/bin location specific for your case, like here:
<jvmarg value="-Dorg.eclipse.update.jarprocessor.pack200=/home/mtaal/myapps/java/jdk1.5.0_15/jre/bin" />
- Solving the warning: [java] WARN: CVS file org.eclipse.emf/org.eclipse.emf.cdo/hibernate/org.eclipse.emf.cdo.tests.hibernate/src/org/eclipse/emf/cdo/tests/hibernate/external_model1_4.persistence.xml either contains invalid line endings on the server (CR/LF instead of just LF) or is a binary file that is not marked as -kb.
- Solution: this can be repaired in Eclipse through the menu: File > Convert Line Delimiters to. Convert all line delimiters to Unix. This menu option can be executed for one or more development projects.
- Solving the warning: [java] Warning: file /opt/users/hudsonbuild/.hudson/jobs/emf-teneo-nightly/workspace/build/build/workspace/plugins/org.eclipse.emf.teneo.annotations/META-INF/MANIFEST.MF,line 9: The JRE container on the classpath is not a perfect match to the 'J2SE-1.5' execution environment
- Solution: do the following: 1) select an explicit JRE/JDK in Hudson (see Hudson configuration), 2) set an explicit jre in the workspace (see examples above in the teneo_build.xml and teneo_test.xml), 3) for plugin projects showing the warning, right-mouse click on the plugin project and then 'PDE Tools > Update Classpath'.
- Out of memory error:
- Solution: see this bugzilla for a solution https://bugs.eclipse.org/bugs/show_bug.cgi?id=306102
- Out of memory error when processing test results: this occurs at the end of the test run. The buckminster junit plugin converts the complete test output to an in-memory String before writing to disk.
- Solution: This can be solved by setting the output log level of testcases to a higher level (for example 'error' instead of 'debug').
Related Links
For the Teneo setup some scripts have been re-used from the Buckminster build itself. The Buckminster releng/build project can be found here in svn.
During the Teneo build setup several newsgroup posts and bugzillas were used to discuss issues and solutions:
- Qualifier replacement not happening
- How to make a build fail on a compile error
- Creating separate zip files with feature and plugins
- How to prevent: JRE container on the classpath is not a perfect match to the 'J2SE-1.5'
- What does promote do?
- mspec to target platform and workspace
- Request for Hudson jobs for EMFT Texo and EMF Teneo
- How to create a SDK feature
- Provide an option to flatten JUnit testReport XMLs
The CDO project also uses buckminster for its build. Some links/information:
- CDO Hudson job
- The CDO Buckminster releng project can be found here:
dev.eclipse.org, /cvsroot/modeling, org.eclipse.emf/org.eclipse.emf.cdo/releng and then the org.eclipse.emf.cdo.releng project
TODO
- Update the modeling update site and download pages with output from the build.
- Hudson configuration: currently the promote and clean.all ant targets will report a fail in Hudson because Hudson expects test results as part of the build output, obviously these two targets will not create test results. Possibly these two targets have to be moved to a separate Hudson job.