Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
OTBuilding
Revision as of 09:29, 14 May 2011 by Unnamed Poltroon (Talk)
The Object Teams Development Tooling is built in two main stages:
Contents
Building and Testing with PDE/Build
This is basically a plain vanilla PDE/Build setup with the following customizations:
- a top-level build.xml ant file initializes a myriad of properties and then
- prepare building:
- unzip a drop of the Eclipse SDK
- put
org.eclipse.team.svn.pde.build
into dropins (patched version to handletrunk
as expected, bug 301045)
- create the OT/J compiler by 1. invocation of PDE/Build
- build the OTDT and tests by 2. invocation of PDE/Build
- preparations before this stage
- unpack
eclipse-test-framework
to make it available while compiling tests - pre-load the output repository location from the previous build including category-less metadata
- unpack
- polish after this stage
- patch
content.xml
re version range in feature reference from patch feature (see also below, Publishing step 5)
- patch
- preparations before this stage
- delegate to the next level for testing
- prepare building:
- the main build.xml is a fairly thin layer for steps 2 & 3 above: it handles some properties and then delegates to one of two final PDE/Build stages:
- OT-Compiler: a PDE/Build with a generated
build.xml
and these customizations:- fetch ot-compiler.map from svn
- install the special
Bootstrap_MANIFEST.MF
in order to createorg.eclipse.jdt.core
in the exact same version number as we have it in our base install. - at end of build insert the resulting
org.eclipse.jdt.core
into the base install (simply replacing the existing plugin jar)
- OTDT: a second PDE/Build with a generated
build.xml
and these customizations:- at this stage, the jdt.core is our version, so we can actually compiler OT/J code
- fetch otdt.map from svn
- by setting
p2.gathering=true
in build.properties we trigger a p2-based build.
- OT-Compiler: a PDE/Build with a generated
- the final test.xml drives setup and execution of tests:
- create the software-under-test:
- unzip the base Eclipse SDK
- call p2 to install the OTDT from
testrun/otdtUpdatesDir
- call p2 to install the tests from
testrun/updateSiteTests/eclipse
- invoke all tests in sequence, has support for parallization (currently unused)
- create the software-under-test:
Unit test source and statistics
- Source for all unit tests are here
- 2 suites (plug-ins) of jdt tests are maintained with slight modifications
- 4 more suites of jdt tests are used unmodified
- 6 test suites have been developed specifically for Object Teams.
- A grand total of more than 50000 individual tests are used.
Publishing a p2 Repository
After building and testing were successful a shell script createRepository.sh performs these steps:
- Jar signing
- zip all jars into a big ball - skip symlinks which only point to artifacts from previous build
- send to the signing daemon, wait for the result to appear
- Populate a new repository
stagingRepo
- symlink artifacts from existing update site -- this ensures exact same bits, so unchanged plugins can be re-used during updating
- would we copy these files they were more difficult to distinguish/skip in subsequent phases
- physically copy bcel jar
- how can we ensure that clients in all update-scenarii will find bcel from original orbit repo?
- symlink artifacts from existing update site -- this ensures exact same bits, so unchanged plugins can be re-used during updating
- Pack200 all jar files
- apparently conditioning is already performed by PDE/Build for p2?
- Generate p2 metadata
- Patch
content.xml
as to widen the version range of references toorg.eclipse.jdt.core.feature.group
:- The patch feature
org.eclipse.objectteams.otdt.core.patch
must refer to an exact version of the feature which it patches, but even the same jdt bits will create different qualifiers in subsequent builds, so when we build against jdt RC3 it wouldn't be compatible with the final release even if jdt has no changes. - see bug 304156 - This is done using patch-content-xml.xsl
- The patch feature
- Archive category-less metadata as the basis for future builds
- cumulative repositories and categories don't play well together, because a category, once created cannot be augmented see bug 251888#c11
- Generate the category
- Add download statistics capability
- use XSL transformation addDownloadStats.xsl, which may not be a very safe way of doing this
- Jar-up metadata
- Remove symlinks which were needed to resolve dependencies on previous builds
Now the stagingRepo is ready to be copied over the previously published repository.
The script accepts these parameters:
createRepository.sh updateMasterRelativePath [ -nosign ] [ statsRepoId statsVersionId ]
- updateMasterRelativePath:
- relative path under
downloads/objectteams/updates
pointing to a repository from which to resolve unchanged bundles/features - specify
none
for an un-parented repository
- relative path under
- -nosign
- skip signing
- statsRepoId
- this identifies the repository in the download statistics, currently either
0.7
orunstable
- this identifies the repository in the download statistics, currently either
- statsVersionId
- this identifies the version in the downloading statistics, currently
0.7.1
- this identifies the version in the downloading statistics, currently
Updating to a new Eclipse version
Throughout the build process various version dependencies must be observed, thus moving to a new version of Eclipse to build against involves a number of adjustments:
- First our SVN branch/tag needs to be selected
- enter this path in
run.properties
(for bootstrapping our map files via variablemapVersionTag
) - if not building from trunk adjust map files, insert the branch using the
tag
property (this requires a further patched version of org.eclipse.svn.pde.build to avoid qualifier replacement using the branch name).
- enter this path in
- Next the exact version of a downloadable SDK build must be selected
- this version goes into file
otdt_prerequisites.sh
, variablesEVERSION
andDROP
- this version goes into file
- on
build.eclipse.org
I have a scriptbin/extractVersions
that extracts these numbers from the SDK tar ball:- versions of pde.build and equinox.launcher: insert in
run.properties
- version of the jdt feature:
- short format into
run.properties
(incl..next
: incremented by one for version range) - long format into
feature.xml
of otdt.core.patch
- short format into
- versions of the jdt.core plugin
- original version into
{EOT}.otdt.feature/ot-compiler-feature/feature.xml
andorg.eclipse.jdt.core/META-INF/Boostrap_MANIFEST.MF
- actual OT version in otdt.core.patch
- original version into
- versions of pde.build and equinox.launcher: insert in
- update test versions:
-
otdt.map
:-
org.eclipse.jdt.core.tests.builder
- org.eclipse.jdt.debug.tests
- org.eclipse.jdt.ui.tests.refactoring
- org.eclipse.jdt.ui.tests
- org.eclipse.jdt.text.tests
- org.eclipse.jface.text.tests
- org.eclipse.text.tests
- org.eclipse.core.filebuffers.tests
-
-
test.properties
:-
org.eclipse.jdt.core.tests.builder
- org.eclipse.jdt.core.tests.compiler
- org.eclipse.jdt.core.tests.model
- org.eclipse.test.performance
- org.eclipse.jdt.debug.tests
- org.eclipse.jdt.ui.tests
- org.eclipse.jdt.ui.tests.refactoring
-
-
- FIXME: avoid redundance, try to automate