Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Modeling Project Releng
This document is directed to the Eclipse Modeling Framework Technologies (EMFT) [1] component owners and was created help them to set up and run their builds. It describes the existing procedures that all EMFT components are required to adopt, describing, for example, how the files should be organized, what the build files are, and the Integration and Milestone release process.
Note that many of the tips and tricks described herein apply to EMF, MDT, and M2T builds as well.
Throughout this document are references to releng (Release Engineering) files which need to be configured along the way. When a project is first starting, these steps are normally skipped until after all the plugin content is in CVS. Then, and only then, the Modeling Project Releng Module can be created for the new project, and builds can begin.
Overwhelmed yet?
Well, you're in luck. As of 2006-07-12, there is a new module template which can be used as a starting point. For more on creating and configuring a Modeling Project Releng Module, see Modeling Project Releng Module. Go there, start with that, then come back here for tips and tricks if necessary.
This is a live document! We will be enriching it as questions are raised.
Contents
- 1 Plugin and Feature Files
- 1.1 Directory Structure
- 1.2 General Recommendations
- 1.3 Branding Icon In About Eclipse Dialog
- 1.4 Branding Plugin <example>
- 1.5 Core Plugins & Features <example>
- 1.6 Documentation Plugin & Feature <example>
- 1.7 Tests Plugin & Feature
- 1.8 Examples Plugin & Feature
- 1.9 SDK Feature
- 1.10 Source Plugin & Feature <example>
- 2 Building Zips & Jars
Plugin and Feature Files
The contents of your features and plugins directories should mimic what is available in the EMF and UML2 projects. Although this section tries to summarize the important points, "learning by example" is the recommended approach.
Directory Structure
<example>
We will ask you to organize your files as we've done for EMF and UML2. Basically you will need to create the following directory structure for each subdirectory you own under the EMFT module:
OLD Style -- features intermixed | NEW Style -- features separated | ||||
Old EMFT project (/cvsroot/technology) emft/[subproject]/ plugins doc tests examples | - or - |
New EMFT project (/cvsroot/modeling/) org.eclipse.*/org.eclipse.*.[subproject]/ plugins doc tests examples |
Old EMFT project (/cvsroot/technology) emft/[subproject]/ plugins doc tests examples features | - or - |
New EMFT project (/cvsroot/modeling/) org.eclipse.*/org.eclipse.*.[subproject]/ plugins doc tests examples features |
|
|
The directories containing features and fragments must be suffixed by -feature and -fragment respectively.
If you want source plugins and features, you will have to create them.
General Recommendations
- Each feature and plugin directory should be also an Eclipse project, containing all the necessary files such as, for example, .project.
- The .project must only refer to builders available to the open-source community. Also, since Eclipse 3M2, it should not refer to other plugin projects. <example>
- We use CVS to backup the files, so...
- You can, and probably should, use the $Id$ CVS tag. For more details and other tags read the CVS documentation
- Add a .cvsignore file to keep unnecessary files (such as the output directory) out of CVS. <example>
- Don't add unnecessary files to your plugins and features. If you use a "non-Eclipse standard" file, please ensure that it has a purpose and that that purpose is clear.
- All plugins should provide the following files:
- The build.properties file is extremely important and must be accurate to allow PDE to build your plugins and features. Please review the EMF build.properties files for plugins with code, documentation and branding plugins, and features.
Features <example>
- The PDE build process is based on features, so all your plugins must be referenced by a feature.
- A feature directory can contain the definition of other features and plugins. As you can see in the example, we use this to define "derived" elements, like EMF's SDK feature, source feature and source plugin.
- A feature's build.properties file can specify that the source plugin and feature should be generated. In EMF, this is done in the SDK feature's build.properties.
Qualifiers (1.0.0.qualifier)
- To add qualifiers to features and plugins, you must have 4 pieces set up correctly.
- 1. Plugins <example>
- Append .qualifier as the 4th field of the plugin version in every MANIFEST.MF file.
- Append .qualifier as the 4th field of the plugin version in every MANIFEST.MF file.
- 2. Features <example>
- Append .qualifier as the 4th field of the feature version in every feature.xml file.
- Ensure that if you have comments before the <feature> tag, they do NOT include the word "feature". If they do, 1.0.0.qualifier will not be replaced by PDE with the build's timestamp/id. This is documented in bug 129868.
- Change the versions in all plugins included in feature.xml files to 0.0.0. PDE will replace 0.0.0 by the appropriate version during the build.
- 3. Doc
- Ensure that org.eclipse.[subproject].doc/build.xml uses ${pluginVersion}.${forceContextQualifier} instead of ${pluginVersion} or a hard-coded version like 1.0.0. You may need to add a new property. <example>
<property name="pluginVersion" value="1.0.0"/>
- Add these lines to the end of the gather.bin.parts target in org.eclipse.[subproject].doc/build.xml (line break added in path attribute). <example>
<eclipse.versionReplacer path="${destination.temp.folder}/org.eclipse.[subproject].doc_\ ${pluginVersion}.${forceContextQualifier}" version="${pluginVersion}.${forceContextQualifier}"/>
- 4. Releng
- Add the following lines to the create.label.properties target of releng/[subproject]/buildAll.xml. <example>
<property name="forceContextQualifier" value="v${timestamp}"/>
<echo file="${buildDirectory}/label.properties" append="true" > forceContextQualifier=${forceContextQualifier} </echo>
- OPTIONAL: If you would like your features' versions to be suffixed with a dash followed by a random string of letters and numbers, you must also add a line to your build.properties files. Why would you want this? It is apparently calculated from the CVS tags to ensure that the version for the features will change if the source changes in the build.
- 5. Releng
Tip: due to property file parser limitations in Eclipse, it is recommended that you always leave an empty line at the end of .properties (and .properties.template) files.
- For more on plugin versioning, see http://www.eclipse.org/equinox/documents/plugin-versioning.html.
Branding Icon In About Eclipse Dialog
If you want to use the Eclipse Modeling icon in the About Eclipse dialog, you must reference it correctly in ALL your about.ini files, eg:
- org.eclipse.xsd.doc/about.ini
- org.eclipse.xsd/about.ini
- org.eclipse.xsd.sdk-feature/sourceTemplatePlugin/about.ini
# Property "featureImage" contains path to feature image (32x32) featureImage=modeling32.png
Note that the latest Eclipse Modeling icon is modeling32.png, as decided by bug 154906. You can copy this new icon from here: modeling32.png
Branding Plugin <example>
- The "branding plugin" is the plugin that provides specific files to describe a feature - check the example to see what these files are.
- It is recommended that the branding plugin be given the same name and id as the feature it describes.
- The "branding plugin" may also contain source, so you don't need to create an additional plugin just to hold the branding files.
Core Plugins & Features <example>
Core Plugins
For plugins with code:
- The EMF artifacts and model specifications (XML schema, for example) should be located in a models directory in the root of the plugin. Don't forget to include the genmodel to allow users to regenerate your code.
- DO commit all generated code. This makes it easier to run your plugin from the workspace.
- The source files should be placed in "source folders" and each "source folder" should have a different "output folder".
- The plugins with code should be deployed as jarred plugins. In order to tip the PDE build to jar a plugin, you need to set the MANIFEST.MF's Bundle-ClassPath attribute to . and use . in the build.properties file. As an example, look at the ecore's manifest and build.properties mentally replacing ecore.jar by ..
Right now you are probably asking yourself why the ecore files are declaring a jar instead of using .. To make a long story short, this is necessary for EMF to bootstrap itself during development (required when you use EMF to develop EMF). When developing, we can easily zip the contents of the output folder into the appropriate jar and restart Eclipse. Our build "fixes" the files before the plugins are packaged. Btw, to jar your plugin, you will also need to add unpack="false" to the plugin reference in your feature. <example>
- Almost all files should have a copyright. This is what is used for the Java files in EMF (please check the other types of files in the example):
/** * <copyright> * * Copyright (c) 2005 IBM Corporation and others. * All rights reserved. This program and the accompanying materials * are made available under the terms of the Eclipse Public License v1.0 * which accompanies this distribution, and is available at * http://www.eclipse.org/legal/epl-v10.html * * Contributors: * IBM - Initial API and implementation * * </copyright> */
- The META-INF/MANIFEST.MF must end with an empty line. Also, don't forget to add the following line to the manifest to ensure the plugin classes are properly initialized:
Eclipse-LazyStart: true
- The .classpath files should only contain references to the JDK and PDE container and the definition of the source and output folders.
- In order to ensure that your plugin.properties files can be translated, you will need to add the following "directives" to them. All strings after a given directive are supposed to conform to its rules:
NLS_MESSAGEFORMAT_ALL | Each string is assumed to be processed by the MessageFormat class (single quote must be coded as 2 consecutive single quotes ''). |
NLS_MESSAGEFORMAT_NONE | All strings are assumed to NOT be processed by the MessageFormat class (single quote must be coded as 1 single quote '). |
NLS_MESSAGEFORMAT_VAR | Strings which contain replacement variables are processed by the MessageFormat class (single quote must be coded as 2 consecutive single quotes ''). Strings which do NOT contain replacement variables are NOT processed by the MessageFormat class (single quote must be coded as 1 single quote '). |
- Add a schema for each extension point you define in your plugin. <example>
Core Features
This is a stub. Contribute to this wiki by adding content here.
Documentation Plugin & Feature <example>
- While the PDE build is able to automatically create the Ant build script for the plugins with source code, you will need to hard code and maintain the script for the documentation plugins. The easiest way of creating the script (called build.xml) is:
- After adding all the files to the plugin directory and configuring the build.properties file, right click the plugin.xml file and select PDE Tools->Create Ant Build File.
- Open the build.xml file and delete the tasks in the gather.bin.parts; target.
- In the build.jars, use the zip ant tasks to zip up the content of the plugin - this is required since Eclipse expects the documentation files to be in an archive file.
- Open the build.properties editor and select the "Custom Build" check box at the top of the page
- Every time you start code a new release, you will need to manually update this build script. See the "Release Process" section for more details on new releases.
Javadoc <example>
- Creating javadoc in your plugin requires a number of files & folders:
- build.xml (including all source), or
build.xml (excluding internal packages & classes) - plugin.xml
- toc.xml (table of contents)
- build.properties
- build/antJavadoc.sh
- build/javadoc.xml.template
- build/overview.html
- images/ (folder, ref'd in build.xml#build.jars)
- references/ (folder, ref'd in build.xml#build.jars)
- tutorials/ (folder, ref'd in build.xml#build.jars)
- build.xml (including all source), or
- Note that unless you have images, references, or tutorials at the time you're creating these files, you'll have to toss a placeholder into them so they won't be pruned (ignored) when extracting from CVS. A simple (blank) .cvsignore file will do nicely.
Javadoc Indexed By Eclipse Help
To get Eclipse to index your javadoc in the Help system, you must revise the following files (see also bug 142558):
1. Specify where extra topics_*.xml files are attached to the main table of contents entry: toc.xml
<topic label="Reference"> <link toc="topics_Reference.xml"/> </topic>
2. Specify which files to include in the doc plugin (custom build script, not PDE-generated): build.xml
<target name="gather.bin.parts" depends="init" if="destination.temp.folder"> ... <fileset dir="${build.result.folder}" includes="about.*,eclipse*.gif,eclipse*.png,eclipse*.jpg,plugin.*,toc*.xml,topics_*.xml,doc.zip,index/**,META-INF/**"/> ... </target>
3. UPDATED Configure javadoc.xml.template, if required. Doclet is no longer used; instead, generation is done simply in antJavadoc.sh
<target name="javadoc" depends="extractPlatformJavadoc"> ... <javadoc ...> <arg value="-J-Xmx256m"/> ... </javadoc> ...
4. Define extension points to allow extra topics_*.xml files to be seen in Eclipse Help: plugin.xml
<!-- ============================= --> <!-- Define TOCs --> <!-- ============================= --> <extension point="org.eclipse.help.toc"> <toc file="topics_Reference.xml" /> </extension> <!-- ============================= --> <!-- Define Javadoc locations --> <!-- ============================= --> <extension point="org.eclipse.pde.core.javadoc"> <javadoc path="references/javadoc"> <!-- defaults to reference/api --> </javadoc> </extension>
Tests Plugin & Feature
This is a stub. Contribute to this wiki by adding content here.
Examples Plugin & Feature
This is a stub. Contribute to this wiki by adding content here.
SDK Feature
This is a stub. Contribute to this wiki by adding content here.
Source Plugin & Feature <example>
- To create a source plugin and feature, you must have 5 pieces set up correctly.
- 1. org.eclipse.[subproject]-feature/feature.xml <example>
- This will define which *src.zip files will be created - one per plugin include in the feature. If you want source for your examples, do the same in your [subproject].examples-feature.
- [update] NOTE: you must include your branding plugin (eg., org.eclipse.net4j) in your branding feature (eg., org.eclipse.net4j-feature/feature.xml), even if it contains NO source, in order to properly suppress unpacking src/* in the .source plugin. Ensure that you've set that plugin to be unpack="false":
<plugin id="org.eclipse.net4j" download-size="0" install-size="0" version="0.0.0" unpack="false"/>
- 2. org.eclipse.[subproject]-feature/sourceTemplateFeature/* <example>
- 3. org.eclipse.[subproject]-feature/sourceTemplatePlugin/* <example>
- These define the contents that will go into your source feature(s) and plugin(s).
- 4. org.eclipse.[subproject]-feature/org.eclipse.[subproject].sdk/feature.xml <example>
<includes id="org.eclipse.[subproject].source" version="1.0.0" match="compatible"/>
- This connects your SDK to your .source plugins/features so that they will be included in the SDK. For more on match rules, see Matching Rules.
- 5. org.eclipse.[subproject].*/build.properties (for ALL applicable plugins) <example>
Bundle-ClassPath With Dot
RECOMMENDED: jarred plugins w/ org/eclipse/.../*.class files
Bundle-ClassPath With Jar
NOT RECOMMENDED: jarred plugins with nested jars
# NLS_MESSAGEFORMAT_VAR source.. = src/ output.. = bin/
src.includes = about.html bin.includes = plugin.xml,\ plugin.properties,\ icons/,\ .,\ META-INF/
Use . instead of a jar name; repeat in MANIFEST.MF files:
Bundle-ClassPath: .
# NLS_MESSAGEFORMAT_VAR source.[Bundle-ClassPath] = src/ output.[Bundle-ClassPath] = bin/
src.includes = about.html bin.includes = plugin.xml,\ plugin.properties,\ icons/,\ [Bundle-ClassPath],\ META-INF/
Note that [Bundle-ClassPath] is the jar name defined in the matching MANIFEST.MF, such as:
Bundle-ClassPath: eodmeditor.jar
These define what gets packaged in the *src.zip files. You should NOT use src.includes = src/ unless you want your source plugin to include BOTH a *src.zip (containing *.java source files) and the unpacked *.java source files themselves.
Multiple Namespaces
- If you want to contribute sources to your SDK, you must create a source plugin and feature for every feature that will contribute source. If you only have one main feature, eg, org.eclipse.emf.[subproject]-feature, there will be one src.zip created for every plugin listed in that feature's feature.xml. If you have more than one, as in the case with org.eclipse.emf.transaction, containing both org.eclipse.emf.transaction-feature and org.eclipse.emf.workspace-feature, each must contain its own sourceTemplateFeature and sourceTemplatePlugin. (???)
- You can also use the generate.feature directive in your SDK's build.properties, if you have source for more than one namespace. (???)
Building Zips & Jars
- We use the Eclipse build process which relies on PDE. See this article if you are interested in the details. It also covers the JUnit automated tests that can be run during the build.
- You can use PDE to configure what will appear in your zips.
Incubation Status
In order to comply with Eclipse.org's Incubation Rules, EMFT components must adhere to these 3 rules.
For example, here are the changes required to make EMF Compare compliant: Search CVS results
- 1. All downloadable zip files for builds and milestones must include the word
incubation
in the filename. For example,emft-compare-SDK-incubation-N200708031343.zip
. The jar files in the download zip file are NOT required (bug 178944) to contain the wordincubation
in the filename.
- To change your build to provide
-incubation
in the zip names, search your .releng project for ".zip
" and fix all files accordingly. Releng Example
- Building: customTargets.xml (all versions of this file)
- Testing: relengbuildgtk.sh, testManifest.xml, testManifest.xml.template, testing.properties, readme.html
- Promoting: promoteToEclipse.*.properties
- 2. All
Bundle-Name
s must include the worldincubation
. Note thatBundle-SymbolicName
s should not includeincubation
because the Bundle-SymbolicName is a technical namespace, not a user namespace. For example,Bundle-Name: Foo Plug-in (Incubation)
.
- Update all your component's MANIFEST.MF and/or plugin.properties files, including tests, examples, and doc (which file to edit depends on where "Bundle-Name" strings are located). MANIFEST.MF Example
Bundle-Name: %pluginName (Incubation) [2]
- 3. The names for update manager features must include the word
incubation
. For example,EMF Compare Documentation (Incubation)
.
- Update all your component's feature.xml and/or feature.properties files, including tests, examples, doc, and SDK features. Add
(Incubation)
to both the feature label and the feature description. Feature.* Example
<feature id="org.eclipse.emf.compare.sdk" label="EMF Compare SDK (Incubation)" version="0.7.0.qualifier" provider-name="Eclipse.org">
<description> EMF Compare SDK includes runtime, source, and documentation (Incubation) </description> [3]
When a project exits incubation, above changes should be reversed to remove the incubation
identifiers from features, plugins, and zip file names.
Customizing Zip Bundles
In the event that you have plugins or features that PDE will not normally bundle together, such as .ui plugins in the runtime zip, you will need to adjust the way PDE behaves in order to ensure all your plugins/features appear in your zips as you need them. If you need a custom zip, like EMF has for Models or Standalone, this is also how you can accomplish this.
There's two ways to customize what gets put into a zip bundle by PDE: the correct way (a) and the shortcut (b)
Option A
The first method involves creating a feature which sets up the included features/plugins that have to be in there, as with .sdk features in the EMFT subprojects (ocl, query, validation, transaction). See details in CVS. An example of this is org.eclipse.emf-feature/org.eclipse.emf.sdk. It is nested (rather than being its own org.eclipse.emf.[subproject].[bundlename]-feature) for cosmetic reasons (it looks in the file system).
When creating a new org.eclipse.emf.[subproject]-feature/org.eclipse.emf.[subproject].[bundlename] (or org.eclipse.emf.[subproject].[bundlename]-feature), you will need to ensure it's properly connected to the build harness in 3 ways:
- 1. First, you'll need a folder under your builder/ directory for the feature build, such as:
org.eclipse.emft/releng/validation/builder/sdk/
Into this folder must go a customTargets.xml and a build.properties file. Then make sure that customTargets.xml refers to the feature correctly, as in customTargets.xml. (Search for "sdk" on lines 9, 20, 187.)
- 2. Then, the buildAll.xml script must be told how to build the new zip, eg, buildAll.xml (line 139):
<target name="buildAll" depends="init"> <ant antfile="build.xml" target="main"> <property name="component" value="builder/sdk"/> </ant> ... </target>
- 3. And finally, you need to add the custom feature to the mapfile & mapfile template, since it won't be generated by PDE into the mapfile automagically. Examples: org.eclipse.emft/releng/validation/maps/validation.map
org.eclipse.emft/releng/validation/templateFiles/validation.map.template
Add an entry such as (line break added for clarity):
feature@org.eclipse.emf.validation.sdk=@cvsTag@,@cvsRoot1@,,org.eclipse.emft/validation/plugins/ org.eclipse.emf.validation-feature/org.eclipse.emf.validation.sdk
Option B (or adding an optional dependency)
The second method is faster (and, arguably, more hackish). This is also a valid method for adding an optional dependency, like for example adding optional support for OCL.
Instead of one feature per zip, as above, you can have custom instructions/rules in the buildAll.xml script which allow you to copy extra files that would normally be excluded from the zip to clean up missing content. These instructions are kept in one place (ie., only one ant script), so maintenance is easier, but this solution should only be used to add files to existing bundles, not to create new, custom ones.
For example, there's a validation.ocl plugin which must be included in the validation runtime, but since the validation-feature makes no mention of it, it's excluded. So, to work around this, once the SDK is built, copy the validaton.ocl plugin & feature from the SDK zip to the runtime zip after its assembly. This is done in the org.eclipse.emft/releng/validation/buildAll.xml script:
<target name="buildAll" depends="init"> <ant antfile="build.xml" target="main"> <property name="component" value="builder/sdk" /> </ant> <ant antfile="build.xml" target="main"> <property name="component" value="builder/runtime" /> </ant> ... <!-- add .ocl feature + plugin from SDK to runtime --> <zip update="true" destfile="${buildDirectory}/${buildLabel}/emft-${subprojectName}-runtime-${buildAlias}.zip"> <zipfileset src="${buildDirectory}/${buildLabel}/emft-${subprojectName}-SDK-${buildAlias}.zip"> <include name="**/org.eclipse.emf.${subprojectName}.ocl*"/> <include name="**/org.eclipse.emf.${subprojectName}.ocl*/*"/> </zipfileset> </zip> ... </target>
Bear in mind that you must ensure that everything you need to compile is available. You will need to edit all the customTargets.xml files that are used to build code which depends on the new reqiurement. See target "postSetup".
<ant target="getDependency" antfile="${getDeps.xml}"> <property name="url" value="${oclURL}"/> <property name="file" value="${oclFile}"/> <property name="isUnpackedFile" value="${buildDirectory}/plugins/org.eclipse.emf.ocl/plugin.xml"/> </ant>
If your tests require addtional plugins in order to compile or run, be sure to add them to your test.xml file. See target "setup".