View source for Modeling Project Releng/Building/Signing And Packing

'''''This document is a work in progress.'''''

== Requirements ==

1. You must have permission to ssh to build.eclipse.org. This is where the signing will occur. 

2. You must be able to ssh from your build server to build.eclipse.org without being prompted for a host key or password. The first time you try to connect from your build server, accept the host key and you should never be prompted again. If you can ssh from your build server to dev.eclipse.org or download1.eclipse.org, you should be able to connect to build.eclipse.org using the same credentials and ssh key.

3. While on build.eclipse.org, you must be able to create files in your staging folder, eg., '''/home/data/httpd/download-staging.priv/modeling/emft'''.

4. While on build.eclipse.org, you must be able to run '''/usr/bin/sign'''.

If you cannot do any of the above things, [https://bugs.eclipse.org/bugs/enter_bug.cgi?product=community&component=Servers open a bug] and ask for access: "''Please add user foo to group signers to permit jar signing for project bar''". cc: your PMC for approval and optionally, [[User:Nickb]].

== Process ==

=== All/Master Feature ===

Create an "all" or "master" feature, which will build SDK, examples, and any other features you might build. This obsoletes the need for an SDK builder, runtime builder, doc builder, and examples builder. You can still use your custom doc plugin builder; this just cleans up some redundancies in your .releng project. 

If your SDK feature already contains all your smaller features (including runtime, source, doc, and examples, but not tests) then you don't need a new "all" feature.

* Example 1: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet/plugins/org.eclipse.jet.all-feature?root=Modeling_Project JET All-In-One ("Master") Feature]
* Example 2: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/features/org.eclipse.gef.all-feature?root=Tools_Project GEF All-In-One ("Master") Feature]
* Example 3: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2/plugins/org.eclipse.uml2.sdk-feature?root=Modeling_Project UML2 SDK Feature] (already includes Examples)
* Example 4: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools/features/org.eclipse.emf.ecoretools.all-feature/?root=Modeling_Project Ecore Tools All-In-One ("Master") Feature]

=== Map File ===

Add this new feature to your map file.

* Example 1: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet.releng/maps/jet.map?root=Modeling_Project JET map]
* Example 2: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/org.eclipse.gef.releng/maps/gef.map?root=Tools_Project GEF map]
* Example 3: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/maps/uml2.map?root=Modeling_Project UML2 map] (no change)
* Example 4: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools.releng/maps/?root=Modeling_Project Ecore Tools map]

=== All/Master Builder ===

Add a new builder/all/ folder in your .releng project. Your old builder/sdk, builder/doc, builder/examples folders are no longer required and can be deleted. As in step 1, if you're reusing your SDK feature, this step is not required. Just update your customTargets.xml to create the Master Zip.

* Example 1: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet.releng/builder/all/?root=Modeling_Project JET's /builder/all/]
* Example 2: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/org.eclipse.gef.releng/builder/all/?root=Tools_Project GEF's /builder/all/]
* Example 3: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/builder/sdk/customTargets.xml?root=Modeling_Project&r1=1.5&r2=1.6 UML2's /builder/sdk/customTargets.xml]
* Example 4: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools.releng/builder/all/?root=Modeling_Project Ecore Tools' /builder/all/]

=== buildAll.xml ===

Change your .releng/buildAll.xml to use new [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/buildAllHelper.xml?root=Modeling_Project&view=markup signing/packing code]. You'll notice that many targets have been removed to buildAllHelper.xml to simplify this file, and some things have been reordered. The most important change is that you must now define your own packaging; however, by only doing a single PDE build for the whole "All" feature (and a second one for your tests), your build will run faster. This is important because jar signing can take a few minutes or as much as an hour, depending on the queue ahead of your build.

:<b style="color:red">NOTE!</b> This step may not be the most efficient way to collect all your build artifacts into one zip. Another approach is to continue to build multiple targets (SDK, runtime, examples, doc, tests) instead of just the single "all" build, then combine the results of those builds into a single zip. After signing, you would update your individual zips from the Master (signed) zip. 

* Example 1: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet.releng/buildAll.xml?root=Modeling_Project&view=markup JET's buildAll.xml] [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet.releng/buildAll.xml?root=Modeling_Project&r1=1.12&r2=1.26 changes]
* Example 2: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/org.eclipse.gef.releng/buildAll.xml?root=Tools_Project&view=markup GEF's buildAll.xml] [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/org.eclipse.gef.releng/buildAll.xml?root=Tools_Project&r1=1.8&r2=1.44 changes]
* Example 3: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/buildAll.xml?root=Modeling_Project&view=markup UML2's buildAll.xml] [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/buildAll.xml?root=Modeling_Project&r1=1.27&r2=1.33 changes]
* Example 4: [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools.releng/buildAll.xml?root=Modeling_Project&view=markup Ecore Tools' buildAll.xml] [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.emf/org.eclipse.emf.ecoretools.releng/buildAll.xml?root=Modeling_Project&r1=1.5&r2=1.16 changes]

:<b style="color:red">NOTE!</b> Make sure that '''all''' your files are properly appearing the the newly-constructed zips. As discovered in {{bug_comment|173651|10}}, you need to be sure that your source plugins' MANIFEST.MF files are properly included.

{{codeblock|<nowiki><include name="**/org.eclipse.gef*" />
<include name="**/org.eclipse.gef*/**" />
<include name="**/org.eclipse.gef*/**/**" /></nowiki>}}

:<b style="color:red">NOTE!</b> Make sure that you don't accidentally add the same file to a zip (eg., rootfiles like eclipse/epl-v10.html) more than once. To prevent this, add the following attribute to your <zip> tasks ({{bugstrike|221590}}):

{{codeblock|<nowiki><zip ... duplicate="preserve"></nowiki>}}

Should you want to disable signing, comment out the "sign" variable declaration:

{{codeblock|<nowiki><!-- <property name="sign" value="true" /> DISABLED FOR NOW --></nowiki>}}

Or, instead of a <code><property></code>, use a <code><condition></code> so that you can enable signing only for certain build types (eg., to make N builds run faster):

{{codeblock|<nowiki><condition property="sign">
  <or>
    <!-- want to enable/disable signing for a given build type? add/remove types here -->
    <equals arg1="${buildType}" arg2="I"/>
    <equals arg1="${buildType}" arg2="M"/>
    <equals arg1="${buildType}" arg2="S"/>
    <equals arg1="${buildType}" arg2="R"/>
  </or>
</condition></nowiki>}}

=== promoteToEclipse.*.properties ===

Use the Master zip as input to the [http://dev.eclipse.org/viewcvs/index.cgi/releng-common/tools/scripts/buildUpdateSite.sh?root=Modeling_Project&view=markup buildUpdateSite.sh] script, by updating your [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2t/org.eclipse.jet.releng/promoteToEclipse.jet.properties?root=Modeling_Project&view=markup promoteToEclipse.*.properties] [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/org.eclipse.gef.releng/promoteToEclipse.gef.properties?root=Tools_Project&view=markup file].

{{codeblock|<nowiki># zip to use for UM jar generation
filePrefixesToUnzipArray=( "$projectName-$subprojectName-Master-" );</nowiki>}}
:- or -
{{codeblock|<nowiki># zip to use for UM jar generation
filePrefixesToUnzipArray=( "$projectName-$subprojectName-Master-incubation-" );</nowiki>}}

=== UPDATE: Signed Sources ===

As discussed in {{bug|235734}}, the above process does not sign *everything* in the zips. So, to correct this, we need to reverse the flow so that we create update jars (features and plugins, all jarred) FIRST and zip contents (folders + jars) SECOND.

For the case of UML2, here are the two steps:

: 1. [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/builder/sdk/build.properties?root=Modeling_Project&r1=1.1&r2=1.2 *.releng/builder/all/build.properties]

: 2. [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.mdt/org.eclipse.uml2.releng/buildAll.xml?root=Modeling_Project&r1=1.34&r2=1.38 *.releng/buildAll.xml]

=== And finally... ===

Promote as before, using build/promo.php or promoteToEclipse.sh. Or, to test creating an update site without publishing a build, you can use buildUpdateSite.sh without the -promote flag to build an update site locally on the build server in /var/www/html/.

== Testing / Install Verification ==

When you install your signed features from the update site, you should see something like this:

[[Image:Gef-3.4-signed-features.png]]

== Bugs ==

* {{bug|173651}}: Support Pack200 in Modeling builds / update sites
* {{bug|217929}}: M2T Signing & Packing
* {{bugstrike|217945}}: Problems with file permissions in signed/packed jars
* {{bug|211613}}: JDK 6's Pack200 is not backward compatible with JDK 5 if packed jars contain no classfiles

== References ==

* [[Platform-releng-signedbuild|Platform Releng Signed Build]]
* [[JAR Signing]]
* [[Pack200]]
* [[JarProcessor Options]]


[[Category:Modeling]] [[Category:Releng]]