Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "MoDisco/Releng/How to Use"

(Manually)
 
(21 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The MoDisco project is built using [[Athena Common Build]], with this [http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng releng project].
+
The MoDisco project is built using [[Buckminster]], with this [http://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng.buckminster releng project]. <span style="color:#606060">(for old Athena build, see [[MoDisco/Releng/Athena/How_to_Use|this page]])</span>
  
 
For more explanation about how the build process works, see also [[MoDisco/Releng/How_it_Works|How it Works?]].
 
For more explanation about how the build process works, see also [[MoDisco/Releng/How_it_Works|How it Works?]].
  
 
== What is built? ==
 
== What is built? ==
In the build.properties file of the releng project, the "mainFeatureToBuildID" property defines the feature that must be built. This feature must include all other features that must be built.
+
<code>org.eclipse.gmt.modisco.all</code> is the feature that is built (defined in buckminster.cspec).
 +
This feature must include all other features that must be built.
  
The releng project contains map files, which specify how to get the plugins and features that are to be built. For each plugin or feature, the map defines its location on a version control system (CVS and SVN are supported), and the version that must be used. Nightlies ignore the version information, and always build from head, whereas other types of builds checkout the tag or branch specified in the map.
+
The releng project contains a Buckminster rmap, which specifies how to get the plugins and features that are to be built. For each plugin or feature, the map defines its location on a version control system (CVS, SVN), and the version that must be used.
  
 
== How to start a build? ==
 
== How to start a build? ==
Line 12: Line 13:
 
=== Automatically ===
 
=== Automatically ===
  
* Nightly builds are run every 6 hours everyday, at 02:10, 08:10, 14:10 and 20:10 [http://en.wikipedia.org/wiki/Eastern_Time_Zone EST], if the [http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco MoDisco SVN] changed since the last build.
+
* Nightly builds are run every 6 hours everyday, at 02:10, 08:10, 14:10 and 20:10 [http://en.wikipedia.org/wiki/Eastern_Time_Zone EST], if the [http://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco MoDisco SVN] changed since the last build.
  
* Integration builds are started every Tuesday at 06:36 EST (11:36 or 12:36 GMT), if the [http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng releng project] changed since the last integration build. That is, a new integration build will only be triggered when a map file changed.
+
* Integration builds are started manually.
  
 
=== Manually ===
 
=== Manually ===
  
 
* Only committers in the MoDisco project, and members of the Callisto-dev group can launch MoDisco build jobs from Hudson.
 
* Only committers in the MoDisco project, and members of the Callisto-dev group can launch MoDisco build jobs from Hudson.
* Nightly build : go to [https://build.eclipse.org/hudson/job/cbi-modisco-nightly/ https://build.eclipse.org/hudson/job/cbi-modisco-nightly/]
+
* Nightly build : go to [https://hudson.eclipse.org/hudson/job/modisco-nightly/ https://hudson.eclipse.org/hudson/job/modisco-nightly/]
* Integration build : go to [https://build.eclipse.org/hudson/job/cbi-modisco-integration/ https://build.eclipse.org/hudson/job/cbi-modisco-integration/]
+
* Integration build : go to [https://hudson.eclipse.org/hudson/job/modisco-integration/ https://hudson.eclipse.org/hudson/job/modisco-integration/]
  
 
Then:
 
Then:
Line 29: Line 30:
 
=== Automatically ===
 
=== Automatically ===
 
Successful '''N'''ightly and '''I'''ntegration builds are automatically published to download.eclipse.org. For example, a nightly build for version 0.8.0, created on October 6 2009 at 11:54 EST would be published to:
 
Successful '''N'''ightly and '''I'''ntegration builds are automatically published to download.eclipse.org. For example, a nightly build for version 0.8.0, created on October 6 2009 at 11:54 EST would be published to:
  http://download.eclipse.org/modeling/gmt/modisco/downloads/drops/0.8.0/N200910061154
+
  http://download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.8.0/N200910061154
These builds can then be seen and downloaded from [http://www.eclipse.org/gmt/modisco/downloads/ http://www.eclipse.org/gmt/modisco/downloads/], where additional information is available (test results, build log, build configuration, build dependencies).
+
These builds can then be seen and downloaded from [http://www.eclipse.org/MoDisco/downloads/ http://www.eclipse.org/MoDisco/downloads/], where additional information is available (test results, build log).
 +
 
 +
=== With a script ===
 +
Run <code>/opt/public/modeling/mdt/modisco/manualPromote.sh</code> and fill in the parameters.
  
 
=== Manually ===
 
=== Manually ===
 
'''S'''table, '''M'''aintenance and '''R'''elease builds are not automatically published. They should be first tested internally before publishing. For example, to publish the 0.8.0M2 milestone build:
 
'''S'''table, '''M'''aintenance and '''R'''elease builds are not automatically published. They should be first tested internally before publishing. For example, to publish the 0.8.0M2 milestone build:
* First, fetch the build archive to test:
+
* First, fetch the build archive to test (from Hudson, or using <code>wget</code> or <code>scp</code> for example).
scp -r <commiterid>@build.eclipse.org:/opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-integration/lastSuccessful/archive/build/* .
+
 
* Then, test the build locally
 
* Then, test the build locally
 
* Then, publish it:
 
* Then, publish it:
 
  ssh <commiterid>@build.eclipse.org
 
  ssh <commiterid>@build.eclipse.org
  cd /opt/users/hudsonbuild/.hudson/jobs/cbi-modisco-integration/lastSuccessful/archive/build/
+
  wget https://hudson.eclipse.org/hudson/job/modisco-integration/113/artifact/S201009291117.zip
  cp -r S200910061155 /home/data/httpd/download.eclipse.org/modeling/gmt/modisco/downloads/drops/0.8.0/
+
  unzip S201009291117.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.9.0/
* Finally, for a release, update the update site with the new build:
+
Add a new update site with the new build to the composite:
rm -rf /home/data/httpd/download.eclipse.org/modeling/gmt/modisco/updates/release/*
+
* unzip the content in a new folder with the release number:
  unzip S200910061155/MODISCO-Update-incubation-0.8.0M2.zip -d /home/data/httpd/download.eclipse.org/modeling/gmt/modisco/updates/release/
+
  unzip MODISCO-Update-0.8.0M2.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/updates/milestones/0.10/M3
* You can enable download stats on the repository by running [http://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng/serverConfiguration/addDownloadStats.sh addDownloadStats.sh]on the update site
+
* update <code>compositeContent.xml</code> and <code>compositeArtifacts.xml</code> to add a reference to your newly added update site
* Check that the new build appears on [http://www.eclipse.org/gmt/modisco/downloads/ http://www.eclipse.org/gmt/modisco/downloads/].
+
** set the value of p2.timestamp to the result of "<code>date +%s000</code>"
 +
* You can enable download stats on the repository by running [http://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng.buckminster/serverConfiguration/addDownloadStats.sh addDownloadStats.sh] on the update site
 +
* Check that the new build appears on [http://www.eclipse.org/MoDisco/downloads/ http://www.eclipse.org/MoDisco/downloads/].
 +
* Builds can be hidden from this page before a release by modifying <code>downloads-scripts.php</code> in <code>www/MoDisco</code> on <code>:pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse</code>
 
* Update the archive site with the new build:
 
* Update the archive site with the new build:
** copy the drop folder (eg. "S200910061155") into <code>/home/data/httpd/archive.eclipse.org/modisco/downloads/drops/x.y.z</code>
+
** copy the drop folder (eg. "S200910061155") into <code>/home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/drops/x.y.z</code>
** update <code>/home/data/httpd/archive.eclipse.org/modisco/downloads/index.html</code> with a link to the newly added update zip
+
** update <code>/home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/index.html</code> with a link to the newly added update zip
 +
* [[#Tagging|Tag the build]]
 +
* [https://hudson.eclipse.org/hudson/job/modisco-integration/wipeOutWorkspace Wipe out the Hudson workspace] to free up room on the server.
  
 
=== Simultaneous Release ===
 
=== Simultaneous Release ===
If the build must be part of the simultaneous release, there are a few more steps to follow:
+
If the build must be part of the simultaneous release, you must also:
* Publish the update site to <code>/home/data/httpd/download.eclipse.org/modeling/gmt/modisco/updates/staging</code>
+
* Use the [[Eclipse_b3/aggregator/manual|B3 Aggregator]] (or a text editor if the modification is trivial) to [[Indigo/Contributing_to_Indigo_Build|update the build model for the aggregator]] ([http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.indigo.build/?root=Callisto after checking it out from CVS]).
* Use the [http://www.eclipse.org/modeling/amalgam/downloads/?project= DSL Toolkit] to [[Helios/Contributing_to_Helios_Build|update the build model for the aggregator]] ([http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.helios.build/?root=Callisto after checking it out from CVS]).
+
* Open juno.b3aggr in the B3 Aggregator editor, right-click on the first root node and select '''Validate Aggregation'''. Check in the error log that there are no errors.
  
 
== Build parameters ==
 
== Build parameters ==
 
Hudson builds expect these parameters:
 
Hudson builds expect these parameters:
* PROJECTID : modeling.gmt.modisco : no reason to change this
 
* VERSION : the version being built, must also be set in the [http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng releng project]'s promote-N.properties and promote-I.properties.
 
 
* BUILDTYPE : the kind of build, represented by a code letter (see [http://www.eclipse.org/modeling/downloads/build-types.php this page]):
 
* BUILDTYPE : the kind of build, represented by a code letter (see [http://www.eclipse.org/modeling/downloads/build-types.php this page]):
 
** '''N''': Nightly
 
** '''N''': Nightly
Line 65: Line 71:
 
** '''S''': Stable (for Milestones and Release Candidate builds)
 
** '''S''': Stable (for Milestones and Release Candidate builds)
 
** '''R''': Release
 
** '''R''': Release
* EXTRAFLAGS : flags that are passed to the build start script (see [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.commonbuilder.releng/tools/scripts/start.sh?root=Technology_Project&view=markup start.sh]). Useful flags are:
+
* BUILD_ALIAS : the name of the release (0.8.0, 0.9.0M1, etc.); blank for nightly and integration builds
** '''-buildAlias''' : for release, maintenance and stable builds, use this option to give a more meaningful name to the build. For example, add <code>-buildAlias 0.8.0M2</code> to get "MODISCO-SDK-incubation-0.8.0M2.zip" instead of "MODISCO-SDK-incubation-S200910070943.zip".
+
* SIGN_UPDATE_SITE : whether to sign the update site (takes about an hour on the Eclipse build server); mandatory for releases
** '''-forceContextQualifier''' : force the ".qualifier" part of a plugin version number to a specific string.
+
* VERSION : the version being built. Dictates in which folder the build will be placed under the download "drops" folder.
** '''-fetchTag''' : Force a specific tag to be used when pulling sources from the VCS. For example, use <code>-fetchTag HEAD</code> to build from HEAD instead of from the versions specified in the releng project's map files ([https://bugs.eclipse.org/bugs/show_bug.cgi?id=292157 doesn't seem to work with SVN maps]).
+
* PROJRELENGROOT : The path on the SVN (trunk, branch or tag) from which to take the version of the releng project that must be used for this build.
** '''-localSourceCheckoutDir''' : get the sources from a local directory instead of from the VCS. The nightly build job script is configured to checkout sources from the head of the trunk, instead of using what's indicated in the maps. This option is then used to indicate the checked out sources directory.
+
  
 
== Building locally ==
 
== Building locally ==
  
First, checkout these projects into your workspace (you can import [http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng/psf/modiscoReleng.psf this team project set]):
+
TODO
* http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng
+
* :pserver:dev.eclipse.org:/cvsroot/technology,org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng,org.eclipse.dash.common.releng
+
* :pserver:dev.eclipse.org:/cvsroot/eclipse,org.eclipse.releng.basebuilder,org.eclipse.releng.basebuilder,r35x_v20090811
+
  
Then, setup the [https://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng/build.properties build.properties] file in the MoDisco releng project with correct information for your system:
+
== Tagging ==
* Set all the <code>JAVA*_HOME</code> properties to the location of your JDK 6 install (eg: <code>/usr/lib/jvm/java-1.6.0-openjdk</code> or <code>C:/Program Files/Java/jdk1.6.0_16</code>)
+
To tag a release, use this script: [https://dev.eclipse.org/svnroot/modeling/org.eclipse.mdt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng.buckminster/utils/tag.sh tag.sh].<br>
* Set the <code>downloadsDir</code> property to a writable directory (eg: <code>/tmp/build/downloads</code> or <code>C:/temp/build/downloads</code>)
+
Releases should have a tag like '''R0_7_1''', and milestones should have a tag like '''S0_9_0_M2'''.
* change the <code>dependencyURLs</code> with the Eclipse SDK zip specific to your platform
+
{{warning|Tag names|Since the tag name is used as a bundle qualifier by PDE Build, it should not contain periods. For example, use '''R0_7_1''' instead of <s>v0.7.1</s>}}
  
Additionally, you should change all the <code>dependencyURLs</code> to point to one of your local Eclipse mirrors, to avoid saturating the main Eclipse download server, and getting extremely slow downloads.
+
[[Category:MoDisco]]
 +
[[Category:Releng]]
  
Alternatively, you can download the dependencies manually and put them in your <code>downloadsDir</code>. The build will then use the files you provided instead of downloading them itself.
+
== Checking ==
 +
=== Check bundles ===
 +
Check that each bundle contains an about.html file:
 +
for f in $( ls *.jar ); do unzip -t $f | grep -q about.html || echo $f; done
 +
or for plug-ins in your workspace:
 +
find $workspaceRoot -name 'build.properties' | while read i; do grep -q about.html "$i" || echo "$i"; done
  
Finally, start the build by right-clicking the <code>build.xml</code> file in the releng project inside Eclipse and choosing '''Run as > Ant Build'''.
+
=== Check licenses ===
 +
find $workspaceRoot -type f -name 'license.html' | while read i; do md5 $i; done
  
=== Building on Windows ===
+
=== Check Manifest.MF ===
On Windows, the build is a little trickier, but it should work if you do this:
+
find $workspaceRoot -name 'MANIFEST.MF' | while read i; do grep -q Bundle-Vendor "$i" || echo "$i"; done
* Add the <code>workspaceDir</code> property to the <code>build.properties</code>, with the location of your Eclipse workspace (for example: <code>D:/workspaces/MoDisco</code>)
+
find $workspaceRoot -name 'feature.xml' | while read i; do grep -q provider-name "$i" || echo "$i"; done
* Install [http://www.cygwin.com/ Cygwin] into <code>C:\cygwin</code> (a directory without spaces is recommended)
+
* Prepend <code>C:\cygwin\bin</code> to the <code>%Path</code> variable (open '''System properties''', click on '''Environment Variables...''', and change the '''Path''' variable so that it reads something like:
+
C:\cygwin\bin;%SystemRoot%\system32;%SystemRoot%;...
+
* Build from local sources instead of fetching from maps (see [[#Building from local sources]])
+
* If the build complains it can't find some tool, then install it into Cygwin by re-running Cygwin's <code>setup.exe</code>
+
* Spaces in the path of the JDK cause problems. There are several possible solutions:
+
** Use the "8.1" form of the path: (eg: "c:/progra~1/Java/jdk1.6.0_16"). You can find it by "dir /x" (thanks to Nick Boldt for this tip!).
+
** (Re-)install your JDK in a path not containing spaces
+
** Make a directory junction like this (on Windows Vista or 7):
+
C:\>mklink /J Java "C:\Program Files\Java\jdk1.6.0_16"
+
Junction created for Java <<===>> C:\Program Files\Java\jdk1.6.0_16
+
And then set the <code>JAVA*_HOME</code> variables to <code>C:/Java</code> in the <code>build.properties</code>.
+
* If tests seem to cause problems, you can disable them by removing <code>buildTests</code> and <code>test</code> from <code>build.steps</code> (and launch tests manually).
+
===== Caveat =====
+
* pack200 is not yet supported on Windows (http://bugs.eclipse.org/268405)
+
* running headless JUnit tests is not yet supported on Windows either (http://bugs.eclipse.org/268420)
+
  
=== Building from local sources ===
+
=== Check simrel reports ===
You can set the <code>localSourceCheckoutDir</code> in the <code>build.properties</code> to a directory where you already have all the MoDisco sources. This directory should contain a "plugins/" and a "features/" directory.
+
for url in \
 +
    "http://build.eclipse.org/juno/simrel/reports/layoutCheck.txt" \
 +
    "http://build.eclipse.org/juno/simrel/reports/verifydiroutput/unsigned.txt" \
 +
    "http://build.eclipse.org/juno/simrel/reports/versionPatternCheck.txt" \
 +
    "http://build.eclipse.org/juno/simrel/reports/breedata.txt" \
 +
    "http://build.eclipse.org/juno/simrel/reports/pack200data.txt" \
 +
    "http://build.eclipse.org/juno/simrel/reports/nonUniqueVersions.txt"
 +
do
 +
  echo "checking $url"
 +
  curl -s -S "$url" | egrep 'modisco|facet'
 +
done
  
You can build it by doing this:
+
url="http://build.eclipse.org/juno/simrel/reports/licenseConsistency.html"
<pre>
+
echo "checking $url"
mkdir MoDiscoTree && cd MoDiscoTree
+
curl -s -S $url > licenseConsistency
svn export http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/features/trunk features
+
n=$(cat licenseConsistency | grep -n "Features with matching" | sed 's/:.*//')
svn export http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/plugins/trunk plugins
+
cat licenseConsistency | head -n $n | egrep 'modisco|facet'
svn export http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/tests/trunk/ plugins --force
+
svn export http://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/tests/trunk/
+
org.eclipse.gmt.modisco.tests.feature features/org.eclipse.gmt.modisco.tests.feature --force
+
</pre>
+
  
==== Tip ====
+
url="http://build.eclipse.org/juno/simrel/reports/featureNames.html"
To avoid having to do a manual <code>svn export</code> before each build, you can use links on Linux or directory junctions on Windows to link both "plugins" and "features" to the root of your workspace:
+
echo "checking $url"
 +
curl -s -S $url > featureNames
 +
n=$(cat featureNames | grep -n "Probably correct names" | sed 's/:.*//')
 +
cat featureNames | head -n $n | egrep 'modisco|facet'
  
===== on Linux =====
+
url="http://build.eclipse.org/juno/simrel/reports/bundleNames.html"
<pre>
+
echo "checking $url"
mkdir /home/work/MoDiscoTree
+
curl -s -S $url > bundleNames
ln -s /home/work/workspaces/MoDisco /home/work/MoDiscoTree/plugins
+
n=$(cat bundleNames | grep -n "Probably correct bundle name" | sed 's/:.*//')
ln -s /home/work/workspaces/MoDisco /home/work/MoDiscoTree/features
+
cat bundleNames | head -n $n | egrep 'modisco|facet'
</pre>
+
And then set <code>localSourceCheckoutDir=/home/work/MoDiscoTree</code> in the <code>build.properties</code>.
+
===== on Windows =====
+
<pre>
+
C:\Users\Travail>mkdir MoDiscoTree
+
C:\Users\Travail>cd MoDiscoTree
+
C:\Users\Travail\MoDiscoTree>mklink /J plugins C:\Users\Travail\workspaces\MoDiscoAll
+
Junction created for plugins <<===>> C:\Users\Travail\workspaces\MoDiscoAll
+
C:\Users\Travail\MoDiscoTree>mklink /J features C:\Users\Travail\workspaces\MoDiscoAll
+
Junction created for features <<===>> C:\Users\Travail\workspaces\MoDiscoAll
+
</pre>
+
And then set <code>localSourceCheckoutDir=C:/Users/Travail/MoDiscoTree</code> in the <code>build.properties</code>.
+
  
== Tagging ==
+
url="http://build.eclipse.org/juno/simrel/reports/providerNames.html"
To tag a release, use this script: [https://dev.eclipse.org/svnroot/modeling/org.eclipse.gmt.modisco/releng/trunk/org.eclipse.gmt.modisco.releng/utils/tag.sh tag.sh].<br>
+
echo "checking $url"
Releases should have a tag like '''R0_7_1''', and milestones should have a tag like '''S0_8_M5'''.
+
curl -s -S $url > providerNames
{{warning|Tag names|Since the tag name is used as a bundle qualifier by PDE Build, it should not contain periods. For example, use '''R0_7_1''' instead of <s>v0.7.1</s>}}
+
n=$(cat providerNames | grep -n "Probably using correct provider name" | sed 's/:.*//')
 +
cat providerNames | head -n $n | egrep 'modisco|facet'
  
[[Category:MoDisco]]
+
url="http://build.eclipse.org/juno/simrel/reports/copyrights.html"
[[Category:Releng]]
+
echo "checking $url"
[[Category:Athena Common Build Users]]
+
curl -s -S $url > copyrights
 +
n=$(cat copyrights | grep -n "Features with copyrights that are probably ok" | sed 's/:.*//')
 +
cat copyrights | head -n $n | egrep 'modisco|facet'
 +
 
 +
url="http://build.eclipse.org/juno/simrel/reports/esdata.txt"
 +
echo "checking $url"
 +
curl -s -S $url > esdata
 +
n=$(cat esdata | grep -n "Bundles without an Eclipse-SourceReference" | sed 's/:.*//')
 +
cat esdata | tail -n +$n | egrep 'modisco|facet'

Latest revision as of 11:25, 9 May 2012

The MoDisco project is built using Buckminster, with this releng project. (for old Athena build, see this page)

For more explanation about how the build process works, see also How it Works?.

What is built?

org.eclipse.gmt.modisco.all is the feature that is built (defined in buckminster.cspec). This feature must include all other features that must be built.

The releng project contains a Buckminster rmap, which specifies how to get the plugins and features that are to be built. For each plugin or feature, the map defines its location on a version control system (CVS, SVN), and the version that must be used.

How to start a build?

Automatically

  • Nightly builds are run every 6 hours everyday, at 02:10, 08:10, 14:10 and 20:10 EST, if the MoDisco SVN changed since the last build.
  • Integration builds are started manually.

Manually

Then:

  • In Hudson, click on Build Now, change the build parameters as needed (see #Build parameters), and click on Build.
  • You can then click on the job name in the Build History section in the left column, and then on Console Output, to follow build progress in real time.

How to publish a build?

Automatically

Successful Nightly and Integration builds are automatically published to download.eclipse.org. For example, a nightly build for version 0.8.0, created on October 6 2009 at 11:54 EST would be published to:

http://download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.8.0/N200910061154

These builds can then be seen and downloaded from http://www.eclipse.org/MoDisco/downloads/, where additional information is available (test results, build log).

With a script

Run /opt/public/modeling/mdt/modisco/manualPromote.sh and fill in the parameters.

Manually

Stable, Maintenance and Release builds are not automatically published. They should be first tested internally before publishing. For example, to publish the 0.8.0M2 milestone build:

  • First, fetch the build archive to test (from Hudson, or using wget or scp for example).
  • Then, test the build locally
  • Then, publish it:
ssh <commiterid>@build.eclipse.org
wget https://hudson.eclipse.org/hudson/job/modisco-integration/113/artifact/S201009291117.zip
unzip S201009291117.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/downloads/drops/0.9.0/

Add a new update site with the new build to the composite:

  • unzip the content in a new folder with the release number:
unzip MODISCO-Update-0.8.0M2.zip -d /home/data/httpd/download.eclipse.org/modeling/mdt/modisco/updates/milestones/0.10/M3
  • update compositeContent.xml and compositeArtifacts.xml to add a reference to your newly added update site
    • set the value of p2.timestamp to the result of "date +%s000"
  • You can enable download stats on the repository by running addDownloadStats.sh on the update site
  • Check that the new build appears on http://www.eclipse.org/MoDisco/downloads/.
  • Builds can be hidden from this page before a release by modifying downloads-scripts.php in www/MoDisco on :pserver:anonymous@dev.eclipse.org:/cvsroot/org.eclipse
  • Update the archive site with the new build:
    • copy the drop folder (eg. "S200910061155") into /home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/drops/x.y.z
    • update /home/data/httpd/archive.eclipse.org/modeling/mdt/modisco/downloads/index.html with a link to the newly added update zip
  • Tag the build
  • Wipe out the Hudson workspace to free up room on the server.

Simultaneous Release

If the build must be part of the simultaneous release, you must also:

Build parameters

Hudson builds expect these parameters:

  • BUILDTYPE : the kind of build, represented by a code letter (see this page):
    • N: Nightly
    • I: Integration
    • M: Maintenance (NOT milestone)
    • S: Stable (for Milestones and Release Candidate builds)
    • R: Release
  • BUILD_ALIAS : the name of the release (0.8.0, 0.9.0M1, etc.); blank for nightly and integration builds
  • SIGN_UPDATE_SITE : whether to sign the update site (takes about an hour on the Eclipse build server); mandatory for releases
  • VERSION : the version being built. Dictates in which folder the build will be placed under the download "drops" folder.
  • PROJRELENGROOT : The path on the SVN (trunk, branch or tag) from which to take the version of the releng project that must be used for this build.

Building locally

TODO

Tagging

To tag a release, use this script: tag.sh.
Releases should have a tag like R0_7_1, and milestones should have a tag like S0_9_0_M2.

Warning2.png
Tag names
Since the tag name is used as a bundle qualifier by PDE Build, it should not contain periods. For example, use R0_7_1 instead of v0.7.1

Checking

Check bundles

Check that each bundle contains an about.html file:

for f in $( ls *.jar ); do unzip -t $f | grep -q about.html || echo $f; done

or for plug-ins in your workspace:

find $workspaceRoot -name 'build.properties' | while read i; do grep -q about.html "$i" || echo "$i"; done

Check licenses

find $workspaceRoot -type f -name 'license.html' | while read i; do md5 $i; done

Check Manifest.MF

find $workspaceRoot -name 'MANIFEST.MF' | while read i; do grep -q Bundle-Vendor "$i" || echo "$i"; done
find $workspaceRoot -name 'feature.xml' | while read i; do grep -q provider-name "$i" || echo "$i"; done

Check simrel reports

for url in \
   "http://build.eclipse.org/juno/simrel/reports/layoutCheck.txt" \
   "http://build.eclipse.org/juno/simrel/reports/verifydiroutput/unsigned.txt" \
   "http://build.eclipse.org/juno/simrel/reports/versionPatternCheck.txt" \
   "http://build.eclipse.org/juno/simrel/reports/breedata.txt" \
   "http://build.eclipse.org/juno/simrel/reports/pack200data.txt" \
   "http://build.eclipse.org/juno/simrel/reports/nonUniqueVersions.txt"
do
  echo "checking $url"
  curl -s -S "$url" | egrep 'modisco|facet'
done
url="http://build.eclipse.org/juno/simrel/reports/licenseConsistency.html"
echo "checking $url"
curl -s -S $url > licenseConsistency
n=$(cat licenseConsistency | grep -n "Features with matching" | sed 's/:.*//')
cat licenseConsistency | head -n $n | egrep 'modisco|facet'
url="http://build.eclipse.org/juno/simrel/reports/featureNames.html"
echo "checking $url"
curl -s -S $url > featureNames
n=$(cat featureNames | grep -n "Probably correct names" | sed 's/:.*//')
cat featureNames | head -n $n | egrep 'modisco|facet'
url="http://build.eclipse.org/juno/simrel/reports/bundleNames.html"
echo "checking $url"
curl -s -S $url > bundleNames
n=$(cat bundleNames | grep -n "Probably correct bundle name" | sed 's/:.*//')
cat bundleNames | head -n $n | egrep 'modisco|facet'
url="http://build.eclipse.org/juno/simrel/reports/providerNames.html"
echo "checking $url"
curl -s -S $url > providerNames
n=$(cat providerNames | grep -n "Probably using correct provider name" | sed 's/:.*//')
cat providerNames | head -n $n | egrep 'modisco|facet'
url="http://build.eclipse.org/juno/simrel/reports/copyrights.html"
echo "checking $url"
curl -s -S $url > copyrights
n=$(cat copyrights | grep -n "Features with copyrights that are probably ok" | sed 's/:.*//')
cat copyrights | head -n $n | egrep 'modisco|facet'
url="http://build.eclipse.org/juno/simrel/reports/esdata.txt"
echo "checking $url"
curl -s -S $url > esdata
n=$(cat esdata | grep -n "Bundles without an Eclipse-SourceReference" | sed 's/:.*//')
cat esdata | tail -n +$n | egrep 'modisco|facet'

Back to the top