https://wiki.eclipse.org/api.php?action=feedcontributions&user=Kmoir.ca.ibm.com&feedformat=atomEclipsepedia - User contributions [en]2024-03-29T01:09:51ZUser contributionsMediaWiki 1.26.4https://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=294868Platform-releng/Obsolete/Platform-releng-basics2012-03-21T21:08:15Z<p>Kmoir.ca.ibm.com: /* Rsync */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
== Test builds for older maintenance streams ==<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
kmoir's crontab on eclipsebuildserv (will be deleted March 23)<br />
<br />
<pre><br />
#rsync<br />
10,20,30,45,50 * * * * cd /home/users/releng/buildTools/scripts; ./ottAll fullmoon.ottawa.ibm.com >> /dev/null<br />
<br />
#nightly and integration builds<br />
0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
#0 20 * * 5 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild<br />
#0 20 * * 6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild<br />
<br />
#maintenance builds<br />
#0 8 * * 3 cd /builds; /home/users/releng/buildTools/eclipse37x/runMBuild<br />
<br />
#test builds<br />
#3.5.x<br />
#10 17 * * 1 cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest<br />
<br />
#milestone builds<br />
#0 8 * * 5 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 20 * * 0,2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 8,13,18 * * 1,3 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild-skipPerf<br />
#extra easter milestone builds<br />
#0 1 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 8,13,18 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild-skipPerf<br />
<br />
#test builds<br />
50 16 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runKimNBuildTest<br />
#4 16 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runKimNBuildTest42<br />
#0 4 * * 5 cd /builds; /home/users/releng/buildTools/eclipse37/runIBuildJava7<br />
#16 17 * * 1 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest<br />
#34 11 * * 2 cd /builds; /home/users/releng/buildTools/eclipse36/runKimIBuildTest<br />
#48 16 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/run36patchesTest<br />
#32 15 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest<br />
#55 10 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTestIES361<br />
#47 11 * * 4 cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest<br />
#12 11 * * 4 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7<br />
<br />
#clean test machines<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl<br />
<br />
#clean git tagging repos<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
#clean build machine of old builds<br />
<br />
#perf tests<br />
#0 18 * * 1 /home/users/releng/buildTools/eclipse.perf/perf35.sh<br />
#0 18 * * 5 /home/users/releng/buildTools/eclipse.perf/perf36.sh<br />
#0 18 * * 5 /home/users/releng/buildTools/eclipse.perf/perf37.sh<br />
<br />
##restart build<br />
#47 15 * * 3 cd /builds/perf37_201203071509/3.7_perf_37x/eclipsePerformanceBuildTools; sh command.txt<br />
#11 16 * * 3 cd /builds/N201203042000/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
<br />
#clean N-builds dir<br />
30 17 * * 3,4,6 rm -rf /builds/transfer/files/updates/3.7-N-builds/*<br />
30 17 * * 3,4,6 rm -rf /builds/transfer/files/updates/3.8-N-builds/*<br />
<br />
#34x build<br />
#26 10 * * 3 cd /home/users/releng/buildTools/eclipse34x; ./runKimTestMBuild<br />
<br />
30 6 * * * ps -eo uid,pid,etime,cmd | egrep '^ *507' | grep rsh | grep epwin | egrep '([1-9]+-)?([0-9]{2}:?){3}' | awk '{print $2}'| xargs kill -9 > /dev/null<br />
<br />
#21 9 * * 5 cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild<br />
<br />
<br />
#52 9 * * 4 cd /builds/I201203120800/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
<br />
# rebuild for PDE bug 2:00 Tuesday<br />
#0 14 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre><br />
<br />
kmoir's crontab on fullmoon<br />
<br />
<pre><br />
#rsync<br />
19,49 * * * * cd /home/users/kmoir/scripts; ./ottAll build.eclipse.org >> /dev/null<br />
<br />
12,50 * * * * rsync -a -e "ssh -l kmoir" --delete --stats /home/www/eclipse/equinox/ kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/equinox/ --exclude devicekit --exclude sat >> /dev/null<br />
<br />
15,45 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.1-I-builds /home/www/eclipse/updates<br />
15,45 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds /home/www/eclipse/updates<br />
20,50 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.1 /home/www/eclipse/updates<br />
31,1 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones /home/www/eclipse/updates<br />
<br />
#clean n-builds directory<br />
18 0 * * 3,6 rm -rf /home/www/eclipse/updates/3.7-N-builds/*<br />
18 0 * * 3,6 rm -rf /home/www/eclipse/updates/3.8-N-builds/*<br />
<br />
30 3 * * * /home/users/releng/buildTools/scripts/cleandrops.pl /home/www/eclipse/downloads/drops<br />
40 3 * * * /home/users/releng/buildTools/scripts/cleandrops.pl /home/www/eclipse/equinox/drops<br />
</pre><br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
Update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
<br />
In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
<br />
Update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
I just copy the milestone child repo /home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds to 4.2milestones and update the compositeContent.jar and compositeArtifacts.jar to point to the new repo. <br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
For weekly integration builds, the build just adds a new child repo automatically. Once a milestone, I go and clean out the integration build repos.<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.<br />
<br />
<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=294867Platform-releng/Obsolete/Platform-releng-basics2012-03-21T21:06:39Z<p>Kmoir.ca.ibm.com: /* Rsync */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
== Test builds for older maintenance streams ==<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
kmoir's crontab on eclipsebuildserv (will be deleted March 23)<br />
<br />
#rsync<br />
10,20,30,45,50 * * * * cd /home/users/releng/buildTools/scripts; ./ottAll fullmoon.ottawa.ibm.com >> /dev/null<br />
<br />
#nightly and integration builds<br />
0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
#0 20 * * 5 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild<br />
#0 20 * * 6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild<br />
<br />
#maintenance builds<br />
#0 8 * * 3 cd /builds; /home/users/releng/buildTools/eclipse37x/runMBuild<br />
<br />
#test builds<br />
#3.5.x<br />
#10 17 * * 1 cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest<br />
<br />
#milestone builds<br />
#0 8 * * 5 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 20 * * 0,2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 8,13,18 * * 1,3 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild-skipPerf<br />
#extra easter milestone builds<br />
#0 1 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
#0 8,13,18 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild-skipPerf<br />
<br />
#test builds<br />
50 16 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runKimNBuildTest<br />
#4 16 * * 4 cd /builds; /home/users/releng/buildTools/eclipse38/runKimNBuildTest42<br />
#0 4 * * 5 cd /builds; /home/users/releng/buildTools/eclipse37/runIBuildJava7<br />
#16 17 * * 1 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest<br />
#34 11 * * 2 cd /builds; /home/users/releng/buildTools/eclipse36/runKimIBuildTest<br />
#48 16 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/run36patchesTest<br />
#32 15 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest<br />
#55 10 * * 3 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTestIES361<br />
#47 11 * * 4 cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest<br />
#12 11 * * 4 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7<br />
<br />
#clean test machines<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl<br />
<br />
#clean git tagging repos<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
#clean build machine of old builds<br />
<br />
#perf tests<br />
#0 18 * * 1 /home/users/releng/buildTools/eclipse.perf/perf35.sh<br />
#0 18 * * 5 /home/users/releng/buildTools/eclipse.perf/perf36.sh<br />
#0 18 * * 5 /home/users/releng/buildTools/eclipse.perf/perf37.sh<br />
<br />
##restart build<br />
#47 15 * * 3 cd /builds/perf37_201203071509/3.7_perf_37x/eclipsePerformanceBuildTools; sh command.txt<br />
#11 16 * * 3 cd /builds/N201203042000/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
<br />
#clean N-builds dir<br />
30 17 * * 3,4,6 rm -rf /builds/transfer/files/updates/3.7-N-builds/*<br />
30 17 * * 3,4,6 rm -rf /builds/transfer/files/updates/3.8-N-builds/*<br />
<br />
#34x build<br />
#26 10 * * 3 cd /home/users/releng/buildTools/eclipse34x; ./runKimTestMBuild<br />
<br />
30 6 * * * ps -eo uid,pid,etime,cmd | egrep '^ *507' | grep rsh | grep epwin | egrep '([1-9]+-)?([0-9]{2}:?){3}' | awk '{print $2}'| xargs kill -9 > /dev/null<br />
<br />
#21 9 * * 5 cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild<br />
<br />
<br />
#52 9 * * 4 cd /builds/I201203120800/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
<br />
# rebuild for PDE bug 2:00 Tuesday<br />
#0 14 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
<br />
<br />
kmoir's crontab on fullmoon<br />
<br />
#rsync<br />
19,49 * * * * cd /home/users/kmoir/scripts; ./ottAll build.eclipse.org >> /dev/null<br />
<br />
12,50 * * * * rsync -a -e "ssh -l kmoir" --delete --stats /home/www/eclipse/equinox/ kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/equinox/ --exclude devicekit --exclude sat >> /dev/null<br />
<br />
15,45 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.1-I-builds /home/www/eclipse/updates<br />
15,45 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds /home/www/eclipse/updates<br />
20,50 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.1 /home/www/eclipse/updates<br />
31,1 * * * * rsync -ar -e "ssh -l kmoir" --delete kmoir@build.eclipse.org:/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones /home/www/eclipse/updates<br />
<br />
#clean n-builds directory<br />
18 0 * * 3,6 rm -rf /home/www/eclipse/updates/3.7-N-builds/*<br />
18 0 * * 3,6 rm -rf /home/www/eclipse/updates/3.8-N-builds/*<br />
<br />
30 3 * * * /home/users/releng/buildTools/scripts/cleandrops.pl /home/www/eclipse/downloads/drops<br />
40 3 * * * /home/users/releng/buildTools/scripts/cleandrops.pl /home/www/eclipse/equinox/drops<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
Update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
<br />
In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
<br />
Update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
I just copy the milestone child repo /home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds to 4.2milestones and update the compositeContent.jar and compositeArtifacts.jar to point to the new repo. <br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
For weekly integration builds, the build just adds a new child repo automatically. Once a milestone, I go and clean out the integration build repos.<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.<br />
<br />
<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=294795Platform-releng-faq-obsolete2012-03-21T15:55:30Z<p>Kmoir.ca.ibm.com: /* I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work? */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==Testing the 4.2 build on build.eclipse.org==<br />
<br />
Just documenting the current issues here (March 21, 2012)<br />
<br />
The builder to build the 4.2 primary build is the R4_2_primary branch of<br />
org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder<br />
<br />
The branch of the map files with the bundles to build are R4_HEAD. There are a<br />
few weeks out of date. This is just for testing. <br />
<br />
e4Build@build:/shared/eclipse/e4> pwd<br />
/shared/eclipse/e4<br />
e4Build@build:/shared/eclipse/e4> ls masterBuildKim.sh<br />
masterBuildKim.sh<br />
<br />
Is the script that invokes the build. It needs to be refactored a bit. Also,<br />
the bits that tag the repos have been commented out because I don't want to<br />
pollute the the 4.2 maps until we have the other issues worked out.<br />
<br />
In our regular 3.8 stream builds we have scripts in the bootstrap script that<br />
update the tags in the map files repo. The<br />
eclipse.platform.releng.maps\org.eclipse.releng\tagging\repositories.txt file<br />
lists the appropriate branches to look for each repo.<br />
<br />
These scripts update the maps with a timestamp in the tags field of the<br />
appropriate project. This is disabled for now for testing purposes.<br />
<br />
Anyways, the issues and notes<br />
1) The current 4.2 stream org.eclipse.platform.doc.isv bundle copies over some<br />
content from the 3.x stream of this same bundle. This needs to be deleted once<br />
we move to 4.2 build primary<br />
2) update.core has been added added to sdk.tests feature because some of the<br />
tests still depend on it. <br />
3) update.tests.core has been removed for org.eclipse.sdk.tests feature because<br />
these tests aren't relevant anymore<br />
4) The 4.2 versions sdk and platform features have parts commented in their<br />
build.properties that are commented out. These need to be uncommented and<br />
released when we move to the 4.2 build primary so the appropriate source<br />
bundles are produced. I've temporarily uncommented them and tagged them for<br />
the R4_HEAD version of org.eclipse.releng.<br />
5) The e4 build part doesn't run yet, I just build the e4 bundles needed for<br />
the platform.<br />
6) Need to add the 4.2 stream tests to the sdk.tests feature and run them,<br />
right now it just includes the 3.8 stream tests.<br />
7) The test results page generation also needs to be updated to include all the<br />
tests.<br />
8)I use a hudson token to invoke the tests via JSON. This needs to be stored<br />
somewhere secure. Right now, I just add it to the command.txt when hacking the<br />
build. <br />
9) The build isn't copied to the correct location, signed or mirrored into the<br />
correct repo because it's just a test build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-basics#Restarting_the_build<br />
<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==Scripts to promote a 4.x stream build==<br />
<br />
ssh to build.eclipse.org as pwebster<br />
<br />
./promote4x.sh ${buildId} ${milestonename}<br />
<br />
Example<br />
./promote4x.sh I20110916-1615 M2<br />
<br />
update the index.html to reflect the new build<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads/index.html<br />
<br>/home/data/httpd/download.eclipse.org/e4/downloads/index.html<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
# xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
# The derby database needs to be running on port 1528 (There is an init script for that)<br />
# X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Git_Workflows&diff=294711Platform-releng/Git Workflows2012-03-21T11:45:11Z<p>Kmoir.ca.ibm.com: /* Clone a repo */</p>
<hr />
<div>We'd like to capture some common CVS workflows used by the Eclipse Project and spell out the git/EGit equivalent. ''Please read this page even if you don't use EGit''. It contains important instructions on how to setup your repository.<br />
<br />
Please read some of [http://progit.org/book/ Pro Git] to get a feel for how git repositories work. Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
= Configure the workspace =<br />
Open the '''Team > Git > Configuration''' preference page and on the '''User Settings''' tab add the '''user.name''' and the '''user.email''' property. If you don't want to share your e-mail you can also use your committer account ID. Note that you will not be able to push changes to the the repository if the latter property is not matching with your records at the Eclipse Foundation.<br />
<br />
Set '''New text file line delimiter''' to '''Unix''' on the '''General > Workspace''' preference page.<br />
<br />
= Getting EGit =<br />
<br />
You can install EGit 1.1.0 from [http://eclipse.org/egit/download/]. <br />
<br />
If you would like to keep up with their current bug fixes, install EGit/JGit from their nightly build site [http://download.eclipse.org/egit/updates-nightly http://download.eclipse.org/egit/updates-nightly].<br />
<br />
= Clone a repo =<br />
<br />
The first step is to clone the one or more repos you need to work on. You want to clone the repo to a location ''outside'' your workspace. Then use the '''EGit Import Projects''' option to import the projects. <br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
#Switch to the '''Git Repository Exploring''' Perspective <br />
#Use '''Clone a Git repository''' [[Image:CloneGit.gif|Clone a git repository]] <br />
#you can paste in your connection URL and it should do the right thing. Some URLs (not all of them contain content right now in the testing phase). The repos that have been migrated are in bold text: <br />
##'''ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.bundles.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.binaries.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.framework.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.incubator.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/equinox/rt.equinox.p2.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.core.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.core.binaries.git''' <br> <br />
##'''ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.debug.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.build.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.incubator.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/pde/eclipse.pde.ui.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.debug.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git'''<br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git'''<br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.resources.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.runtime.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.team.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.text.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.ua.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.ui.git&nbsp;''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.swt.git&nbsp;''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.swt.binaries.git''' <br />
##'''ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.common.git<br>''' <br />
#''Next'' <br />
#Select all branches to clone <br />
#''Next'' <br />
#Confirm the location that it will clone the repository into. <br />
#an initial branch of '''master''' and a remote name of '''origin''' are standard. <br />
#''Finish'' - Now just sit back while git copies the entire repo to your harddrive&nbsp;:-)<br />
<br />
== Configuring the repo ==<br />
<br />
Unless you are working on topic branches, we work in a fairly linear history. Please set branch.'''branchname'''.rebase = true (see instructions below).<br />
<br />
Make sure that you set '''core.autocrlf=false''' and on Windows '''core.filemode=false'''. If you use EGit to clone the repository then this is done automatically for you.<br />
<br />
Once you've cloned a repository, you can go to the '''Preferences &gt; Team &gt; Git &gt; Configuration''' page. Select your repository, select the branch you picked when you cloned the repository and click '''New Entry...'''. Append "rebase" to the text in the 'Key' field and enter "true" as value. <br />
<br />
[[Image:RepositoryConfigurationSettings.png]] <br />
<br />
To automate the setting of "branch.'''branchname'''.rebase = true" if you use command line git, add "branch.autosetuprebase = always" to your global user settings. Unfortunately, this does not yet work properly in EGit, see [https://bugs.eclipse.org/345536 Bug 345536].<br />
<br />
== Importing the projects ==<br />
<br />
We need to get the projects from the repo into our workspace: <br />
<br />
#right click on your newly cloned repo and select '''Import Projects''' <br />
#you want '''Import existing projects''' from the ''Working Directory'' <br />
#''Next'' <br />
#Select the projects that you want to import from the repository <br />
#''Finish''<br />
<br />
Now you can start working. <br />
<br />
== A note on deleting projects ==<br />
<br />
Typically you will only want to have a subset of the projects from a given repository in your workspace. When you are no longer interested in a project, you can delete it from your workspace. However, '''NEVER select 'Delete project contents on disk'''' for a project in a git repository. If you do, Git will consider this an outgoing deletion to be committed to the remote branch. Later while working on a completely unrelated project you may accidentally commit this deletion (and you wouldn't be the first to do so).<br />
<br />
= Start working in HEAD =<br />
<br />
To start working in '''HEAD''' you must clone your repository and checkout a working copy. By default, cloning the repo checks out the '''master''' branch, which is the same as '''HEAD''' in CVS. <br />
<br />
See [[#Clone_a_repo]]. <br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
The constant '''HEAD''' is used in GIT as well, but has a completely different meaning. In GIT '''HEAD''' means the pointer to the latest commit in your currently checked-out branch (more or less). <br />
<br />
= Start working in a new branch =<br />
<br />
For example, create the R3_7_maintenance branch for your repo. This example is for the case that your branch doesn't already exist as '''refs/remotes/origin/R3_7_maintenance'''. <br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
#right click on one of your projects and choose '''Team&gt;Switch To&gt;New Branch''' <br />
#you need to pick a source ref. <br />
##'''HEAD''' == current checked out commit <br />
##'''refs/heads/master''' means your '''master''' branch. <br />
##'''refs/remotes/origin/R3_7_maintenance''' - the existing remote branch. If you pick this one and name your local branch the same, EGit will automatically create a ''tracked branch''. <br />
##'''refs/tags/R3_7''' is the tags to branch from <br />
#name the branch '''R3_7_maintenance''' <br />
#select the '''Rebase''' merge option <br />
#leave "Check out the new branch" selected.<br />
<br />
This will create a new branch for you to work on. Once you've made your initial commits, you need [[#Commit_changes_to_the_main_repo]]. Pushing up to the repo will push any new branches you've created as well. <br />
<br />
= Create a patch =<br />
<br />
We have 2 options for accepting contributions from the community. See [[Development Resources/Handling Git Contributions]] (prefered) and [[Git#IP_process_implications_of_DVCS]]. Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
To create a patch: <br />
<br />
#You need to show the commits in the history view. <br />
##Either right-click on a file you just committed and '''Show in history''' <br />
##or right-click on the project and '''Show In&gt;History''', then find the commit you want <br />
#right click on the commit in the ''History'' view, and select '''Create Patch''' <br />
#use ''Next'' if you'd like the patch in the standard git mbox format<br />
<br />
= Apply a patch =<br />
<br />
A normal workspace patch will apply in the same fashion it does for CVS. Simply copy the file to the clipboard and paste it into the ''Package Explorer'' or use '''Team&gt;Apply Patch'''. <br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
Patches created with git/EGit have a different pattern. They are diff statements that look like: <br />
<br />
diff --git a/bundles/org.eclipse.e4.ui.workbench.renderers.swt/src/org/eclipse/e4/ui/workbench/renderers/swt/HandledContributionItem.java b/bundles/org.eclipse.e4.ui.workbench.renderers.swt/src/org/eclipse/e4/ui/workbench/renderers/swt/HandledContributionItem.java<br />
index 99d339f..37bcf68 100644<br />
--- a/bundles/org.eclipse.e4.ui.workbench.renderers.swt/src/org/eclipse/e4/ui/workbench/renderers/swt/HandledContributionItem.java<br />
+++ b/bundles/org.eclipse.e4.ui.workbench.renderers.swt/src/org/eclipse/e4/ui/workbench/renderers/swt/HandledContributionItem.java<br />
<br />
To apply a patch like this, you should: <br />
<br />
#copy and paste the patch into the ''Package Explorer'' <br />
#Select '''Apply patch to the workspace root''' <br />
#''Next'' <br />
#under '''Patch Options''' set '''Ignore leading path name segments''' to ''2'' <br />
#now you can examine the patch and apply is normally.<br />
<br />
= Contributing to a build =<br />
<br />
Daily development occurs in the '''master''' branch, but weekly integration builds are done out of a branch called '''integration'''. The builder takes care of tagging the build inputs and updating bundle qualifiers for those bundles that have changed. To contribute a repository to the build, do the following:<br />
<br />
# Make sure you do a fetch on your repository<br />
# Ensure all desired changes for the build are in branch '''master'''<br />
# In the <b>Git Repositories</b> view, expand <b>Branches &gt; Local</b>, and select '''integration'''. Right click and select '''checkout'''.<br />
# Select branch '''master''' and select '''merge'''<br />
# Right-click on the repository and select '''Push to Upstream'''<br />
<br />
To contribute to a build from the git command line, first ensure your local '''master''' stream contains the contents you want to contribute to the build. Then, do the following:<br />
<br />
git pull<br />
git checkout integration<br />
git merge master<br />
git push origin integration<br />
<br />
This will cause a fast-forward merge, so that both '''master''' and '''integration''' refer to the same commit. Past commits that were involved in builds will be identifiable by the build tag on that commit.<br />
<br />
== Changing what branch is built ==<br />
<br />
The builder automatically takes a branch of each repository and contributes the latest branch contents to the build. The branch used for each repository is configured in the following file:<br />
<br />
http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/tree/org.eclipse.releng/tagging/repositories.txt<br />
<br />
To build a different branch, edit this file and enter the branch name to be built on the line next to your repository. Commit and push the change. Use branch '''master''' for current development stream builds, and Rx_y_maintenance for maintenance builds.<br />
<br />
== Creating an integration branch ==<br />
<br />
If your repository does not yet have a branch called '''integration''', it will need to be created and pushed. This is done as follows:<br />
<br />
# Open the '''Git Repositories''' view, and expand '''Branches &gt; Local'''.<br />
# Right-click on '''master''', and select '''Create Branch...'''<br />
# For the branch name, enter '''integration'''. Leave the pull strategy as "None".<br />
# Click Finish<br />
# Right-click on the repository, and select '''Push...'''<br />
# In the Push dialog, ensure the correct remote is selected and click '''Next'''<br />
# In the '''Source ref''' field, select '''integration''' in the drop-down list. The destination spec will be filled in automatically. Click '''Add Spec''' to include this specification in the push operation.<br />
# Click '''Finish''' to complete the push operation<br />
<br />
For command line users, do the following:<br />
<br />
git checkout master<br />
git branch integration<br />
git push origin integration<br />
<br />
= Manual tagging =<br />
<br />
Tagging for weekly integration builds is not needed, but sometimes you still need to tag. For example, tagging with a Rx_y tag when a release is done. To create a tag, use '''git tag mynewtag''' from the command line or the [[EGit/User_Guide#Tagging|EGit Tag Dialog]]. Once a tag has been added, you must push the tag to the repository.<br />
<br />
To push a specific tag, use '''git push origin mynewtag''' from the command line or use the following steps in EGit:<br />
<br />
# Team > Remote > Push...<br />
# Click Next to get to the Specifications Page<br />
# In the Source ref box, enter in '''refs/tags/mynewtag''' (content assist is available to quickly find a tag)<br />
# Click Add Spec<br />
# Click Next<br />
# Assuming the dry run is successful, click Finish<br />
<br />
You can push all tags using '''git push --tags''' from the command line or pressing the Add All Tags Spec on the Specifications Page in EGit. However, this is risky as you will push all local tags and you will replace any tags that have been deleted from the remote repository.<br />
<br />
The [[E4/Git|e4 Git page]] has some helpful scripts and additional information on tagging<br />
<br />
= Commit changes to the main repo =<br />
<br />
Committing a change to the main repo is a 2-step process in git. In git, a commit creates a commit with the changed files in your local clone repository. A push will put that commit in our main repo. Committing and pushing are distinct operations in git. <br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
To get your changes to the main repo in EGit: <br />
<br />
#Do a '''pull''' or a '''fetch''' and a '''merge''' into your working branch <br />
#right-click on your project and use '''Team&gt;Commit''' <br />
#Your commit message should include the bug number you are using for your fix/work. <br />
#check the files that should be included in the commit in the ''Files'' section <br />
#''Commit''<br />
<br />
Then you need to push your changes to make sure they're visible to everyone else <br />
<br />
#right-click on your project and use '''Team&gt;Push to Upstream''' <br />
#it should provide a status dialog with the refs that were updated, or a failure if the main repo has commits that you haven't either merged or rebased on. <br />
#if there's a failure, you need to [[#Update_to_pull_in_the_latest_changes_to_HEAD]] or the relevant branch.<br />
<br />
<br> Common commit message: <br />
<br />
Bug 349177 - [releng] stitch ui.workbench fork back into main<br />
Updating some code to reflect the real change<br />
<br />
The [[EGit/User_Guide#Git_Staging_View|eGit Staging View]] provides a way to see all changed code in your workspace. You can select a subset of the changed files to commit.<br />
<br />
= Update to pull in the latest changes to HEAD =<br />
<br />
To make changes visible from our main repo is a 2 step process in git: <br />
<br />
#'''fetch''' which updates your cloned repo with all of the objects and remote branches from the main repo. <br />
#'''merge''' which updates your local branch to point to the correct commit <br />
##the most common case is called the "fast forward merge". That's where your repo has no local changes, and git can simply update your local '''master''' to the commit pointed to by '''origin/master''' <br />
##if you have a local commit that has not yet been pushed, you might have to deal with merge conflicts: <br />
###you will either have to merge the new '''origin/master''' into your '''master''' (which will lead to a merge commit with 2 parent commits) <br />
###or rebase your local commit onto that last commit coming from '''origin/master'''. This leaves the history of the main repo as a simple line, and is prefered ... I think.<br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
Git can do both '''fetch''' and '''merge''' for the current branch at once: <br />
<br />
#right click on your project and select '''Team&gt;Pull'''<br />
<br />
= Merge conflicting changes =<br />
<br />
Refer to the [[EGit/User Guide]] for more detailed instructions and pictures. <br />
<br />
If you '''pull''' changes for your branch and there are conflicts, you must merge them before you can commit your changes. <br />
<br />
By default EGit merges conflicts into the local files using the old merge syntax [http://www.kernel.org/pub/software/scm/git/docs/git-merge.html#_how_conflicts_are_presented Text Merge Markers]. To switch to a compare editor merge, use the '''Team&gt;Merge Tool''' menu item. See [[EGit/User Guide#Resolving_a_merge_conflict]]<br />
<br />
= Dealing with line terminators =<br />
<br />
Git has various options for normalizing line terminators between Windows (which uses CR/LF), and other platforms that use only LF. After trying various approaches, the Eclipse project has settled on the following approach:<br />
<br />
* Set core.autocrlf=false. If you use EGit to clone the repository then this is done automatically for you.<br />
* In your workspaces set General > Workspace > 'New text file line delimiter' to 'Unix'.<br />
* In the current dev branches (do not do this in maintenance branches) set 'New text file line delimiter' to 'Unix' on the 'Resource' property page of your projects.<br />
* If you don't work with EGit then it is your responsibility to avoid committing wrong line delimiters to the repository.<br />
* If using Windows, set core.fileMode=false. Again, if you use EGit to clone the repository then this is done automatically for you.<br />
<br />
References:<br />
* http://dev.eclipse.org/mhonarc/lists/eclipse-dev/msg09216.html<br />
* http://dev.eclipse.org/mhonarc/lists/eclipse-dev/msg09224.html<br />
<br />
= Cross-referencing between bugzilla and git =<br />
<br />
Some committers like to attach patches to bugzilla for all changes, so they have a record of exactly what changes occurred to fix the bug. With git there is a better way: <br />
<br />
*Push the change to git.eclipse.org <br />
*Navigate to git.eclipse.org in your web browser <br />
*Find your repository, and click on the "Log" tab <br />
*Click on your commit, which should be near the top of the log at this point <br />
*You are now on a page that shows a graphical diff of call changes that occurred in that commit. Copy the URL of this commit page into a bugzilla comment.<br />
<br />
Now if someone in the future wants to see what changes were made to fix the bug, they have a one click link to see all the changes. <br />
<br />
<br> <br />
<br />
= Pruning corrupt repository branches =<br />
<br />
Sometimes a branch in your repository might get corrupted and you may see errors such as "Could not read an object while parsing commit ...." in Egit or "Branch .... does not point to a valid object". The following command comes handy in such scenarios:<br />
git remote prune origin<br />
<br />
(Always prefix a --dry-run to see what this command is going to do before you go ahead and execute it.)<br />
<br />
<br> <br />
<br />
= References =<br />
<br />
For background reading on Git, see [[Git#Resources]].<br />
<br />
[[Category:Git]][[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=294308Platform-releng-faq-obsolete2012-03-19T17:53:06Z<p>Kmoir.ca.ibm.com: /* Scripts to promote a 4.x stream build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==Scripts to promote a 4.x stream build==<br />
<br />
ssh to build.eclipse.org as pwebster<br />
<br />
./promote4x.sh ${buildId} ${milestonename}<br />
<br />
Example<br />
./promote4x.sh I20110916-1615 M2<br />
<br />
update the index.html to reflect the new build<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads/index.html<br />
<br>/home/data/httpd/download.eclipse.org/e4/downloads/index.html<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
# xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
# The derby database needs to be running on port 1528 (There is an init script for that)<br />
# X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=294307Platform-releng-faq-obsolete2012-03-19T17:52:49Z<p>Kmoir.ca.ibm.com: /* Scripts to promote a 4.x stream build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==Scripts to promote a 4.x stream build==<br />
<br />
ssh to build.eclipse.org as pwebster<br />
<br />
./promote4x.sh ${buildId} ${milestonename}<br />
<br />
Example<br />
./promote4x.sh I20110916-1615 M2<br />
<br />
update the index.html to reflect the new build<br />
<br />
<br>/home/data/httpd/download.eclipse.org/eclipse/downloads/index.html<br />
<br>/home/data/httpd/download.eclipse.org/e4/downloads/index.html<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
# xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
# The derby database needs to be running on port 1528 (There is an init script for that)<br />
# X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=294306Platform-releng-faq-obsolete2012-03-19T17:52:33Z<p>Kmoir.ca.ibm.com: /* Scripts to promote a 3.x stream build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==Scripts to promote a 4.x stream build==<br />
<br />
ssh to build.eclipse.org as pwebster<br />
<br />
./promote4x.sh ${buildId} ${milestonename}<br />
<br />
Example<br />
./promote4x.sh I20110916-1615 M2<br />
<br />
update the index.html to reflect the new build<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads/index.html<br />
/home/data/httpd/download.eclipse.org/e4/downloads/index.html<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
# xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
# The derby database needs to be running on port 1528 (There is an init script for that)<br />
# X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=294086Platform-releng/Obsolete/Platform-releng-basics2012-03-16T15:16:30Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
== Test builds for older maintenance streams ==<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
Update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
<br />
In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
<br />
Update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
I just copy the milestone child repo /home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds to 4.2milestones and update the compositeContent.jar and compositeArtifacts.jar to point to the new repo. <br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
For weekly integration builds, the build just adds a new child repo automatically. Once a milestone, I go and clean out the integration build repos.<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.<br />
<br />
<br />
[[Category:Eclipse_Platform_Releng| ]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293977Platform-releng-faq-obsolete2012-03-15T21:50:14Z<p>Kmoir.ca.ibm.com: /* How performance tests are invoked in the build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
# xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
# The derby database needs to be running on port 1528 (There is an init script for that)<br />
# X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293976Platform-releng-faq-obsolete2012-03-15T21:49:37Z<p>Kmoir.ca.ibm.com: /* How performance tests are invoked in the build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
When the performances tests complete, the results are generated. <br />
<br />
<pre><br />
<target name="generatePerformanceResults"><br />
<mkdir dir="${buildDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${postingDirectory}/${buildLabel}/performance" /><br />
<taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /><br />
<condition property="configArgs" value="-ws gtk -arch ppc"><br />
<equals arg1="${os.arch}" arg2="ppc64" /><br />
</condition><br />
<condition property="configArgs" value="-ws gtk -arch x86"><br />
<equals arg1="${os.arch}" arg2="i386" /><br />
</condition><br />
<property name="configArgs" value="" /><br />
<br />
<java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"><br />
<arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator<br />
-current ${buildId}<br />
-jvm ${eclipse.perf.jvm}<br />
-print <br />
-output ${postingDirectory}/${buildLabel}/performance/<br />
-config eplnx1,eplnx2,epwin2,epwin3<br />
-dataDir ${postingDirectory}/../../data/v38<br />
-config.properties ${eclipse.perf.config.descriptors}<br />
-scenario.pattern org.eclipse.%.test%" /><br />
<!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed...<br />
-baseline ${eclipse.perf.ref}<br />
-baseline.prefix R-3.4_200806172000<br />
--><br />
<!-- add this argument to list above when there are milestone builds to highlight <br />
-highlight.latest 3.3M1_<br />
--><br />
<env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /><br />
<sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /><br />
</java><br />
</pre><br />
<br />
Two important things about generating performance results:<br />
1) xhost+ needs to be enabled in a terminal of the user generating the performance results<br />
2) The derby database needs to be running on port 1528 (There is an init script for that)<br />
3) X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293972Platform-releng-faq-obsolete2012-03-15T21:35:55Z<p>Kmoir.ca.ibm.com: /* How performance tests are invoked in the build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293969Platform-releng-faq-obsolete2012-03-15T21:20:59Z<p>Kmoir.ca.ibm.com: /* How performance tests are invoked in the build */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
<pre><br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
</pre><br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293965Platform-releng-faq-obsolete2012-03-15T21:20:00Z<p>Kmoir.ca.ibm.com: /* How to regenerate performance results from build artifacts */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
<br />
==How performance tests are invoked in the build==<br />
<br />
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml<br />
<br />
<pre><target name="testAll" unless="skip.tests"><br />
<waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"><br />
<and><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /><br />
<available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /><br />
</and><br />
</waitfor><br />
<br />
<property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /><br />
<antcall target="configure.team.cvs.test" /><br />
<br />
<!--replace buildid in vm.properties for JVM location settings--><br />
<replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /><br />
<br />
<antcall target="addnoperfmarker" /><br />
<br />
<parallel><br />
<antcall target="test-JUnit" /><br />
<antcall target="test-performance" /><br />
</parallel><br />
</target><br />
</pre><br />
<br />
The test-performance target in the buildAll.xml looks like this<br />
<br />
<pre><br />
<target name="test-performance" unless="skip.performance.tests"><br />
<br />
<echo message="Starting performance tests." /><br />
<property name="dropLocation" value="${postingDirectory}" /><br />
<ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /><br />
<antcall target="generatePerformanceResults" /><br />
</target><br />
</pre><br />
<br />
<br />
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs<br />
<br />
<pre><br />
<target name="performanceTests"><br />
<br />
<condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"><br />
<isset property="performance.base" /><br />
</condition><br />
<br />
<property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /><br />
<mkdir dir="${testResults}" /><br />
<br />
<parallel><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp-perf" /><br />
<param name="cfgKey" value="win32xp-perf" /><br />
<param name="markerName" value="eclipse-win32xp-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/win32xp2-perf" /><br />
<param name="cfgKey" value="win32xp2-perf" /><br />
<param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/rhelws5-perf" /><br />
<param name="sleep" value="120" /><br />
<param name="cfgKey" value="rhelws5-perf" /><br />
<param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /><br />
</antcall><br />
<antcall target="test"><br />
<param name="tester" value="${basedir}/sled10-perf" /><br />
<param name="sleep" value="300" /><br />
<param name="cfgKey" value="sled10-perf" /><br />
<param name="markerName" value="eclipse-sled10-perf-${buildId}" /><br />
</antcall><br />
</pre><br />
<br />
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.<br />
<br />
#Windows XP<br />
win32xp-perf=epwin2<br />
win32xp2-perf=epwin3<br />
<br />
#RedHat Enterprise Linux WS 5<br />
rhelws5-perf=eplnx2<br />
<br />
#sled 10 <br />
sled10-perf=eplnx1<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293942Platform-releng-faq-obsolete2012-03-15T20:11:25Z<p>Kmoir.ca.ibm.com: /* How do I find data for old performance results on existing build page? */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293941Platform-releng-faq-obsolete2012-03-15T20:10:25Z<p>Kmoir.ca.ibm.com: /* How do I find data for old performance results on existing build page? */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293940Platform-releng-faq-obsolete2012-03-15T20:09:50Z<p>Kmoir.ca.ibm.com: /* How do I set up performance tests? */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng<br />
*3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng<br />
*3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://fullmoon.ottawa.ibm.com/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293839Platform-releng/Obsolete/Platform-releng-basics2012-03-14T20:26:56Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
Update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
<br />
In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
<br />
Update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
I just copy the milestone child repo /home/data/httpd/download.eclipse.org/eclipse/updates/4.2-I-builds to 4.2milestones and update the compositeContent.jar and compositeArtifacts.jar to point to the new repo. <br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
For weekly integration builds, the build just adds a new child repo automatically. Once a milestone, I go and clean out the integration build repos.<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293838Platform-releng/Obsolete/Platform-releng-basics2012-03-14T20:20:44Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
Update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
<br />
In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
<br />
Update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293837Platform-releng/Obsolete/Platform-releng-basics2012-03-14T20:20:14Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines (Currently 6 PM, on Sunday)<br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session. (And/or, use 'screen' so work continues on host, even if connection lost.)<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
'''Move to latest EMF and ECF contributions'''<br />
<br />
::[TODO: Kim, can you elaborate? correct?]<br />
<br />
: update maps for pre-req'd version. For the 4.2 build we consume EMF from http://download.eclipse.org/modeling/emf/emf/updates/2.8milestones/base/. This repo has to have the Juno EMF contribution for that milestone. Kenn Hussey is the one that we contact for this.<br />
: In the 3.8 build we do that same thing, consume new versions of ECF. Not very often these days. Scott Lewis is the contact for this and Ian Bull should know about the need to consume a new version. Again, similar to EMF, we consume a version of ECF at +0, where they are higher up on the release train stack. So they usually provide a special repo for us to consume.<br />
: update build.properties to point to right location (needed for JavaDoc builds to work right)<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
'''Update b3aggrcon file for aggregation'''<br />
<br />
This includes both ep and equinox b3aggrcon files. Must do after milestone is "final". Often helps to update with "near milestone" results a few days or week early, to give others an "early look" at what's coming.<br />
<br />
'''Update composite artifacts to add new milestone repository'''<br />
<br />
Every milestone, once final, update the platform's composite sites, such as update compositeContent.jar and compositeArtifacts.jar at <br />
<br />
<pre><nowiki>/home/data/httpd/download.eclipse.org/eclipse/updates/4.2milestones</nowiki></pre><br />
<br />
[TODO: Kim, how/when do the "subdirectory repos" get copied there? (i.e. built there? or copied from build location? or mirrored from build location? Is there a script? ]<br />
<br />
[TODO: Similar for weekly integration builds?]<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293817Platform-releng-faq-obsolete2012-03-14T13:57:31Z<p>Kmoir.ca.ibm.com: /* How to regenerate performance results from build artifacts */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://fullmoon.ottawa.ibm.com/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293646Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:43:33Z<p>Kmoir.ca.ibm.com: /* Other common tasks */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change, where the version of ECF was updated.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293645Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:43:01Z<p>Kmoir.ca.ibm.com: /* Other common tasks */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of a similar change.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293644Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:42:45Z<p>Kmoir.ca.ibm.com: /* Other common tasks */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre><br />
<br />
Bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=367794 bug 367794] is an example of this change.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293643Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:40:40Z<p>Kmoir.ca.ibm.com: /* Other common tasks */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
#Update orbit.map with pointer to new location<br />
#Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
#Modify the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre></div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293642Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:40:06Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.<br />
<br />
<br />
== Other common tasks ==<br />
<br />
Move to a newer version of an Orbit bundle<br />
<br />
Update orbit.map with pointer to new location<br />
Update appropriate platformOptions.txt file in org.eclipse.platform.doc.isv or jdtOptions.txt in org.eclipse.jdt.doc.isv to reflect the new name of the bundle as appropriate. <br />
Update the appropriate build.properties in the feature that generates the source bundles for that Orbit bundle. Source bundles for Orbit bundles should not be generated because they are contributed in binary form. For instance, if you look at this build.properties for the Eclipse 3.8 SDK feature, you can see that the orbit bundles are all listed as "unpack=true". <br />
<br />
<pre><br />
###############################################################################<br />
# Copyright (c) 2000, 2009 IBM Corporation and others.<br />
# All rights reserved. This program and the accompanying materials<br />
# are made available under the terms of the Eclipse Public License v1.0<br />
# which accompanies this distribution, and is available at<br />
# http://www.eclipse.org/legal/epl-v10.html<br />
# <br />
# Contributors:<br />
# IBM Corporation - initial API and implementation<br />
###############################################################################<br />
bin.includes=eclipse_update_120.jpg,feature.xml,feature.properties<br />
<br />
generate.feature@org.eclipse.platform.source=org.eclipse.platform,feature@org.eclipse.rcp.source,feature@org.eclipse.equinox.p2.user.ui.source;optional="true",plugin@org.eclipse.platform.doc.isv;unpack="false",\<br />
plugin@org.apache.ant.source;version=1.8.2.qualifier;unpack="false",\<br />
plugin@com.jcraft.jsch.source;version=0.1.44.qualifier;unpack="false",\<br />
exclude@org.eclipse.platform.doc.user<br />
<br />
generate.feature@org.eclipse.jdt.source=org.eclipse.jdt, plugin@org.eclipse.jdt.doc.isv;unpack="false",\<br />
plugin@org.junit.source;version=3.8.2.qualifier;unpack="false",\<br />
plugin@org.junit.source;version=4.8.2.qualifier;unpack="false",\<br />
plugin@org.hamcrest.core.source;version=1.1.0.qualifier;unpack="false",\<br />
exclude@org.eclipse.jdt.doc.user<br />
generate.feature@org.eclipse.pde.source=org.eclipse.pde,plugin@org.objectweb.asm.source;version=3.3.1.qualifier;unpack="false",\exclude@org.eclipse.pde.doc.user<br />
generate.feature@org.eclipse.cvs.source=org.eclipse.cvs<br />
generate.feature@org.eclipse.help.source=org.eclipse.help,\<br />
plugin@javax.servlet.source;version=3.0.0.qualifier;unpack="false",\<br />
plugin@javax.servlet.jsp.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.jasper.glassfish.source;version=2.2.2.qualifier;unpack="false",\<br />
plugin@com.sun.el.source;version=2.2.0.qualifier;unpack="false",\<br />
plugin@org.apache.commons.logging.source;version=1.0.4.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.analysis.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.apache.lucene.core.source;version=2.9.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.continuation.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.http.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.io.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.security.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.server.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.servlet.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.jetty.util.source;version=8.1.1.qualifier;unpack="false",\<br />
plugin@org.eclipse.javax.el.source;version=2.2.0.qualifier;unpack="false"<br />
<br />
generatedVersionLength=45<br />
</pre></div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293641Platform-releng/Obsolete/Platform-releng-basics2012-03-12T21:22:24Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.basebuilder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293634Platform-releng-faq-obsolete2012-03-12T19:53:08Z<p>Kmoir.ca.ibm.com: /* When is the next build? */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
David Williams and John Arthorne have rights to update this calendar.<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://fullmoon.ottawa.ibm.com/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293625Platform-releng/Obsolete/Platform-releng-basics2012-03-12T19:07:30Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<pre>clean git tagging repos on eclipsebuildserv<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37x<br />
0 18 * * 0 /home/users/releng/buildTools/scripts/cleandrops.pl /builds/eclipse/sdk37<br />
</pre><br />
<br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.base.builder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293624Platform-releng/Obsolete/Platform-releng-basics2012-03-12T18:58:44Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.pl</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.base.builder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293616Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:25:38Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test milestone bundles in org.eclipse.releng.base.builder'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293615Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:20:31Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
'''Move to next Orbit milestone build''' <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]<br />
<br />
<br />
'''Test new bundles during test day of milestone week'''<br />
<br />
Release the relevant subset of bundles to org.eclipse.releng.basebuilder and run a test build to ensure the milestone build can build Eclipse. Update the builder to the milestone build after it's released, run test build.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293612Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:12:42Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
Move to next Orbit milestone build <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
<br>This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293611Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:12:27Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
Move to next Orbit milestone build <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
<br>This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b git commit to update to Orbit M6 build]<br />
<br />
<br>Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293610Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:12:03Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
Move to next Orbit milestone build <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2 bug 373958 comment 2]<br />
<br />
<br>This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b]<br />
<br />
<br>Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293609Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:11:44Z<p>Kmoir.ca.ibm.com: /* Common tasks during a milestone */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
Move to next Orbit milestone build <br />
<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2]<br />
<br />
<br>This is a very simple change this milestone. I just replaced the old Orbit buildId with the new one. <br />
<br />
[http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b]<br />
<br />
<br>Usually there if there are new Orbit bundles released during a milestone that a team needs to consume, we run test builds before milestone week to ensure that the new Orbit bundles are the right shape etc. For example, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174 bug 368174]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293606Platform-releng/Obsolete/Platform-releng-basics2012-03-12T17:09:21Z<p>Kmoir.ca.ibm.com: /* Restarting the build */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.<br />
<br />
== Common tasks during a milestone ==<br />
<br />
Move to next Orbit milestone build<br />
<br />
See https://bugs.eclipse.org/bugs/show_bug.cgi?id=373958#c2<br />
<br />
<br />
This is a very simple change this milestone. I just replaced the old Orbit<br />
buildId with the new one. <br />
<br />
http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?id=55ec07487a54e6c4ba7392e88c917823e283d76b<br />
<br />
Usually there if there are new Orbit bundles released during a milestone that a<br />
team needs to consume, we run test builds before milestone week to ensure that<br />
the new Orbit bundles are the right shape etc. For example, see bug <br />
https://bugs.eclipse.org/bugs/show_bug.cgi?id=368174</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293595Platform-releng/Obsolete/Platform-releng-basics2012-03-12T16:01:03Z<p>Kmoir.ca.ibm.com: /* Common build failures */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependencies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in Git doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale Git clone processes on eclipsebuildserv.&nbsp; ps -ef | grep git.&nbsp; If there is a git process that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependencies due to errors in Manifest or missing dependencies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependencies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse38/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293594Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:58:48Z<p>Kmoir.ca.ibm.com: /* Build scripts */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse38/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293593Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:47:27Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Location of Git and CVS repos ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293592Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:46:40Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293591Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:46:10Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
Common git repos that the eclipse team uses<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows]] <br />
<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293588Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:44:30Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
Common git repos that the eclipse team uses<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
The complete list of Git repos for the Eclipse and Equinox projects are here [[Platform-releng/Git_Workflows Git Workflows]] <br />
<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293587Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:42:08Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
Common git repos that the eclipse team uses<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
<br>branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
<br>features in features/<br />
<br>branding plugins in bundles/<br />
<br>platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
<br>map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
<br>doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293586Platform-releng/Obsolete/Platform-releng-basics2012-03-12T15:41:37Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Git and CVS repos for releng related activity<br />
<br />
Common git repos that the eclipse team uses<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.git<br />
branding plugins in platform/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git<br />
features in features/<br />
branding plugins in bundles/<br />
platform feature for 3.x stream builds in oldfeatures/<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git <br />
map files<br />
<br />
ssh://userid@git.eclipse.org/gitroot/platform/eclipse.common.git<br />
doc bundles in common/<br />
<br />
The org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder projects are still in CVS (/cvsroot/eclipse)<br />
They need to be migrated to Git by the end of 2012. The basebuilder project should really be 1) In its own repo because of it's binary content or 2) Convert the build to use a product build from the repo of SDK bundles and consume the custom bundles separately or 3) Just extract a SDK and add custom bundles to it<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293422Platform-releng/Obsolete/Platform-releng-basics2012-03-09T16:41:04Z<p>Kmoir.ca.ibm.com: /* Build scripts */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse37/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse38/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.8-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20120305<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293421Platform-releng/Obsolete/Platform-releng-basics2012-03-09T16:38:53Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng in the integration branch of org.eclipse.releng<br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from mastter of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
<br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse36/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.6-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
You will notice that the integration build has a compareMaps flag passed to the build. This paramater takes the current version of the map file project, and compares it to the version here in the the file "/home/users/releng/buildTools/eclipse36/mapTag.properties"<br />
<br />
-bash-3.00$ cat /home/users/releng/buildTools/eclipse36/mapTag.properties<br />
<br />
lastMapTag=I20100429-1549<br />
<br />
The bootstrap script that initiates 3.6 stream builds is located on the build machine here...<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20100430<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293420Platform-releng-faq-obsolete2012-03-09T16:33:59Z<p>Kmoir.ca.ibm.com: </p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
== Basic overview of Platform releng tasks and troubleshooting tips<br> ==<br />
<br />
[[Platform-releng-basics|Platform releng basics]]<br />
<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://fullmoon.ottawa.ibm.com/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng-faq-obsolete&diff=293419Platform-releng-faq-obsolete2012-03-09T16:33:34Z<p>Kmoir.ca.ibm.com: /* Basic overview of Platform releng tasks and troubleshooting tips */</p>
<hr />
<div>'''Eclipse Platform Release Engineering FAQ'''<br />
<br />
If you would like additional content added to this FAQ, please send a note to the [https://dev.eclipse.org/mailman/listinfo/platform-releng-dev platform releng developers mailing list].<br />
<br />
==What hardware comprises the platform-releng build infrastructure?==<br />
The eclipse platform build infrastructure consists of<br />
#junit test machines, <br />
##ejwin1 (winxp with 1.5 vm) 5 on KVM on table<br />
##ejwin2 (winxp with 1.6 vm 6 on KBM on table <br />
##lb6mac (macosx Leopard) mac on table on LHS of lab<br />
##ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack<br />
##ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack<br />
#performance machines<br />
##epwin2 (winxp with 1.5 vm) G on large KVM in rack<br />
##epwin3 (winxp with 1.6 vm) 7 on large KVM in rack<br />
##eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack <br />
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack <br />
#Other<br />
##minsky (database server with apache derby installed to store performance test results), (cvs test server)<br />
##eclipsebuildserv (build machine) Linux machine on far back table of the room<br />
<br />
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.<br />
<br />
==Is the Eclipse platform build process completely automated?==<br />
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.<br />
<br />
*org.eclipse.releng.eclipsebuilder -&gt; scripts to build each type of drop<br />
*org.eclipse.releng.basebuilder -&gt; a subset of eclipse plugins required to run the build.<br />
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks<br />
<br />
Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder.<br />
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.<br />
<br />
==What's the difference between an integration and nightly build?==<br />
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.<br />
<br />
==How do I use the releng plugin?==<br />
The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page.<br />
<br />
This plug-in provides features that will help with the Eclipse development process. Installing the plug-in will add the following actions. The source is contained in the *.jar files so please feel free to submit patches for features or bug fixes. To install simply unzip the file into root of your eclipse install and restart Eclipse. <br />
'''Please use the Release feature of this plug-in to do your build submissions.'''<br />
*'''Release''' to the Team menu. This action will Tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.<br />
*''''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.<br />
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.<br />
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.<br />
*'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.<br />
<br />
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?==<br />
This document provides an overview of <br />
[http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&amp;content-type=text/html how to build eclipse components using the releng scripts.] However, it is really out of date. Your best bet is to use the source build to recompile the SDK.<br />
<br />
==What is the latest version of org.eclipse.releng.basebuilder?==<br />
See this document which describes correct tag of [[Platform-releng-basebuilder|org.eclipse.releng.basebuilder]] for use in your builds.<br />
<br />
==How do I automate my builds with PDE Build?==<br />
See the [[PDE/Build]] wiki page.<br />
<br />
==How do I contribute a JUnit Plug-in Test to the build?==<br />
#The tests need to be contributed as one or more plugins that use the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test/testframework.html?rev=HEAD&content-type=text/html org.eclipse.test] automated testing framework. JUnit tests can also be written as performance tests. Click [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html here] for the latest instructions.<br />
#Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.<br />
#Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.<br />
#Install the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-releng-home/dev.html?rev=HEAD&content-type=text/html org.eclipse.releng tools] plug-in, and use the &quot;Release&quot; action in the context menu of the test plug-in project to update and commit map file.<br />
#Open a bug against platform releng to add the plugins to the build process. Include the following information:<br />
*the plug-in id(s)<br />
*the plug-ins that contain a test.xml<br />
*update the build.properties to include the test.xml<br />
*additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK<br />
<br />
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.<br />
<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ui.tests/ (with UI)]<br />
*Example [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ant.tests.core/ (headless)]<br />
<br />
==How do I contribute an example plug-in to the build?==<br />
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.<br />
<br />
Next, open a bug against platform releng with the plug-in's id.<br />
<br />
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.<br />
<br />
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?==<br />
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure].<br />
<br />
==How does the platform team create the source build?==<br />
See [[Platform-releng-sourcebuild]]<br />
<br />
==How do you run the source build for 3.5 builds?==<br />
See [[Platform-releng-sourcebuild35]] (document still in progress)<br />
<br />
==How does the platform team sign their builds?==<br />
See [[Platform-releng-signedbuild]]<br />
<br />
==How long does the build take to complete?==<br />
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.<br />
<br />
==When is the next build?==<br />
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule].<br />
<br />
==When is the next milestone?==<br />
Please refer to the [http://www.eclipse.org/eclipse/development/eclipse_project_plan_3_4.html Eclipse Platform Project Plan].<br />
<br />
==I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?==<br />
<br />
See {{bug|134413}}.<br />
<br />
We have a number of tests that intermittently fail. The reasons are <br />
*issues with the tests themselves<br />
*tests subject to timing issues<br />
*tests that fail intermittently due to various conditions<br />
<br />
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.<br />
<br />
==Scripts to promote a 3.x stream build==<br />
<br />
*Disable rsync<br />
<br />
*cd /home/users/releng/buildTools/extraTools on eclipsebuildserv<br />
<br />
*See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild<br />
**Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName><br />
<br />
*Promote eclipse milestone build <br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4<br />
<br />
*Promote equinox build<br />
**./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4<br />
<br />
*Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.<br />
<br />
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)<br />
<br />
*Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.<br />
<br />
*Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org<br />
<br />
*Enjoy post-milestone chocolate.<br />
<br />
==How to hide a build to allow it to sync to the mirrors==<br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd<br />
<br />
/home/data/httpd/download.eclipse.org/eclipse/downloads<br />
<br />
php eclipse3x.php > eclipse3x.html<br />
<br />
update <br />
<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php<br />
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html<br />
<br />
to point to eclipse3x.html instead of eclipse3x.php<br />
<br />
revert the changes when the build is ready to release<br />
<br />
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?==<br />
There are several ways to approach this issue. <br />
*Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.<br />
*In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.<br />
*Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.<br />
<br />
==I'm looking for an older build but can't find them on the download page?==<br />
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.<br />
<br />
==How do I run a test build on the hudson install at eclipse.org ?==<br />
<br />
Login to this web page with your committer id and password.<br />
<br />
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/<br />
<br />
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).<br />
<br />
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393<br />
<br />
==How do I run the JUnit tests used in the build?==<br />
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice.<br />
<br />
==How do you run the tests on the Windows machine via rsh==<br />
To run the windows tests from the build machine,<br />
<br><br />
<tt><br />
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt<br />
</tt><br />
<br />
==Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)==<br />
<br />
The default Mac install only includes a JRE, not a JDK. The JDK is listed under [https://connect.apple.com Apple Connect] under J2SE 5.0 Release 5 Developer Documentation.<br />
Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home<br />
<br />
[http://tech.puredanger.com/2007/09/21/java-source-mac/link Reference]<br />
<br />
==How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?==<br />
This [[Platform-releng-cvs-tests | document]] outlines the steps required to setup a test CVS repository on Linux.<br />
<br />
==How do I set up performance tests?==<br />
Refer to the [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html performance tests how-to] or the even better [http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf Performance First talk] at Eclipse Con 2007.<br />
<br />
Baseline tests are run from a branch of the builder. <br />
<br />
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng<br />
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng<br />
*3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng<br />
<br />
==How do I find data for old performance results on existing build page?==<br />
Refer to the "Raw data and Stats" link for a specific test.<br />
<br />
For instance for 3.3.1.1 results, [http://fullmoon.ottawa.ibm.com/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here]<br />
<br />
Then look at the [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests].<br />
<br />
Then click on a [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test].<br />
<br />
Then click "Raw data and Stats". <br />
<br />
You will see the data for [http://download.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds].<br />
<br />
==How do I run the performance tests from the zips provided on the download page?==<br />
See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=91229#c3 here].<br />
<br />
==Process to implement a new baseline==<br />
Implement new performance [[Platform-releng-new-baseline | baseline]] after a release.<br />
<br />
==How to regenerate performance results from build artifacts==<br />
<br />
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.<br />
<br />
*cd to build directory on build machine<br />
*cd org.eclipse.releng.eclipsebuilder<br />
*apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]]<br />
*rerun generation on hacked builder<br />
*on build machine<br />
**at now<br />
**cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
**ctrl d<br />
<br />
<br />
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.<br />
<br />
==Why should I package plugins and features as jars?==<br />
See the [http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Running Eclipse from JARs] document from the Core team.<br />
<br />
==Why should I use qualifiers?==<br />
See the [[Version_Numbering | Versioning Guidelines ]]<br />
<br />
==How do I incorporate qualifiers into the build process?==<br />
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.<br />
<br />
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.<br />
<br />
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.<br />
<br />
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build<br />
<br />
<source lang="xml"><br />
<condition property="forceContextQualifier" value="${buildId}"><br />
<equals arg1="${buildType}" arg2="N" /><br />
</condition><br />
</source><br />
<br />
Also, in the <code>bootstrap.sh</code> file which kicks off our build, we specify <code>-DgenerateFeatureVersionSuffix=true</code><br />
<br />
buildCommand="$antRunner<br />
-q<br />
-buildfile buildAll.xml<br />
$mail<br />
$testBuild<br />
$compareMaps<br />
-DmapVersionTag=$mapVersionTag<br />
-DpostingDirectory=$postingDirectory<br />
-Dbootclasspath=$bootclasspath<br />
-DbuildType=$buildType<br />
-D$buildType=true<br />
-DbuildId=$buildId<br />
-DbuildLabel=$buildLabel<br />
-Dtimestamp=$timestamp<br />
-DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse<br />
$skipPerf<br />
$skipTest<br />
$tagMaps<br />
-DJ2SE-1.5=$bootclasspath_15<br />
-DJ2SE-1.4=$bootclasspath<br />
-DCDC-1.0/Foundation-1.0=$bootclasspath_foundation<br />
-DlogExtension=.xml<br />
$javadoc<br />
$updateSite<br />
$sign<br />
-DgenerateFeatureVersionSuffix=true<br />
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre<br />
-listener org.eclipse.releng.build.listeners.EclipseBuildListener"<br />
<br />
<br />
==How can I run the update manager from a command line to mirror a remote site?==<br />
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.<br />
<br />
<br />
==I have questions about PDEBuild - where should I go for answers?==<br />
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.<br />
<br />
==Debugging pde build with missing dependancies==<br />
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint<br />
In display view,<br />
state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false))<br />
print the errors for the bundle in question.<br />
<br />
==I'd like to incoporate source bundles in my build, how do I do it?==<br />
<br />
See [[PDEBuild/Individual_Source_Bundles | Individual Source Bundles]]<br />
<br />
==Are there RSS feeds for builds?==<br />
Yes, for I-Builds only. They are available [http://download.eclipse.org/eclipse/downloads/builds-eclipse-3.3.xml here].<br />
<br />
==If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.==<br />
See the [[How_to_add_things_to_the_Eclipse_doc | adding javadoc]] document.<br />
<br />
<br />
<br />
==Troubleshooting test failures==<br />
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.<br />
<br />
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.<br />
<br />
<source lang="xml"><br />
<property<br />
name="extraVMargs" <br />
value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/><br />
</source><br />
<br />
Then,<br />
*create a remote debugging launch target in your dev environment<br />
*put a breakpoint somewhere in your code<br />
*launch the tests from the command line, using the -noclean option to preserve the modified test.xml<br />
*launch the debug target<br />
<br />
==I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?==<br />
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.<br />
<br />
==Eclipse Release checklist==<br />
[[Eclipse/Release_checklist | Release checklist]]<br />
<br />
==New JUnit/performance machine deployment checklist==<br />
<br />
Steps to take after OS install from IT<br />
<br />
===All platforms===<br />
*verify hostname is set on machine<br />
*verify hostname is registered in DNS and both forward and reverse lookups resolve correctly<br />
*disable screen saver<br />
*disable all power management options so that the machine, disk, monitor etc is always on<br />
*create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it<br />
*install ganglia and configure to interact with appropriate eclipse build node<br />
*verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.<br />
*disable real time virus scanning of build directories<br />
*ensure that machines starts as build user automatically upon reboot<br />
<br />
====Windows====<br />
*install rshd<br />
**set rshd user equivalency to build user<br />
**ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup<br />
*ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them<br />
<br />
===Linux/Mac===<br />
*copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention<br />
*ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests<br />
*update /etc/hosts to include localhost<br />
*disable iptables in chkconfig and stop the service<br />
<br />
<br />
<br />
==Process to release builds to Simultaneous Release==<br />
*Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. <br />
*For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon<br />
*Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.<br />
*Watch build message to ensure coordinated release build completes successfully.<br />
<br />
<br />
<br />
==How do I verify the signatures of packed jars on an update site?==<br />
See {{bug|193568}} for the tool and instructions.<br />
<br />
==Where are the p2 update sites for the Eclipse Project?==<br />
See [http://wiki.eclipse.org/Eclipse_Project_Update_Sites the list]<br />
<br />
==How do I use the p2 zipped repos on the build page to provision my install or a pde target?==<br />
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos<br />
<br />
==Where can I download foundation libraries?==<br />
<br />
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.<br />
<br />
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage<br />
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily<br />
<br />
==Platform Releng Helios Plan==<br />
[[Platform-releng/Helios | Helios plan]]<br />
<br />
==Platform Indigo Plan==<br />
[[Platform-releng/Indigo | Indigo plan]]<br />
<br />
==How to assemble the deltapack using the p2 slicer==<br />
<br />
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.<br />
<br />
<pre><br />
<project default="build"><br />
<target name="build"><br />
<property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /><br />
<property name="tempDir" value="D:\\deltapack" /><br />
<br />
<p2.mirror source="${repo}"><br />
<destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /><br />
<iu id="org.eclipse.platform.feature.group" version="" /><br />
<iu id="org.eclipse.rcp.feature.group" version="" /><br />
<iu id="org.eclipse.jdt.feature.group" version="" /><br />
<iu id="org.eclipse.equinox.executable" version="" /><br />
<slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /><br />
</p2.mirror><br />
</target><br />
</project><br />
</pre><br />
<br />
Unzip 3.5RC2 to a directory and run the script using the AntRunner.<br />
<br />
<code><br />
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.<br />
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml<br />
</code><br />
<br />
The resulting files will be in the d:\deltapack directory on your file system.<br />
<br />
<br />
==How to verify the packed jars in a repo==<br />
<br />
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder><br />
<br />
== Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4 ==<br />
<br />
[[http://wiki.eclipse.org/Eclipse/Testing/JUnit4_Changes Junit4 Changes]]<br />
<br />
<br />
<br />
==How to avoid breaking the build==<br />
<br />
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build]<br />
<br />
<br />
<br />
== Submitting content to the Helios build ==<br />
<br />
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.<br />
<br />
/builds/transfer/files/updates/3.6milestones/<br />
<br />
I just ls the features directory of the current milestone, update the build files, then commit.<br />
<br />
== Invoking the signing process from the command line ==<br />
# Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)<br />
# Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse<br />
# Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.<br />
<br />
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput<br />
<br />
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.<br />
<br />
== Connecting to the derby database using ij ==<br />
<br />
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre<br />
<br />
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin<br />
<br />
cd /builds/db-derby-10.4.2.0-bin/bin<br />
<br />
connect 'jdbc:derby://localhost:1528/perfDb36';<br />
<br />
== Are the Eclipse and Equinox projects converting from CVS to git? ==<br />
<br />
This was completed in the fall of 2012. Here's the planning document<br />
<br />
[[Platform-releng/Juno_Git_Migration | Juno Git Migration]]<br />
<br />
Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration.<br />
<br />
== How do you incorporate the p2MirrorURL into your repo at build time ==<br />
<br />
See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann.<br />
<br />
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org<br />
bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.<br />
<br />
==Eclipsecon presentations==<br />
===2010===<br />
<br />
*From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction<br />
<br />
===2007===<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3635 PDE Build and Build Clinic]<br />
*[http://www.eclipsecon.org/2007/index.php?page=sub/&id=3726 Putting your Build to the Test]<br />
<br />
===2006===<br />
*[http://www.eclipsecon.org/2006/Sub.do?id=254 From developer to Download: A Tour of the Platform Build Factory], an updated version is available [http://www.eclipse.org/eclipse/platform-releng/eclipsecon2006/tour.zip here].<br />
<br />
===2005===<br />
Best releng practices from our Eclipsecon 2005 poster:<br />
#Automate the build process from end-to-end and automate early.<br />
#Use PDE Build in Eclipse to generate build scripts.<br />
#Automate JUnit testing and performance monitoring.<br />
#Automate build notifications (email).<br />
#Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)<br />
#Ensure stability in the build process by thorough testing of builder changes.<br />
#Schedule builds at regular intervals and enforce rebuild policies.<br />
#Use map files in a central CVS repository.<br />
#Build on Linux.<br />
#Have fun!<br />
<br />
==Useful reference documents==<br />
*Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html<br />
*Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html<br />
*The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf<br />
*[[Building]]<br />
<br />
==Who are the platform-releng committers?==<br />
Kim Moir<br />
<br />
==As the holiday season approaches, what gifts are appropriate for your favourite release engineer?==<br />
We enjoy [http://www.louisesbelgianchocs.com fine chocolate], [http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-ui-home/curries.png Shaan], and [http://en.wikipedia.org/wiki/Strongbow_Cider apple juice].<br />
<br />
==What does the platform-releng team do when they're not running the build farm and wrangling bugs?==<br />
We herd cats. s/eds/ibm/g<br />
<br />
http://video.google.com/videoplay?docid=4057591681481453187<br />
<br />
==What factors contribute to the low percentage of women participating in free software/open source?==<br />
Cambridge University recently completed a study on this very topic with [http://www.flosspols.org/deliverables/FLOSSPOLS-D16-Gender_Integrated_Report_of_Findings.pdf interesting results].<br />
<br />
Here is another [http://infohost.nmt.edu/~val/review/flosspols.pdf one] written by Val Henson, a Linux kernel committer.<br />
<br />
[http://www.catalyst.org/knowledge/titles/title.php?page=women_in_high_tech08 2008 report from Catalyst]<br />
<br />
==Deprecated contents from Eclipse Platform Release Engineering FAQ==<br />
<br />
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated<br />
<br />
[[Category:FAQ]]<br />
[[Category:Releng]]</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293416Platform-releng/Obsolete/Platform-releng-basics2012-03-09T15:47:26Z<p>Kmoir.ca.ibm.com: </p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng <br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from HEAD of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
3.7.x maintenance builds run from R3_7_maintenance branch of org.eclipse.releng <br />
<pre>0 8 * * 3 cd /builds; /home/users/releng/buildTools/eclipse37x/runMBuild<br />
</pre> <br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh</pre> <br />
<br />
<br>Eclipse 3.7.2+ test builds from R3_7_maintenance branch of org.eclipse.releng in Git (automatically tagged from R3_7_maintenance branch)<br />
<pre>20 9 * * 4 cd /builds; /home/users/releng/buildTools/eclipse37x/runKimMBuildtest</pre><br />
<br />
<br>Eclipse 3.6.2+ test builds from R3_6_maintenance branch of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 16 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runMKimBuildTest</pre><br />
<br />
<br>Eclipse 3.6.2+ + Java7 from R3_6_maintenance_Java7 of org.eclipse.releng in Git (not tagged automatically, have to update maps manually)<br />
<pre>26 18 * * 1 cd /builds; /home/users/releng/buildTools/eclipse36x/runM362Java7</pre><br />
<br />
<br>Eclipse 3.5.x test builds from R3_5_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest</pre> <br />
<br />
<br> 3.4.2 test builds from R3_4_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild</pre> <br />
<br />
<br> 3.2.x test builds from R3_2_maintenance branch of org.eclipse.releng in CVS<br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse36/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.6-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
You will notice that the integration build has a compareMaps flag passed to the build. This paramater takes the current version of the map file project, and compares it to the version here in the the file "/home/users/releng/buildTools/eclipse36/mapTag.properties"<br />
<br />
-bash-3.00$ cat /home/users/releng/buildTools/eclipse36/mapTag.properties<br />
<br />
lastMapTag=I20100429-1549<br />
<br />
The bootstrap script that initiates 3.6 stream builds is located on the build machine here...<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20100430<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.comhttps://wiki.eclipse.org/index.php?title=Platform-releng/Obsolete/Platform-releng-basics&diff=293415Platform-releng/Obsolete/Platform-releng-basics2012-03-09T15:30:59Z<p>Kmoir.ca.ibm.com: /* Basic overview of platform builds */</p>
<hr />
<div>== Basic overview of platform builds ==<br />
<br />
Integration builds are in crontab of build user on build machine <br />
<br />
Integration builds run from the tags of org.eclipse.releng <br />
<pre>0 8 * * 2 cd /builds; /home/users/releng/buildTools/eclipse38/runIBuild<br />
</pre> <br />
Nightly builds run from HEAD of org.eclipse.releng <br />
<pre>0 20 * * 1,2,3,5,0 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
0 20 * * 4,6 cd /builds; /home/users/releng/buildTools/eclipse38/runNBuild-skipPerf<br />
</pre> <br />
3.7.x maintenance builds run from R3_7_maintenance branch of org.eclipse.releng <br />
<pre>0 8 * * 3 cd /builds; /home/users/releng/buildTools/eclipse37x/runMBuild<br />
</pre> <br />
The machines used in the [http://wiki.eclipse.org/Platform-releng-faq#What_hardware_comprises_the_platform-releng_build_infrastructure.3F builds] and their locations are listed in the FAQ. <br />
<br />
A script runs once a week to clean the test machines <br />
<pre>0 18 * * 0 /home/users/releng/buildTools/scripts/buildblaster.sh<br />
</pre> <br />
<br />
<br />
3.5.x test builds <br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse35x/runMKimBuildTest<br />
</pre> <br />
<br> 3.4.2 test builds <br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse34x/runKimTestMBuild<br />
</pre> <br />
3.2.x test builds <br />
<pre>cd /builds; /home/users/releng/buildTools/eclipse32x/runMTestBuild-skipPerf<br />
</pre><br />
<br />
==Build scripts==<br />
<br />
Here is an overview of the build scripts used in the build.<br />
<br />
http://wiki.eclipse.org/Platform-releng-faq#Is_the_Eclipse_platform_build_process_completely_automated.3F<br />
<br />
As mentioned above, the builds are initiated by cron.<br />
<br />
The integration build is run as follows<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/runIBuild<br />
</pre><br />
<br />
<pre><br />
# !/bin/sh<br />
<br />
cd /builds<br />
<br />
command="/home/users/releng/buildTools/eclipse36/bootstrap.sh -notify platform-releng-dev@eclipse.org -buildDirectory /builds/I -tagMapFiles -compareMaps -sign -updateSite /builds/transfer/files/updates/3.6-I-builds I"<br />
<br />
/home/users/releng/buildTools/extraTools/monitor.sh $command<br />
<br />
</pre><br />
<br />
You will notice that the integration build has a compareMaps flag passed to the build. This paramater takes the current version of the map file project, and compares it to the version here in the the file "/home/users/releng/buildTools/eclipse36/mapTag.properties"<br />
<br />
-bash-3.00$ cat /home/users/releng/buildTools/eclipse36/mapTag.properties<br />
<br />
lastMapTag=I20100429-1549<br />
<br />
The bootstrap script that initiates 3.6 stream builds is located on the build machine here...<br />
<br />
<pre><br />
/home/users/releng/buildTools/eclipse36/bootstrap.sh<br />
</pre><br />
<br />
If you look search for "buildProjectTags" in this file, you'll see something like this<br />
This indicates the tag of the the three releng projects (org.eclipse.releng.eclipsebuilder, org.eclipse.releng.basebuilder, eclipseInternalBuildTools) used to run the build. If you need to run the build with a new version of the builder, update this tag.<br />
buildProjectTags=v20100430<br />
<br />
== Rsync ==<br />
<br />
The build is rsynced from eclipsebuildserv to fullmoon.ottawa.ibm.com to download.eclipse.org. Look in kmoir's crontab for the scripts.<br />
<br />
== Common build failures ==<br />
<br />
*Fail to fetch bundles from Orbit - symptom - java net url timeout in build failure message. Start a new build -&gt; www.eclipse.org timeouts are intermittent. <br />
*"Error occurred while transforming repository" usually means that the bundle couldn't be fetched from Orbit. For example<br />
<pre>Build M20090825-1330 (Timestamp: 200908251330): The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/buildAll.xml:166: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/build.xml:78: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line:<br />
/builds/M200908251330/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.5.1.R35x_20090721/scripts/genericTargets.xml:59: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_master.xml:73: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.sdk.xml:41: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:11: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.platform.xml:904: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:10: The following error occurred while executing this line:<br />
/builds/M200908251330/src/fetch_org.eclipse.equinox.p2.user.ui.xml:304: Error occurred while transforming repository.<br />
</pre> <br />
Since our build runs in an IBM lab, the http gets for orbit bundles from eclipse.org are usually redirected from download.eclipse.org to fullmoon.ottawa.ibm.com by the foundation. This redirection can be avoided by changing the url in the orbit.map from download.eclipse.org to www.eclipse.org/external. Sometimes it will take a day or so for the internal mirror to get the latest orbit build. <br />
<br />
*Missing dependancies due to erroneous map file submission.&nbsp; Check that the map file refers to a version of a project that exists in the repo. If the version of a bundle in cvs doesn't match the one in the map files, ask team to resubmit and start a new build. <br />
*Build doesn't proceed - connent not updated etc.&nbsp; Check for stale cvs checkouts on eclipsebuildserv.&nbsp; ps -ef | grep cvs.&nbsp; If there is a cvs connection that's over an hour old, kill it and allow the build to proceed.<br />
*SDKs fail to provision due to missing dependancies. There are releng logs on each test results page of the build that provide logs of the from the call to the p2 director to provision the zips. The Equinox team is helpful in finding the cause of the problem. See this page for an example. http://download.eclipse.org/eclipse/downloads/drops/R-3.5-200906111540/buildlogs.php <br />
*Missing dependancies due to errors in Manifest or missing dependancies in manifest. In the both cases, the error sent to the releng list will look something like http://dev.eclipse.org/mhonarc/lists/platform-releng-dev/msg09778.html <br />
*Dependancies expressed in manifest and feature don't match. This means that the p2 director cannot resolve the dependencies for the product it is trying to build. The resultant zips for that platform are 0 size. For instance, see bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=258489 | bug 258489]. Also, you can see the .log files that the directory built by following the ''Release Engineering build logs'' link off the ''Test results'' build page.<br />
*Message "Build failed, map files unchanged". This means either one of two things. One - a build ran and the map files were unchanged so it didn't need to run. Two - you restarted a failed build but the map files didn't change. In the second case, you should update this /home/users/releng/buildTools/eclipse36/mapTag.properties to reflect the build id of the last successful build. <br />
*Message "A problem occured while invoking the director". If there is a compile error in the build, the bundle with the compile error will not be published to the repository. Thus when the director is invoked to build zips, the director operation will fail. <br />
*The director logs, mirror logs etc. are located on the test results pages, under release engineering logs. [http://download.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/buildlogs.php Example of 3.6 release engineering logs. ] The location on the build machine filesystem is<br />
/builds/transfer/files/master/download/drops/<buildId>/buildlogs. If the build has failed invoking the director, the director logs may not be copied to this location yet. In this case, there will be director logs in /builds/<buildId>/org.eclipse.releng.basebuilder/configuration directory.<br />
*All build tasks are invoked via the userid kmoir on the internal build machine, test machines and eclipse.org.<br />
<br />
<br><br />
<br />
== Missing test results ==<br />
<br />
*The test results page of the build isn't updated until all the test results have completed. To see what machines are currently being used for tests look the the following directory. There will be marker files that correspond to the machine operating system and build number.<br />
<pre>-bash-3.00$ ls /home/users/releng/buildTools/markers/*.marker<br />
eclipse-macosx-M20090824-0800.marker<br />
eclipse-rhelws5-6.0-M20090824-0800.marker<br />
eclipse-rhelws5-M20090824-0800.marker<br />
eclipse-rhelws5-perf-M20090824-0800.marker<br />
eclipse-sled10-perf-M20090824-0800.marker<br />
eclipse-win32xp-6.0-M20090824-0800.marker<br />
eclipse-win32xp-M20090824-0800.marker<br />
eclipse-win32xp-perf-M20090824-0800.marker<br />
</pre> <br />
If you cat the marker files you can see the hostname of the machine that corresponds to the marker file <br />
<pre>-bash-3.00$ cat /home/users/releng/buildTools/markers/*.marker<br />
0=lb6mac<br />
0=ejlnx2<br />
0=ejlnx1<br />
0=eplnx2<br />
0=eplnx1<br />
0=ejwin2<br />
0=ejwin1<br />
0=epwin2<br />
</pre> <br />
The windows machines run tests via rsh. The linux and mac machines run tests via ssh. The windows machines need to be rebooted every so often. The old build artifacts are cleaned automatically by cron jobs. <br />
<br />
*The windows tests machines crash every so often. The event logs don't indicate that anything is wrong with the build. They are on the KVM next to the Eclipse rack in lab, on number 5 and 6. They can just be rebooted and logged into as the build user. I usually reboot these machines once a week so this doesn't happen.<br />
<br />
== Restarting the build ==<br />
<br />
Each build has a corresponding directory on eclipsebuildserv in /builds. For instance /builds/I201004291549/. If you cd to the org.eclipse.releng.eclipsebuilder directory, there is a script called buildAll.xml. The main target looks something like this<br />
<br />
<pre><br />
<target name="main" depends="init"><br />
<antcall target="buildEclipseSourceDrops" /><br />
<antcall target="buildMasterFeature" /><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="updatePackProperties" /><br />
<antcall target="signMasterFeature" /><br />
</sequential><br />
<sequential><br />
<antcall target="buildSdkTestFeature" /><br />
<ant antfile="${eclipse.build.configs}/../helper.xml" target="verifyCompile" /><br />
</sequential><br />
</parallel><br />
<parallel failonany="true"><br />
<sequential><br />
<antcall target="packageEclipseDistributables" /><br />
<ant antfile="${equinox.build.configs}/equinox.prov/run.xml" /><br />
<antcall target="packageRepos" /><br />
<antcall target="packageEquinoxDistributables" /><br />
<br />
<antcall target="apiTooling" /><br />
<antcall target="publishEclipse" /><br />
<antcall target="publishEquinox" /><br />
</sequential><br />
<antcall target="testEclipse" /><br />
</parallel><br />
<antcall target="publishRSS" /> <br />
</pre><br />
<br />
If the build fails, you can restart it by commenting out the sections of the build that have already completed in the main target. Then cd to /builds/I201004291549/org.eclipse.releng.eclipsebuilder, for example, and run sh command.txt. This will allow the build to continue. Alternatively, you can add a cron entry to restart the build. For example,<br />
<br />
<pre><br />
36 20 * * 3 cd /builds/I201004291549/org.eclipse.releng.eclipsebuilder; sh command.txt<br />
</pre><br />
<br />
It's better to start the build from cron if you need the tests to proceed without having to retain your ssh session.</div>Kmoir.ca.ibm.com