Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Platform-releng-faq-obsolete"
m (→Is the Eclipse platform build process completely automated?) |
(→How do I use the releng plugin?) |
||
(53 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
'''Eclipse Platform Release Engineering FAQ''' | '''Eclipse Platform Release Engineering FAQ''' | ||
− | + | == Basic overview of Platform releng tasks and troubleshooting tips<br> == | |
+ | |||
+ | [[Platform-releng-basics|Platform releng basics]] | ||
+ | |||
==What hardware comprises the platform-releng build infrastructure?== | ==What hardware comprises the platform-releng build infrastructure?== | ||
Line 17: | Line 20: | ||
##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack | ##eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack | ||
#Other | #Other | ||
− | ## | + | ##minsky (database server with apache derby installed to store performance test results), (cvs test server) |
− | + | ||
##eclipsebuildserv (build machine) Linux machine on far back table of the room | ##eclipsebuildserv (build machine) Linux machine on far back table of the room | ||
+ | |||
+ | The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory. | ||
==Is the Eclipse platform build process completely automated?== | ==Is the Eclipse platform build process completely automated?== | ||
Line 28: | Line 32: | ||
*we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks | *we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks | ||
− | Fetch and build scripts are generated automatically by the org.eclipse.pde.build bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder. | + | Fetch and build scripts are generated automatically by the [[PDE/Build | org.eclipse.pde.build]] bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder. |
After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated. | After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated. | ||
Line 35: | Line 39: | ||
==How do I use the releng plugin?== | ==How do I use the releng plugin?== | ||
− | The RelEng Plug-in is included in the Eclipse builds and is available at the bottom of the download page. | + | The RelEng Plug-in repository is included in the Eclipse builds and is available at the bottom of the download page. You can install the new software from the downloaded repository or simply unzip the the 'features' and 'plugins' folder into the 'dropins' folder of your Eclipse install. |
− | This plug-in provides features that will help with the Eclipse development process | + | This plug-in provides the following features that will help with the Eclipse development process: |
− | ''' | + | |
− | *'''Release''' to the Team menu. This action will | + | *'''Fix Copyright''' to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at. |
− | * | + | *'''Copyright Tool''' preference page. |
+ | *'''POM Version Tool''' preference page. The [[Version_Numbering#pom.xml_Versions|POM Version tool]] will compare the artifact version in a pom.xml file at the root of the project to the plug-in version in the manifest. If the two versions do not match, a problem marker will be created. The option is set to 'Ignore' by default. Eclipse bundle developers should set this to ''''Error''''. | ||
+ | |||
+ | The following functionality is no longer used for Eclipse build inputs that moved to Git/CBI/Maven: | ||
+ | <div style="font-size:smaller; margin-left:2em"> | ||
+ | *'''Release''' to the Team menu. This action will tag selected projects with the specified version '''and''' update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done. | ||
+ | *'''Load Map Projects''' to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded. | ||
*'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify. | *'''Tag Map Projects''' to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify. | ||
*'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files. | *'''Compared with Released''' to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files. | ||
*'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files. | *'''Replace with Released''' to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files. | ||
− | + | </div> | |
==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?== | ==How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?== | ||
Line 54: | Line 64: | ||
==How do I automate my builds with PDE Build?== | ==How do I automate my builds with PDE Build?== | ||
− | [ | + | See the [[PDE/Build]] wiki page. |
==How do I contribute a JUnit Plug-in Test to the build?== | ==How do I contribute a JUnit Plug-in Test to the build?== | ||
Line 78: | Line 88: | ||
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page. | The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page. | ||
+ | |||
+ | ==Testing the 4.2 build on build.eclipse.org== | ||
+ | |||
+ | Just documenting the current issues here (March 21, 2012) | ||
+ | |||
+ | The builder to build the 4.2 primary build is the R4_2_primary branch of | ||
+ | org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder | ||
+ | |||
+ | The branch of the map files with the bundles to build are R4_HEAD. There are a | ||
+ | few weeks out of date. This is just for testing. | ||
+ | |||
+ | e4Build@build:/shared/eclipse/e4> pwd | ||
+ | /shared/eclipse/e4 | ||
+ | e4Build@build:/shared/eclipse/e4> ls masterBuildKim.sh | ||
+ | masterBuildKim.sh | ||
+ | |||
+ | Is the script that invokes the build. It needs to be refactored a bit. Also, | ||
+ | the bits that tag the repos have been commented out because I don't want to | ||
+ | pollute the the 4.2 maps until we have the other issues worked out. | ||
+ | |||
+ | In our regular 3.8 stream builds we have scripts in the bootstrap script that | ||
+ | update the tags in the map files repo. The | ||
+ | eclipse.platform.releng.maps\org.eclipse.releng\tagging\repositories.txt file | ||
+ | lists the appropriate branches to look for each repo. | ||
+ | |||
+ | These scripts update the maps with a timestamp in the tags field of the | ||
+ | appropriate project. This is disabled for now for testing purposes. | ||
+ | |||
+ | Anyways, the issues and notes | ||
+ | 1) The current 4.2 stream org.eclipse.platform.doc.isv bundle copies over some | ||
+ | content from the 3.x stream of this same bundle. This needs to be deleted once | ||
+ | we move to 4.2 build primary | ||
+ | 2) update.core has been added added to sdk.tests feature because some of the | ||
+ | tests still depend on it. | ||
+ | 3) update.tests.core has been removed for org.eclipse.sdk.tests feature because | ||
+ | these tests aren't relevant anymore | ||
+ | 4) The 4.2 versions sdk and platform features have parts commented in their | ||
+ | build.properties that are commented out. These need to be uncommented and | ||
+ | released when we move to the 4.2 build primary so the appropriate source | ||
+ | bundles are produced. I've temporarily uncommented them and tagged them for | ||
+ | the R4_HEAD version of org.eclipse.releng. | ||
+ | 5) The e4 build part doesn't run yet, I just build the e4 bundles needed for | ||
+ | the platform. | ||
+ | 6) Need to add the 4.2 stream tests to the sdk.tests feature and run them, | ||
+ | right now it just includes the 3.8 stream tests. | ||
+ | 7) The test results page generation also needs to be updated to include all the | ||
+ | tests. | ||
+ | 8)I use a hudson token to invoke the tests via JSON. This needs to be stored | ||
+ | somewhere secure. Right now, I just add it to the command.txt when hacking the | ||
+ | build. | ||
+ | 9) The build isn't copied to the correct location, signed or mirrored into the | ||
+ | correct repo because it's just a test build. | ||
+ | |||
+ | http://wiki.eclipse.org/Platform-releng-basics#Restarting_the_build | ||
+ | |||
==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?== | ==I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?== | ||
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure]. | The best approach is use the source build drop and modify the scripts to support that you are interested in. Then [https://bugs.eclipse.org open a bug] with product ''Platform'' and component ''Releng'' attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the [http://www.eclipse.org/eclipse/platform-releng/contributingEclipsePorts.html procedure]. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
==How does the platform team sign their builds?== | ==How does the platform team sign their builds?== | ||
Line 92: | Line 152: | ||
==How long does the build take to complete?== | ==How long does the build take to complete?== | ||
− | It takes | + | It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated. |
==When is the next build?== | ==When is the next build?== | ||
Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule]. | Please refer to the [http://www.eclipse.org/eclipse/platform-releng/buildSchedule.html build schedule]. | ||
+ | |||
+ | David Williams and John Arthorne have rights to update this calendar. | ||
==When is the next milestone?== | ==When is the next milestone?== | ||
Line 111: | Line 173: | ||
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact. | The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact. | ||
− | + | ==Scripts to promote a 3.x stream build== | |
− | ==Scripts to promote a build== | + | |
*Disable rsync | *Disable rsync | ||
Line 125: | Line 186: | ||
*Promote equinox build | *Promote equinox build | ||
− | **./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3. | + | **./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4 |
− | *Extract | + | *Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link. |
*Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours) | *Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours) | ||
− | *Send out message to eclipse-dev, platform-releng-dev@eclipse.org | + | *Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property. |
+ | |||
+ | *Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org | ||
*Enjoy post-milestone chocolate. | *Enjoy post-milestone chocolate. | ||
+ | |||
+ | ==Scripts to promote a 4.x stream build== | ||
+ | |||
+ | ssh to build.eclipse.org as pwebster | ||
+ | |||
+ | ./promote4x.sh ${buildId} ${milestonename} | ||
+ | |||
+ | Example | ||
+ | ./promote4x.sh I20110916-1615 M2 | ||
+ | |||
+ | update the index.html to reflect the new build | ||
+ | |||
+ | /home/data/httpd/download.eclipse.org/eclipse/downloads/index.html | ||
+ | <br>/home/data/httpd/download.eclipse.org/e4/downloads/index.html | ||
+ | |||
+ | ==How to hide a build to allow it to sync to the mirrors== | ||
+ | |||
+ | kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd | ||
+ | |||
+ | /home/data/httpd/download.eclipse.org/eclipse/downloads | ||
+ | |||
+ | php eclipse3x.php > eclipse3x.html | ||
+ | |||
+ | update | ||
+ | |||
+ | kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php | ||
+ | kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html | ||
+ | |||
+ | to point to eclipse3x.html instead of eclipse3x.php | ||
+ | |||
+ | revert the changes when the build is ready to release | ||
==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?== | ==I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?== | ||
Line 144: | Line 238: | ||
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds. | Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds. | ||
− | ==How do I run a test build on the hudson install | + | ==How do I run a test build on the hudson install at eclipse.org ?== |
Login to this web page with your committer id and password. | Login to this web page with your committer id and password. | ||
− | https:// | + | https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/ |
− | On the left hand side, select "Build Now". Your build will be available in | + | On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/). |
− | (/builds/transfer/files/ | + | |
− | This | + | This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393 |
− | https://bugs.eclipse.org/bugs/show_bug.cgi?id= | + | |
==How do I run the JUnit tests used in the build?== | ==How do I run the JUnit tests used in the build?== | ||
− | With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/ | + | With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can [http://download.eclipse.org/eclipse/downloads/drops4/R-4.2.1-201209141800/automatedtesting.html follow the instructions] associated with this zip to run the JUnit tests on the platform of your choice. |
==How do you run the tests on the Windows machine via rsh== | ==How do you run the tests on the Windows machine via rsh== | ||
Line 181: | Line 273: | ||
Baseline tests are run from a branch of the builder. | Baseline tests are run from a branch of the builder. | ||
+ | *3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng | ||
+ | *3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng | ||
+ | *3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng | ||
*3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng | *3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng | ||
*3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng | *3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng | ||
Line 188: | Line 283: | ||
Refer to the "Raw data and Stats" link for a specific test. | Refer to the "Raw data and Stats" link for a specific test. | ||
− | For instance for 3.3.1.1 results, [http:// | + | For instance for 3.3.1.1 results, [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/performance.php click here] |
− | Then look at the [http:// | + | Then look at the [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/org.eclipse.jdt.ui.php jdt ui tests]. |
− | Then click on a [http:// | + | Then click on a [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp().html specific test]. |
Then click "Raw data and Stats". | Then click "Raw data and Stats". | ||
− | You will see the data for [http:// | + | You will see the data for [http://archive.eclipse.org/eclipse/downloads/drops/R-3.3.1.1-200710231652/performance/eclipseperflnx1_R3.3/org.eclipse.jdt.ui.tests.performance.views.CleanUpPerfTest.testSortMembersCleanUp()_raw.html previous builds]. |
==How do I run the performance tests from the zips provided on the download page?== | ==How do I run the performance tests from the zips provided on the download page?== | ||
Line 220: | Line 315: | ||
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt. | The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt. | ||
+ | The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine. | ||
+ | ==How to regenerate performance results from build artifacts== | ||
− | + | This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine. | |
− | + | ||
− | + | *cd to build directory on build machine | |
− | + | *cd org.eclipse.releng.eclipsebuilder | |
+ | *apply patch to builder in bug [[https://bugs.eclipse.org/bugs/attachment.cgi?id=118705 256297]] | ||
+ | *rerun generation on hacked builder | ||
+ | *on build machine | ||
+ | **at now | ||
+ | **cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt | ||
+ | **ctrl d | ||
− | |||
− | The | + | The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt. |
+ | The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine. | ||
+ | ==How performance tests are invoked in the build== | ||
+ | |||
+ | Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml | ||
+ | |||
+ | <pre><target name="testAll" unless="skip.tests"> | ||
+ | <waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"> | ||
+ | <and> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /> | ||
+ | <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /> | ||
+ | </and> | ||
+ | </waitfor> | ||
+ | |||
+ | <property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /> | ||
+ | <antcall target="configure.team.cvs.test" /> | ||
+ | |||
+ | <!--replace buildid in vm.properties for JVM location settings--> | ||
+ | <replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /> | ||
+ | |||
+ | <antcall target="addnoperfmarker" /> | ||
+ | |||
+ | <parallel> | ||
+ | <antcall target="test-JUnit" /> | ||
+ | <antcall target="test-performance" /> | ||
+ | </parallel> | ||
+ | </target> | ||
+ | </pre> | ||
+ | |||
+ | The test-performance target in the buildAll.xml looks like this | ||
+ | |||
+ | <pre> | ||
+ | <target name="test-performance" unless="skip.performance.tests"> | ||
+ | |||
+ | <echo message="Starting performance tests." /> | ||
+ | <property name="dropLocation" value="${postingDirectory}" /> | ||
+ | <ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /> | ||
+ | <antcall target="generatePerformanceResults" /> | ||
+ | </target> | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs | ||
+ | |||
+ | <pre> | ||
+ | <target name="performanceTests"> | ||
+ | |||
+ | <condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"> | ||
+ | <isset property="performance.base" /> | ||
+ | </condition> | ||
+ | |||
+ | <property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /> | ||
+ | <mkdir dir="${testResults}" /> | ||
+ | |||
+ | <parallel> | ||
+ | <antcall target="test"> | ||
+ | <param name="tester" value="${basedir}/win32xp-perf" /> | ||
+ | <param name="cfgKey" value="win32xp-perf" /> | ||
+ | <param name="markerName" value="eclipse-win32xp-perf-${buildId}" /> | ||
+ | </antcall> | ||
+ | <antcall target="test"> | ||
+ | <param name="tester" value="${basedir}/win32xp2-perf" /> | ||
+ | <param name="cfgKey" value="win32xp2-perf" /> | ||
+ | <param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /> | ||
+ | </antcall> | ||
+ | <antcall target="test"> | ||
+ | <param name="tester" value="${basedir}/rhelws5-perf" /> | ||
+ | <param name="sleep" value="120" /> | ||
+ | <param name="cfgKey" value="rhelws5-perf" /> | ||
+ | <param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /> | ||
+ | </antcall> | ||
+ | <antcall target="test"> | ||
+ | <param name="tester" value="${basedir}/sled10-perf" /> | ||
+ | <param name="sleep" value="300" /> | ||
+ | <param name="cfgKey" value="sled10-perf" /> | ||
+ | <param name="markerName" value="eclipse-sled10-perf-${buildId}" /> | ||
+ | </antcall> | ||
+ | </pre> | ||
+ | |||
+ | This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh. | ||
+ | |||
+ | <pre> | ||
+ | #Windows XP | ||
+ | win32xp-perf=epwin2 | ||
+ | win32xp2-perf=epwin3 | ||
+ | |||
+ | #RedHat Enterprise Linux WS 5 | ||
+ | rhelws5-perf=eplnx2 | ||
+ | |||
+ | #sled 10 | ||
+ | sled10-perf=eplnx1 | ||
+ | </pre> | ||
+ | |||
+ | This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database. | ||
+ | |||
+ | When the performances tests complete, the results are generated. | ||
+ | |||
+ | <pre> | ||
+ | <target name="generatePerformanceResults"> | ||
+ | <mkdir dir="${buildDirectory}/${buildLabel}/performance" /> | ||
+ | <mkdir dir="${postingDirectory}/${buildLabel}/performance" /> | ||
+ | <taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /> | ||
+ | <condition property="configArgs" value="-ws gtk -arch ppc"> | ||
+ | <equals arg1="${os.arch}" arg2="ppc64" /> | ||
+ | </condition> | ||
+ | <condition property="configArgs" value="-ws gtk -arch x86"> | ||
+ | <equals arg1="${os.arch}" arg2="i386" /> | ||
+ | </condition> | ||
+ | <property name="configArgs" value="" /> | ||
+ | |||
+ | <java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"> | ||
+ | <arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator | ||
+ | -current ${buildId} | ||
+ | -jvm ${eclipse.perf.jvm} | ||
+ | -print | ||
+ | -output ${postingDirectory}/${buildLabel}/performance/ | ||
+ | -config eplnx1,eplnx2,epwin2,epwin3 | ||
+ | -dataDir ${postingDirectory}/../../data/v38 | ||
+ | -config.properties ${eclipse.perf.config.descriptors} | ||
+ | -scenario.pattern org.eclipse.%.test%" /> | ||
+ | <!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed... | ||
+ | -baseline ${eclipse.perf.ref} | ||
+ | -baseline.prefix R-3.4_200806172000 | ||
+ | --> | ||
+ | <!-- add this argument to list above when there are milestone builds to highlight | ||
+ | -highlight.latest 3.3M1_ | ||
+ | --> | ||
+ | <env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /> | ||
+ | <sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /> | ||
+ | </java> | ||
+ | </pre> | ||
+ | Two important things about generating performance results: | ||
+ | # xhost+ needs to be enabled in a terminal of the user generating the performance results | ||
+ | # The derby database needs to be running on port 1528 (There is an init script for that) | ||
+ | # X has to be open on the machine generating the results or you'll get the "SWT - no more handles error" | ||
==Why should I package plugins and features as jars?== | ==Why should I package plugins and features as jars?== | ||
Line 287: | Line 528: | ||
-Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre | -Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre | ||
-listener org.eclipse.releng.build.listeners.EclipseBuildListener" | -listener org.eclipse.releng.build.listeners.EclipseBuildListener" | ||
− | |||
Line 293: | Line 533: | ||
Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature. | Refer to [http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/update_standalone.html running update manager from command line] in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature. | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
==I have questions about PDEBuild - where should I go for answers?== | ==I have questions about PDEBuild - where should I go for answers?== | ||
Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup. | Consult the [[PDEBuild |PDE Build faq ]] or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup. | ||
Line 362: | Line 576: | ||
==Eclipse Release checklist== | ==Eclipse Release checklist== | ||
− | [[ | + | [[Eclipse/Release_checklist | Release checklist]] |
==New JUnit/performance machine deployment checklist== | ==New JUnit/performance machine deployment checklist== | ||
Line 394: | Line 608: | ||
==Process to release builds to Simultaneous Release== | ==Process to release builds to Simultaneous Release== | ||
− | *Check out org.eclipse. | + | *Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto. |
− | *Copy | + | *For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon |
− | + | *Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo. | |
− | * | + | *Watch build message to ensure coordinated release build completes successfully. |
− | + | ||
− | |||
− | |||
− | |||
− | |||
− | |||
==How do I verify the signatures of packed jars on an update site?== | ==How do I verify the signatures of packed jars on an update site?== | ||
Line 421: | Line 630: | ||
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage | Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage | ||
The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily | The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily | ||
− | |||
− | |||
==Platform Releng Helios Plan== | ==Platform Releng Helios Plan== | ||
[[Platform-releng/Helios | Helios plan]] | [[Platform-releng/Helios | Helios plan]] | ||
+ | |||
+ | ==Platform Indigo Plan== | ||
+ | [[Platform-releng/Indigo | Indigo plan]] | ||
==How to assemble the deltapack using the p2 slicer== | ==How to assemble the deltapack using the p2 slicer== | ||
Line 474: | Line 684: | ||
[http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build] | [http://wiki.eclipse.org/Platform-releng-avoid-build-breakage Avoid breaking the build] | ||
− | |||
− | + | ||
+ | == Submitting content to the Helios build == | ||
+ | |||
+ | The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine. | ||
+ | |||
+ | /builds/transfer/files/updates/3.6milestones/ | ||
+ | |||
+ | I just ls the features directory of the current milestone, update the build files, then commit. | ||
== Invoking the signing process from the command line == | == Invoking the signing process from the command line == | ||
Line 492: | Line 708: | ||
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre | export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre | ||
+ | |||
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin | export DERBY_HOME=/builds/db-derby-10.4.2.0-bin | ||
cd /builds/db-derby-10.4.2.0-bin/bin | cd /builds/db-derby-10.4.2.0-bin/bin | ||
+ | |||
connect 'jdbc:derby://localhost:1528/perfDb36'; | connect 'jdbc:derby://localhost:1528/perfDb36'; | ||
+ | |||
+ | == Are the Eclipse and Equinox projects converting from CVS to git? == | ||
+ | |||
+ | This was completed in the fall of 2012. Here's the planning document | ||
+ | |||
+ | [[Platform-releng/Juno_Git_Migration | Juno Git Migration]] | ||
+ | |||
+ | Here is the umbrella [https://bugs.eclipse.org/bugs/show_bug.cgi?id=345479 bug 345479] that was used to coordinate the migration. | ||
+ | |||
+ | == How do you incorporate the p2MirrorURL into your repo at build time == | ||
+ | |||
+ | See this excellent document [[Equinox/p2/p2.mirrorsURL | p2.mirrorsURL]] written by Stephan Herrmann. | ||
+ | |||
+ | The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org | ||
+ | bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto. | ||
==Eclipsecon presentations== | ==Eclipsecon presentations== | ||
Line 549: | Line 782: | ||
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated | Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated | ||
− | |||
[[Category:FAQ]] | [[Category:FAQ]] | ||
[[Category:Releng]] | [[Category:Releng]] | ||
+ | [[Category:Eclipse_Platform_Releng| ]] |
Revision as of 08:23, 15 September 2014
Eclipse Platform Release Engineering FAQ
Contents
- 1 Basic overview of Platform releng tasks and troubleshooting tips
- 2 What hardware comprises the platform-releng build infrastructure?
- 3 Is the Eclipse platform build process completely automated?
- 4 What's the difference between an integration and nightly build?
- 5 How do I use the releng plugin?
- 6 How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?
- 7 What is the latest version of org.eclipse.releng.basebuilder?
- 8 How do I automate my builds with PDE Build?
- 9 How do I contribute a JUnit Plug-in Test to the build?
- 10 How do I contribute an example plug-in to the build?
- 11 Testing the 4.2 build on build.eclipse.org
- 12 I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?
- 13 How does the platform team sign their builds?
- 14 How long does the build take to complete?
- 15 When is the next build?
- 16 When is the next milestone?
- 17 I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?
- 18 Scripts to promote a 3.x stream build
- 19 Scripts to promote a 4.x stream build
- 20 How to hide a build to allow it to sync to the mirrors
- 21 I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?
- 22 I'm looking for an older build but can't find them on the download page?
- 23 How do I run a test build on the hudson install at eclipse.org ?
- 24 How do I run the JUnit tests used in the build?
- 25 How do you run the tests on the Windows machine via rsh
- 26 Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)
- 27 How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?
- 28 How do I set up performance tests?
- 29 How do I find data for old performance results on existing build page?
- 30 How do I run the performance tests from the zips provided on the download page?
- 31 Process to implement a new baseline
- 32 How to regenerate performance results from build artifacts
- 33 How to regenerate performance results from build artifacts
- 34 How performance tests are invoked in the build
- 35 Why should I package plugins and features as jars?
- 36 Why should I use qualifiers?
- 37 How do I incorporate qualifiers into the build process?
- 38 How can I run the update manager from a command line to mirror a remote site?
- 39 I have questions about PDEBuild - where should I go for answers?
- 40 Debugging pde build with missing dependancies
- 41 I'd like to incoporate source bundles in my build, how do I do it?
- 42 Are there RSS feeds for builds?
- 43 If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.
- 44 Troubleshooting test failures
- 45 I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?
- 46 Eclipse Release checklist
- 47 New JUnit/performance machine deployment checklist
- 48 Process to release builds to Simultaneous Release
- 49 How do I verify the signatures of packed jars on an update site?
- 50 Where are the p2 update sites for the Eclipse Project?
- 51 How do I use the p2 zipped repos on the build page to provision my install or a pde target?
- 52 Where can I download foundation libraries?
- 53 Platform Releng Helios Plan
- 54 Platform Indigo Plan
- 55 How to assemble the deltapack using the p2 slicer
- 56 How to verify the packed jars in a repo
- 57 Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4
- 58 How to avoid breaking the build
- 59 Submitting content to the Helios build
- 60 Invoking the signing process from the command line
- 61 Connecting to the derby database using ij
- 62 Are the Eclipse and Equinox projects converting from CVS to git?
- 63 How do you incorporate the p2MirrorURL into your repo at build time
- 64 Eclipsecon presentations
- 65 Useful reference documents
- 66 Who are the platform-releng committers?
- 67 As the holiday season approaches, what gifts are appropriate for your favourite release engineer?
- 68 What does the platform-releng team do when they're not running the build farm and wrangling bugs?
- 69 What factors contribute to the low percentage of women participating in free software/open source?
- 70 Deprecated contents from Eclipse Platform Release Engineering FAQ
Basic overview of Platform releng tasks and troubleshooting tips
What hardware comprises the platform-releng build infrastructure?
The eclipse platform build infrastructure consists of
- junit test machines,
- ejwin1 (winxp with 1.5 vm) 5 on KVM on table
- ejwin2 (winxp with 1.6 vm 6 on KBM on table
- lb6mac (macosx Leopard) mac on table on LHS of lab
- ejlnx1 (RHEL 5.3 with 1.5 vm) 5 on large KVM in rack
- ejlnx2 (RHEL 5.3 with 1.6 vm) E on large KVM in rack
- performance machines
- epwin2 (winxp with 1.5 vm) G on large KVM in rack
- epwin3 (winxp with 1.6 vm) 7 on large KVM in rack
- eplnx1 (SLES 10 with 1.6 vm) H on large KVM in rack
- eplnx2 (RHEL 5.2 with 1.6 vm) F on large KVM in rack
- Other
- minsky (database server with apache derby installed to store performance test results), (cvs test server)
- eclipsebuildserv (build machine) Linux machine on far back table of the room
The linux machines should be logged in as kmoir. The windows machines should be logged in as the default user. If the build machines don't have the correct user logged in, the ui tests will fail. All the test machines have 2 x 3.00Ghz processors and 3GB of memory.
Is the Eclipse platform build process completely automated?
Yes, the Eclipse build process starts with a cron job on our main build machine that initiates a shell script that checks out the builder projects.
- org.eclipse.releng.eclipsebuilder -> scripts to build each type of drop
- org.eclipse.releng.basebuilder -> a subset of eclipse plugins required to run the build.
- we also use several small cvs projects on an internal server that store login credentials, publishing information and jdks
Fetch and build scripts are generated automatically by the org.eclipse.pde.build bundle in org.eclipse.releng.basebuilder. Fetch scripts specify the version of code to retrieve from the repository. Build scripts are also generated automatically to compile all java code, determine compile order and manage dependencies. We create a master feature of all the non-test bundles and features used in the build. This feature is signed and packed, and the we create a p2 repo from the packed master feature. Metadata for the products is published to the repository. We then use the director application to provision the contents of the SDKs etc to different directories and then zip or tar them up. We also use custom build scripts that you can see in org.eclipse.releng.eclipsebuilder. After the SDKs are built, the automated junit and performance testing begins. Tests occur over ssh for Linux machines, and rsh for Windows machines. Each component team contributes their own tests. Once the tests are completed,the results are copied back to the build machine. Also, the images for the performance tests are generated.
What's the difference between an integration and nightly build?
With integration builds, we specify a version of the plug-in to retrieve in the map files. (org.eclipse.releng/maps). With nightly builds, we retrieve the plug-ins specified in the map files from the HEAD stream. The types of builds we run are described here http://download.eclipse.org/eclipse/downloads/build_types.html.
How do I use the releng plugin?
The RelEng Plug-in repository is included in the Eclipse builds and is available at the bottom of the download page. You can install the new software from the downloaded repository or simply unzip the the 'features' and 'plugins' folder into the 'dropins' folder of your Eclipse install.
This plug-in provides the following features that will help with the Eclipse development process:
- Fix Copyright to the Resource Perspective Projects context menu. Select one or more projects in the Resource Perspective. This action will sanity check the copyright notices in all the *.java and *.properties files. Copyrights will be updated automatically where the tool deems appropriate. A copyright.log file will be written to the workspace directory noting odd conflicts that need to be looked at.
- Copyright Tool preference page.
- POM Version Tool preference page. The POM Version tool will compare the artifact version in a pom.xml file at the root of the project to the plug-in version in the manifest. If the two versions do not match, a problem marker will be created. The option is set to 'Ignore' by default. Eclipse bundle developers should set this to 'Error'.
The following functionality is no longer used for Eclipse build inputs that moved to Git/CBI/Maven:
- Release to the Team menu. This action will tag selected projects with the specified version and update the appropriate loaded *.map files with the version. The user must have the *.map files loaded in their workspace and the use must commit the map file changes to the repository when done.
- Load Map Projects to the Team menu. Select one or more *.map file and this action will load the projects listed in the *.map file into your workspace. Naturally the versions specified in the *.map file will be loaded.
- Tag Map Projects to the Team menu. Select one or more *Map files and this action will tag the projects listed in the *Map files with a tag you specify.
- Compared with Released to the Compare menu. Compare the selected projects with the versions referenced in the currently loaded map files.
- Replace with Released to the Replace menu. Replace the selected projects with the versions referenced in the currently loaded map files.
How do I run a build using the scripts in org.eclipse.releng.eclipsebuilder?
This document provides an overview of how to build eclipse components using the releng scripts. However, it is really out of date. Your best bet is to use the source build to recompile the SDK.
What is the latest version of org.eclipse.releng.basebuilder?
See this document which describes correct tag of org.eclipse.releng.basebuilder for use in your builds.
How do I automate my builds with PDE Build?
See the PDE/Build wiki page.
How do I contribute a JUnit Plug-in Test to the build?
- The tests need to be contributed as one or more plugins that use the org.eclipse.test automated testing framework. JUnit tests can also be written as performance tests. Click here for the latest instructions.
- Open bug for for PMC approval for new plug-in projects on dev.eclipse.org:/cvsroot/eclipse. CC webmaster on the bug once the plug-ins are created. Or you may want to include the new plug-ins under an existing project and in this case, the webmasters don't need to get involved.
- Add the new test plugin to the org.eclipse.releng project in the appropriate map file for your component.
- Install the org.eclipse.releng tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit map file.
- Open a bug against platform releng to add the plugins to the build process. Include the following information:
- the plug-in id(s)
- the plug-ins that contain a test.xml
- update the build.properties to include the test.xml
- additional steps to setup the tests outside of installing the test plugins and framework in an Eclipse SDK
The Eclipse release engineering team will update the appropriate feature and scripts to build and run the new tests.
- Example (with UI)
- Example (headless)
How do I contribute an example plug-in to the build?
Once you have your plug-in ready and available in CVS, install the org.eclipse.releng.tools plug-in, and use the "Release" action in the context menu of the test plug-in project to update and commit the map file.
Next, open a bug against platform releng with the plug-in's id.
The Eclipse release engineering team will update the org.eclipse.sdk.examples-feature to include the new tests. They will also update org.eclipse.releng.eclipsebuilder/all/publishing/templateFiles/testManifest.xml to indicate the zip that should get a red X if there are compile errors associated with that plug-in on the build page.
Testing the 4.2 build on build.eclipse.org
Just documenting the current issues here (March 21, 2012)
The builder to build the 4.2 primary build is the R4_2_primary branch of org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder
The branch of the map files with the bundles to build are R4_HEAD. There are a few weeks out of date. This is just for testing.
e4Build@build:/shared/eclipse/e4> pwd /shared/eclipse/e4 e4Build@build:/shared/eclipse/e4> ls masterBuildKim.sh masterBuildKim.sh
Is the script that invokes the build. It needs to be refactored a bit. Also, the bits that tag the repos have been commented out because I don't want to pollute the the 4.2 maps until we have the other issues worked out.
In our regular 3.8 stream builds we have scripts in the bootstrap script that update the tags in the map files repo. The eclipse.platform.releng.maps\org.eclipse.releng\tagging\repositories.txt file lists the appropriate branches to look for each repo.
These scripts update the maps with a timestamp in the tags field of the appropriate project. This is disabled for now for testing purposes.
Anyways, the issues and notes 1) The current 4.2 stream org.eclipse.platform.doc.isv bundle copies over some content from the 3.x stream of this same bundle. This needs to be deleted once we move to 4.2 build primary 2) update.core has been added added to sdk.tests feature because some of the tests still depend on it. 3) update.tests.core has been removed for org.eclipse.sdk.tests feature because these tests aren't relevant anymore 4) The 4.2 versions sdk and platform features have parts commented in their build.properties that are commented out. These need to be uncommented and released when we move to the 4.2 build primary so the appropriate source bundles are produced. I've temporarily uncommented them and tagged them for the R4_HEAD version of org.eclipse.releng. 5) The e4 build part doesn't run yet, I just build the e4 bundles needed for the platform. 6) Need to add the 4.2 stream tests to the sdk.tests feature and run them, right now it just includes the 3.8 stream tests. 7) The test results page generation also needs to be updated to include all the tests. 8)I use a hudson token to invoke the tests via JSON. This needs to be stored somewhere secure. Right now, I just add it to the command.txt when hacking the build. 9) The build isn't copied to the correct location, signed or mirrored into the correct repo because it's just a test build.
http://wiki.eclipse.org/Platform-releng-basics#Restarting_the_build
I would like to recompile eclipse on a platform that is not currently supported. What is the best approach to this? How can I ensure that the community can use my work?
The best approach is use the source build drop and modify the scripts to support that you are interested in. Then open a bug with product Platform and component Releng attaching the patches to the scripts that were required to build the new drop. If you are interested in contributing a port to eclipse, here is the procedure.
How does the platform team sign their builds?
See Platform-releng-signedbuild
How long does the build take to complete?
It takes three hours and 20 minutes for all the drops to be produced. JUnit (eight hours) and performance tests (twelve hours) run in parallel after the windows, mac, linux and sdk tests drops have been built. It takes another two hours for the performance results to be generated.
When is the next build?
Please refer to the build schedule.
David Williams and John Arthorne have rights to update this calendar.
When is the next milestone?
Please refer to the Eclipse Platform Project Plan.
I noticed that you promoted an integration build to a milestone build. The integration build had test failures but the milestone builds doesn't show any. Why is this?
See bug 134413.
We have a number of tests that intermittently fail. The reasons are
- issues with the tests themselves
- tests subject to timing issues
- tests that fail intermittently due to various conditions
The component teams are always trying to fix their tests but unfortunately they are still some issues. When we promote a build to a milestone, we rerun the tests that failed. Many pass on the second time because the tests initially failed due to a timing issue or intermittent condition. Or a team will have a broken test that doesn't warrant a rebuild for a milestone. In that case, the releng team sprinkles pixie dust over the build page to erase the red Xes, but leaves the appropriate build failures intact.
Scripts to promote a 3.x stream build
- Disable rsync
- cd /home/users/releng/buildTools/extraTools on eclipsebuildserv
- See parameters... ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild
- Usage: PromoteBuild <oldBuildDirectory> <newTypePrefix> <newName>
- Promote eclipse milestone build
- ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/downloads/drops/I20080925-0800 S 3.5M4
- Promote equinox build
- ./builds/tools/jdk/linux/jdk1.4/bin/java -cp promote33.jar PromoteBuild.PromoteBuild /builds/transfer/files/master/equinox/drops/I20080925-0800 S 3.4M4
- Extract new and noteworthy zip to S-... directory on eclipsebuildserv. Update index.php to include New and Noteworthy link.
- Enable rsync, wait until build is synched to eclipse.org (Takes 1-2 hours)
- Copy child repo to appropriate milestone or release repo. Update compositeArtifacts.jar and compositeContent.jar to include properties to point to the child repo. Update the child repo's artifacts.jar to include the correct link the the p2.mirrorsURL property.
- Send out message to eclipse-dev, platform-releng-dev@eclipse.org, equinox-dev@eclipse.org
- Enjoy post-milestone chocolate.
Scripts to promote a 4.x stream build
ssh to build.eclipse.org as pwebster
./promote4x.sh ${buildId} ${milestonename}
Example ./promote4x.sh I20110916-1615 M2
update the index.html to reflect the new build
/home/data/httpd/download.eclipse.org/eclipse/downloads/index.html
/home/data/httpd/download.eclipse.org/e4/downloads/index.html
How to hide a build to allow it to sync to the mirrors
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> pwd
/home/data/httpd/download.eclipse.org/eclipse/downloads
php eclipse3x.php > eclipse3x.html
update
kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi createIndex4x.php kmoir@build:/home/data/httpd/download.eclipse.org/eclipse/downloads> vi index.html
to point to eclipse3x.html instead of eclipse3x.php
revert the changes when the build is ready to release
I'd like to run a build with the version of the code that was used for build X? How do I know what versions of code in the repository were used?
There are several ways to approach this issue.
- Look at the directory.txt linked from the top of every build page. The directory.txt concatenates all the map files used in a build into a single file. Please note that every nightly build runs from HEAD. Therefore it is impossible to replicate a nightly build.
- In addition, with every maintenance and integration build, the dev.eclipse.org:/cvsroot/eclipse org.eclipse.releng project, which contains the map files for each project, is tagged with a version corresponding to that build. For instance, the version I20051018-0932 corresponds to the integration build of the same name. Please note that the builder projects, (org.eclipse.releng.eclipsebuilder and org.eclipse.releng.basebuilder) are also tagged during an integration build with the corresponding build name, but are not reflected in the directory.txt because they are not included in the eclipse distribution itself.
- Finally, after each release the releng team tags all projects from the map files used in the release. So, all the projects used to construct version 3.1.1 are tagged R_3_1_1.
I'm looking for an older build but can't find them on the download page?
Check out archive site the http://archive.eclipse.org/eclipse/downloads for vintage builds.
How do I run a test build on the hudson install at eclipse.org ?
Login to this web page with your committer id and password.
https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-equinox-test-N/
On the left hand side, select "Build Now". Your test nightly build will be available in about three hours, depending on the load average on the eclipse Hudson slaves. This build isn't signed or packed to save time. Once the build is finished, select "Build Artifacts" to to look at the SDKs etc (/builds/transfer/files/bogus/download/drops/${buildId}) or p2 repo (/builds/transfer/files/repo/).
This build will also run tests on three platforms (Windows 7, Linux, and Mac OS X10.6). These take six to eight hours to complete. There are still a few remaining issues respect to running several test suites on a some platforms. For more details, see bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=295393
How do I run the JUnit tests used in the build?
With every build, there is a eclipse-Automated-Tests-${buildId}.zip file. You can follow the instructions associated with this zip to run the JUnit tests on the platform of your choice.
How do you run the tests on the Windows machine via rsh
To run the windows tests from the build machine,
rsh ejwin2 start /min /wait c:\\buildtest\\N20081120-2000\\eclipse-testing\\testAll.bat c:\\buildtest\\N20081120-2000\\eclipse-testing winxplog.txt
Where do you find the Java SDK for the Mac (This is required to run the JDT debug JUnit tests)
The default Mac install only includes a JRE, not a JDK. The JDK is listed under Apple Connect under J2SE 5.0 Release 5 Developer Documentation. Once the .dmg is installed, you should see a src.jar under /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
How do I set up a test cvs server to run the cvs tests portion of the JUnit tests?
This document outlines the steps required to setup a test CVS repository on Linux.
How do I set up performance tests?
Refer to the performance tests how-to or the even better Performance First talk at Eclipse Con 2007.
Baseline tests are run from a branch of the builder.
- 3.8 builds are compared against 3.4 baselines in the perf_37x branch of org.eclipse.releng
- 3.7 builds are compared against 3.4 baselines in the perf_36x branch of org.eclipse.releng
- 3.6 builds are compared against 3.4 baselines in the perf_35x branch of org.eclipse.releng
- 3.5 builds are compared against 3.4 baselines in the perf_34x branch of org.eclipse.releng
- 3.4 builds are compared against 3.3 baselines in the perf_33x branch of org.eclipse.releng
- 3.3 builds are compared against 3.2 baselines in the perf_32x branch of org.eclipse.releng
How do I find data for old performance results on existing build page?
Refer to the "Raw data and Stats" link for a specific test.
For instance for 3.3.1.1 results, click here
Then look at the jdt ui tests.
Then click on a specific test.
Then click "Raw data and Stats".
You will see the data for previous builds.
How do I run the performance tests from the zips provided on the download page?
See here.
Process to implement a new baseline
Implement new performance baseline after a release.
How to regenerate performance results from build artifacts
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.
- cd to build directory on build machine
- cd org.eclipse.releng.eclipsebuilder
- apply patch to builder in bug [256297]
- rerun generation on hacked builder
- on build machine
- at now
- cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt
- ctrl d
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.
How to regenerate performance results from build artifacts
This assumes that the artifacts used to run the build and the resulting build directory itself still exist on the build machine.
- cd to build directory on build machine
- cd org.eclipse.releng.eclipsebuilder
- apply patch to builder in bug [256297]
- rerun generation on hacked builder
- on build machine
- at now
- cd /builds/${buildId}/org.eclipse.releng.eclipsebuilder; sh command.txt
- ctrl d
The process needs to be executed using at or cron if you are logged in remotely because the swt libraries are needed to run the generation. If you run the process while logged in remotely via ssh, the process will fail with "No more handles". The output logs for this process are in the posting directory under buildlogs/perfgen*.txt.
The user that's running the tests on the build machine needs run "xhost +" in a terminal on the build machine.
How performance tests are invoked in the build
Performance tests run if the -skipTest or -skipPerf parameters isn't passed to the build when running it. Both JUnit and performance tests are invoked in the build by the testAll target in the org.eclipse.releng.eclipsebuilder/buildAll.xml
<target name="testAll" unless="skip.tests"> <waitfor maxwait="4" maxwaitunit="hour" checkevery="1" checkeveryunit="minute"> <and> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-Automated-Tests-${buildId}.zip.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-win32.zip.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-linux-gtk.tar.gz.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-SDK-${buildId}-macosx-cocoa.tar.gz.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-${buildId}-delta-pack.zip.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-win32.zip.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-linux-gtk.tar.gz.md5" /> <available file="${postingDirectory}/${buildLabel}/checksum/eclipse-platform-${buildId}-macosx-cocoa.tar.gz.md5" /> </and> </waitfor> <property name="cvstest.properties" value="${base.builder}/../eclipseInternalBuildTools/cvstest.properties" /> <antcall target="configure.team.cvs.test" /> <!--replace buildid in vm.properties for JVM location settings--> <replace dir="${eclipse.build.configs}/sdk.tests/testConfigs" token="@buildid@" value="${buildId}" includes="**/vm.properties" /> <antcall target="addnoperfmarker" /> <parallel> <antcall target="test-JUnit" /> <antcall target="test-performance" /> </parallel> </target>
The test-performance target in the buildAll.xml looks like this
<target name="test-performance" unless="skip.performance.tests"> <echo message="Starting performance tests." /> <property name="dropLocation" value="${postingDirectory}" /> <ant antfile="testAll.xml" dir="${eclipse.build.configs}/sdk.tests/testConfigs" target="performanceTests" /> <antcall target="generatePerformanceResults" /> </target>
This calls the testAll.xml in org.eclipse.releng.eclipsebuilder/sdk.tests/testConfigs
<target name="performanceTests"> <condition property="internalPlugins" value="../../../eclipsePerformanceBuildTools/plugins"> <isset property="performance.base" /> </condition> <property name="testResults" value="${postingDirectory}/${buildLabel}/performance" /> <mkdir dir="${testResults}" /> <parallel> <antcall target="test"> <param name="tester" value="${basedir}/win32xp-perf" /> <param name="cfgKey" value="win32xp-perf" /> <param name="markerName" value="eclipse-win32xp-perf-${buildId}" /> </antcall> <antcall target="test"> <param name="tester" value="${basedir}/win32xp2-perf" /> <param name="cfgKey" value="win32xp2-perf" /> <param name="markerName" value="eclipse-win32xp2-perf-${buildId}" /> </antcall> <antcall target="test"> <param name="tester" value="${basedir}/rhelws5-perf" /> <param name="sleep" value="120" /> <param name="cfgKey" value="rhelws5-perf" /> <param name="markerName" value="eclipse-rhelws5-perf-${buildId}" /> </antcall> <antcall target="test"> <param name="tester" value="${basedir}/sled10-perf" /> <param name="sleep" value="300" /> <param name="cfgKey" value="sled10-perf" /> <param name="markerName" value="eclipse-sled10-perf-${buildId}" /> </antcall>
This invokes the tests in parallel on the performance test machines. In this case, there is a machine.cfg file in the same directory as the above file that maps the "cfgKey" value written above to the hostname of the machine. The tests are invoked on the windows machines via rsh and on the linux machines via ssh.
#Windows XP win32xp-perf=epwin2 win32xp2-perf=epwin3 #RedHat Enterprise Linux WS 5 rhelws5-perf=eplnx2 #sled 10 sled10-perf=eplnx1
This invokes all the tests in the org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testScripts\test.xml on each machine. If the test bundle has a performance target in the test.xml, the performance tests for that machine will run. The test scripts use the values in the (for example when running on window xp) org.eclipse.releng.eclipsebuilder\eclipse\buildConfigs\sdk.tests\testConfigs\win32xp2-perf\vm.properties which specifies the database to write to as well as the port, and url of that database.
When the performances tests complete, the results are generated.
<target name="generatePerformanceResults"> <mkdir dir="${buildDirectory}/${buildLabel}/performance" /> <mkdir dir="${postingDirectory}/${buildLabel}/performance" /> <taskdef name="performanceResults" classname="org.eclipse.releng.performance.PerformanceResultHtmlGenerator" /> <condition property="configArgs" value="-ws gtk -arch ppc"> <equals arg1="${os.arch}" arg2="ppc64" /> </condition> <condition property="configArgs" value="-ws gtk -arch x86"> <equals arg1="${os.arch}" arg2="i386" /> </condition> <property name="configArgs" value="" /> <java jar="${basedir}/../org.eclipse.releng.basebuilder/plugins/org.eclipse.equinox.launcher.jar" fork="true" maxmemory="512m" error="${buildlogs}/perfgenerror.txt" output="${buildlogs}/perfgenoutput.txt"> <arg line="${configArgs} -consolelog -nosplash -data ${buildDirectory}/perf-workspace -application org.eclipse.test.performance.ui.resultGenerator -current ${buildId} -jvm ${eclipse.perf.jvm} -print -output ${postingDirectory}/${buildLabel}/performance/ -config eplnx1,eplnx2,epwin2,epwin3 -dataDir ${postingDirectory}/../../data/v38 -config.properties ${eclipse.perf.config.descriptors} -scenario.pattern org.eclipse.%.test%" /> <!-- baselines arguments are no longer necessary since bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=209322 has been fixed... -baseline ${eclipse.perf.ref} -baseline.prefix R-3.4_200806172000 --> <!-- add this argument to list above when there are milestone builds to highlight -highlight.latest 3.3M1_ --> <env key="LD_LIBRARY_PATH" value="${basedir}/../org.eclipse.releng.basebuilder" /> <sysproperty key="eclipse.perf.dbloc" value="${dbloc}" /> </java>
Two important things about generating performance results:
- xhost+ needs to be enabled in a terminal of the user generating the performance results
- The derby database needs to be running on port 1528 (There is an init script for that)
- X has to be open on the machine generating the results or you'll get the "SWT - no more handles error"
Why should I package plugins and features as jars?
See the Running Eclipse from JARs document from the Core team.
Why should I use qualifiers?
See the Versioning Guidelines
How do I incorporate qualifiers into the build process?
Update your plug-ins and features to use qualifiers as described in the previous FAQ entry.
Additional steps that the platform releng team uses to incorporate qualifiers into the build process.
For nightly builds, we set the qualifier to the buildId. For integration builds, pde build sets it to the tag in the map file for plugins, and generates feature qualifiers.
In our from org.eclipse.releng.eclipsebuilder/eclipse/buildAll.xml, forceContextQualifier is set to the buildId if is a nightly build
<condition property="forceContextQualifier" value="${buildId}"> <equals arg1="${buildType}" arg2="N" /> </condition>
Also, in the bootstrap.sh
file which kicks off our build, we specify -DgenerateFeatureVersionSuffix=true
buildCommand="$antRunner -q -buildfile buildAll.xml $mail $testBuild $compareMaps -DmapVersionTag=$mapVersionTag -DpostingDirectory=$postingDirectory -Dbootclasspath=$bootclasspath -DbuildType=$buildType -D$buildType=true -DbuildId=$buildId -DbuildLabel=$buildLabel -Dtimestamp=$timestamp -DmapCvsRoot=:ext:sdimitro@dev.eclipse.org:/cvsroot/eclipse $skipPerf $skipTest $tagMaps -DJ2SE-1.5=$bootclasspath_15 -DJ2SE-1.4=$bootclasspath -DCDC-1.0/Foundation-1.0=$bootclasspath_foundation -DlogExtension=.xml $javadoc $updateSite $sign -DgenerateFeatureVersionSuffix=true -Djava15-home=$builderDir/jdk/linuxppc/ibm-java2-ppc-50/jre -listener org.eclipse.releng.build.listeners.EclipseBuildListener"
How can I run the update manager from a command line to mirror a remote site?
Refer to running update manager from command line in Eclipse Help for the latest information. It copies the appropriate versions of features and associated plugins you specify and generates the site.xml. For instance, if you wanted to create an update site for the platform feature.
I have questions about PDEBuild - where should I go for answers?
Consult the PDE Build faq or ask on the org.eclipse.platform http://www.eclipse.org/newsgroups/ newsgroup.
Debugging pde build with missing dependancies
Breakpoint in BuildTimeSite class, in missingPlugins method, set breakpoint In display view, state.getState().getResolverErrors(state.getBundle("org.eclipse.ui.workbench", null, false)) print the errors for the bundle in question.
I'd like to incoporate source bundles in my build, how do I do it?
Are there RSS feeds for builds?
Yes, for I-Builds only. They are available here.
If I add a new plugin to a build, how do I ensure that javadoc will be included in the build.
See the adding javadoc document.
Troubleshooting test failures
If tests pass in a dev workspace but fail in the automated test harness, check to make sure that your build.properties is exporting all the necessary items; check that plug-in dependencies are correct, since the test harness environment will only include depended-upon plug-ins; and check that file references do not depend on the current working directory, since that may be different in the test harness.
To debug tests in the context of the automated test harness, add the following element to the test.xml file in your plug-in's directory in the test harness.
<property name="extraVMargs" value="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000 -Djava.compiler=NONE"/>
Then,
- create a remote debugging launch target in your dev environment
- put a breakpoint somewhere in your code
- launch the tests from the command line, using the -noclean option to preserve the modified test.xml
- launch the debug target
I would like a plaform releng committer to comment on a bug assigned to another component. What mailbox do I add to the cc list in bugzilla?
Please cc the generic mail alias platform-releng-inbox@eclipse.org. Please do not just cc your favourite release engineer using their individual email address. They might be on a beach somewhere enjoying a sunny vacation and therefore unable to comment.
Eclipse Release checklist
New JUnit/performance machine deployment checklist
Steps to take after OS install from IT
All platforms
- verify hostname is set on machine
- verify hostname is registered in DNS and both forward and reverse lookups resolve correctly
- disable screen saver
- disable all power management options so that the machine, disk, monitor etc is always on
- create c:\buildtest or /buildtest directory and ensure the build user owns it and can write to it
- install ganglia and configure to interact with appropriate eclipse build node
- verify that the machines can access all ports required for testing, port 2401,22 to cvs test server, port 1527, 1528 to database server, and access via http to download.eclipse.org, archive.eclipse.org.
- disable real time virus scanning of build directories
- ensure that machines starts as build user automatically upon reboot
Windows
- install rshd
- set rshd user equivalency to build user
- ensure rshd daemon is run in normal mode (default is hidden) and starts automatically upon startup
- ensure cvs, zip and unzip binaries are installed and update the path environment variable to include them
Linux/Mac
- copy ssh keys from build machine to test machines and ensure that ssh logins occur without user intervention
- ensure that correct mozilla/xulrunner libraries are installed in the build and match those defined in the build test scripts for SWT tests
- update /etc/hosts to include localhost
- disable iptables in chkconfig and stop the service
Process to release builds to Simultaneous Release
- Check out org.eclipse.indigo.build or org.eclipse.juno.build from /cvsroot/callisto.
- For eclipse platform, update versions in ep.b3aggrcon, for equinox equinox.b3aggrcon
- Copy child repo from integration build site or maintenance build site into the appropriate milestone or release site. Update compositeContent.jar and compositeArtifacts.jar to reflect the new child repo.
- Watch build message to ensure coordinated release build completes successfully.
How do I verify the signatures of packed jars on an update site?
See bug 193568 for the tool and instructions.
Where are the p2 update sites for the Eclipse Project?
See the list
How do I use the p2 zipped repos on the build page to provision my install or a pde target?
http://wiki.eclipse.org/Equinox/p2/Equinox_p2_zipped_repos
Where can I download foundation libraries?
Anyone can get access to the foundation 1.1 jar from the OSGi R4.1 specification at http://www.osgi.org/Download/Release4V41. They do not need to be a member, but they must fill out a form so an e-mail can be sent with special links to download the jars from OSGi.
Members can access the content or the OSGi R4.1 specification at https://www.osgi.org/members/Release41/HomePage The latest builds of the OSGi R4.2 specification can be accessed by members at https://www.osgi.org/members/Build/Daily
Platform Releng Helios Plan
Platform Indigo Plan
How to assemble the deltapack using the p2 slicer
Create a build script that looks like this. This example is built using the 3.5RC2 bundles and 3.5RC2 p2 repository.
<project default="build"> <target name="build"> <property name="repo" value="http://download.eclipse.org/eclipse/updates/3.5milestones/S-3.5RC2-200905221710" /> <property name="tempDir" value="D:\\deltapack" /> <p2.mirror source="${repo}"> <destination kind="metadata" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /> <destination kind="artifact" location="file://${tempDir}" name="RCP Delta Pack Repo" format="${repo}" /> <iu id="org.eclipse.platform.feature.group" version="" /> <iu id="org.eclipse.rcp.feature.group" version="" /> <iu id="org.eclipse.jdt.feature.group" version="" /> <iu id="org.eclipse.equinox.executable" version="" /> <slicingOptions includeOptional="false" includeNonGreedy="false" followStrict="true" followOnlyFilteredRequirements="true" /> </p2.mirror> </target> </project>
Unzip 3.5RC2 to a directory and run the script using the AntRunner.
D:\3.5RC2plugins\eclipse>java -jar plugins\org.eclipse.equinox.launcher_1.0.200.
v20090520.jar -application org.eclipse.ant.core.antRunner -buildfile -d D:\temp\build.xml
The resulting files will be in the d:\deltapack directory on your file system.
How to verify the packed jars in a repo
java -cp plugins/org.eclipse.equinox.p2.jarprocessor_1.0.100.v20090520-1905.jar org.eclipse.equinox.internal.p2.jarprocessor.verifier.Verifier -dir <tempWorkingDir> <inputfolder>
Overview of JUnit4 changes released to Eclipse test framework in Eclipse 3.6M4
How to avoid breaking the build
Submitting content to the Helios build
The helios build files are in org.eclipse.helios.build in /cvsroot/callisto. We update the ep.build and equinox.build files to reflect the versions of the components each milestones. This is the location of the milestones directory on the build machine.
/builds/transfer/files/updates/3.6milestones/
I just ls the features directory of the current milestone, update the build files, then commit.
Invoking the signing process from the command line
- Login via ssh to build.eclipse.org and ensure you're in the signer's group (groups myuserid | grep signers)
- Copy the file you want to sign to the signing staging area. In our case, it's /home/data/httpd/download-staging.priv/eclipse
- Invoke the signing script as follows /usr/bin/sign /home/data/httpd/download-staging.priv/eclipse/mybundles.zip mail /home/data/httpd/download-staging.priv/eclipse/myOutput
Note: You're bundles don't have to be in zip file. You can also just sign an individual jar.
You'll receive an email when the signing is complete and available in /home/data/httpd/download-staging.priv/eclipse/myOutput
You can also tail -f /tmp/signing/signer.log to see the output as the bundles are signed.
Connecting to the derby database using ij
export JAVA_HOME=/builds/Cloudscape_10.0/ibm-jre-n142p/jre
export DERBY_HOME=/builds/db-derby-10.4.2.0-bin
cd /builds/db-derby-10.4.2.0-bin/bin
connect 'jdbc:derby://localhost:1528/perfDb36';
Are the Eclipse and Equinox projects converting from CVS to git?
This was completed in the fall of 2012. Here's the planning document
Here is the umbrella bug 345479 that was used to coordinate the migration.
How do you incorporate the p2MirrorURL into your repo at build time
See this excellent document p2.mirrorsURL written by Stephan Herrmann.
The p2.mirrorsURL should be added to your metadata so that p2 will see the list of available mirrors to choose during installation. Mirrors = less eclipse.org bandwidth utilization = happy Eclipse.org webmasters. Always try to keep the sysadmins happy is my motto.
Eclipsecon presentations
2010
- From build to assembly to deployment: Using p2 to facilitate agile development http://www.slideshare.net/irbull/p2-introduction
2007
2006
- From developer to Download: A Tour of the Platform Build Factory, an updated version is available here.
2005
Best releng practices from our Eclipsecon 2005 poster:
- Automate the build process from end-to-end and automate early.
- Use PDE Build in Eclipse to generate build scripts.
- Automate JUnit testing and performance monitoring.
- Automate build notifications (email).
- Use gentle humiliation to encourage developers to contribute carefully. (Clown nose technique.)
- Ensure stability in the build process by thorough testing of builder changes.
- Schedule builds at regular intervals and enforce rebuild policies.
- Use map files in a central CVS repository.
- Build on Linux.
- Have fun!
Useful reference documents
- Build and Test Automation for plug-ins and features http://eclipse.org/articles/Article-PDE-Automation/automation.html
- Eclipse's culture of shipping http://www.artima.com/lejava/articles/eclipse_culture.html
- The Eclipse Way: Proccesses that Adapt http://eclipsecon.org/2005/presentations/econ2005-eclipse-way.pdf
- Building
Who are the platform-releng committers?
Kim Moir
As the holiday season approaches, what gifts are appropriate for your favourite release engineer?
We enjoy fine chocolate, Shaan, and apple juice.
What does the platform-releng team do when they're not running the build farm and wrangling bugs?
We herd cats. s/eds/ibm/g
http://video.google.com/videoplay?docid=4057591681481453187
What factors contribute to the low percentage of women participating in free software/open source?
Cambridge University recently completed a study on this very topic with interesting results.
Here is another one written by Val Henson, a Linux kernel committer.
Deprecated contents from Eclipse Platform Release Engineering FAQ
Old content from the faq can be found here http://wiki.eclipse.org/Platform-releng-faq-deprecated