Difference between revisions of "E4/Builds"

From Eclipsepedia

< E4
Jump to: navigation, search
(Milestone Builds)
(CVS)
(36 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= Builds =
+
= Builds =
  
== Milestone Builds ==
+
In e4 (like in the other projects under the Eclipse top-level project), we produce regular builds. One important kind of build is called ''integration build'' - usually run once a week to produce a version that is good enough to for purposes of integrating with other teams' work, and for self-hosting. Integration builds do not just take whatever is in HEAD at the time of running the build - given the amount of changes produced by the committers, this is not predictable enough. Instead, each team needs to make a ''build submission'', which basically amounts to tagging their projects at a known good state with a tag that then gets put into the team's map file. The map files are located in so-called releng projects. These projects are checked out from HEAD by the build system, followed by checking out the referenced projects using the specified tags. This means that if a team forgets to submit to the integration build, the build will just check out whatever was submitted last time.
  
<table border="1">
+
To make a build submission, one team member would first check out (or update) all the team's projects, and run their tests locally. Then, to tag all changed projects, and updating the map file to contain those tags, they would use the ''releng tools'' (see below for installation instructions). By using the releng tools instead of tagging and updating the map file manually, the process is made less error-prone, and as an additional benefit, change log information will be generated for you at the same time. The change log is produced by searching for bugzilla ids in commit comments, and doing a lookup in the Bugzilla system to retrieve the bugs' descriptions. Sending the change log information to the mailing list makes everyone aware of what the team has worked on, and can help later if you find regressions and would like to be able to map back to what has been changed and why.
<tr>
+
<th>Date</th>
+
<th>Build</th>
+
</tr>
+
  
<tr>
+
== Plug-in Versioning  ==
<td>Tuesday, 08:30 EDT</td>
+
<td>M3 Candidate</td>
+
</tr>
+
  
<tr>
+
Our current release, [http://download.eclipse.org/e4/downloads/drops/R-0.11-201106201631/index.html e4 0.11], has all of our plugins marked as (Incubation) and new plugins should have a version 0.11.0.qualifier.
<td>Tuesday, 19:30 EDT</td>
+
<td>M3 Candidate</td>
+
</tr>
+
  
<tr>
+
Now that HEAD is open for our 0.12 release we are following the standard [[Version Numbering]] guidelines. For example, for existing bundles:  
<td>Wednesday, 08:30 EDT</td>
+
<td>M3 Candidate</td>
+
</tr>
+
  
<tr>
+
*if you contribute bug fixes for a bundle, please increment the number to '''0.9.1.qualifier'''
<td>Wednesday, 19:30 EDT</td>
+
*if you change the public API or public classes (to add new API or to refactor API for the 0.12 release), please increment the number to '''0.10.0.qualifier'''
<td>M3 Candidate</td>
+
</tr>
+
  
<tr>
+
For new bundles, please mark them as '''0.12.0.qualifier'''
<td>Thursday, 08:30 EDT</td>
+
<td>M3 Candidate</td>
+
</tr>
+
  
<tr>
+
When we graduate bundles so they are ready for Eclipse 4.0 they will retain their &lt;1.0 version.  When we vet the API and promote it, they will change their version number to '''1.0.0.qualifier'''
<td>Thursday, 19:30 EDT</td>
+
<td>M3 Candidate</td>
+
</tr>
+
  
 +
== Milestone Builds  ==
  
<tr>
+
{| border="1"
<td>Friday, 08:30 EDT</td>
+
|-
<td>Final build towards M3 - we'll need a go on this build</td>
+
! Milestone
</tr>
+
! Delivery Date
 
+
|-
</table>
+
| 0.11
 +
| 06/22/2011
 +
|}
  
== Integration Builds ==
+
== Integration Builds ==
  
 
{| border="1"
 
{| border="1"
|+ E4 Integration Builds
+
|+ E4 Integration Builds  
 
|- align="left"
 
|- align="left"
! Date !! Time !! Frequency
+
! Date  
 +
! Time  
 +
! Frequency
 
|-
 
|-
| Thursdays
+
| Thursdays  
| 19:30 EST
+
| 22:00 EST  
 
| Weekly
 
| Weekly
 
|}
 
|}
Line 61: Line 45:
 
[http://download.eclipse.org/e4/downloads/ e4 Downloads Page]
 
[http://download.eclipse.org/e4/downloads/ e4 Downloads Page]
  
== Submitting For the Build ==
+
== Submitting For the Build ==
  
In General:
+
In General:  
  
'''Checking in code''': The checkin comment should at a minimum include "bug #", and preferably be of the form "bug # summary"
+
'''Checking in code''': The checkin comment should at a minimum include "bug #", and preferably be of the form "bug # summary"  
  
For the build:
+
For the build:  
  
'''Tool''': Teams should use the "Eclipse Releng Tools" to update their releng project for each build. The releng tools can be installed from the update sites already included in 3.5.
+
'''Tool''': Teams should use the "Eclipse Releng Tools" to update their releng project for each build. The releng tools can be installed from the update sites already included in 3.7 and/or 4.1. If you cannot find it, try [http://download.eclipse.org/eclipse/updates/3.7 http://download.eclipse.org/eclipse/updates/3.7].
  
[[Image:Install-Releng-Tools.png]]
+
[[Image:Install-Releng-Tools.png]]  
  
'''Tags''': tags should be of the form v<date>, so v20081120 or v20081120-1300
+
Releng tools are in the eclipse updates site now that Helios is released and can be installed from the director app as well:
  
'''Procedure''':
+
eclipse/eclipse -application org.eclipse.equinox.p2.director \
<ol>
+
-consoleLog -noSplash \
<li>check out your team releng project from e4/releng</li>
+
  -repository http://download.eclipse.org/eclipse/updates/3.7 \
<li>use the psf files to check out your core plugins, tests, and examples</li>
+
  -installIU org.eclipse.releng.tools.feature.group
<li>Run Team>Release from the context menu:</li>
+
<ol>
+
<li>Select your team's releng project, like '''org.eclipse.e4.ui.releng'''.<br />
+
[[Image:Release-1.png]]
+
<br />
+
</li>
+
<li>Select the projects you want to release from your map file. <br />
+
[[Image:Release-2.png]]
+
<br />
+
</li>
+
<li>You can review the changed projects in this pane. Make sure 'Generate Build Notes' is selected. <br />
+
[[Image:Release-3.png]]
+
<br />
+
</li>
+
<li>Copy out the build note changes so you can send an email to ''e4-dev@eclipse.org'',
+
with a subject that starts with [releng].  You don not have to select a build
+
notes file to continue. <br />
+
[[Image:Release-4.png]]
+
<br />
+
</li>
+
<li>Enter the release tag for you build. This would normally be the build time and date with a v in front, '''v20090101-1930''' <br />
+
  [[Image:Release-5.png]]
+
<br />
+
</li>
+
<li>You can review the changes to your map file here, or simply continue. <br />
+
[[Image:Release-6.png]]
+
<br />
+
</li>
+
<li>Commit the changes so they will be picked up by the I-build</li>
+
</ol>
+
</ol>
+
  
  
 +
'''Tags''': tags should be of the form v&lt;date&gt;, so v20081120 or v20081120-1300
  
'''Versioning Plugins''': Teams should try and follow the standard versioning rules posted at [[Version_Numbering]].  If you need to copy over an existing eclipse project to make changes, please increment the minor number by 100.  i.e. 3.5.0 becomes 3.105.0.  When you need to break API, then update the major number. i.e. 3.105.0 becomes 4.0.0
+
'''Procedure''':  
  
== Build Infrastructure ==
+
#check out your team releng project from e4/releng
 +
#use the psf files to check out your core plugins, tests, and examples
 +
#Run Team&gt;Release from the context menu:
 +
##Select your team's releng project, like '''org.eclipse.e4.ui.releng'''.<br>[[Image:Release-1.png]] <br>
 +
##Select the projects you want to release from your map file. <br>[[Image:Release-2.png]] <br>
 +
##You can review the changed projects in this pane. Make sure 'Generate Build Notes' is selected. <br>[[Image:Release-3.png]] <br>
 +
##Copy out the build note changes so you can send an email to ''e4-dev@eclipse.org'', with a subject that starts with [releng]. You don not have to select a build notes file to continue. <br>[[Image:Release-4.png]] <br>
 +
##Enter the release tag for you build. This would normally be the build time and date with a v in front, '''v20090101-1930''' <br>[[Image:Release-5.png]] <br>
 +
##You can review the changes to your map file here, or simply continue. <br>[[Image:Release-6.png]] <br>
 +
##Commit the changes so they will be picked up by the I-build
  
We run our builds on build.eclipse.org with a local userid, '''e4Build'''.
+
<br>
  
The builds run and publish tests results.
+
'''Versioning Plugins''': Teams should try and follow the standard versioning rules posted at [[Version Numbering]]. If you need to copy over an existing eclipse project to make changes, please increment the minor number by 100. i.e. 3.5.0 becomes 3.105.0. When you need to break API, then update the major number. i.e. 3.105.0 becomes 4.0.0
  
=== Build Scripts ===
+
== Build Infrastructure  ==
The main build project is in e4/releng/org.eclipse.e4.builder.  We use 2 main bash scripts for building.  '''start.sh''' will build a feature and generate repo data if necessary.  '''runBuild.sh''' controls the build.  It checks out the eclipse basebuilder and the e4 builder project.  It requests the building of all of the e4 features, and starts the testing, and publishes the results to http://download.eclipse.org/e4/downloads/
+
  
'''start.sh''' runs the PDE builder using org.eclipse.e4.builder/builder/general as the ${builder} property, and builds the different  features requested by '''runBuild.sh'''.
+
We run our builds on build.eclipse.org (a ppc linux machine) with a local userid, '''e4Build'''.  
  
=== Machine Setup ===
+
=== Build Scripts  ===
  
There are a number of directories that need to be available to run a build.
+
The main build project is in <code>e4/releng/org.eclipse.e4.builder</code>. It contains the <code>masterBuild.sh</code> script that can be modified to run a test build or a real build (tagged and uploaded to download.eclipse.org). It also contains the PDE build directory, <code>e4/releng/org.eclipse.e4.builder/builder/general</code>, that builds our master feature. Our PDE build directory also contains modified code/XML to support using p2 to run our automated tests.  
  
<ul>
+
<br>
<li>/opt/public/common: the java installs can be appropriate for your architecture.
+
 
<pre>
+
=== Machine Setup  ===
apache-ant-1.7.0/        ibm-java2-ppc-50@      ibm-java-jdk-ppc-60@
+
 
 +
There are a number of directories that need to be available to run a build.
 +
 
 +
*/opt/public/common: the java installs can be appropriate for your architecture. <pre>apache-ant-1.7.0/        ibm-java2-ppc-50@      ibm-java-jdk-ppc-60@
 
flex_sdk_3.2.0.3794_mpl/  ibm-java2-ppc-50-old@  ibm-java-ppc-60@
 
flex_sdk_3.2.0.3794_mpl/  ibm-java2-ppc-50-old@  ibm-java-ppc-60@
 
ibm-java2-142@            ibm-java2-ppc64-50@
 
ibm-java2-142@            ibm-java2-ppc64-50@
</pre></li>
+
</pre>
<li>/shared/eclipse/e4/flex_sdk_3.2.0.3794_mpl</li>
+
*/shared/eclipse/e4/flex_sdk_3.2.0.3794_mpl  
<li>/shared/eclipse/e4/build/e4</li>
+
*/shared/eclipse/e4/build/e4
</ul>
+
  
To run the tests, we need a virtual X server on :8:
+
To run the tests, we need a virtual X server on&nbsp;:8:  
  
  Xvfb :8 -screen 0 1280x1024x24 -auth auth.cfg
+
  Xvfb&nbsp;:8 -screen 0 1280x1024x24 -auth auth.cfg
 +
metacity --display=:8.0 --replace --sm-disable
  
Where auth.cfg contains at least:
+
Where auth.cfg contains at least:  
  
 
  localhost
 
  localhost
  
 +
== Build Requirements  ==
  
=== TODO ===
+
The build is currently controlled from the <code>masterBuild.sh</code> script. There are a number of dependencies that make this a linux only build, and the machine has to be set up the same way as build.eclipse.org. This section will capture the requirements/process so that all of the needed steps are executed. See bug [https://bugs.eclipse.org/bugs/show_bug.cgi?id=281407 releng - Run a build from any platform] for any discussions about this section.
  
*sign the build.
+
=== Setup ===
*generate a zip with 2 projects, org.eclipse.swt.e4 and org.eclispe.swt.e4.jcl, that can be imported into a workspace in an eclipse with the ActionScript tools installed.
+
*Work with the [[Common_Build_Infrastructure]] to host our builds.
+
*Teams need to generate source features
+
*Update the build email to include
+
**did the tests pass or fail, and
+
**for a compile error, where should people look
+
  
= CVS =
+
In [http://dev.eclipse.org/viewcvs/index.cgi/e4/releng/org.eclipse.e4.builder/scripts/masterBuild.sh?view=co&content-type=text/plain masterBuild.sh]
  
The resources, ui, and swt teams now have plugins in their team area in the e4 project.  Our project is under :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse
+
We have a set of variables that are needed in order to make the system go.  
  
 +
{| border="1"
 +
|-
 +
! Variable
 +
! Comments
 +
|-
 +
| writableBuildRoot
 +
|
 +
The base directory that contains all build related files except the JREs.
  
== Resources ==
+
|-
 +
| projRelengBranch
 +
|
 +
The branch to check out the releng project (this contains subprojects that contain all our maps). Default is HEAD.
  
The resources CVS structure is:
+
|-
 +
| projRoot
 +
|
 +
The CVS root used to check out the releng project. For anonymous test builds, it is <code>:pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse</code>
  
*e4
+
|-
**org.eclipse.e4.resources
+
| arch
***bundles
+
|
***doc
+
Build architecture, currently x86 or ppc.
***examples
+
***features
+
***tests
+
**releng
+
***org.eclipse.e4.resources.releng
+
  
The '''org.eclipse.e4.resources.releng''' contains the map files for building as well as Team Project Sets that should be used for checking out the projects to be worked on.
+
|-
 +
| supportDir
 +
|
 +
The directory that contains the eclipse install used to build (by default we use the eclipse basebuilder), and usually <code>org.eclipse.e4.builder</code> directory that we launch the build against.
  
Resources currently has 4 features.  2 feature patches for the plugins (one for rcp, one for platform), a feature that contains the 2 patches, and a test feature for the tests.
+
|-
 +
| builderDir
 +
|
 +
Points to the <code>org.eclipse.e4.builder</code> so that it can be in a different place than the <code>supportDir</code>.
  
=== Setup/Restrictions ===
+
|-
 +
| builddate
 +
|
 +
Build date in the format YYYYMMDD, ex: 20090625
  
Resources can currently update a 3.5 SDK I build and run.
+
|-
 +
| buildtime
 +
|
 +
Build time in the 24 hour clock format HHMM, ex: 1930
  
== UI ==
+
|-
 +
| tagMaps
 +
|
 +
Controls if we tag the map files after we check out our releng project.
  
The UI CVS structure is:
+
|-
 +
| publishDir
 +
|
 +
Controls if we should upload the finished build. Currently this is the form of an SCP destination directory.
  
*e4
+
|-
**org.eclipse.e4.ui
+
| basebuilderBranch
***bundles
+
|
***doc
+
The tag that determines which base builder we use for the build.
***examples
+
***features
+
***tests
+
**releng
+
***org.eclipse.e4.ui.releng
+
  
The '''org.eclipse.e4.ui.releng''' contains the map files for building as well as Team Project Sets that should be used for checking out the projects to be worked on.
+
|-
 +
| eclipseIBuild
 +
|
 +
The ID of the eclipse build we use as a target platform. With the changes to use a zipped repo for the eclipse SDK, this ID is only used to generate URLs on the download page. It should potentially be used to collect the correct p2 repo for the eclipse/equinox SDK.
  
UI currently has one main feature for the modelled workbench and one feature for the demos. The build master feature, org.eclipse.e4.master, is also in UI for now.
+
|-
 +
| javaHome
 +
|
 +
The java that is used to launch JVMs from the masterBuild.sh script.
  
=== Setup/Restrictions ===
+
|-
 +
| buildTimestamp
 +
|
 +
The date and time together: 20090625-1930
  
UI currently requires EMF 2.4.1 or 2.5.0 in installed on 3.5 in order to run.  They can be installed from the update site included with the I builds.
+
|-
 +
| buildDir
 +
|
 +
The directory to build in, by default <code>$writableBuildRoot/build/e4/downloads/drops/4.0.0</code>.
  
== SWT ==
+
|-
 +
| targetDir
 +
|
 +
A directory that contains "target platforms" to work against. It contains a zip download cache, any eclipse target platforms, an untransformed repo directory (a directory with zipped repos to use in the build), and the runnable repo generated by the build.
  
The SWT CVS structure is:
+
|-
 +
| buildDirectory
 +
|
 +
The directory for the current build, <code>$buildDir/I$buildTimestamp</code> by default.
  
*e4
+
|-
**org.eclipse.e4.swt
+
| testDir
***bundles
+
|
***doc
+
The directory used for running the automated tests.
***examples
+
***features
+
***tests
+
**releng
+
***org.eclipse.e4.swt.releng
+
  
The '''org.eclipse.e4.swt.releng''' contains the map files for building as well as Team Project Sets that should be used for checking out the projects to be worked on.
+
|-
 +
| buildResults
 +
|
 +
The directory for the build results. This will contain the index.html, compile logs, p2 repo, platform zips, etc. Defaults to <code>$buildDirectory/I$buildTimestamp</code>. So for example, <code>.../4.0.0/I20090625-1930/I20090625-1930</code>.
  
SWT has 2 features, one for the ActionScript tools and another for the ActionScript tests.
+
|-
 +
| relengBaseBuilderDir
 +
|
 +
The root directory for the eclipse install that will run the build. By default we use the eclipse basebuilder.
  
=== Setup/Restrictions ===
+
|-
 +
| writableBuildRoot
 +
|
 +
|}
  
To update to the ActionScript tools you must first download and install the Open Source Flex SDK.
+
<br>
  
'''Setup Flex environment:'''
+
In [http://dev.eclipse.org/viewcvs/index.cgi/e4/releng/org.eclipse.e4.builder/builder/general/build.properties?view=co general/build.properties]
  
# Download and install the Adobe Open Source Flex SDK (available from http://opensource.adobe.com). NOTE: The path where you install the Flex SDK must contain no spaces. This is due to a bug in FCSH.
+
We also code/re-generate a number of properties in the PDE builder.  
# update your eclipse.ini file and add another line at the end, -Dflex.sdk=&lt;your path to the installed sdk&gt;
+
  
Then you can update to the ActionScript tools.  To work on the tools themselves, you must also set the FLEX_SDK classpath variable in Window>Preferences>Java>Build Path>Classpath Variables to point to the sdk location.
+
{| border="1"
 +
|-
 +
! Variable
 +
! Comments
 +
|-
 +
| buildType
 +
|
 +
Set to '''I''', but not currently used to do more than create the matching build results directory and correct build "tag".
 +
 
 +
|-
 +
| eclipseBaseURL
 +
|
 +
The URL of the eclipse SDK base used for this build. This was used to download targets and automated test frameworks, but is now only used to generate links in the download page.
 +
 
 +
|-
 +
| emfBaseURL
 +
|
 +
Now only used to generate links in the download page.
 +
 
 +
|-
 +
| gefBaseURL
 +
|
 +
Now only used to generate links in the download page.
 +
 
 +
|-
 +
| wstBaseURL
 +
|
 +
Now only used to generate links in the download page.
 +
 
 +
|-
 +
| JAVA60_HOME
 +
|
 +
A base java 6 install, set to <code>/opt/public/common/ibm-java-jdk-ppc-60</code> to match build.eclipse.org.
 +
 
 +
|-
 +
| JAVA50_64_HOME
 +
|
 +
A base java 5 64-bit install, set to <code>/opt/public/common/ibm-java2-ppc64-50</code> to match build.eclipse.org.
 +
 
 +
|-
 +
| JAVA50_HOME
 +
|
 +
A base java 5 install, set to <code>/opt/public/common/ibm-java2-ppc-50</code> to match build.eclipse.org.
 +
 
 +
|-
 +
| JAVA14_HOME
 +
|
 +
A base java 1.4.2 install, set to <code>/opt/public/common/ibm-java2-142</code> to match build.eclipse.org.
 +
 
 +
|-
 +
| JavaSE-1.6
 +
|
 +
Definition of the jars that make up the JSE 1.6 BREE.
 +
 
 +
|-
 +
| J2SE-1.5
 +
|
 +
Definition of the jars that make up the J2SE 1.5 BREE.
 +
 
 +
|-
 +
| J2SE-1.4
 +
|
 +
Definition of the jars that make up the J2SE 1.4 BREE.
 +
 
 +
|-
 +
| flex.sdk
 +
|
 +
A property to the base directory for a flex SDK, needed to build the actionScript supporting tools.
 +
 
 +
|-
 +
| repoBaseLocation
 +
|
 +
This property points to the directory of zipped repos to be used by the built. It is processed through the repo2runnable ant task.
 +
 
 +
|-
 +
| transformedRepoLocation
 +
|
 +
The runnable repo produced by repo2runnable. Defaults to ${repoBaseLocation}-trans
 +
 
 +
|}
 +
 
 +
=== Manual Setup ===
 +
 
 +
A lot of the directory structures must be set up before you can run a build. Perhaps the directory structure needs to be flattened so that it can be more easily created.
 +
 
 +
The basebuilder (eclipse install used to run the build) and org.eclipse.e4.builder project have to be on disk somewhere. A build updates them to the latest required versions and links to them using symlinks so that they can be used to run the build, drive the build, and provide the testing instructions.
 +
 
 +
The runnable repo is used for the build as well as to produce the platform zips and install supporting bundles for the automated tests. The untransformed repo location must be created, and populated with repos downloaded by hand. Currently, the eclipse SDK repo must be hand rolled as well. It would be nice to have the runnable repo populated by downloading the needed zipped repos.
 +
 
 +
The other possibility is to use the repo2runnable task directly and not through <code>repoBaseLocation</code>. Then multiple repo locations can be specified either as local files or URLs (to zipped repos or p2 repos).
 +
 
 +
There must be at least 3 JREs/JDKs installed on a machine used to build, 1.4.2, 1.5, and 1.6. A bundles BREE instructs PDE build which set of libraries to use during compile.
 +
 
 +
=== Steps  ===
 +
 
 +
In [http://dev.eclipse.org/viewcvs/index.cgi/e4/releng/org.eclipse.e4.builder/scripts/masterBuild.sh?view=co&content-type=text/plain masterBuild.sh]
 +
 
 +
The steps we go through to run a build are controlled by the <code>masterBuild.sh</code> script.
 +
 
 +
{| border="1"
 +
|-
 +
! Step
 +
! Description
 +
|-
 +
| realBuildProperties
 +
|
 +
Set all of the build properties so that the build will be tagged and copied over to download.eclipse.org when complete.
 +
 
 +
|-
 +
| testBuildProperties
 +
|
 +
Set the build properties to run a test build. Complete with tests, this is not tagged or copied to download.eclipse.org. Only one of <code>testBuildProperties</code> or <code>realBuildProperties</code> should be set.
 +
 
 +
|-
 +
| commonProperties
 +
|
 +
Just common properties that are set for both types of builds.
 +
 
 +
|-
 +
| updateBaseBuilder
 +
|
 +
Update the basebuilder to the correct version. Not done in certain types of test builds, like when using the basebuilder checked out into your workspace.
 +
 
 +
|-
 +
| updateE4Builder
 +
|
 +
Get the latest of the templates and general builder. This is not done in certain types of test builds, when testing changes that are not checked in.
 +
 
 +
|-
 +
| updateBaseBuilderInfo
 +
|
 +
Collect info on the basebuilder to run the build. This is done in such a way that the launcher jar can be used to launch other eclipse applications (like the publisher or director) as part of the PDE build.
 +
 
 +
|-
 +
| buildMasterFeature
 +
|
 +
This runs the PDE build. It runs repo2runnable, builds all of the features and plugins in e4 and generates the e4 p2 repository, and generates platform zips.
 +
 
 +
|-
 +
| copyCompileLogs
 +
|
 +
Script that generates an html page that links to any compile logs generated during the compile.
 +
 
 +
|-
 +
| generateRepoHtml
 +
|
 +
Generates a default index for the repo URL ... to stop 404 complaints when people click on it in their browser.
 +
 
 +
|-
 +
| generateSwtZip
 +
|
 +
Generates the 2 SWT action script projects, that can be imported into the workspace for use with the ActionScript tools. Checks out 2 projects, fiddles with some of the text files, and then zips them up into the results directory.
 +
 
 +
|-
 +
| runTheTests
 +
|
 +
Copy the modified test.xml/runtests (linux) scripts to the testing directory along with one of the generated platform zips. Run the automated tests on the virtual X machine, Xvfb.
 +
 
 +
|}
  
 +
= CVS  =
  
To work on org.eclipse.swt, org.eclipse.swt.e4.jcl, org.eclipse.swt.e4.examples, and org.eclipse.swt.examples you need to create a Linked Resource variable, WORKSPACE, that points to your current workspace root, from Window>Preferences>General>Workspace>Linked Resources. Also if .classpath_flex exists it must be copied to the .classpath entry.
+
The resources, ui, and swt teams now have plugins in their team area in the e4 project. Our project is under&nbsp;:pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse - module e4

Revision as of 10:58, 22 June 2011

Contents

Builds

In e4 (like in the other projects under the Eclipse top-level project), we produce regular builds. One important kind of build is called integration build - usually run once a week to produce a version that is good enough to for purposes of integrating with other teams' work, and for self-hosting. Integration builds do not just take whatever is in HEAD at the time of running the build - given the amount of changes produced by the committers, this is not predictable enough. Instead, each team needs to make a build submission, which basically amounts to tagging their projects at a known good state with a tag that then gets put into the team's map file. The map files are located in so-called releng projects. These projects are checked out from HEAD by the build system, followed by checking out the referenced projects using the specified tags. This means that if a team forgets to submit to the integration build, the build will just check out whatever was submitted last time.

To make a build submission, one team member would first check out (or update) all the team's projects, and run their tests locally. Then, to tag all changed projects, and updating the map file to contain those tags, they would use the releng tools (see below for installation instructions). By using the releng tools instead of tagging and updating the map file manually, the process is made less error-prone, and as an additional benefit, change log information will be generated for you at the same time. The change log is produced by searching for bugzilla ids in commit comments, and doing a lookup in the Bugzilla system to retrieve the bugs' descriptions. Sending the change log information to the mailing list makes everyone aware of what the team has worked on, and can help later if you find regressions and would like to be able to map back to what has been changed and why.

Plug-in Versioning

Our current release, e4 0.11, has all of our plugins marked as (Incubation) and new plugins should have a version 0.11.0.qualifier.

Now that HEAD is open for our 0.12 release we are following the standard Version Numbering guidelines. For example, for existing bundles:

  • if you contribute bug fixes for a bundle, please increment the number to 0.9.1.qualifier
  • if you change the public API or public classes (to add new API or to refactor API for the 0.12 release), please increment the number to 0.10.0.qualifier

For new bundles, please mark them as 0.12.0.qualifier

When we graduate bundles so they are ready for Eclipse 4.0 they will retain their <1.0 version. When we vet the API and promote it, they will change their version number to 1.0.0.qualifier

Milestone Builds

Milestone Delivery Date
0.11 06/22/2011

Integration Builds

E4 Integration Builds
Date Time Frequency
Thursdays 22:00 EST Weekly

e4 Downloads Page

Submitting For the Build

In General:

Checking in code: The checkin comment should at a minimum include "bug #", and preferably be of the form "bug # summary"

For the build:

Tool: Teams should use the "Eclipse Releng Tools" to update their releng project for each build. The releng tools can be installed from the update sites already included in 3.7 and/or 4.1. If you cannot find it, try http://download.eclipse.org/eclipse/updates/3.7.

Install-Releng-Tools.png

Releng tools are in the eclipse updates site now that Helios is released and can be installed from the director app as well:

eclipse/eclipse -application org.eclipse.equinox.p2.director \
-consoleLog -noSplash \
-repository http://download.eclipse.org/eclipse/updates/3.7 \
-installIU org.eclipse.releng.tools.feature.group


Tags: tags should be of the form v<date>, so v20081120 or v20081120-1300

Procedure:

  1. check out your team releng project from e4/releng
  2. use the psf files to check out your core plugins, tests, and examples
  3. Run Team>Release from the context menu:
    1. Select your team's releng project, like org.eclipse.e4.ui.releng.
      Release-1.png
    2. Select the projects you want to release from your map file.
      Release-2.png
    3. You can review the changed projects in this pane. Make sure 'Generate Build Notes' is selected.
      Release-3.png
    4. Copy out the build note changes so you can send an email to e4-dev@eclipse.org, with a subject that starts with [releng]. You don not have to select a build notes file to continue.
      Release-4.png
    5. Enter the release tag for you build. This would normally be the build time and date with a v in front, v20090101-1930
      Release-5.png
    6. You can review the changes to your map file here, or simply continue.
      Release-6.png
    7. Commit the changes so they will be picked up by the I-build


Versioning Plugins: Teams should try and follow the standard versioning rules posted at Version Numbering. If you need to copy over an existing eclipse project to make changes, please increment the minor number by 100. i.e. 3.5.0 becomes 3.105.0. When you need to break API, then update the major number. i.e. 3.105.0 becomes 4.0.0

Build Infrastructure

We run our builds on build.eclipse.org (a ppc linux machine) with a local userid, e4Build.

Build Scripts

The main build project is in e4/releng/org.eclipse.e4.builder. It contains the masterBuild.sh script that can be modified to run a test build or a real build (tagged and uploaded to download.eclipse.org). It also contains the PDE build directory, e4/releng/org.eclipse.e4.builder/builder/general, that builds our master feature. Our PDE build directory also contains modified code/XML to support using p2 to run our automated tests.


Machine Setup

There are a number of directories that need to be available to run a build.

  • /opt/public/common: the java installs can be appropriate for your architecture.
    apache-ant-1.7.0/         ibm-java2-ppc-50@      ibm-java-jdk-ppc-60@
    

flex_sdk_3.2.0.3794_mpl/ ibm-java2-ppc-50-old@ ibm-java-ppc-60@ ibm-java2-142@ ibm-java2-ppc64-50@

  • /shared/eclipse/e4/flex_sdk_3.2.0.3794_mpl
  • /shared/eclipse/e4/build/e4

To run the tests, we need a virtual X server on :8:

Xvfb :8 -screen 0 1280x1024x24 -auth auth.cfg
metacity --display=:8.0 --replace --sm-disable 

Where auth.cfg contains at least:

localhost

Build Requirements

The build is currently controlled from the masterBuild.sh script. There are a number of dependencies that make this a linux only build, and the machine has to be set up the same way as build.eclipse.org. This section will capture the requirements/process so that all of the needed steps are executed. See bug releng - Run a build from any platform for any discussions about this section.

Setup

In masterBuild.sh

We have a set of variables that are needed in order to make the system go.

Variable Comments
writableBuildRoot

The base directory that contains all build related files except the JREs.

projRelengBranch

The branch to check out the releng project (this contains subprojects that contain all our maps). Default is HEAD.

projRoot

The CVS root used to check out the releng project. For anonymous test builds, it is :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse

arch

Build architecture, currently x86 or ppc.

supportDir

The directory that contains the eclipse install used to build (by default we use the eclipse basebuilder), and usually org.eclipse.e4.builder directory that we launch the build against.

builderDir

Points to the org.eclipse.e4.builder so that it can be in a different place than the supportDir.

builddate

Build date in the format YYYYMMDD, ex: 20090625

buildtime

Build time in the 24 hour clock format HHMM, ex: 1930

tagMaps

Controls if we tag the map files after we check out our releng project.

publishDir

Controls if we should upload the finished build. Currently this is the form of an SCP destination directory.

basebuilderBranch

The tag that determines which base builder we use for the build.

eclipseIBuild

The ID of the eclipse build we use as a target platform. With the changes to use a zipped repo for the eclipse SDK, this ID is only used to generate URLs on the download page. It should potentially be used to collect the correct p2 repo for the eclipse/equinox SDK.

javaHome

The java that is used to launch JVMs from the masterBuild.sh script.

buildTimestamp

The date and time together: 20090625-1930

buildDir

The directory to build in, by default $writableBuildRoot/build/e4/downloads/drops/4.0.0.

targetDir

A directory that contains "target platforms" to work against. It contains a zip download cache, any eclipse target platforms, an untransformed repo directory (a directory with zipped repos to use in the build), and the runnable repo generated by the build.

buildDirectory

The directory for the current build, $buildDir/I$buildTimestamp by default.

testDir

The directory used for running the automated tests.

buildResults

The directory for the build results. This will contain the index.html, compile logs, p2 repo, platform zips, etc. Defaults to $buildDirectory/I$buildTimestamp. So for example, .../4.0.0/I20090625-1930/I20090625-1930.

relengBaseBuilderDir

The root directory for the eclipse install that will run the build. By default we use the eclipse basebuilder.

writableBuildRoot


In general/build.properties

We also code/re-generate a number of properties in the PDE builder.

Variable Comments
buildType

Set to I, but not currently used to do more than create the matching build results directory and correct build "tag".

eclipseBaseURL

The URL of the eclipse SDK base used for this build. This was used to download targets and automated test frameworks, but is now only used to generate links in the download page.

emfBaseURL

Now only used to generate links in the download page.

gefBaseURL

Now only used to generate links in the download page.

wstBaseURL

Now only used to generate links in the download page.

JAVA60_HOME

A base java 6 install, set to /opt/public/common/ibm-java-jdk-ppc-60 to match build.eclipse.org.

JAVA50_64_HOME

A base java 5 64-bit install, set to /opt/public/common/ibm-java2-ppc64-50 to match build.eclipse.org.

JAVA50_HOME

A base java 5 install, set to /opt/public/common/ibm-java2-ppc-50 to match build.eclipse.org.

JAVA14_HOME

A base java 1.4.2 install, set to /opt/public/common/ibm-java2-142 to match build.eclipse.org.

JavaSE-1.6

Definition of the jars that make up the JSE 1.6 BREE.

J2SE-1.5

Definition of the jars that make up the J2SE 1.5 BREE.

J2SE-1.4

Definition of the jars that make up the J2SE 1.4 BREE.

flex.sdk

A property to the base directory for a flex SDK, needed to build the actionScript supporting tools.

repoBaseLocation

This property points to the directory of zipped repos to be used by the built. It is processed through the repo2runnable ant task.

transformedRepoLocation

The runnable repo produced by repo2runnable. Defaults to ${repoBaseLocation}-trans

Manual Setup

A lot of the directory structures must be set up before you can run a build. Perhaps the directory structure needs to be flattened so that it can be more easily created.

The basebuilder (eclipse install used to run the build) and org.eclipse.e4.builder project have to be on disk somewhere. A build updates them to the latest required versions and links to them using symlinks so that they can be used to run the build, drive the build, and provide the testing instructions.

The runnable repo is used for the build as well as to produce the platform zips and install supporting bundles for the automated tests. The untransformed repo location must be created, and populated with repos downloaded by hand. Currently, the eclipse SDK repo must be hand rolled as well. It would be nice to have the runnable repo populated by downloading the needed zipped repos.

The other possibility is to use the repo2runnable task directly and not through repoBaseLocation. Then multiple repo locations can be specified either as local files or URLs (to zipped repos or p2 repos).

There must be at least 3 JREs/JDKs installed on a machine used to build, 1.4.2, 1.5, and 1.6. A bundles BREE instructs PDE build which set of libraries to use during compile.

Steps

In masterBuild.sh

The steps we go through to run a build are controlled by the masterBuild.sh script.

Step Description
realBuildProperties

Set all of the build properties so that the build will be tagged and copied over to download.eclipse.org when complete.

testBuildProperties

Set the build properties to run a test build. Complete with tests, this is not tagged or copied to download.eclipse.org. Only one of testBuildProperties or realBuildProperties should be set.

commonProperties

Just common properties that are set for both types of builds.

updateBaseBuilder

Update the basebuilder to the correct version. Not done in certain types of test builds, like when using the basebuilder checked out into your workspace.

updateE4Builder

Get the latest of the templates and general builder. This is not done in certain types of test builds, when testing changes that are not checked in.

updateBaseBuilderInfo

Collect info on the basebuilder to run the build. This is done in such a way that the launcher jar can be used to launch other eclipse applications (like the publisher or director) as part of the PDE build.

buildMasterFeature

This runs the PDE build. It runs repo2runnable, builds all of the features and plugins in e4 and generates the e4 p2 repository, and generates platform zips.

copyCompileLogs

Script that generates an html page that links to any compile logs generated during the compile.

generateRepoHtml

Generates a default index for the repo URL ... to stop 404 complaints when people click on it in their browser.

generateSwtZip

Generates the 2 SWT action script projects, that can be imported into the workspace for use with the ActionScript tools. Checks out 2 projects, fiddles with some of the text files, and then zips them up into the results directory.

runTheTests

Copy the modified test.xml/runtests (linux) scripts to the testing directory along with one of the generated platform zips. Run the automated tests on the virtual X machine, Xvfb.

CVS

The resources, ui, and swt teams now have plugins in their team area in the e4 project. Our project is under :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse - module e4