Difference between revisions of "Platform-releng/Platform Build"

From Eclipsepedia

Jump to: navigation, search
(Submitting for aggregator builds)
(Building)
(24 intermediate revisions by 9 users not shown)
Line 1: Line 1:
This page provides instructions for building the Eclipse Platform using preferred technologies identified as part of the [[CBI]] initiative.
+
This page provides instructions for building the Eclipse Platform using preferred technologies identified as part of the [[CBI]] initiative.  
  
== Prerequisites ==
+
== Prerequisites ==
 +
 
 +
=== Free HDD space  ===
  
=== Free HDD space ===
 
 
:~25GB is recommended.
 
:~25GB is recommended.
  
=== Oracle Java 1.6 or higher ===
+
=== Oracle Java 1.6 or higher ===
* Oracle 1.6 JDK needs to be on PATH (OpenJDK also works and is becoming officially supported on Red Hat Enterprise Linux for Kepler. See [http://www.eclipse.org/projects/project-plan.php?planurl=http://www.eclipse.org/eclipse/development/plans/eclipse_project_plan_4_3.xml#target_environments plan]).
+
 
* Verify correct version of java is used
+
*Oracle 1.6 JDK needs to be on PATH (OpenJDK also works and is becoming officially supported on Red Hat Enterprise Linux for Kepler. See [http://www.eclipse.org/projects/project-plan.php?planurl=http://www.eclipse.org/eclipse/development/plans/eclipse_project_plan_4_3.xml#target_environments plan]).  
* Set JAVA_HOME to point to your JDK
+
*Verify correct version of java is used  
* Ensure your java is set to run in Server mode
+
*Set JAVA_HOME to point to your JDK  
 +
*Ensure your java is set to run in Server mode
 +
 
 +
Some of the inner build callouts, like the SWT fragment build, depend on having an Oracle JVM.
 +
 
 +
(Seems that currently all JDKs from 1.4 to 7 with a configured toolchains.xml are needed to get a successful build. See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=409790#c32 Bug report] and [http://dev.eclipse.org/mhonarc/lists/cbi-dev/msg01025.html Mailing List])
  
Some of the inner build callouts, like the SWT fragment build, depend on having an Oracle JVM.
+
<br>
  
==== Java Server Mode ====
+
==== Java Server Mode ====
  
A parameter used by the build "-XX:-UseLoopPredicate" is not recognized by java when running in Client mode. To check which mode you are running in by default run:
+
A parameter used by the build "-XX:-UseLoopPredicate" is not recognized by java when running in Client mode. To check which mode you are running in by default run:  
  
 
   $ java -version
 
   $ java -version
  java version "1.7.0_09"
+
java version "1.7.0_09"
  Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
+
Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
  Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)
+
Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)
  
If the last line says "Client VM" instead of "Server VM" then you are running in client mode. In which case you will need to modify the file '''/jdk1.7.0_09/jre/lib/amd64/jvm.cfg''' and ensure that the line '''-server KNOWN''' is the first line in the file.
+
If the last line says "Client VM" instead of "Server VM" then you are running in client mode. In which case you will need to modify the file '''/jdk1.7.0_09/jre/lib/amd64/jvm.cfg''' and ensure that the line '''-server KNOWN''' is the first line in the file.  
  
=== Maven 3.0.4 ===
+
=== Maven 3.0.4 ===
* Download from http://maven.apache.org/download.html or use your Linux distribution provided version (whether it works depends on the distribution).
+
* make sure '''mvn''' is available in your PATH
+
  
=== Git ===
+
*Download from http://maven.apache.org/download.html or use your Linux distribution provided version (whether it works depends on the distribution).
 +
*make sure '''mvn''' is available in your PATH
  
==== Linux ====
+
=== Git  ===
  
* You can install Git using your distro's package manager.
+
==== Linux  ====
  
==== Using Maven on Ubuntu ====
+
*You can install Git using your distro's package manager.
  
The Maven included in Ubuntu sometimes has issues. See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=394664 Bug report]. If you face such issues, install Maven from the Maven Homepage.
+
==== Using Maven on Ubuntu ====
  
==== Using Maven on Fedora ====
+
The Maven included in Ubuntu sometimes has issues. See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=394664 Bug report]. If you face such issues, install Maven from the Maven Homepage.
  
Maven package included in Fedora is working without a problem and is used to build the Eclipse Platform package shipped with Fedora.
+
==== Using Maven on Fedora ====
  
==== Mac OSX ====
+
Maven package included in Fedora is working without a problem and is used to build the Eclipse Platform package shipped with Fedora.
  
* Download from http://code.google.com/p/git-osx-installer/
+
==== Mac OSX  ====
* Install using the downloaded DMG file
+
  
==== Windows ====
+
*Download from http://code.google.com/p/git-osx-installer/
 +
*Install using the downloaded DMG file
  
* Download Git for Windows from http://msysgit.github.com/
+
==== Windows ====
* Install using the downloaded exe file
+
* We recommend using the "Git Bash" application provided by Git for Windows as your commandline shell throughout the build
+
  
 +
*Download Git for Windows from http://msysgit.github.com/
 +
*Install using the downloaded exe file
 +
*We recommend using the "Git Bash" application provided by Git for Windows as your commandline shell throughout the build
  
'''Important: Per Bug 376400 we discovered that msysgit has a max character limit somewhere around 256 which causes cloning files with a path longer than that to fail'''
+
<br> '''Important: Per Bug 376400 we discovered that msysgit has a max character limit somewhere around 256 which causes cloning files with a path longer than that to fail'''  
  
See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=376400#c4
+
See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=376400#c4  
  
Workaround: Put your repo in the root of a drive and give it a short name. For example: C:\z
+
Workaround: Put your repo in the root of a drive and give it a short name. For example: C:\z  
  
Example:
+
Example:  
  
 
   cd c:
 
   cd c:
  git clone -b R4_2_maintenance --recurse-submodules \
+
git clone -b master --recursive \
  git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git \
+
git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git \
  z
+
z
  
'''Note:''' The final "z" parameter at the end of the command is important as tells git to checkout the repository and rename it to "z". This reduces the path length of the repository to be short enough to workaround Bug 376400.
+
'''Note:''' The final "z" parameter at the end of the command is important as tells git to checkout the repository and rename it to "z". This reduces the path length of the repository to be short enough to workaround Bug 376400.  
  
== Build environment setup ==
+
== Build environment setup ==
  
''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms''
+
''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms''  
  
=== review maven settings.xml ===
+
=== review maven settings.xml ===
To avoid getting hit by {{bug|365727}} and to make sure your local environment configuration does not interfere with CBI build, make sure build user account does not have maven settings.xml
+
 
 +
To avoid getting hit by {{bug|365727}} and to make sure your local environment configuration does not interfere with CBI build, make sure build user account does not have maven settings.xml  
  
 
  igor@desktop:~$ ls -l ~/.m2/settings.xml
 
  igor@desktop:~$ ls -l ~/.m2/settings.xml
 
  ls: cannot access /home/igor/.m2/settings.xml: No such file or directory
 
  ls: cannot access /home/igor/.m2/settings.xml: No such file or directory
  
On Windows XP this is located in:
+
On Windows XP this is located in:  
  
 
  C:\Documents and Settings\your_user\.m2\settings.xml
 
  C:\Documents and Settings\your_user\.m2\settings.xml
 
  File should not exist, if it does delete it.
 
  File should not exist, if it does delete it.
  
On Windows 7 this is located in:
+
On Windows 7 this is located in:  
 +
 
 
  C:\Users\your_user\.m2\settings.xml
 
  C:\Users\your_user\.m2\settings.xml
 
  File should not exist, if it does delete it.
 
  File should not exist, if it does delete it.
  
=== give maven JVM more ram ===
+
=== give maven JVM more ram ===
  
   export MAVEN_OPTS=-Xmx2048m
+
   export MAVEN_OPTS=-Xmx2048m <ref>In some cases, even more. Such as for Luna (4.4) Eclipse builds, we have found we needed 2560m for signed builds. See {{bug|413691}}. </ref>
  
If your using Windows:
+
If you are using Windows:  
  
 
   SET MAVEN_OPTS=-Xmx2048m
 
   SET MAVEN_OPTS=-Xmx2048m
  
== Building ==
+
=== disable checking mirrors for dependencies ===
  
''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms''
+
Along with making more memory available to Maven, you also might want to disable checking mirrors for dependencies, especially if you start seeing build errors like the following:
 +
<pre>
 +
Caused by: java.lang.RuntimeException: "Messages while reading artifacts from child repositories":
 +
  ["Problems while reading artifacts from http://download.eclipse.org/<...>":
 +
  ["Retry another mirror":
 +
  ["Artifact not found: <...>/org.eclipse.emf.ecore.xmi_2.9.0.v20130528-0742.jar.pack.gz."],  
 +
  "Retry another mirror":
 +
  ["Artifact not found: <...>/org.eclipse.emf.common_2.9.0.v20130528-0742.jar.pack.gz."]]]
 +
</pre>
  
We're currently working on the ''R4_2_maintenance'' and ''R3_8_maintenance'' branches.  When we're confident in them, we'll merge the changes into ''master''.
+
To turn off mirror checking do the following (assuming you want to add it at the same time as adding more memory):
  
A script that captures the steps is available in [http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/tree/scripts/build.sh?h=R4_2_maintenance build.sh]
+
export MAVEN_OPTS="-Xmx2048m -Declipse.p2.mirrors=false"
  
=== cloning platform source tree ===
+
If you are using Windows:
  
Replace ''R4_2_maintenance'' with the branch you'd like to checkout.
+
SET MAVEN_OPTS="-Xmx2048m -Declipse.p2.mirrors=false"
  
Clone the repository and checkout the branch:
+
== Building  ==
  
  git clone -b R4_2_maintenance --recurse-submodules \
+
''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms''
  git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git
+
  
 +
We're currently working on the ''master'' branch (for Luna) and are backporting patches to ''R4_3_maintenance'' (for Kepler SRs).
  
To update this clone and all submodules:
+
A script that captures the steps is available in [http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/tree/scripts/build.sh]
 
+
  git fetch origin
+
  git merge origin/R4_2_maintenance
+
  git pull --recurse-submodules
+
  git submodule update
+
  
If you want to switch from another branch to this one, replace '''git merge origin/R4_2_maintenance''' with:
+
=== cloning platform source tree  ===
  
  git checkout -b R4_2_maintenance origin/R4_2_maintenance
+
Replace ''master'' with the branch you'd like to checkout.
  
=== Running the build ===
+
Clone the repository and checkout the branch:
  
   mvn -f maven-cbi-plugin/pom.xml clean install
+
   git clone -b master --recursive \
  mvn -f eclipse-parent/pom.xml clean install
+
git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git
  
 +
<br> To update this clone and all submodules:
  
Once the parent and the plugin are installed in the local repo, you can run the build from the aggregator root:
+
  cd eclipse.platform.releng.aggregator
 +
git pull --recurse-submodules
 +
git submodule update
  
  mvn clean install  -Pno-bree-libs -Dmaven.test.skip=true
+
If you want to switch from another branch to this one, replace '''git merge origin/master''' with:
  
 +
  git checkout -b master origin/master
  
'''NOTE:''' If you build multiple streams on the same system, you'll want to add ''-Dmaven.repo.local=/some/directory/somewhere'' to the end of the 3 mvn commands to avoid collisions (Using a different local repo for each stream). Most casual developers won't be affected.
+
=== Check for Necessary pom.xml updates  ===
* On Windows use ''-Dmaven.repo.local=C:\path\to\somewhere''
+
  
=== Build result===
+
From time to time committers will forget to update the bundle version contained in a pom.xml when a bundle version change is made in a bundle project manifest. If building from master you should likely check the latest I-Build from the eclipse project [http://download.eclipse.org/eclipse/downloads/ downloads page] for a link called '''''POM updates made''''' to see if there are any outstanding patches that you should apply before allowing a successful local build.
  
Once the build finishes, the following artifacts are created.
+
=== Running the build ===
  
Packaged SDK zip files are located in your '''eclipse.platform.releng.aggregator''' directory under
+
From the aggregator root, run:
 +
 
 +
  mvn clean verify
 +
 
 +
<br> '''NOTE:''' If you build multiple streams on the same system, you'll want to add ''-Dmaven.repo.local=/some/directory/somewhere'' to the end of the 3 mvn commands to avoid collisions (Using a different local repo for each stream). Most casual developers won't be affected.
 +
 
 +
*On Windows use ''-Dmaven.repo.local=C:\path\to\somewhere''
 +
 
 +
=== Build result  ===
 +
 
 +
Once the build finishes, the following artifacts are created.
 +
 
 +
Packaged SDK zip files are located in your '''eclipse.platform.releng.aggregator''' directory under  
  
 
   org.eclipse.releng.tychoeclipsebuilder/sdk/target/products/
 
   org.eclipse.releng.tychoeclipsebuilder/sdk/target/products/
  
P2 repository
+
P2 repository  
  
 
   org.eclipse.repository/target/repository
 
   org.eclipse.repository/target/repository
  
Source bundles
+
Source bundles  
  
 
   org.eclipse.repository/target/repository/plugins
 
   org.eclipse.repository/target/repository/plugins
  
Packaged JUnit Plugin Tests and Automated Testing Framework
+
Packaged JUnit Plugin Tests and Automated Testing Framework  
  
 
   org.eclipse.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-1.0.0-SNAPSHOT-bundle.zip
 
   org.eclipse.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-1.0.0-SNAPSHOT-bundle.zip
  
=== Cleaning the build repo ===
+
=== Cleaning the build repo ===
  
To completely clean a build repo (before a new build or before updating the aggregator and submodules):
+
To completely clean a build repo (before a new build or before updating the aggregator and submodules):  
  
 
   git submodule foreach git clean -f -d
 
   git submodule foreach git clean -f -d
  git submodule foreach git reset --hard HEAD
+
git submodule foreach git reset --hard HEAD
  git clean -f -d
+
git clean -f -d
   git reset --hard HEAD
+
git reset --hard HEAD
 +
 
 +
To get a pristine build repository, after a build, you will have to specify the -x option for git clean.
 +
 
 +
   git submodule foreach git clean -f -d -x
 +
git submodule foreach git reset --hard HEAD
 +
git clean -f -d -x
 +
git reset --hard HEAD
 +
 
 +
The reason is that /targets is specified in the .gitignore file, so git ... ignores them. They are included in .gitignore so they are not accidentally committed and don't clutter visual displays of "changes". Normally Maven will clean those up before a build, but in case you want to be positive you do not get an "old" copy, of one of the target artifacts, use -x.
  
 
=== Using BREE Libs  ===
 
=== Using BREE Libs  ===
Line 173: Line 208:
 
BREE libs can be used to build using the same BREE as what is used on build.eclipse.org. You will first need to download and install the ee.zip attached to Bug 386649 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=386649) and follow the instructions in the description of the bug.  
 
BREE libs can be used to build using the same BREE as what is used on build.eclipse.org. You will first need to download and install the ee.zip attached to Bug 386649 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=386649) and follow the instructions in the description of the bug.  
  
Here's an [[Platform-releng/toolchainsExample| example of toolchains.xml]]. It currently only works with Oracle JREs (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=389856 bug 389856]). Someone running the platform build would need their own version of toolchains.xml in their build ids home directory's .m2 directory (~/.m2/toolchains.xml) which points to the location on their file system of the tools required (BREE libs and JDKs).  
+
Here's an [[Platform-releng/toolchainsExample|example of toolchains.xml]]. It currently only works with Oracle JREs (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=389856 bug 389856]). Someone running the platform build would need their own version of toolchains.xml in their build ids home directory's .m2 directory (~/.m2/toolchains.xml) which points to the location on their file system of the tools required (BREE libs and JDKs).  
  
Once setup you can inform the build to use it by passing '''-Pbree-libs''' instead of '''-Pno-bree-libs''' on the mvn build command.
+
Once setup you can inform the build to use it by passing '''-Pbree-libs''' on the mvn build command.  
  
=== Pack200 & Signing ===
+
=== Pack200 &amp; Signing ===
  
Pack200 & Signing is supported when built using build.eclipse.org and is disabled by default.
+
Pack200 &amp; Signing is supported when built using build.eclipse.org and is disabled by default.  
  
* Install eclipse-jarsigner-plugin to local maven repository used by the build (i.e. /var/tmp/lts/R4_localrepo).
+
*Simply Run the build with '''-Peclipse-sign''' parameter.
* Run the build with '''-Peclipse-sign''' parameter.  
+
  
Eclipse Jarsigner can be built from: http://git.eclipse.org/c/cbi/org.eclipse.cbi.maven.plugins.git
+
==== Advanced  ====
  
  git clone -n git://git.eclipse.org/gitroot/cbi/org.eclipse.cbi.maven.plugins.git
+
If you'd like to build your own version of the eclipse-jarsigner-plugin you can use the instructions below.  
  cd org.eclipse.cbi.maven.plugins
+
  git checkout eclipse-jarsigner-plugin-1.0.1
+
  mvn -f eclipse-jarsigner-plugin/pom.xml \
+
    clean install
+
  
=== Building natives ===
+
Eclipse Jarsigner can be built from: http://git.eclipse.org/c/cbi/org.eclipse.cbi.maven.plugins.git
  
Optionally, you can pass -Dnative= parameter with one of the following options to compile the natives for the specified native.
+
  git clone -n git://git.eclipse.org/gitroot/cbi/org.eclipse.cbi.maven.plugins.git
 +
cd org.eclipse.cbi.maven.plugins
 +
git checkout master
 +
mvn -f eclipse-jarsigner-plugin/pom.xml \
 +
  clean install
  
<pre>
+
=== Building natives  ===
- cocoa.macosx.x86
+
 
 +
Optionally, you can pass -Dnative= parameter with one of the following options to compile the natives for the specified native.
 +
<pre>- cocoa.macosx.x86
 
- cocoa.macosx.x86_64
 
- cocoa.macosx.x86_64
 
- gtk.linux.x86
 
- gtk.linux.x86
Line 204: Line 240:
 
- win32.win32.x86_64
 
- win32.win32.x86_64
 
- win32.wce_ppc.arm
 
- win32.wce_ppc.arm
</pre>
+
</pre>  
 
+
Example:  
Example:
+
  
   mvn clean install \
+
   mvn clean verify \
    -Dmaven.test.skip=true -Dnative=gtk.linux.x86_64
+
  -Dmaven.test.skip=true -Dnative=gtk.linux.x86_64
  
== running the tests ==
+
== running the tests ==
  
Copy the junit tests and the CBI SDK (pick the one for your platform) that was built to a testing directory. Also unzip the junit tests.
+
Copy the junit tests and the CBI SDK (pick the one for your platform) that was built to a testing directory. Also unzip the junit tests.  
  
 
   mkdir -p /var/tmp/lts/R3_platform-tests
 
   mkdir -p /var/tmp/lts/R3_platform-tests
  cp eclipse.platform.tychoeclipsebuilder/sdk/target/products/org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz /var/tmp/lts/R3_platform-tests
+
cp eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products/org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz /var/tmp/lts/R3_platform-tests
  cp eclipse.platform.tychoecilpsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-bundle.zip /var/tmp/lts/R3_platform-tests
+
cp eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-bundle.zip /var/tmp/lts/R3_platform-tests
  cd /var/tmp/lts/R3_platform-tests
+
cd /var/tmp/lts/R3_platform-tests
  unzip eclipse-junit-tests-bundle.zip
+
unzip eclipse-junit-tests-bundle.zip
  
Modify the file equinoxp2tests.properties to point to the CBI built repository.  
+
Modify the file equinoxp2tests.properties to point to the CBI built repository. (This example uses /home/user/R3_platform-aggregator as the CBI platform root)  
(This example uses /home/user/R3_platform-aggregator as the CBI platform root)  
+
  
 
   org.eclipse.equinox.p2.tests.current.build.repo=file:/home/user/R3_platform-aggregator/eclipse.platform.repository/target/repository/
 
   org.eclipse.equinox.p2.tests.current.build.repo=file:/home/user/R3_platform-aggregator/eclipse.platform.repository/target/repository/
  
Note down the org.eclipse.equinox.p2.reconciler.tests.platform.archive.linux-x86_64= file name for your architecture. (In this example linux x86_64)
+
Note down the org.eclipse.equinox.p2.reconciler.tests.platform.archive.linux-x86_64= file name for your architecture. (In this example linux x86_64)  
  
 
   eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
 
   eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
  
Rename your org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz you copied earlier to match the name you jotted down.
+
Rename your org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz you copied earlier to match the name you jotted down.  
  
 
   mv org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
 
   mv org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
  
Copy your eclipse-platform to eclipse SDK
+
Copy your eclipse-platform to eclipse SDK  
  
 
   cp eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
 
   cp eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
  
Download a copy of the latest released Eclipse Classic from http://www.eclipse.org/downloads/.
+
Download a copy of the latest released Eclipse Classic from http://www.eclipse.org/downloads/.  
  
 
   http://www.eclipse.org/downloads/
 
   http://www.eclipse.org/downloads/
  
Copy the Eclipse Classic SDK to eclipse-platform
+
Copy the Eclipse Classic SDK to eclipse-platform  
  
 
   cp eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
 
   cp eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
  
You should now have 4 copies of the platform, 2 from the latest released and 2 from the CBI build.
+
You should now have 4 copies of the platform, 2 from the latest released and 2 from the CBI build.  
  
 
   eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz  
 
   eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz  
  eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
+
eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
  eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz  
+
eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz  
  eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
+
eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
  
Unzip the CBI platform
+
Unzip the CBI platform  
  
 
   tar zxvf eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
 
   tar zxvf eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
  
Run the tests
+
Run the tests  
  
 
   ./runtests -os linux -ws gtk -arch x86_64
 
   ./runtests -os linux -ws gtk -arch x86_64
  
== Troubleshooting ==
+
== Troubleshooting ==
  
=== Bug 384482 - Apple JVM renamed rt.jar to classes.jar causing CBI Platform build issues ===
+
=== Bug 384482 - Apple JVM renamed rt.jar to classes.jar causing CBI Platform build issues ===
  
Apple JVM renamed rt.jar to classes.jar causing the CBI Platform build to fail when building on macosx. Running the build a 2nd time after the failure however produces a working build. It isn't ideal to have the user run the build twice though.
+
Apple JVM renamed rt.jar to classes.jar causing the CBI Platform build to fail when building on macosx. Running the build a 2nd time after the failure however produces a working build. It isn't ideal to have the user run the build twice though.  
  
A workaround for this issue is to create a symbolic link for classes.jar to rt.jar.
+
A workaround for this issue is to create a symbolic link for classes.jar to rt.jar.  
  
 
   cd /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib
 
   cd /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib
  sudo ln -s ../../Classes/classes.jar rt.jar
+
sudo ln -s ../../Classes/classes.jar rt.jar
  
 +
<br>
  
== Submitting for aggregator builds ==
+
== Submitting for aggregator builds ==
  
[Note: see [[Platform-releng/Automated_Platform_Build#Contributing_to_a_build]] for the "how to" using the automatic processing done by the [[Platform-releng/Automated_Platform_Build]]. But, this "manual" approach is still important to know how to do, especially as "good citizens" if the aggregator build is broken, a "fix" can often be made before the next automated build.]  
+
[Note: see [[Platform-releng/Automated Platform Build#Contributing_to_a_build]] for the "how to" using the automatic processing done by the [[Platform-releng/Automated Platform Build]]. But, this "manual" approach is still important to know how to do, especially as "good citizens" if the aggregator build is broken, a "fix" can often be made before the next automated build.]  
  
In PDE build the features specify what plugins to build and the map files specify where to get the plugin projects and which version to check out.
+
In PDE build the features specify what plugins to build and the map files specify where to get the plugin projects and which version to check out.  
  
In the CBI build the pom.xml files and the directory structure specify what plugins to build and the aggregator git repo specifies which checkout is the build input by capturing the commit for each component repo as a submodule entry.
+
In the CBI build the pom.xml files and the directory structure specify what plugins to build and the aggregator git repo specifies which checkout is the build input by capturing the commit for each component repo as a submodule entry.  
  
The manual process for submitting Platform UI for a build involves updating the submodule in the aggregator:
+
The manual process for submitting Platform UI for a build involves updating the submodule in the aggregator:  
  
 
   # starting in eclipse.platform.releng.aggregator
 
   # starting in eclipse.platform.releng.aggregator
  cd eclipse.platform.ui
+
cd eclipse.platform.ui
  git checkout R4_2_maintenance
+
git checkout R4_2_maintenance
  git pull
+
git pull
  cd ..
+
cd ..
  git status
+
git status
  
The status should look something like:
+
The status should look something like:  
  
 
   # On branch R4_2_maintenance
 
   # On branch R4_2_maintenance
  # Changed but not updated:
+
# Changed but not updated:
  #  (use "git add <file>..." to update what will be committed)
+
#  (use "git add &lt;file&gt;..." to update what will be committed)
  #  (use "git checkout -- <file>..." to discard changes in working directory)
+
#  (use "git checkout -- &lt;file&gt;..." to discard changes in working directory)
  #  (commit or discard the untracked or modified content in submodules)
+
#  (commit or discard the untracked or modified content in submodules)
  #
+
#
  #      modified:  eclipse.platform.ui (new commits)
+
#      modified:  eclipse.platform.ui (new commits)
  #
+
#
  
To pull in the new commits, just add the repo:
+
To pull in the new commits, just add the repo:  
  
 
   git add eclipse.platform.ui
 
   git add eclipse.platform.ui
  git commit
+
git commit
  git push origin HEAD
+
git push origin HEAD
 
+
  
These steps are be driven by a file named repositories.txt used by [[Platform-releng/Automated_Platform_Build]] at the beginning of an automatedd build.
+
<br> These steps are be driven by a file named repositories.txt used by [[Platform-releng/Automated Platform Build]] at the beginning of an automated build.  
  
 +
<references/>
  
[[Category:Eclipse_Platform_Releng| ]]
+
[[Category:Eclipse_Platform_Releng| Eclipse_Platform_Releng]]

Revision as of 10:19, 2 September 2013

This page provides instructions for building the Eclipse Platform using preferred technologies identified as part of the CBI initiative.

Contents

Prerequisites

Free HDD space

~25GB is recommended.

Oracle Java 1.6 or higher

  • Oracle 1.6 JDK needs to be on PATH (OpenJDK also works and is becoming officially supported on Red Hat Enterprise Linux for Kepler. See plan).
  • Verify correct version of java is used
  • Set JAVA_HOME to point to your JDK
  • Ensure your java is set to run in Server mode

Some of the inner build callouts, like the SWT fragment build, depend on having an Oracle JVM.

(Seems that currently all JDKs from 1.4 to 7 with a configured toolchains.xml are needed to get a successful build. See Bug report and Mailing List)


Java Server Mode

A parameter used by the build "-XX:-UseLoopPredicate" is not recognized by java when running in Client mode. To check which mode you are running in by default run:

 $ java -version
java version "1.7.0_09"
Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)

If the last line says "Client VM" instead of "Server VM" then you are running in client mode. In which case you will need to modify the file /jdk1.7.0_09/jre/lib/amd64/jvm.cfg and ensure that the line -server KNOWN is the first line in the file.

Maven 3.0.4

  • Download from http://maven.apache.org/download.html or use your Linux distribution provided version (whether it works depends on the distribution).
  • make sure mvn is available in your PATH

Git

Linux

  • You can install Git using your distro's package manager.

Using Maven on Ubuntu

The Maven included in Ubuntu sometimes has issues. See Bug report. If you face such issues, install Maven from the Maven Homepage.

Using Maven on Fedora

Maven package included in Fedora is working without a problem and is used to build the Eclipse Platform package shipped with Fedora.

Mac OSX

Windows

  • Download Git for Windows from http://msysgit.github.com/
  • Install using the downloaded exe file
  • We recommend using the "Git Bash" application provided by Git for Windows as your commandline shell throughout the build


Important: Per Bug 376400 we discovered that msysgit has a max character limit somewhere around 256 which causes cloning files with a path longer than that to fail

See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=376400#c4

Workaround: Put your repo in the root of a drive and give it a short name. For example: C:\z

Example:

 cd c:
git clone -b master --recursive \
git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git \
z

Note: The final "z" parameter at the end of the command is important as tells git to checkout the repository and rename it to "z". This reduces the path length of the repository to be short enough to workaround Bug 376400.

Build environment setup

Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms

review maven settings.xml

To avoid getting hit by bug 365727 and to make sure your local environment configuration does not interfere with CBI build, make sure build user account does not have maven settings.xml

igor@desktop:~$ ls -l ~/.m2/settings.xml
ls: cannot access /home/igor/.m2/settings.xml: No such file or directory

On Windows XP this is located in:

C:\Documents and Settings\your_user\.m2\settings.xml
File should not exist, if it does delete it.

On Windows 7 this is located in:

C:\Users\your_user\.m2\settings.xml
File should not exist, if it does delete it.

give maven JVM more ram

 export MAVEN_OPTS=-Xmx2048m [1]

If you are using Windows:

 SET MAVEN_OPTS=-Xmx2048m

disable checking mirrors for dependencies

Along with making more memory available to Maven, you also might want to disable checking mirrors for dependencies, especially if you start seeing build errors like the following:

Caused by: java.lang.RuntimeException: "Messages while reading artifacts from child repositories": 
  ["Problems while reading artifacts from http://download.eclipse.org/<...>": 
  ["Retry another mirror": 
  ["Artifact not found: <...>/org.eclipse.emf.ecore.xmi_2.9.0.v20130528-0742.jar.pack.gz."], 
  "Retry another mirror": 
  ["Artifact not found: <...>/org.eclipse.emf.common_2.9.0.v20130528-0742.jar.pack.gz."]]]

To turn off mirror checking do the following (assuming you want to add it at the same time as adding more memory):

export MAVEN_OPTS="-Xmx2048m -Declipse.p2.mirrors=false"

If you are using Windows:

SET MAVEN_OPTS="-Xmx2048m -Declipse.p2.mirrors=false"

Building

Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms

We're currently working on the master branch (for Luna) and are backporting patches to R4_3_maintenance (for Kepler SRs).

A script that captures the steps is available in [1]

cloning platform source tree

Replace master with the branch you'd like to checkout.

Clone the repository and checkout the branch:

 git clone -b master --recursive \
git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git


To update this clone and all submodules:

 cd eclipse.platform.releng.aggregator
git pull --recurse-submodules
git submodule update

If you want to switch from another branch to this one, replace git merge origin/master with:

 git checkout -b master origin/master

Check for Necessary pom.xml updates

From time to time committers will forget to update the bundle version contained in a pom.xml when a bundle version change is made in a bundle project manifest. If building from master you should likely check the latest I-Build from the eclipse project downloads page for a link called POM updates made to see if there are any outstanding patches that you should apply before allowing a successful local build.

Running the build

From the aggregator root, run:

 mvn clean verify


NOTE: If you build multiple streams on the same system, you'll want to add -Dmaven.repo.local=/some/directory/somewhere to the end of the 3 mvn commands to avoid collisions (Using a different local repo for each stream). Most casual developers won't be affected.

  • On Windows use -Dmaven.repo.local=C:\path\to\somewhere

Build result

Once the build finishes, the following artifacts are created.

Packaged SDK zip files are located in your eclipse.platform.releng.aggregator directory under

 org.eclipse.releng.tychoeclipsebuilder/sdk/target/products/

P2 repository

 org.eclipse.repository/target/repository

Source bundles

 org.eclipse.repository/target/repository/plugins

Packaged JUnit Plugin Tests and Automated Testing Framework

 org.eclipse.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-1.0.0-SNAPSHOT-bundle.zip

Cleaning the build repo

To completely clean a build repo (before a new build or before updating the aggregator and submodules):

 git submodule foreach git clean -f -d
git submodule foreach git reset --hard HEAD
git clean -f -d
git reset --hard HEAD

To get a pristine build repository, after a build, you will have to specify the -x option for git clean.

 git submodule foreach git clean -f -d -x
git submodule foreach git reset --hard HEAD
git clean -f -d -x
git reset --hard HEAD

The reason is that /targets is specified in the .gitignore file, so git ... ignores them. They are included in .gitignore so they are not accidentally committed and don't clutter visual displays of "changes". Normally Maven will clean those up before a build, but in case you want to be positive you do not get an "old" copy, of one of the target artifacts, use -x.

Using BREE Libs

BREE libs can be used to build using the same BREE as what is used on build.eclipse.org. You will first need to download and install the ee.zip attached to Bug 386649 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=386649) and follow the instructions in the description of the bug.

Here's an example of toolchains.xml. It currently only works with Oracle JREs (see bug 389856). Someone running the platform build would need their own version of toolchains.xml in their build ids home directory's .m2 directory (~/.m2/toolchains.xml) which points to the location on their file system of the tools required (BREE libs and JDKs).

Once setup you can inform the build to use it by passing -Pbree-libs on the mvn build command.

Pack200 & Signing

Pack200 & Signing is supported when built using build.eclipse.org and is disabled by default.

  • Simply Run the build with -Peclipse-sign parameter.

Advanced

If you'd like to build your own version of the eclipse-jarsigner-plugin you can use the instructions below.

Eclipse Jarsigner can be built from: http://git.eclipse.org/c/cbi/org.eclipse.cbi.maven.plugins.git

 git clone -n git://git.eclipse.org/gitroot/cbi/org.eclipse.cbi.maven.plugins.git
cd org.eclipse.cbi.maven.plugins
git checkout master
mvn -f eclipse-jarsigner-plugin/pom.xml \
 clean install

Building natives

Optionally, you can pass -Dnative= parameter with one of the following options to compile the natives for the specified native.

- cocoa.macosx.x86
- cocoa.macosx.x86_64
- gtk.linux.x86
- gtk.linux.x86_64
- win32.win32.x86
- win32.win32.x86_64
- win32.wce_ppc.arm

Example:

 mvn clean verify \
 -Dmaven.test.skip=true -Dnative=gtk.linux.x86_64

running the tests

Copy the junit tests and the CBI SDK (pick the one for your platform) that was built to a testing directory. Also unzip the junit tests.

 mkdir -p /var/tmp/lts/R3_platform-tests
cp eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products/org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz /var/tmp/lts/R3_platform-tests
cp eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-bundle.zip /var/tmp/lts/R3_platform-tests
cd /var/tmp/lts/R3_platform-tests
unzip eclipse-junit-tests-bundle.zip

Modify the file equinoxp2tests.properties to point to the CBI built repository. (This example uses /home/user/R3_platform-aggregator as the CBI platform root)

 org.eclipse.equinox.p2.tests.current.build.repo=file:/home/user/R3_platform-aggregator/eclipse.platform.repository/target/repository/

Note down the org.eclipse.equinox.p2.reconciler.tests.platform.archive.linux-x86_64= file name for your architecture. (In this example linux x86_64)

 eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz

Rename your org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz you copied earlier to match the name you jotted down.

 mv org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz

Copy your eclipse-platform to eclipse SDK

 cp eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz

Download a copy of the latest released Eclipse Classic from http://www.eclipse.org/downloads/.

 http://www.eclipse.org/downloads/

Copy the Eclipse Classic SDK to eclipse-platform

 cp eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz

You should now have 4 copies of the platform, 2 from the latest released and 2 from the CBI build.

 eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz 
eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz 
eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz

Unzip the CBI platform

 tar zxvf eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz

Run the tests

 ./runtests -os linux -ws gtk -arch x86_64

Troubleshooting

Bug 384482 - Apple JVM renamed rt.jar to classes.jar causing CBI Platform build issues

Apple JVM renamed rt.jar to classes.jar causing the CBI Platform build to fail when building on macosx. Running the build a 2nd time after the failure however produces a working build. It isn't ideal to have the user run the build twice though.

A workaround for this issue is to create a symbolic link for classes.jar to rt.jar.

 cd /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib
sudo ln -s ../../Classes/classes.jar rt.jar


Submitting for aggregator builds

[Note: see Platform-releng/Automated Platform Build#Contributing_to_a_build for the "how to" using the automatic processing done by the Platform-releng/Automated Platform Build. But, this "manual" approach is still important to know how to do, especially as "good citizens" if the aggregator build is broken, a "fix" can often be made before the next automated build.]

In PDE build the features specify what plugins to build and the map files specify where to get the plugin projects and which version to check out.

In the CBI build the pom.xml files and the directory structure specify what plugins to build and the aggregator git repo specifies which checkout is the build input by capturing the commit for each component repo as a submodule entry.

The manual process for submitting Platform UI for a build involves updating the submodule in the aggregator:

 # starting in eclipse.platform.releng.aggregator
cd eclipse.platform.ui
git checkout R4_2_maintenance
git pull
cd ..
git status

The status should look something like:

 # On branch R4_2_maintenance
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#   (commit or discard the untracked or modified content in submodules)
#
#       modified:   eclipse.platform.ui (new commits)
#

To pull in the new commits, just add the repo:

 git add eclipse.platform.ui
git commit
git push origin HEAD


These steps are be driven by a file named repositories.txt used by Platform-releng/Automated Platform Build at the beginning of an automated build.

  1. In some cases, even more. Such as for Luna (4.4) Eclipse builds, we have found we needed 2560m for signed builds. See bug 413691.