Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Platform-releng/Platform Build"
(migrated to https://github.com/eclipse-platform/eclipse.platform.releng/wiki/Platform-Build) |
|||
| (134 intermediate revisions by 22 users not shown) | |||
| Line 1: | Line 1: | ||
| − | |||
| − | + | {{warning|Note: Migrated to https://github.com/eclipse-platform/eclipse.platform.releng/wiki/Platform-Build}} | |
| − | + | This page provides instructions for building the Eclipse Platform using preferred technologies identified as part of the [[CBI]] initiative. | |
| − | + | ||
| − | === | + | == Prerequisites == |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | === Free HDD space === | |
| + | :~25GB is recommended. | ||
| − | === | + | === Java 17 JVM === |
| − | + | ||
| − | + | ||
| − | + | *Java 17 JDK needs to be on PATH as the VM that "runs the build". | |
| + | *Verify correct version of java is used | ||
| + | *Set JAVA_HOME to point to your JDK | ||
| + | *Ensure your java is set to run in Server mode | ||
| − | + | * Notes: | |
| − | + | :* Some of the inner build callouts, like the SWT fragment build, depend on having an Oracle JVM. | |
| + | :* It is highly recommended to have all JDKs, from CDC-1.0/Foundation-1.0 to Java 11, in a configured toolchains.xml file. See [https://bugs.eclipse.org/bugs/show_bug.cgi?id=409790#c32 Bug report] and [http://dev.eclipse.org/mhonarc/lists/cbi-dev/msg01025.html Mailing List]) and see the Section below titled 'Using BREE Libs'. | ||
| − | + | === Maven 3.6.3 === | |
| − | + | ||
| + | *Download from http://maven.apache.org/download.html or use your Linux distribution provided version (whether it works depends on the distribution). | ||
| + | *make sure '''mvn''' is available in your PATH | ||
| − | === | + | === Git === |
| − | + | ==== Linux ==== | |
| − | + | *You can install Git using your distro's package manager. | |
| − | + | ==== Mac OSX ==== | |
| − | + | *Download from http://code.google.com/p/git-osx-installer/ | |
| + | *Install using the downloaded DMG file | ||
| − | + | ==== Windows ==== | |
| − | + | *Download Git for Windows from http://msysgit.github.com/ | |
| + | *Install using the downloaded exe file | ||
| + | *We recommend using the "Git Bash" application provided by Git for Windows as your commandline shell throughout the build | ||
| − | + | <br> '''Important: Per Bug 376400 we discovered that msysgit has a max character limit somewhere around 256 which causes cloning files with a path longer than that to fail''' | |
| − | + | See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=376400#c4 | |
| − | + | Workaround: Put your repo in the root of a drive and give it a short name. For example: C:\z | |
| − | + | Example: | |
| − | + | cd c: | |
| + | git clone -b master --recursive \ | ||
| + | https://github.com/eclipse-platform/eclipse.platform.releng.aggregator.git \ | ||
| + | z | ||
| − | + | '''Note:''' The final "z" parameter at the end of the command is important as tells git to checkout the repository and rename it to "z". This reduces the path length of the repository to be short enough to workaround Bug 376400. | |
| − | + | == Build environment setup == | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms'' | |
| − | + | ||
| − | + | === give maven JVM more ram === | |
| − | + | export MAVEN_OPTS=-Xmx2048m | |
| − | + | If you are using Windows: | |
| − | + | SET MAVEN_OPTS=-Xmx2048m | |
| − | + | == Building == | |
| − | + | ''Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms'' | |
| − | + | === cloning platform source tree === | |
| − | + | Replace ''master'' with the branch you'd like to checkout. | |
| − | + | Clone the repository and checkout the branch: | |
| − | + | git clone -b master --recursive \ | |
| + | https://github.com/eclipse-platform/eclipse.platform.releng.aggregator.git | ||
| − | + | <br> To update this clone and all submodules: | |
| − | + | cd eclipse.platform.releng.aggregator | |
| + | git pull --recurse-submodules | ||
| + | git submodule update | ||
| − | + | If you want to switch from another branch to this one, replace '''git merge origin/master''' with: | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | git checkout -b master origin/master | |
| − | + | === Running the build === | |
| − | + | From the aggregator root, run: | |
| − | + | mvn clean verify | |
| − | = | + | :'''NOTE:''' If you build multiple streams on the same system, you'll want to add ''-Dmaven.repo.local=/some/directory/somewhere'' to the end of the 3 mvn commands to avoid collisions (Using a different local repo for each stream). Most casual developers won't be affected. |
| − | + | ::*On Windows use ''-Dmaven.repo.local=C:\path\to\somewhere'' | |
| − | <pre> | + | === Running the build without tests === |
| − | - cocoa.macosx. | + | |
| + | From the aggregator root, run: | ||
| + | |||
| + | mvn clean verify -DskipTests=true | ||
| + | |||
| + | :'''NOTE''' "-Dmaven.test.skip=true" cannot be used in CBI, see {{bug|456510}} | ||
| + | |||
| + | === Build result === | ||
| + | |||
| + | Once the build finishes, the following artifacts are created. | ||
| + | |||
| + | Packaged SDK zip files are located in your '''eclipse.platform.releng.aggregator''' directory under | ||
| + | |||
| + | org.eclipse.releng.tychoeclipsebuilder/sdk/target/products/ | ||
| + | |||
| + | P2 repository | ||
| + | |||
| + | org.eclipse.repository/target/repository | ||
| + | |||
| + | Source bundles | ||
| + | |||
| + | org.eclipse.repository/target/repository/plugins | ||
| + | |||
| + | Packaged JUnit Plugin Tests and Automated Testing Framework | ||
| + | |||
| + | org.eclipse.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-1.0.0-SNAPSHOT-bundle.zip | ||
| + | |||
| + | === Cleaning the build repo === | ||
| + | |||
| + | To completely clean a build repo (before a new build or before updating the aggregator and submodules): | ||
| + | |||
| + | git submodule foreach git clean -f -d | ||
| + | git submodule foreach git reset --hard HEAD | ||
| + | git clean -f -d | ||
| + | git reset --hard HEAD | ||
| + | |||
| + | To get a pristine build repository, after a build, you will have to specify the -x option for git clean. | ||
| + | |||
| + | git submodule foreach git clean -f -d -x | ||
| + | git submodule foreach git reset --hard HEAD | ||
| + | git clean -f -d -x | ||
| + | git reset --hard HEAD | ||
| + | |||
| + | The reason is that /targets is specified in the .gitignore file, so git ... ignores them. They are included in .gitignore so they are not accidentally committed and don't clutter visual displays of "changes". Normally Maven will clean those up before a build, but in case you want to be positive you do not get an "old" copy, of one of the target artifacts, use -x. | ||
| + | |||
| + | === Using BREE Libs === | ||
| + | |||
| + | BREE libs can be used to build using the same BREE as what is used on build.eclipse.org. You will first need to download and install the ee.zip attached to Bug 386649 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=386649) and follow the instructions in the description of the bug. | ||
| + | |||
| + | Here's an [[Platform-releng/toolchainsExample|example of toolchains.xml]]. It currently only works with Oracle JREs (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=389856 bug 389856]). Someone running the platform build would need their own version of toolchains.xml in their build ids home directory's .m2 directory (~/.m2/toolchains.xml) which points to the location on their file system of the tools required (BREE libs and JDKs). | ||
| + | |||
| + | Once setup you can inform the build to use it by passing '''-Pbree-libs''' on the mvn build command. | ||
| + | |||
| + | === Signing === | ||
| + | |||
| + | Signing is supported when built using build.eclipse.org and is disabled by default. | ||
| + | |||
| + | *Simply Run the build with '''-Peclipse-sign''' parameter. | ||
| + | |||
| + | === Building natives === | ||
| + | |||
| + | Optionally, you can pass -Dnative= parameter with one of the following options to compile the natives for the specified native. | ||
| + | <pre>- cocoa.macosx.aarch64 | ||
- cocoa.macosx.x86_64 | - cocoa.macosx.x86_64 | ||
| − | - gtk.linux. | + | - gtk.linux.aarch64 |
| + | - gtk.linux.ppc64le | ||
- gtk.linux.x86_64 | - gtk.linux.x86_64 | ||
| − | |||
- win32.win32.x86_64 | - win32.win32.x86_64 | ||
| − | - | + | </pre> |
| − | </pre> | + | Example: |
| + | |||
| + | mvn clean verify \ | ||
| + | -DskipTests=true -Dnative=gtk.linux.x86_64 | ||
| + | |||
| + | === Building Selected Projects === | ||
| + | |||
| + | Instead of a full build, selected projects can be built by going to the desired root directory of the project(s) and specifying the <pre>build-individual-bundles</pre> profile. | ||
Example: | Example: | ||
| − | + | (from the eclipse.platform.releng.aggregator repository root) \ | |
| − | + | cd rt.equinox.framework \ | |
| + | mvn -Pbuild-individual-bundles clean verify | ||
| − | + | This would build all projects under rt.equinox.framework. | |
| − | + | Example: | |
| − | + | (from the eclipse.platform.releng.aggregator repository root) \ | |
| − | + | cd rt.equinox.framework/bundles/org.eclipse.osgi \ | |
| + | mvn -Pbuild-individual-bundles clean verify | ||
| − | + | This would build only the org.eclipse.osgi project. | |
| − | == | + | == Build output: p2 repo and RCP products == |
| − | === | + | Once build is complete, look in the following folders for the usual output artifacts: |
| + | * <tt>eclipse.platform.releng.aggregator/eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products</tt> for the archives of the Eclipse SDK. (Siblings of <tt>sdk</tt> folder contain other RCP products produced by Platform Build). Those are the packages that are typically published at http://download.eclipse.org/eclipse/downloads/drops4/ . | ||
| + | * <tt>eclipse.platform.releng.aggregator/eclipse.platform.releng.tychoeclipsebuilder/eclipse.platform.repository/target/repository</tt> for the typical p2 repository containing installable unit shipped by Platform Build. This is the repository that get typically added to http://download.eclipse.org/eclipse/updates/ . | ||
| + | |||
| + | == Running the Eclipse platform tests == | ||
| + | |||
| + | Copy the junit tests and the CBI SDK (pick the one for your platform) that was built to a testing directory. Also unzip the junit tests and enter them. | ||
| + | |||
| + | mkdir -p /var/tmp/lts/R3_platform-tests | ||
| + | cp eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products/org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz /var/tmp/lts/R3_platform-tests | ||
| + | cp eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-bundle.zip /var/tmp/lts/R3_platform-tests | ||
| + | cd /var/tmp/lts/R3_platform-tests | ||
| + | unzip eclipse-junit-tests-bundle.zip | ||
| + | cd eclipse-testing | ||
| + | |||
| + | Modify the file equinoxp2tests.properties to point to the CBI built repository. (This example uses /home/user/R3_platform-aggregator as the CBI platform root) | ||
| + | |||
| + | org.eclipse.equinox.p2.tests.current.build.repo=file:/home/user/R3_platform-aggregator/eclipse.platform.repository/target/repository/ | ||
| + | |||
| + | Note down the org.eclipse.equinox.p2.reconciler.tests.platform.archive.linux-x86_64= file name for your architecture. (In this example linux x86_64) | ||
| + | |||
| + | eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | Rename your org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz you copied earlier to match the name you jotted down. | ||
| + | |||
| + | mv ../org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | Copy your eclipse-platform to eclipse SDK | ||
| + | |||
| + | cp eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | Download a copy of the latest released Eclipse Classic (Eclipse IDE for Java Developers) from http://www.eclipse.org/downloads/. | ||
| + | |||
| + | http://www.eclipse.org/downloads/ | ||
| + | |||
| + | Copy the Eclipse Classic SDK to eclipse-platform | ||
| + | |||
| + | cp eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | You should now have 4 copies of the platform, 2 from the latest released and 2 from the CBI build. | ||
| + | |||
| + | eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz | ||
| + | eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz | ||
| + | eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | Unzip the CBI platform | ||
| + | |||
| + | tar zxvf eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz | ||
| + | |||
| + | Run the tests | ||
| + | |||
| + | ./runtests -os linux -ws gtk -arch x86_64 | ||
| − | + | In case you get "testing.properties: No such file or directory", rename or symlink test.properties | |
| − | + | ln -s test.properties testing.properties | |
| − | + | and re-run the tests. | |
| − | + | == Submitting for aggregator builds == | |
| − | + | See [[Platform-releng/Automated Platform Build#Contributing_to_a_build]] for the "how to" using the automatic processing done by the [[Platform-releng/Automated Platform Build]]. | |
| − | + | [[Category:Eclipse_Platform_Releng| ]] | |
| − | + | ||
Latest revision as of 06:28, 7 July 2023
This page provides instructions for building the Eclipse Platform using preferred technologies identified as part of the CBI initiative.
Contents
Prerequisites
Free HDD space
- ~25GB is recommended.
Java 17 JVM
- Java 17 JDK needs to be on PATH as the VM that "runs the build".
- Verify correct version of java is used
- Set JAVA_HOME to point to your JDK
- Ensure your java is set to run in Server mode
- Notes:
- Some of the inner build callouts, like the SWT fragment build, depend on having an Oracle JVM.
- It is highly recommended to have all JDKs, from CDC-1.0/Foundation-1.0 to Java 11, in a configured toolchains.xml file. See Bug report and Mailing List) and see the Section below titled 'Using BREE Libs'.
Maven 3.6.3
- Download from http://maven.apache.org/download.html or use your Linux distribution provided version (whether it works depends on the distribution).
- make sure mvn is available in your PATH
Git
Linux
- You can install Git using your distro's package manager.
Mac OSX
- Download from http://code.google.com/p/git-osx-installer/
- Install using the downloaded DMG file
Windows
- Download Git for Windows from http://msysgit.github.com/
- Install using the downloaded exe file
- We recommend using the "Git Bash" application provided by Git for Windows as your commandline shell throughout the build
Important: Per Bug 376400 we discovered that msysgit has a max character limit somewhere around 256 which causes cloning files with a path longer than that to fail
See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=376400#c4
Workaround: Put your repo in the root of a drive and give it a short name. For example: C:\z
Example:
cd c: git clone -b master --recursive \ https://github.com/eclipse-platform/eclipse.platform.releng.aggregator.git \ z
Note: The final "z" parameter at the end of the command is important as tells git to checkout the repository and rename it to "z". This reduces the path length of the repository to be short enough to workaround Bug 376400.
Build environment setup
Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms
give maven JVM more ram
export MAVEN_OPTS=-Xmx2048m
If you are using Windows:
SET MAVEN_OPTS=-Xmx2048m
Building
Note: Unless otherwise stated, these instructions apply to Linux, MacOSX, and Windows platforms
cloning platform source tree
Replace master with the branch you'd like to checkout.
Clone the repository and checkout the branch:
git clone -b master --recursive \ https://github.com/eclipse-platform/eclipse.platform.releng.aggregator.git
To update this clone and all submodules:
cd eclipse.platform.releng.aggregator git pull --recurse-submodules git submodule update
If you want to switch from another branch to this one, replace git merge origin/master with:
git checkout -b master origin/master
Running the build
From the aggregator root, run:
mvn clean verify
- NOTE: If you build multiple streams on the same system, you'll want to add -Dmaven.repo.local=/some/directory/somewhere to the end of the 3 mvn commands to avoid collisions (Using a different local repo for each stream). Most casual developers won't be affected.
- On Windows use -Dmaven.repo.local=C:\path\to\somewhere
Running the build without tests
From the aggregator root, run:
mvn clean verify -DskipTests=true
- NOTE "-Dmaven.test.skip=true" cannot be used in CBI, see bug 456510
Build result
Once the build finishes, the following artifacts are created.
Packaged SDK zip files are located in your eclipse.platform.releng.aggregator directory under
org.eclipse.releng.tychoeclipsebuilder/sdk/target/products/
P2 repository
org.eclipse.repository/target/repository
Source bundles
org.eclipse.repository/target/repository/plugins
Packaged JUnit Plugin Tests and Automated Testing Framework
org.eclipse.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-1.0.0-SNAPSHOT-bundle.zip
Cleaning the build repo
To completely clean a build repo (before a new build or before updating the aggregator and submodules):
git submodule foreach git clean -f -d git submodule foreach git reset --hard HEAD git clean -f -d git reset --hard HEAD
To get a pristine build repository, after a build, you will have to specify the -x option for git clean.
git submodule foreach git clean -f -d -x git submodule foreach git reset --hard HEAD git clean -f -d -x git reset --hard HEAD
The reason is that /targets is specified in the .gitignore file, so git ... ignores them. They are included in .gitignore so they are not accidentally committed and don't clutter visual displays of "changes". Normally Maven will clean those up before a build, but in case you want to be positive you do not get an "old" copy, of one of the target artifacts, use -x.
Using BREE Libs
BREE libs can be used to build using the same BREE as what is used on build.eclipse.org. You will first need to download and install the ee.zip attached to Bug 386649 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=386649) and follow the instructions in the description of the bug.
Here's an example of toolchains.xml. It currently only works with Oracle JREs (see bug 389856). Someone running the platform build would need their own version of toolchains.xml in their build ids home directory's .m2 directory (~/.m2/toolchains.xml) which points to the location on their file system of the tools required (BREE libs and JDKs).
Once setup you can inform the build to use it by passing -Pbree-libs on the mvn build command.
Signing
Signing is supported when built using build.eclipse.org and is disabled by default.
- Simply Run the build with -Peclipse-sign parameter.
Building natives
Optionally, you can pass -Dnative= parameter with one of the following options to compile the natives for the specified native.
- cocoa.macosx.aarch64 - cocoa.macosx.x86_64 - gtk.linux.aarch64 - gtk.linux.ppc64le - gtk.linux.x86_64 - win32.win32.x86_64
Example:
mvn clean verify \ -DskipTests=true -Dnative=gtk.linux.x86_64
Building Selected Projects
Instead of a full build, selected projects can be built by going to the desired root directory of the project(s) and specifying thebuild-individual-bundlesprofile.
Example:
(from the eclipse.platform.releng.aggregator repository root) \ cd rt.equinox.framework \ mvn -Pbuild-individual-bundles clean verify
This would build all projects under rt.equinox.framework.
Example:
(from the eclipse.platform.releng.aggregator repository root) \ cd rt.equinox.framework/bundles/org.eclipse.osgi \ mvn -Pbuild-individual-bundles clean verify
This would build only the org.eclipse.osgi project.
Build output: p2 repo and RCP products
Once build is complete, look in the following folders for the usual output artifacts:
- eclipse.platform.releng.aggregator/eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products for the archives of the Eclipse SDK. (Siblings of sdk folder contain other RCP products produced by Platform Build). Those are the packages that are typically published at http://download.eclipse.org/eclipse/downloads/drops4/ .
- eclipse.platform.releng.aggregator/eclipse.platform.releng.tychoeclipsebuilder/eclipse.platform.repository/target/repository for the typical p2 repository containing installable unit shipped by Platform Build. This is the repository that get typically added to http://download.eclipse.org/eclipse/updates/ .
Running the Eclipse platform tests
Copy the junit tests and the CBI SDK (pick the one for your platform) that was built to a testing directory. Also unzip the junit tests and enter them.
mkdir -p /var/tmp/lts/R3_platform-tests cp eclipse.platform.releng.tychoeclipsebuilder/sdk/target/products/org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz /var/tmp/lts/R3_platform-tests cp eclipse.platform.releng.tychoeclipsebuilder/eclipse-junit-tests/target/eclipse-junit-tests-bundle.zip /var/tmp/lts/R3_platform-tests cd /var/tmp/lts/R3_platform-tests unzip eclipse-junit-tests-bundle.zip cd eclipse-testing
Modify the file equinoxp2tests.properties to point to the CBI built repository. (This example uses /home/user/R3_platform-aggregator as the CBI platform root)
org.eclipse.equinox.p2.tests.current.build.repo=file:/home/user/R3_platform-aggregator/eclipse.platform.repository/target/repository/
Note down the org.eclipse.equinox.p2.reconciler.tests.platform.archive.linux-x86_64= file name for your architecture. (In this example linux x86_64)
eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
Rename your org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz you copied earlier to match the name you jotted down.
mv ../org.eclipse.sdk.ide-linux.gtk.x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
Copy your eclipse-platform to eclipse SDK
cp eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
Download a copy of the latest released Eclipse Classic (Eclipse IDE for Java Developers) from http://www.eclipse.org/downloads/.
http://www.eclipse.org/downloads/
Copy the Eclipse Classic SDK to eclipse-platform
cp eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz
You should now have 4 copies of the platform, 2 from the latest released and 2 from the CBI build.
eclipse-SDK-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-3.7.2-linux-gtk-x86_64.tar.gz eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz eclipse-SDK-201204121421-linux-gtk-x86_64.tar.gz
Unzip the CBI platform
tar zxvf eclipse-platform-201204121421-linux-gtk-x86_64.tar.gz
Run the tests
./runtests -os linux -ws gtk -arch x86_64
In case you get "testing.properties: No such file or directory", rename or symlink test.properties
ln -s test.properties testing.properties
and re-run the tests.
Submitting for aggregator builds
See Platform-releng/Automated Platform Build#Contributing_to_a_build for the "how to" using the automatic processing done by the Platform-releng/Automated Platform Build.