Skip to main content
Jump to: navigation, search

Difference between revisions of "MemoryAnalyzer/Building MAT With Tycho"

m (Release Builds)
 
(38 intermediate revisions by 4 users not shown)
Line 2: Line 2:
  
 
This page describes how Memory Analyzer can be built using Maven/Tycho. The build will
 
This page describes how Memory Analyzer can be built using Maven/Tycho. The build will
* build all bundles and features
+
* build all MAT bundles
 
* execute all tests
 
* execute all tests
* (optional) run findbugs static checks
+
* (optional) run [http://findbugs.sourceforge.net/ FindBugs] static checks
* produce a p2 repository with MAT features
+
* build eclipse features containing the MAT plugins and produce an update site (p2 repository) with them
* produce RCPs for different platforms
+
* produce standalone (Eclipse RCP based) products for different OS platforms
 +
* sign and upload the produced artifacts (when executed on the Eclipse Jenkins server)
  
 
== Prerequisites ==
 
== Prerequisites ==
  
=== SVN Team Provider Installation ===
+
=== Clone the Source Code from Git ===
To import MAT sources into the IDE one can use the SVN Team Provider. Have a look at the first several steps here: [[MemoryAnalyzer/Contributor_Reference#Setup]]
+
MAT sources are in a Git repository, therefore you need a git client. Have a look at [[MemoryAnalyzer/Contributor_Reference#Get the source]]
  
=== Maven Installation ===
+
=== Use Java 1.8 for the Build ===
The Memory Analyzer build requires a Maven 3.0.* installation. It is already present on the Hudson server at Eclipse. For local build one can download it from here http://maven.apache.org/download.html.
+
The MAT build has to be run with Java 1.8. Make sure the JAVA_HOME environment variable is set to point to a JDK 1.8 installation.
  
=== Proxy Settings ===
+
=== Install and Configure Maven ===
If you need to set proxy for Maven, a snippet like this can be added to the Maven settings file:
+
The Memory Analyzer build requires a Maven 3.0.* installation. It is already present on the Jenkins server at Eclipse. For local build one can download it from [http://maven.apache.org/download.html here].
 +
 
 +
If you need to set a proxy for Maven, a snippet like this can be added to the Maven settings file:
 +
<source lang="xml">
 
   <proxies>
 
   <proxies>
 
     <proxy>
 
     <proxy>
Line 27: Line 31:
 
     </proxy>
 
     </proxy>
 
   </proxies>
 
   </proxies>
 
+
</source>
 
More information on Maven settings: http://maven.apache.org/ref/3.0.3/maven-settings/settings.html
 
More information on Maven settings: http://maven.apache.org/ref/3.0.3/maven-settings/settings.html
  
== Checkout MAT Sources from SVN ==
+
== Building MAT from Sources ==
If you want to use/edit MAT sources in the Eclipse IDE and build the same sources with Maven, then you need to take care that the directory structure is preserverd while importing the MAT projects using the SVN team provider. The following steps describe a way to do this:
+
 
+
* Open the 'SVN Repository Exploring' perspective
+
* Add the MAT Subversion repository at: https://dev.eclipse.org/svnroot/tools/org.eclipse.mat
+
* Select 'trunk' and open the context menu
+
* Select 'Find/Check Out As' from the context menu
+
* Select 'Find projects in the children of the selected resource'
+
* Select 'recursively'
+
* Press 'Finish'
+
* After some time, you'll get a window with the message: 'Following list contains projects that were automatically found on the repository. You can select project set you wish to be checked out into the workspace.'
+
* Keep all projects selected
+
* Select 'Check out as a projects into workspace'
+
* Select 'Respect projects hierarchy'
+
* Press 'Finish'
+
 
+
== Preparation Steps ==
+
=== Which preparation steps are performed? ===
+
Parts of MAT depend on the IBM's DTFJ libraries, therefore one needs the libraries during the build.
+
This preparation step will do the following:
+
* download DTFJ (zipped "old" update site)
+
* publish a p2 repository with DTFJ
+
* modify the target platform definition used for the rest of the build
+
 
+
=== Execute the preparation step ===
+
* Go in the ''<mat_src>/prepare_build/'' directory
+
* Execute
+
<pre>mvn clean install</pre>
+
* This maven build is using the maven-antrun plugin to download the zip containing DTFJ. If you need to set proxy you can do it using the standard Java settings, i.e. the command line will look like this:
+
<pre>mvn -Dhttp.proxyHost=myProxyHost -Dhttp.proxyPort=myProxyPort clean install</pre>
+
 
+
On the Hudson boxes at Eclipse the settings are defined as $ANT_OPTS
+
 
+
=== Verify ===
+
If the build was successful, then:
+
* ''<mat_src>/prepare_build/target/repository/'' will contain the p2 repository with DTFJ
+
* ''<mat_src>/org.eclipse.mat.targetdef/mat-site.target'' will be modified to point to this repository
+
 
+
== Building MAT ==
+
Once the [[#Preparation Steps]] are done (normally only once) one can run the maven build for MAT.
+
  
 
=== Execute the build ===
 
=== Execute the build ===
* Go to the root of the sources (there is the parent pom.xml), i.e. ''<mat_src>/pom.xml''
+
* Open a console and go into the ''<mat_src>/parent'' folder (it contains the parent pom.xml)
* To simply build MAT execute
+
* To build MAT with the default profile (build-snapshot) simply execute
 
<pre>mvn clean install</pre>
 
<pre>mvn clean install</pre>
* To build MAT and perform findbugs checks execut
+
* This will cause a fresh build of all bundles, execute tests, build eclipse features, an update site (p2 repository) and standalone products
 +
* If you want to also FindBugs checks are performed as part of the build, then execute
 
<pre>mvn clean install findbugs:findbugs</pre>
 
<pre>mvn clean install findbugs:findbugs</pre>
 +
 
=== Where to find the results? ===
 
=== Where to find the results? ===
 
You can find the results of the build in the corresponding .../target/ directories for each plugin, feature, etc... Of particular interest are:
 
You can find the results of the build in the corresponding .../target/ directories for each plugin, feature, etc... Of particular interest are:
 
* ''<mat_src>/org.eclipse.mat.updatesite/target/site/'' - it contains a p2 repository with MAT features
 
* ''<mat_src>/org.eclipse.mat.updatesite/target/site/'' - it contains a p2 repository with MAT features
* ''<mat_src>/product/target/products/'' - it contains all standalone RCP applications
+
* ''<mat_src>/org.eclipse.mat.product/target/products/'' - it contains all standalone RCP applications
 +
 
 +
== Building MAT Standalone RCPs from an Existing MAT Update Site ==
 +
 
 +
=== Configure and execute the build ===
 +
* Open a console and go into the ''<mat_src>/parent'' folder (it contains the parent pom.xml)
 +
* To produce only the standalone products, using an already existing MAT repository (i.e. without building the bundles again) specify that the ''build-release-rcp'' profile is used when you start maven:
 +
<pre>mvn clean install -P build-release-rcp</pre>
 +
* It will take the already existing MAT plugins/features from the repository specified by the ''mat-release-repo-url'' property in ''<mat_src>/parent/pom.xml''. One can overwrite this location when calling maven. For example, to build products with the older 1.5.0 release, use:
 +
<pre>mvn clean install -P build-release-rcp -Dmat-release-repo-url=http://download.eclipse.org/mat/1.5/update-site/</pre>
 +
 
 +
=== Where to find the results? ===
 +
You can find the standalone products under ''<mat_src>/org.eclipse.mat.product/target/products/''
  
 
== Further Information ==
 
== Further Information ==
* The platforms for which RCPs are built are specified in the ''<mat_src>/pom.xml'' file
+
* The platforms for which RCPs are built are specified in the ''<mat_src>/parent/pom.xml'' file
* There are two script files (''build.bat'' and ''build.sh'') which should combine the preparation steps and the real build. However, both still need some improvements
+
  
 
== Known Problems ==
 
== Known Problems ==
=== Can't build RCPs for Mac ===
 
The problem is reported here: https://bugs.eclipse.org/bugs/show_bug.cgi?id=313997. Putting a patched o.e.equinox.p2.publisher.jar under ''<maven_repository>/org/sonatype/tycho/tycho-p2-runtime/0.11.0/eclipse/plugins/'' helps for local build.
 
 
=== Wrong file permissions ===
 
=== Wrong file permissions ===
When building MAT on a Windows box, the RCPs for any other OS will not have the proper permissions (e.g. the executables won't have the x flag). Building under Lunix or other non-Windows OS helps.
+
When building MAT on a Windows box, the RCPs for any other OS will not have the proper permissions (e.g. the executables won't have the x flag). Building under Linux or other non-Windows OS helps.
 +
 
 +
== Jenkins Job at Eclipse ==
 +
The Jenkins continuous integration instance of the Memory Analyzer Project at the Eclipse is https://ci.eclipse.org/mat/.
 +
 
 +
=== Snapshot / Nightly builds ===
 +
The [https://ci.eclipse.org/mat/job/tycho-mat-nightly/ ''tycho-mat-nightly''] job checks regularly for changes to the Git repository and produces a snapshot build (see Building MAT from Sources) above. [2020: The automatic detection of changes is now working after problems in 2018,2019]
 +
 
 +
The job is additionally configured to sign the plugins and features in the update site, and to upload all artifacts to the download server.
 +
One can download such nightly/snapshot builds here: http://www.eclipse.org/mat/snapshotBuilds.php
 +
 
 +
Info:
 +
* Signing is activated by the build-server profile (i.e. with parameter '-P build-server' added to the maven command)
 +
* The macOS builds are also notarized by the [https://ci.eclipse.org/mat/job/mac-sign/ mac-sign job], which is automatically triggered after a successful snapshot build.
 +
 
 +
=== Release Builds ===
 +
The job [https://ci.eclipse.org/mat/job/mat-standalone-packages/ ''mat-standalone-packages''] can only be triggered manually to build the MAT standalone packages/products using an already existing MAT update site. This can be used in the following scenario - MAT has contributed its bundles and features to the simultaneous Eclipse release as part of a milestone or a release candidate. After the simultaneous release is complete, we would like to have exactly these bundles also packed in the standalone packages, potentially including also the final version of the dependencies (part of the same simultaneous release).
 +
 
 +
The job is configured to use the ''build-release-rcp'' profile when calling maven.
 +
 
 +
The job may need to be changed for each new release.
 +
 
 +
After building the packages the macOS build needs to be notarized using the [https://ci.eclipse.org/mat/job/mac-sign/ ''mac-sign job''], with the parameter of the actual relative location of the dmg file on the download server.
 +
 
 +
The downloads can then be tested.
 +
 
 +
The job [https://ci.eclipse.org/mat/job/mat-promote-release/ ''mat-promote-release''] copies the files to their final location so they can be downloaded by all the users.
  
== Hudson Job at Eclipse ==
+
The job [https://ci.eclipse.org/mat/job/publish-latest-update-site/ ''publish-latest-update-site''] copies a particular release update site to the /mat/latest/update-site
The job at the Eclise Hudson server that performs this build is https://hudson.eclipse.org/hudson/job/tycho-mat-nightly/.
+
  
There are still a number of TODOs:
+
[[Category:Memory Analyzer]]
* sign the jars of the update site automatically. Hopefully this can help (haven't tried yet): http://aniszczyk.org/2011/05/18/eclipse-org-signing-support-for-maven-tycho/
+
* publish nightly results on the download server
+

Latest revision as of 03:03, 3 September 2020

Introduction

This page describes how Memory Analyzer can be built using Maven/Tycho. The build will

  • build all MAT bundles
  • execute all tests
  • (optional) run FindBugs static checks
  • build eclipse features containing the MAT plugins and produce an update site (p2 repository) with them
  • produce standalone (Eclipse RCP based) products for different OS platforms
  • sign and upload the produced artifacts (when executed on the Eclipse Jenkins server)

Prerequisites

Clone the Source Code from Git

MAT sources are in a Git repository, therefore you need a git client. Have a look at MemoryAnalyzer/Contributor_Reference#Get the source

Use Java 1.8 for the Build

The MAT build has to be run with Java 1.8. Make sure the JAVA_HOME environment variable is set to point to a JDK 1.8 installation.

Install and Configure Maven

The Memory Analyzer build requires a Maven 3.0.* installation. It is already present on the Jenkins server at Eclipse. For local build one can download it from here.

If you need to set a proxy for Maven, a snippet like this can be added to the Maven settings file:

  <proxies>
    <proxy>
      <active>true</active>
      <protocol>http</protocol>
      <port>8080</port>
      <host>myproxy_host</host>
      <nonProxyHosts>non_proxy_hosts</nonProxyHosts>
    </proxy>
  </proxies>

More information on Maven settings: http://maven.apache.org/ref/3.0.3/maven-settings/settings.html

Building MAT from Sources

Execute the build

  • Open a console and go into the <mat_src>/parent folder (it contains the parent pom.xml)
  • To build MAT with the default profile (build-snapshot) simply execute
mvn clean install
  • This will cause a fresh build of all bundles, execute tests, build eclipse features, an update site (p2 repository) and standalone products
  • If you want to also FindBugs checks are performed as part of the build, then execute
mvn clean install findbugs:findbugs

Where to find the results?

You can find the results of the build in the corresponding .../target/ directories for each plugin, feature, etc... Of particular interest are:

  • <mat_src>/org.eclipse.mat.updatesite/target/site/ - it contains a p2 repository with MAT features
  • <mat_src>/org.eclipse.mat.product/target/products/ - it contains all standalone RCP applications

Building MAT Standalone RCPs from an Existing MAT Update Site

Configure and execute the build

  • Open a console and go into the <mat_src>/parent folder (it contains the parent pom.xml)
  • To produce only the standalone products, using an already existing MAT repository (i.e. without building the bundles again) specify that the build-release-rcp profile is used when you start maven:
mvn clean install -P build-release-rcp
  • It will take the already existing MAT plugins/features from the repository specified by the mat-release-repo-url property in <mat_src>/parent/pom.xml. One can overwrite this location when calling maven. For example, to build products with the older 1.5.0 release, use:
mvn clean install -P build-release-rcp -Dmat-release-repo-url=http://download.eclipse.org/mat/1.5/update-site/

Where to find the results?

You can find the standalone products under <mat_src>/org.eclipse.mat.product/target/products/

Further Information

  • The platforms for which RCPs are built are specified in the <mat_src>/parent/pom.xml file

Known Problems

Wrong file permissions

When building MAT on a Windows box, the RCPs for any other OS will not have the proper permissions (e.g. the executables won't have the x flag). Building under Linux or other non-Windows OS helps.

Jenkins Job at Eclipse

The Jenkins continuous integration instance of the Memory Analyzer Project at the Eclipse is https://ci.eclipse.org/mat/.

Snapshot / Nightly builds

The tycho-mat-nightly job checks regularly for changes to the Git repository and produces a snapshot build (see Building MAT from Sources) above. [2020: The automatic detection of changes is now working after problems in 2018,2019]

The job is additionally configured to sign the plugins and features in the update site, and to upload all artifacts to the download server. One can download such nightly/snapshot builds here: http://www.eclipse.org/mat/snapshotBuilds.php

Info:

  • Signing is activated by the build-server profile (i.e. with parameter '-P build-server' added to the maven command)
  • The macOS builds are also notarized by the mac-sign job, which is automatically triggered after a successful snapshot build.

Release Builds

The job mat-standalone-packages can only be triggered manually to build the MAT standalone packages/products using an already existing MAT update site. This can be used in the following scenario - MAT has contributed its bundles and features to the simultaneous Eclipse release as part of a milestone or a release candidate. After the simultaneous release is complete, we would like to have exactly these bundles also packed in the standalone packages, potentially including also the final version of the dependencies (part of the same simultaneous release).

The job is configured to use the build-release-rcp profile when calling maven.

The job may need to be changed for each new release.

After building the packages the macOS build needs to be notarized using the mac-sign job, with the parameter of the actual relative location of the dmg file on the download server.

The downloads can then be tested.

The job mat-promote-release copies the files to their final location so they can be downloaded by all the users.

The job publish-latest-update-site copies a particular release update site to the /mat/latest/update-site

Back to the top