Difference between revisions of "EclipseLink/Building"

From Eclipsepedia

Jump to: navigation, search
(ANT)
m (Environment Setup)
(9 intermediate revisions by one user not shown)
Line 1: Line 1:
== SVN Structure ==
+
== Git Structure ==
  
At the moment, all code can be found in the org.eclipse.persistence/trunk directory on the technology SVN server.
+
EclipseLink code has migrated from SVN to Git. The SVN repository has been split into several repositories. The repository list, descriptions, and urls can be found at: http://git.eclipse.org under the EclipseLink project list.
  
 
=== Runtime===
 
=== Runtime===
Line 72: Line 72:
 
====== Environment Setup ======
 
====== Environment Setup ======
 
Most of the required dependencies are already existent in one of the previously discussed plugin folders. However, there are some third-party files that are not supplied. Specifically:
 
Most of the required dependencies are already existent in one of the previously discussed plugin folders. However, there are some third-party files that are not supplied. Specifically:
* Ant 1.7.x. If not already installed, you will need to download and install it.
+
* Ant '''1.7.x'''. Our build scripts don't work properly under 1.8. If not already installed, you will need to download and install it. Make sure you have an "ANT_HOME" env variable that points to the installation, and a "JAVA_HOME" variable that points to your Java installation.
 
* Junit
 
* Junit
 
* Any library licensed under an agreement that is not EPL or Eclipse approved. (This includes Oracle libraries needed to compile oracle specific extensions)
 
* Any library licensed under an agreement that is not EPL or Eclipse approved. (This includes Oracle libraries needed to compile oracle specific extensions)
* Maven 3. Tycho requires Maven 3 functionality. So to build EclipseLink 2.4 and beyond you will need to download and install it if it is not already installed. Make sure you set an "M2_HOME" env variable that points to the installation.
+
* Maven 3. Tycho requires Maven 3 functionality. So to build EclipseLink 2.4 and beyond you will need to download and install it if it is not already installed. <font color="red">'''***Important***</font> Make sure you set an "M2_HOME" env variable that points to the installation.'''
* [http://www.aqute.biz/Code/Download#bnd BND] tool - we use bnd to generate our OSGi plugins in EclipseLink 2.3.x and earlier. Version .366 or later should work fine if placed in the extension.depend.dir or the buildsystem dir. In addition, if bnd is not available the build will attempt to generate meaningful OSGi manifests, but there are known shortcomings and issues with this approach.  
+
* [http://www.aqute.biz/Code/Download#bnd BND] tool '''(2.3.x and earlier only)'''- we use bnd to generate our OSGi plugins in EclipseLink 2.3.x and earlier. Version .366 or later should work fine if placed in the extension.depend.dir or the buildsystem dir. In addition, if bnd is not available the build will attempt to generate meaningful OSGi manifests, but there are known shortcomings and issues with this approach.  
**Add the following entry to your '''user-dir/build.properties''' as of 20100513
+
**Add the following entry to your '''user-dir/build.properties'''  
***<font color="red">''bndtool.jar=bnd-0.0.384.jar''</font>
+
***<font color="red">''bndtool.jar=bnd-0.0.384.jar''</font> as of 20100513
 +
***<font color="red">''bndtool.lib=bnd-0.0.384.jar''</font> as of at least 2.4.2
 +
 
  
 
The build provides mechanisms to incorporate these dependencies outside of SVN.
 
The build provides mechanisms to incorporate these dependencies outside of SVN.
Line 87: Line 89:
 
* library properties: Most dependant libraries are defined using a property definition that can be overridden
 
* library properties: Most dependant libraries are defined using a property definition that can be overridden
  
Overriding Defaults
+
====== Overriding Defaults ======
  
 
Basic Setup
 
Basic Setup
Line 103: Line 105:
 
* Ensure you have ant 1.7 set up to run
 
* Ensure you have ant 1.7 set up to run
 
* Open a command prompt in your checked out trunk directory
 
* Open a command prompt in your checked out trunk directory
* Run "ant" for a full build of all core and test code
+
* Run "ant" '''(or "ant -f antbuild.xml" in 2.4 or later)''' for a full build of all core and test code
 
* or
 
* or
* Run "ant build-eclipselink-jar" for a full build of only core code
+
* use the "build-eclipselink-jar" task for a full build of only core code
  
 
These commands will build the Runtime and/or the tests. Tests can also be run by adding a test execution target (like test-srg).
 
These commands will build the Runtime and/or the tests. Tests can also be run by adding a test execution target (like test-srg).

Revision as of 17:59, 14 January 2013

Contents

Git Structure

EclipseLink code has migrated from SVN to Git. The SVN repository has been split into several repositories. The repository list, descriptions, and urls can be found at: http://git.eclipse.org under the EclipseLink project list.

Runtime

The below is a description of the content of the directories. Note: All projects are Eclipse PDE projects and therefore classpath dependencies are defined in the manifest rather than the project metadata.

Plugins

The plugins directory contains all the common dependencies needed by all the projects. You can make use of them by either importing the Eclipse projects for all the depenencies or by copying the bundle jars (all the jars) in this directory in your Eclipse installation's plugin's directory. Note: The org.eclipse.persistence.asm and org.eclipse.persistence.antlr jars are not checked in and must be either be built by running the EclipseLink build or imported as projects.

The following dependencies can also be obtained from Orbit.

  • commonj.sdo 2.1.1
  • javax.activation 1.1.0
  • javax.mail 1.4.0
  • javax.persistence 1.0.0
  • javax.servlet 2.4.0
  • javax.wsdl 1.6.2
  • javax.xml.bind 2.0.0
  • javax.xml.rpc 1.1.0
  • javax.xml.stream 1.0.1
  • javax.xml.ws 2.1.0
  • org.apache.ant 1.7.0

Instructions about getting bundles from Orbit

Foundation

  • org.eclipse.persistence.core - All other EclipseLink components depend on this bundle. You will need to import this project to work on other EclipseLink components. All other components can be imported separately.
  • org.eclipse.persistence.oracle - extensions that require Oracle-specific libraries for compilation. If you do not have the required oracle libraries, you can get the compiled classes since they are checked in. This bundle is only used for advanced Oracle-specific functionality and not required unless you are using that functionality.
    • The oracle bundle requires that you configure the oracle target within your Eclipse. To do that, you will need to do the following:
      • Find the target.oracle directory in foundation/targets
      • Import the oracle.libs directory and its contents into Eclipse (In Ganymede, that means copying oracle.libs and its contents into your Eclipse dropins directory)
      • Obtain the Oracle dependency jars. There is a list of them in the manifest found in oracle.libs. Note: These are not currently available in open-source format and you will have to obtain them from Oracle
      • Put those jars in the oracle.libs directory you have copied to Eclipses
      • Restart Eclipse

JPA

  • org.eclipse.persistence.jpa - EclipseLink's JPA support
  • plugins - The additional dependencies for JPA support
  • org.eclipse.persistence.equinox - fragment for OSGi weaving on Equinox that plugs into org.eclispe.persistence.jpa. Only required for people doing weaving on Equinox
  • org.eclipse.persistence.equinox.weaving - fragment for OSGi weaving on Equinox that plugs into Equinox itself. Only required for people doing weaving on Equinox

MOXY

  • org.eclipse.persistence.moxy - EclipseLink's Object-XML support
  • plugins - The additional dependencies for Object-XML support

SDO

  • org.eclipse.persistence.sdo - EclipseLink's SDO support
  • plugins - The additional dependencies for SDO support

DBWS

  • org.eclipse.persistence.dbws - EclipseLink's Database Webservice support
  • plugins - The additional dependencies for DBWS support

Build

Importing the projects into Eclipse will build the class files for the PDE projects. Most functionality will be accessible after those builds. Ant files are also provided both for use in a non-Eclipse IDE, and for commandline use. If you want to do a complete build of the EclipseLink artifacts, the steps are outlined In the "building with Ant" section below.

Ant

Ant is used to drive all builds for the EclipseLink Project. Furthermore, it is assumed that EclipseLink will be built (ant tested) in environments other than the Eclipse IDE (PDE build). Therefore, the Ant buildfiles are designed to be used from many entry points with varying configurations. However, starting with 2.4, we have moved to a manifest-first approach to build our OSGi bundles. Therefore, rather than using BND to generate the OSGi manifest post build we are using Tycho (a maven plugin) to generate our OSGi bundles.

Building with Ant

Below are the steps to setup for and then run a basic build of EclipseLink using the Ant buildfiles from the trunk.

Environment Setup

Most of the required dependencies are already existent in one of the previously discussed plugin folders. However, there are some third-party files that are not supplied. Specifically:

  • Ant 1.7.x. Our build scripts don't work properly under 1.8. If not already installed, you will need to download and install it. Make sure you have an "ANT_HOME" env variable that points to the installation, and a "JAVA_HOME" variable that points to your Java installation.
  • Junit
  • Any library licensed under an agreement that is not EPL or Eclipse approved. (This includes Oracle libraries needed to compile oracle specific extensions)
  • Maven 3. Tycho requires Maven 3 functionality. So to build EclipseLink 2.4 and beyond you will need to download and install it if it is not already installed. ***Important*** Make sure you set an "M2_HOME" env variable that points to the installation.
  • BND tool (2.3.x and earlier only)- we use bnd to generate our OSGi plugins in EclipseLink 2.3.x and earlier. Version .366 or later should work fine if placed in the extension.depend.dir or the buildsystem dir. In addition, if bnd is not available the build will attempt to generate meaningful OSGi manifests, but there are known shortcomings and issues with this approach.
    • Add the following entry to your user-dir/build.properties
      • bndtool.jar=bnd-0.0.384.jar as of 20100513
      • bndtool.lib=bnd-0.0.384.jar as of at least 2.4.2


The build provides mechanisms to incorporate these dependencies outside of SVN. build defaults: The build assumes certain defaults for dependencies. These can be overridden, but are provided to allow minimal configuration setup.

  • external dependency location: The build assumes that all dependant libraries external to SVN are store in one of two default locations.
  • extensions.depend.dir: This property defines where external dependencies can be found. (default: .\trunk\..\extension.lib.external)
  • oracle.extensions.depend.dir: This is a legacy property for compatability with Oracle testing scripts. It is used to define the location of Oracle specific jars. (default: .\trunk\..\extension.oracle.lib.external)
  • library properties: Most dependant libraries are defined using a property definition that can be overridden
Overriding Defaults

Basic Setup

  • It may be necessary to create a user specific properties file to override the default values in <trunk>/build.properties. To do so, create a build.properties file in your user home directory (usually C:\Documents and Settings\<username> on windows). Variables commonly needing redefinition in it are:
    • For build:
    • For Test
      • tools.lib=<full path of tools jar in your JDK installation - usually in JAVA_HOME/lib>
      • jdbc.driver.jar=<full path to your JDBC driver>
      • db.driver=<classname of your JDBC driver>
      • db.url=<JDBC url of your DB>
      • db.user=<db username>
      • db.pwd=<db password>
      • (optional) db.platform=<EclipseLink Database Platform to use> - this is only necessary if EclipseLink does not properly dectect you DB with its database detection
  • Ensure you have ant 1.7 set up to run
  • Open a command prompt in your checked out trunk directory
  • Run "ant" (or "ant -f antbuild.xml" in 2.4 or later) for a full build of all core and test code
  • or
  • use the "build-eclipselink-jar" task for a full build of only core code

These commands will build the Runtime and/or the tests. Tests can also be run by adding a test execution target (like test-srg).

Note: Several DBWS component's build files are dependant upon JDK 1.6 (JDK 5 can be used but these components won't build)

Testing

Tests can be run through Ant or in Eclipse

From Ant

Make sure you are using the latest (at least v20090423) version of JUnit 4.5 In the trunk directory, you can use the following targets to run tests:

  • test-srg - Basic "smoke tests"
  • test-lrg - Full test model - must be run sucessfully on either MySQL or Oracle before checking in any changes to core classes
  • test-core - subset of tests just for the core
  • test-core-srg - smoke tess for the core
  • test-jpa - runs JPA tests - required prior to check-in for any changes to JPA included in "test-lrg"
    • JPA tests require some artifacts from the build in order to be able to run. There are two ways to get them.
      • Run the full build as indicated above
      • From the org.eclipse.persistence.jpa directory, run "ant"
    • This will create a directory required by the tests and the jars that contain the models. You only need to rebuild if you are adding or subtracting from the models (Package org.eclipse.persistence.testing.models.jpa and its subpackages).
  • test-wdf - runs the "WDF" JPA test suite (JPA tests contributed by SAP (Walldorf, Germany)).
  • test-moxy - runs MOXY tests - required prior to check-in for any changes to MOXY included in "test-lrg"
  • test-jaxb - required prior to check-in for any changes to JAXB included in "test-moxy"
  • test-oxm - required prior to check-in for any changes to OXM included in "test-moxy"
  • test-sdo - required prior to check-in for any changes to SDO included in "test-lrg"

From Eclipse

The following directories contain projects with tests

  • foundation/eclipselink.core.test - the core testing for EclipseLink
    • Testing Browser.launch - runs legacy test browser. Requires definition of JDBC_LIB and TOOLS_LIB classpath variables.
    • SRGTestModel.launch - runs the SRG. Requires definition of JDBC_LIB classpath variable
    • LRGTestModel.launch - runs the LRG. Requires definition of JDBC_LIB and TOOLS_LIB classpath variables.
  • foundation/eclipselink.extension.oracle.test - the testing for EclipseLink's Oracle-specific functionality.
    • Contains Testing Browser (Oracle).launch - Launches Legacy test browser in a manner that can run Legacy Oracle tests
  • jpa/eclipselink.jpa.test - the JPA tests for EclipseLink
    • See the description of the test-jpa target above for a prerequisite.
    • FullRegressionTestSuite.launch - runs JPA tests
      • Needs DB properties to run. either of the following can be done to give the DB properties:
        • Add a JVM argument: -Dtest.properties=<full path to build.properties in your user home>
        • update the test.properties file in jpa/eclipselink.jpa.test directory to point at correct DB
    • Testing Browser (JPA).launch - Launches Legacy test browser in a manner that can run JPA tests through that browser
      • Note: Recompilation of eclipselink.jpa.test test jars are required only if new classes were added/deleted since the last build.
  • jpa/eclipselink.spring.test - Spring specific JPA testing
  • moxy/eclipselink.moxy.test - MOXY testing
  • sdo/eclipselink.sdo.test - SDO testing
  • dbws/eclipselink.dbws.test - DBWS Testing

Minimum SVN Trunk projects for JPA or Core Eclipse Development and Testing

  • The following import screen capture from Eclipse 3.5 for the SVN trunk details the minimum projects you will need on top of the 10 Orbit projects in order to do JPA or Core development in Eclipse.

Minimum svn trunk projects for jpa core development.JPG

  • Note: Also add the
    • org.eclipse.persistence.moxy project (a requirement of eclipselink.jpa.test)
    • org.eclipse.persistence.moxy.dynamic (a requirement of moxy.test)

Projects and Dependencies

When using the Eclipse IDE - a Europa or later EE version is required. If you are using OSGi, a Ganymede EE version is required. See the Eclipse download site for information http://www.eclipse.org/downloads

All current dependencies can be found in the manifests for the PDE projects or in the bundles.

PDE Conversion

The following is inforation about our work to convert to PDE projects and is still here for people that had EclipseLink projects before our conversion. All other developers should be able to ignore it.

There is a new PDE structure being finalized that will require a rebuild of all existing EclipseLink projects in Eclipse. See [EclipseLink/Development/EclipseLinkPDEConversionPlan http://wiki.eclipse.org/EclipseLink/Development/EclipseLinkPDEConversionPlan]

These notes will be incorporated into the main doc and formalized when this process is complete.

  • Migrating Eclipse Projects to PDE #2 as of 20080508 - currently editing
  • This section is for developers that did the initial PDE refresh after 20080430.
    • Remove existing lib projects and reimport as plugin projects (delete project but not workspace contents)
      • All javax.*, org.apache.*, org.*antlr, org.*.asm
    • Perform actions below

Note: If you are getting compilation errors that are due to a missing plugin like javax.resource or javax.jms - and you have these projects in your workspace. The fix seems to be to delete these projects (not the filesystem) and reimport the project with the problem as well as the plugin that it is complaining it cannot find - together in a single import (from the <root>).

  1. Prerequisites
    1. Eclipse 3.3+ EE version required - you will need the EE version of eclipse
      1. Internal: eclipse-jee-europa-winter-win32, eclipse-jee-europa-fall2-win32 or eclipse-jee-ganymede-M6-win32
    2. Remove any existing eclipselink.* projects (delete project but not workspace contents)
    3. Note: eclipselink.*.test projects are not migrated yet, these can be retained
    4. Keep existing classpath variables
  2. Update SVN at the <trunk> root
  3. Importing Projects
    1. Import | General | Existing projects into Workspace
      1. root directory = <trunk>/plugins
      2. Leave "Copy projects into workspace" unselected
      3. select all of javax.* and org.*
    2. Import jpa/plugins/javax.persistence
    3. Import foundation projects in the same way
    4. Import jpa projects in the same way
    5. Import moxy/plugins projects so jpa test project compiles
    6. Import sdo/plugins
    7. Refresh/Rebuild
      1. If automatic build is not checked - check it and clean all projects
    8. core dependency on jpa lib project
      1. see javax.persistence;bundle-version="1.99.0";visibility:=reexport in the core project
      2. If you get a compile error on javax.persistence it is because we are referencing the 1.99 version instead of 1.0 in the manifest of org.eclipse.persistence.core - make sure the jpa project javax.persistence.1.99 is loaded in your workspace.
  4. Building
    1. Run ant at the <trunk> root so that all the test project jars are updated and built
    2. BUILD SUCCESSFUL Total time: 8 minutes 4 seconds
    3. Or run ant build-eclipselink-jar at the <trunk> root so that all the non-test projects are compiled and eclipselink.jar is built in under 60 seconds.
    4. BUILD SUCCESSFUL Total time: 8 minutes 4 seconds
  5. Running
    1. Test the build by running test.jpa
    2. <trunk>/ant test-jpa
    3. Note: if db.platform is not set in your <user-home>/test.properties then you may get a "ClassNotFoundException: ${db.platform}" - set this variable as follows (using your own platform)
      1. db.platform=org.eclipse.persistence.platform.database.oracle.Oracle11Platform
      2. Normally we do not set a dependecy on the platform though - this will be fixed shortly in our script.
      3. [junit] Tests run: 911, Failures: 0, Errors: 0, Time elapsed: 583.35 sec Total time: 10 minutes 4 Seconds
  6. Refresh eclipse workspace to optionally run the testing browser targets
  7. Pending:
  8. Launch Targets

Port TODO List

List of task, items, issues, refactorig left to due from the initial port.

  • Get all tests passing. - JPA JUnit tests and legacy LRG model currently pass
  • Move left-over internal classes to internal packages.
  • Internationalization.
  • Server tests.
  • Sun Corba and RMI-IIOP - are these elements still required in EclipseLink. If so, make sure tests work. Else, remove.
  • Make all server platforms work. i.e. address the fact that they all need to be reflective to compile

How To