EclipseLink/DesignDocs/328404 new

Design Specification: Persistence Unit Composition

This feature adds support for persistence units being composed. This means a user can define multiple persistence units with a unique set of entity types and expose them through a composite persistence unit which combines the set of classes across multiple data sources to expose a single persistence context.

ER 328404

Document History

old version of the doc

Date	committer	description
2011/04/06	Andrei Ilitchev	Initial New Version

Project overview

The goal is to combine several persistence units into a single one.

em.persist(new A(..));
em.persist(new Z(..));
// insert A into db1; insert Z into db2:
// the two different data bases possibly from different vendors
em.flush();

Concepts

Terminology

• Data Source: For the purposes of this discussion a data-source will include JTA and non-JTA data sources as well as EclipseLink's native JDBC connection pools.
• Composite PU: Refers to a PU which combines 2 or more Composite Member PUs to provide a persistence context which allows queries and transactions (with limitations) across the combined set of entity types.

Requirements

#	Description
1	Add support for a single persistence context accessing entities stored in different Data Sources (schemas/table-space/database) where application developers can perform queries and transactions across the complete set of entities transparently.
2	Support mapping relationships between entities in different Data Sources.
3	Provide easy to use JPA configuration.
4	Clearly capture usage limitations and exceptions expected in Java Docs and user documentation. The most basic example of this is the ability to join across tables in different Data Sources. This limitation effects mapping and query execution and optimizations. These issues need to be identified and documented.
5	Support container managed and application bootstrap JPA usage with dynamic, static, and no weaving.

User Bugs

The following potentially related bugs should be taken into consideration in this feature work.

• JPA enhancement requests.
  • bug 260258 - Add JPA EntityManagerFactoryBroker support

• Core bugs
  • bug 269213 - SessionBroker handling of ExceptionHandler broken because stale copy/paste
  • bug 281569 - Integration of JPA using transaction-type="JTA" but without EJB container, set-up incomplete commit cleanup
  • bug 326649 - DatabaseSessionImpl.finalize() leads to ClassCastException
  • bug 329185 - SessionBroker: ClassCastException in Cursor.buildAndRegisterObject between ServerSession and UnitOfWorkImpl
  • bug 332581 - SessionBroker session event issues to consider and resolve when implementing JPA Session Broker
  • bug 331712 - Allow Isolated classes to work with client session broker

Testing

• The tests are (as always) in trunk\jpa\eclipselink.jpa.test, they are added to FullRegressionTestSuite.
• New test models were defined that mirror some existing test models.
• Each of the new models was divided into three composite members.
  • Division was done in such a way that there are lots of references between classes that belong to different composite members.
• For each new model a new test suite was created that mirrors existing test suite.
• It might be useful to compare the sources of the original and composite versions of mapped classes and tests to see composite specifics and limitations.
• By default all three composite member persistence unit in each model will use the same data base (specified by db.* properties in test.properties file or passed to TestBrowser)
  • To test with three different databases, in test.properties file specify
    • either single.db=false - that would cause member_2 and member_3 to use connection parameters specified in their persistence.xml files;
    • or db2.* and db3.* properties (following the pattern of db.* properties).
    • member_1 always uses the main data base (specified by db.* properties in test.properties file or passed to TestBrowser).

Annotations

• Original
  • Source (src)
    • org.eclipse.persistence.testing.models.jpa.advanced.*
    • org.eclipse.persistence.testing.tests.jpa.advanced.EntityManagerJUnitTestSuite
  • Persistence Unit "default"
    • Resource
      • eclipselink-annotation-model
    • Model jar file
      • eclipselink-annotation-model.jar

• Composite
  • Source (src)
    • org.eclipse.persistence.testing.models.jpa.composite.advanced.*
    • org.eclipse.persistence.testing.models.jpa.composite.advanced.member_1.*
    • org.eclipse.persistence.testing.models.jpa.composite.advanced.member_2.*
    • org.eclipse.persistence.testing.models.jpa.composite.advanced.member_3.*
    • org.eclipse.persistence.testing.tests.jpa.composite.advanced.EntityManagerJUnitTestSuite
  • Persistence Unit "composite-advanced"
    • Resource
      • eclipselink-composite-advanced-model
      • eclipselink-composite-advanced-model-member_1
      • eclipselink-composite-advanced-model-member_2
      • eclipselink-composite-advanced-model-member_3
    • Model jar file
      • eclipselink-composite-advanced-model.jar
    • Member jar files
      • eclipselink-composite-advanced-model-member_1.jar
      • eclipselink-composite-advanced-model-member_2.jar
      • eclipselink-composite-advanced-model-member_3.jar
  • Members' database objects' (tables and sequences) names
    • prefixed with "MBR1_", "MBR2_", "MBR3_" respectively.

XML

• Original
  • Source (src)
    • org.eclipse.persistence.testing.models.jpa.xml.advanced
    • org.eclipse.persistence.testing.tests.jpa.xml.advanced.EntityMappingsAdvancedJUnitTestCase
      • to run it with pu "extended-advanced" (uses eclipselink-orm.xml) use -Dorm.testing=eclipselink
  • Persistence Unit "default"
    • Resource
      • eclipselink-xml-only-model
    • Model jar file
      • eclipselink-annotation-model.jar
  • Persistence Unit "extended-advanced" (uses eclipselink-orm.xml)
    • Resource
      • eclipselinkorm\eclipselink-xml-extended-model
    • Model jar file
      • eclipselink-xml-extended-model.jar

• Composite
  • Source (src)
    • org.eclipse.persistence.testing.models.jpa.xml.composite.advanced.*
    • org.eclipse.persistence.testing.models.jpa.xml.composite.advanced.member_1.*
    • org.eclipse.persistence.testing.models.jpa.xml.composite.advanced.member_2.*
    • org.eclipse.persistence.testing.models.jpa.xml.composite.advanced.member_3.*
    • org.eclipse.persistence.testing.tests.jpa.composite.xml.composite.advanced.EntityMappingsAdvancedJUnitTestCase
      • to run it with pu "xml-extended-composite-advanced" (uses eclipselink-orm.xml) use -Dorm.testing=eclipselink
  • Persistence Unit "xml-composite-advanced"
    • Resource
      • eclipselink-xml-composite-advanced-model
      • eclipselink-xml-composite-advanced-model-member_1
      • eclipselink-xml-composite-advanced-model-member_2
      • eclipselink-xml-composite-advanced-model-member_3
    • Model jar file
      • eclipselink-xml-composite-advanced-model.jar
    • Member jar files
      • eclipselink-xml-composite-advanced-model-member_1.jar
      • eclipselink-xml-composite-advanced-model-member_2.jar
      • eclipselink-xml-composite-advanced-model-member_3.jar
  • Persistence Unit "xml-extended-composite-advanced"
    • Resource
      • eclipselinkorm\eclipselink-xml-extended-composite-advanced-model
      • eclipselinkorm\eclipselink-xml-extended-composite-advanced-model-member_1
      • eclipselinkorm\eclipselink-xml-extended-composite-advanced-model-member_2
      • eclipselinkorm\eclipselink-xml-extended-composite-advanced-model-member_3
    • Model jar file
      • eclipselink-xml-extended-composite-advanced-model.jar
    • Member jar files
      • eclipselink-xml-extended-composite-advanced-model-member_1.jar
      • eclipselink-xml-extended-composite-advanced-model-member_2.jar
      • eclipselink-xml-extended-composite-advanced-model-member_3.jar
  • Members' database objects' (tables and sequences) names
    • prefixed with "XML_MBR1_", "XML_MBR2_", "XML_MBR3_" respectively.

Tooling

The Dali team should be made aware of this feature work and track an enhancement request to uptake it.

Config files

• Composite PU must be configured in persistence.xml using property:

/**
* Indicates if it's a composite persistence unit ("true").
* The property must be specified in persistence.xml of a composite persistence unit.
* The property passed to createEntityManagerFactory method or in system properties is ignored.
* Composite persistence unit would contain all persistence units found in jar files specified by <jar-file> elements in persistence.xml. 
*    <jar-file>member1.jar</jar-file>
*    <jar-file>member2.jar</jar-file>
*    <properties>
*        <property name="eclipselink.composite-unit" value="true"/>
*    </properties>
* @see #COMPOSITE_UNIT_MEMBER
* @see #COMPOSITE_UNIT_PROPERTIES
*/
public static final String COMPOSITE_UNIT = "eclipselink.composite-unit";

• Composite Member PU may specify in persistence.xml that it can't be used as an independent persistence unit:

/**
* Indicates if the persistence unit must be a member of a composite persistence unit ("true"),
* can't be used as an independent persistence unit.
* That happens if persistence unit has dependencies on other persistence unit(s).
* The property may be specified in persistence.xml.
* The property passed to createEntityManagerFactory method or in system properties is ignored.
* If this property is set to true, EntityManagerFactory still could be created,
* but it can't be connected: an attempt to create entity manager would cause an exception.
* @see #COMPOSITE_UNIT
* @see #COMPOSITE_UNIT_PROPERTIES
*/
public static final String COMPOSITE_UNIT_MEMBER = "eclipselink.composite-unit.member";

• While creating Composite PU EntityManagerFactory properties could be passed to Composite Member PUs:

/**
* The property may be passed to createEntityManagerFactory method of a composite persistence unit
* to pass properties to member persistence units.
* The value is a map: 
* the key is a member persistence unit's name,
* the value is a map of properties to be passed to this persistence unit. 
* "eclipselink.composite-unit.properties" -> (
*   ("memberPu1" -> (   "javax.persistence.jdbc.user" -> "user1", 
*                          "javax.persistence.jdbc.password" -> "password1",
*                          "javax.persistence.jdbc.driver" -> "oracle.jdbc.OracleDriver",
*                          "javax.persistence.jdbc.url" -> "jdbc:oracle:thin:@oracle_db_url:1521:db",
*                    ) , 
*   ("memberPu2" -> (   "javax.persistence.jdbc.user" -> "user2",
*                          "javax.persistence.jdbc.password" -> "password2"
*                          "javax.persistence.jdbc.driver" -> "com.mysql.jdbc.Driver",
*                          "javax.persistence.jdbc.url" -> "jdbc:mysql://my_sql_db_url:3306/user2",
*                    )
* )
* @see #COMPOSITE_UNIT
*/
public static final String COMPOSITE_UNIT_PROPERTIES = "eclipselink.composite-unit.properties";

Example

• compositePu specifies transaction type and server platform. It will contain all persistence units defined in member1.jar and member2.jar files.

<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence persistence_1_0.xsd" version="1.0">
    <persistence-unit name="compositePu" transaction-type="JTA">
        <provider>
            org.eclipse.persistence.jpa.PersistenceProvider
        </provider>
 
        <jar-file>member1.jar</jar-file>
        <jar-file>member2.jar</jar-file>
 
        <properties>
            <property name="eclipselink.composite-unit" value="true"/>
            <property name="eclipselink.target-server" value="WebLogic_10"/>
        </properties>
    </persistence-unit>
</persistence>

• memberPu1 defined in member1.jar file. It could be used independently as well as inside composite persistence unit.

<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence persistence_1_0.xsd" version="1.0">
    <persistence-unit name="memberPu1" transaction-type="JTA">
        <provider>
            org.eclipse.persistence.jpa.PersistenceProvider
        </provider>
        <mapping-file>META-INF/advanced-entity-mappings1.xml</mapping-file>
        <jta-data-source>jdbc/OracleJtaDS</jta-data-source>
        <exclude-unlisted-classes>false</exclude-unlisted-classes>
        <properties>
            <property name="eclipselink.target-server" value="WebLogic_10"/>
            <property name="eclipselink.target-database" value="org.eclipse.persistence.platform.database.oracle.Oracle11Platform"/>
        </properties>
    </persistence-unit>
</persistence>

• memberPu2 defined in member2.jar file. It has dependency on a class defined in member1.jar and can't be used independently.

<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence persistence_1_0.xsd" version="1.0">
    <persistence-unit name="memberPu2">
        <provider>
            org.eclipse.persistence.jpa.PersistenceProvider
        </provider>
        <mapping-file>META-INF/advanced-entity-mappings2.xml</mapping-file>
        <jta-data-source>jdbc/MySqlJtaDS</jta-data-source>
        <exclude-unlisted-classes>false</exclude-unlisted-classes>
        <properties>
            <property name="eclipselink.composite-unit.member" value="true"/>
            <property name="eclipselink.target-database" value="org.eclipse.persistence.platform.database.MySQLPlatform"/>
        </properties>
    </persistence-unit>
</persistence>

All three jars - composite and the two members - should be deployed on the same class loader. In case of application server the jars should by packed in an ear file; in standalone case put on the class pass. As usual compositePu could be accessed either through injection (in app. server case):

@PersistenceUnit(unitName="compositePu")
EntityManagerFactory entityManagerFactory;
@PersistenceUnit(unitName="compositePu")
EntityManager entityManager;

or using Persistence:

EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory("compositePu", properties);

Limitations

• Main limitations' cause is that tables in different data bases cannot be joined in a query. Examples:
  • Entities mapped in different composite members could not be joined in a query.
  • If deletion of an entity causes deletion from a table that is mapped by another composite member then delete all query can't be performed.
    • Therefore delete all cannot be used if an entity
      • has ElementCollection mapped in another composite member,
      • owns bidirectional relationship with a JoinTable with target mapped in another composite member,
      • has any reference mapping that is privately owned with target mapped in another composite member.
• Inheritance hierarchy cannot be shared between different composite members.
• If target of a reference mapping is defined in a different composite member and JoinTable is used, then the join table must belong to the target composite member.
  • Such mappings must be unidirectional - the "invert" mapping cannot use "mappedBy".
    • That's because JoinTable must be defined in the same composite member with target entity: master mapping requires it to be in slave's member, slave mapping - in master's.
  • Workaround is to define independent invert mapping with its own JoinTable.
    • If user maintains both sides of the relationship then the two join tables will be in sync.
• Native query created without result class must specify the target composite member persistence unit.

em.createNativeQuery("SELECT F_NAME FROM MBR2_EMPLOYEE").setHint(QueryHints.COMPOSITE_UNIT_MEMBER, memberPuName).getResultList();

Documentation

Still to come.

Open Issues

This section lists the open issues that are still pending that must be decided prior to fully implementing this project's requirements.

Issue #	Description / Notes
1	Should this feature include both split and aggregated persistence units? NO split support.
2	In the case of split persistence units where should the additional Data Sources be defined? PU properties in persistence.xml, eclipselink-orm.xml, both? Since it is possible for different Data Sources to be different database vendors or versions it should be possible to configure all JDBC/connection/pool level options on each data source. NO split support.
3	Should this feature include support to customize additional Data Sources with persistence unit properties. NO split support.
4	In the case of aggregate persistence units, should persistence units be usable on their own. YES, if independent.

Decisions

This section lists decisions made. These are intended to document the resolution of open issues or constraints added to the project that are important.

Issue #	Description / Notes	Decision