Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/Development/Dynamic"

 
(147 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= EclipseLink Incubator: Dynamic Persistence =
+
<css>
<table><tr>
+
  .source-sql {padding:1em;border:1px solid black; background-color: white;}
<td width="75%">
+
  .source-java5 {padding:1em;border:1px solid black; background-color: white;}
This page captures the functional requirements and design of a dynamic persistence solution that will allow consumers to specify their mappings. The purpose of dynamic persistence is to enable simplified data access where only mapping information is required and no concrete Java model is required.
+
  .source-xml {padding:1em;border:1px solid black; background-color: white;}
 +
  .source-text {padding:1em;border:1px solid black; background-color: white;}
 +
</css>
 +
__NOTOC__
 +
= EclipseLink Dynamic Persistence =
 +
This page captures the functional requirements and design of a Dynamic Persistence solution that will be added to EclipseLink from the EclipseLink Incubator. The work is being tracked by {{bug|200045}}.
  
The work is being developed within the [[EclipseLink/Development/Incubator]] and is being tracked by {{bug|200045}}.
+
== Document History ==
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Date
 +
! Author
 +
! Version Description & Notes
 +
|-
 +
| 090917
 +
| Doug Clarke, Mike Norman
 +
| 1.0 (most issued close)
 +
|-
 +
| 100311
 +
| Mike Norman
 +
| 2.0 (Refactoring for Sparse Merge - Phase I)
 +
|}
 +
<br/>
  
===Terminology===
+
__TOC__
====Dynamic Persistence====
+
=== Definition ===
This refers to the ability within an application to define a persistent entity class and use it within the application without the entity class existing (no.java or .class files). This is accomplished by creating a class on the fly using byte-codes.
+
{{:EclipseLink/Development/Dynamic/def1}}
 +
==== Configuration ====
 +
{{:EclipseLink/Development/Dynamic/def2}}
 +
{{:EclipseLink/Development/Dynamic/def3}}
  
====Static Configuration====
+
== Functional Requirements ==
This refers to the dynamic entity's structure and O-R mappings being defined in XML within the packaged application. This is used with Dynamic Persistence when the application is being generated before deployment and no runtime entity class generation/compilation/packaging is wanted.
+
* JPA (ORM)
 +
*# Support for defining dynamic types using deployment XML (with or without sessions.xml) with EclipseLink native ORM API
 +
*# Support for defining dynamic types using deployment XML with sessions.xml with EclipseLink JPA using PU properties to specify sessions.xml location and session name
 +
*# Support for defining dynamic entities at runtime for use with EclipseLink native ORM API
 +
*# Support for defining dynamic entities at runtime for use with EclipseLink JPA
 +
*# Support for removal of a dynamic type - removing the descriptor and any entity caching
 +
*# Support for heterogeneous models where some entity types are persistent and some are dynamic including relationships between these
 +
*# Mappings Supported
 +
*#* Basic/DirectToField
 +
*#* Relationships: 1:1, 1:M, M:M, Basic/DirectCollection
 +
*#* Multi-table and composite PK
 +
*#* Single table and Joined inheritance
 +
*# Schema Creation
 +
*#* Support for creation of new tables - requires a non JTA JDBC connection
 +
* Extension/Customization of Dynamic Persistence capabilities
 +
*# DBWS uses dynamic entities in a 'static' configuration, i.e. once the Web service is deployed and initialized, no new properties can be introduced. In addition, the public <code>DynamicEntity</code> API will never be exposed. With these simpler/different requirements, ensure DBWS can function within the Dynamic Persistence framework as well as use DBWS-specific custom classes.
  
====Dynamic Configuration====
+
The targeted Java version for this functionality is Java SE 5. <br />NB - The build machine uses Java SE 6, but it is configured to generated version 49 byte codes (49 = Java SE 5, 50 = Java SE 6,
This refers to the structure and mappings of the Dynamic entities being defined within the runtime of the application. This generally requires the structure to be persisted so it can be shared between application instances and preserved when the application is shutdown and later restarted.
+
51 = Java SE 7 ...) which can run un-modified on all >5 JVMs.
 +
=== Limitations ===
 +
The following are the limitations of the initial EclipseLink Dynamic Persistence capabilities:
 +
# JPA/ORM Configuration
 +
#* Support will NOT be available for usage of JPA's orm.xml or eclipselink-orm.xml. This will require the attribute-classification to be added to the eclipselink-orm XSD and the metadata processing to support trusting this classification instead of accessing the attribute type from the entity class.
 +
#** {{bug|241659}}: Add support to disable class validation in JPA metadata processing - 
 +
#** {{bug|267217}}: Add Named Access Type to EclipseLink ORM - enabling ValueAccessor config directly
 +
#* There will be NO support for dynamic entity types having any static state (I.e. Hybrid classes). This means you cannot have a static class with additional dynamic attributes or a static entity superclass or mapped superclass
 +
#* Mapping Limitations
 +
#** EmbeddedId, Maps
 +
#** Relationship management
 +
# Dynamic Type Storage: This solution supports the usage of entity types defined at runtime but does not store or share these definitions across multiple application instances or executions of the application. It is the responsibility of the application to maintain theior definitions and create them using the provided API when required.
 +
# MOXy
 +
#* Support for defining dynamic types using deployment XML with sessions.xml to create a JAXB Context
 +
#* Support for defining dynamic types using deployment XML with sessions.xml to create a native XML Context (future)
 +
# JPA/ORM with MOXy (planned for future release)
 +
#* Support all of the JPA/ORM configuration options above with additional support for native/JAXB MOXy mappings of the same dynamic entities
 +
#** The MOXy mappings are limited to be a subset of the JPA/ORM mappings
 +
# DBWS
 +
#* Upgrade the DBWS runtime to use the common DynamicClassLoader and DynamicClassWriter (done)
 +
#* If possible share a common base class and support for MOXy with JPA (done)
 +
# SDO (planned for future release)
 +
#* Upgrade the SDO runtime for dynamic data objects to use the common DynamicClassLoader and DynamicClassWriter
 +
#** SDO will require its own base class with a pluggable value-store as well as an extended DynamicClassWriter to handle the additional interfaces necessary
  
</td>
+
== Design ==
<td style="background-color: #cff8d9; border-width: 1px; border-style: solid; border-color: #999999; padding: 10px;">
+
The design of this new public functionality of EclipseLink is based on existing solutions consumers have used. These existing customers include Oracle BPEL/ESB, EclipseLink DBWS, and EclipseLink SDO. The design of this feature is divided into several pieces each described on their own page:
__TOC__
+
:[[EclipseLink/Development/Dynamic/Design_DynamicClassCreation | Dynamic Class Creation]] - How EclipseLink will support creating classes
</td>
+
:[[EclipseLink/Development/Dynamic/Design_PublicAPI | Public API for runtime JPA and Native ORM Usage]]
</tr>
+
::New Public API: <code>org.eclipse.persistence.dynamic</code>
</table>
+
::New Private implementation: <code>org.eclipse.persistence.internal.dynamic</code>
 +
:[[EclipseLink/Development/Dynamic/Design_NativeXML | Native XML Configuration]] - deployment/sessions XML usage
 +
:[[EclipseLink/Development/Dynamic/Design_RuntimeDefinition | Defining/Modifying/Removing dynamic entity types at runtime]]
  
== Functional Requirements ==
+
== Usage Examples ==
  
# Ability to use dynamic persistence through standard JPA metadata and interfaces as well as leveraging advanced EclipseLink JPA features
+
The following usage examples illustrate how users of the new API can leverage dynamic persistence in their application.
#* Specify mapping metadata using persistence.xml and eclipselink-orm.xml.
+
#* Create dynamic types at runtime
+
#* Access dynamic types through standard (JPA2) metamodel + helpers
+
#* Support removal of dynamic types
+
# Ability to use dynamic persistence through MOXy mappings
+
# Ability to use dynamic persistence within DBWS to share common infrastructure
+
# Ability to use dynamic persistence within SDO to share common infrastructure
+
  
=== Future Features (Current Limitations) ===
+
=== Creating Types in Application ===
  
In order to control the scope of this feature the following limitations have been intended and are listed here as potential future enhancements.
+
Application developers can define new dynamic entity types, configure their mappings, and add the new type to the active session.
  
# '''Flexible Base Class''': This feature assumes the dynamic entity base class. In the future users may want the flexibility to specify a class of their own.
+
<source lang="java5">
# '''LimitedMapping Capability''': This feature does not provide support for all mapping types. The mappings currently not supported include:
+
DynamicClassLoader dcl = DynamicClassLoader.lookup(session);
#* Inheritance
+
Class<?> javaType = dcl.createDynamicClass("model.Simple");
#* Embedded/EmbeddedId
+
#* Composite primary keys
+
# '''Hybrid Classes''': This feature only supports ''DynamicEntity'' persistent classes. In the future developers may want the ability to combine static java classes with one or more attributes accessed dynamically. The ''DynamicEntityImpl'' class provides support for lazy loading, fetch-groups, and attribute change-tracking. Combining this with traditional static attributes and weaving complex.
+
# '''Native ORM''': This feature relies on the JPA metadata processing of eclipselink-orm.xml. When this XML is supported through the native API then dynamic persistence should function as well
+
  
=== Incubator Exit Criteria ===
+
EntityTypeBuilder typeBuilder = new JPAEntityTypeBuilder(javaType, null, "SIMPLE_TYPE");
 +
typeBuilder.setPrimaryKeyFields("SID");
 +
typeBuilder.addDirectMapping("id", int.class, "SID");
 +
typeBuilder.addDirectMapping("value1", String.class, "VAL_1");
 +
typeBuilder.addDirectMapping("value2", boolean.class, "VAL_2");
 +
typeBuilder.addDirectMapping("value3", Calendar.class, "VAL_3");
 +
typeBuilder.addDirectMapping("value4", Character.class, "VAL_4");
  
In order to exit the incubator and become part of the Core/JPA/MOXy/SDO/DBWS components we must get consensus on the usage of this functionality and it must be able to function as the base dynamic solution in all components.
+
typeBuilder.addToSession(session, true, true);
 +
</source>
  
== Design ==
+
==== JPA CRUD Examples ====
  
The design of this feature involves several enhancements to the foundation (Core) component as well as JPA. The high level design involves:
+
<source lang="java5">
 +
// Creating a new Employee with 1:1 to Address and 1:M to PhoneNumber
 +
EntityType empType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Employee");
 +
EntityType addrType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Address");
 +
EntityType phoneType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "PhoneNumber");
  
* '''XML Configuration''': Extension of the eclipselink_orm_1_1.xsd
+
DynamicEntity emp = empType.newInstance();
** Add support for DYNAMIC access type in addition to FIELD and PROPERTY. ''Note: DYNAMIC can only be specified within this XML''
+
emp.set("firstName", "Sample");
** Add support for @attribute-type on all mapping types
+
emp.set("lastName", "Employee");
 +
emp.set("gender", "Male");
 +
emp.set("salary", 123456);
  
* '''Dynamic Model'''
+
DynamicEntity address = addrType.newInstance();
** Public interfaces: ''org.eclipse.persistence.dynamic''
+
emp.set("address", address);
** Internal implementation: ''org.eclipse.persistence.internal.dynamic''
+
  
* '''Metadata Processing Enhancements'''
+
DynamicEntity phone = phoneType.newInstance();
** Handling additional access type
+
phone.set("type", "Mobile");
** Using attribute-type instead of reflective type checking on classes
+
phone.set("areaCode", "613");
** Error handling for configuration limitations
+
phone.set("number", "555-1212");
 +
phone.set("owner", emp);
 +
emp.<Collection<DynamicEntity>>get("phoneNumbers").add( phone);
  
=== XML Configuration ===
+
em.getTransaction().begin();
 +
em.persist(emp);
 +
em.getTransaction().commit();
 +
</source>
  
The change to the eclipselink_orm_1_1.xsd to enable dynamic persistence include:
+
'''Example querying using JPQL and the generic get accessor'''
* Adding '''DYNAMIC''' Access Type. Since this implementation only supports DYNAMIC at the class level it must be set in the persistence unit defaults or individually on each entity. If '''DYNAMIC''' is specified for an entity then no attribute can specify a different access type or an exception will be thrown during validation of the metadata.
+
<source lang="java5">
* Adding of '''@access-type''' mappings to specify the Java type to be used
+
Query query = em.createQuery("SELECT e FROM Employee e WHERE e.manager.address.city = 'Ottawa' ORDER BY e.lastName ASC, e.firstName ASC");
** basic, id, version: specifies the 'basic' type to use
+
List<DynamicEntity> emps = query.getResultList();
** one-to-many, many-to-many, basic-collection, basic-map: In these mappings the attribute-type specifies the collection/map type to use
+
  
<source lang="xml">
+
for (DynamicEntity emp : emps) {
<?xml version="1.0" encoding="windows-1252" ?>
+
    int id = emp.<Integer> get("id");
<entity-mappings
+
    String firstName = emp.<String> get("firstName");
xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/orm"
+
    String lastName = emp.<String> get("lastName");
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0">
+
    int numPhones = emp.<DynamicEntity> get("manager").<Collection> get("phoneNumbers").size();
<persistence-unit-metadata>
+
<xml-mapping-metadata-complete />
+
<exclude-default-mappings />
+
<persistence-unit-defaults>
+
<access>DYNAMIC</access>
+
</persistence-unit-defaults>
+
</persistence-unit-metadata>
+
  
<object-type-converter name="gender" object-type="java.lang.String"
+
    System.out.println("Employee(" + lastName + ", " + firstName + " - " + numPhones + ")");
data-type="java.lang.String">
+
}
<conversion-value object-value="Male" data-value="M" />
+
</source>
<conversion-value object-value="Female" data-value="F" />
+
</object-type-converter>
+
  
<named-query name="Employee.findAll">
+
=== Dynamic Types defined in XML ===
<query>SELECT e FROM Employee e ORDER BY e.id</query>
+
</named-query>
+
  
<entity class="model.Address">
+
==== Native ORM using Project.XML ====
<table name="D_ADDR" />
+
 
<attributes>
+
==== Native ORM using Sessions.xml and Project.xml ====
<id name="id" attribute-type="java.lang.Integer">
+
 
<column name="ADDRESS_ID" />
+
==== JPA using Sessions.XML and Project.XML ====
<generated-value />
+
 
</id>
+
== Open Issues ==
<basic name="city" attribute-type="java.lang.String" />
+
 
<basic name="country" attribute-type="java.lang.String" />
+
The following open issues need to be addressed or documented.
<basic name="province" attribute-type="java.lang.String" />
+
 
<basic name="postalCode" attribute-type="java.lang.String">
+
* DyanmicException needs to be properly integrated with error codes and resource bundle usage. (done)
<column name="P_CODE" />
+
* The EmployeeProject tests (eclipselink.core.test.dynamic/src/org/eclipse/persistence/testing/tests/dynamic/orm/projectxml/EmployeeProject.java) fail due to configuration issues. May be around Inheritance (fixed)
</basic>
+
* The custom Inheritance policy that handles maintaining the class indicators as String and only converting them to classes in the convertClassNamesToClasses call and now pre-initialize needs to be carefully reviewed.
<basic name="street" attribute-type="java.lang.String" />
+
* Need tests added for using sessions.xml with project.xml and both the SessionManager and SessionFactory approaches. (done)
</attributes>
+
 
</entity>
+
= Refactoring (Phase I) =
<entity class="model.PhoneNumber">
+
The current implementation of Dynamic Persistence has a few limitations that need to be resolved in order to handle sparse merge ([https://bugs.eclipse.org/bugs/show_bug.cgi?id=280667 bug 280667]). The Phase I work is specifically designed to enable future work on sparse merge (custom Fetch Groups as a <b>write</b> plan).
<table name="D_PHONE" />
+
 
<attributes>
+
== Changes to public API ==
<id name="id" attribute-type="java.lang.Integer">
+
To better support adding new properties at runtime, the backing store has been changed from a fixed-length <code>Array</code> to a <code>Map</code>. Accordingly, the following APIs are deprecated:
<column name="EMP_ID" updatable="false" insertable="false" />
+
* index-based access no longer supported:
</id>
+
<source lang="java5">
<id name="type" attribute-type="java.lang.String">
+
/**
<column updatable="false" />
+
* @deprecated
</id>
+
* <p>
<basic name="areaCode" attribute-type="java.lang.String">
+
* Index-based access no longer supported
<column name="AREA_CODE" />
+
*
</basic>
+
* @param propertyName
<basic name="number" attribute-type="java.lang.String">
+
* @return
<column name="P_NUMBER" />
+
*/
</basic>
+
public int getPropertyIndex(String propertyName);
<many-to-one name="owner" target-entity="model.Employee" fetch="LAZY">
+
 
<join-column name="EMP_ID" />
+
/**
</many-to-one>
+
* @deprecated
</attributes>
+
* <p>
</entity>
+
* Index-based access no longer supported
<entity class="model.Employee">
+
*
<table name="D_EMP" />
+
* @param propertyIndex
<attributes>
+
* @return
<id name="id" attribute-type="java.lang.Integer">
+
*/
<column name="EMP_ID" />
+
public Class<?> getPropertyType(int propertyIndex);
<generated-value />
+
</source>
</id>
+
A best-effort computation will be performed based on the position of the key (a.k.a. propertyName) in the <code>Map</code>. In a subsequent release, these methods will be removed.
<basic name="firstName" attribute-type="java.lang.String">
+
* new <code>get</code> method on <code>DynamicEntity</code> with an additional parameter, the Java class type to which the property is converted ([https://bugs.eclipse.org/bugs/show_bug.cgi?id=300398 bug 300398])
<column name="F_NAME" />
+
<source lang="java5">
</basic>
+
/**
<basic name="middleInitial" attribute-type="java.lang.String">
+
* Converts the value of the property identified by the specified property name
<column name="INITIAL" />
+
* to the provided target class.
</basic>
+
*
<basic name="lastName" attribute-type="java.lang.String">
+
* @param targetClass
<column name="L_NAME" />
+
*      if the conversion is supported, the property value will be converted to this class.
</basic>
+
* @param propertyName
<basic name="startTime" attribute-type="java.sql.Time" fetch="LAZY">
+
*      the path to a valid object and property.
<column name="START_TIME" />
+
* @return
</basic>
+
*      the converted value of the specified property.
<basic name="endTime" attribute-type="java.sql.Time" fetch="LAZY">
+
* @see #get(String)
<column name="END_TIME" />
+
*/
</basic>
+
public <T> T get(String propertyName, Class<T> targetClass) throws DynamicException;
<basic name="gender" attribute-type="java.lang.String">
+
<column name="GENDER" />
+
<convert>gender</convert>
+
</basic>
+
<basic name="salary" attribute-type="java.lang.Double" />
+
<version name="version" attribute-type="java.lang.Long">
+
</version>
+
<many-to-one name="manager" fetch="LAZY" target-entity="model.Employee">
+
<join-column name="MANAGER_ID" />
+
</many-to-one>
+
<one-to-many name="employees" mapped-by="manager"
+
target-entity="model.Employee" attribute-type="java.util.ArrayList" />
+
<one-to-many name="phoneNumbers" target-entity="model.PhoneNumber"
+
mapped-by="owner" attribute-type="java.util.ArrayList">
+
<cascade>
+
<cascade-all />
+
</cascade>
+
<private-owned />
+
</one-to-many>
+
<one-to-one name="address" target-entity="model.Address"
+
fetch="LAZY">
+
<join-column name="ADDR_ID" />
+
<cascade>
+
<cascade-all />
+
</cascade>
+
<private-owned />
+
</one-to-one>
+
</attributes>
+
</entity>
+
</entity-mappings>
+
 
</source>
 
</source>
  
=== Dynamic Model ===
+
== Separating out behaviour into policies ==
 +
The current implementation incorporates setting default values for certain properties, based on information contained in the EclipseLink meta-data (default values for primitive types, initializing containers and <code>ValueHolders</code>, etc.) However, DBWS does not require default values and actually has some code to work around this as well as a large segment of duplicated code (certain classes could not be inherited). After the refactoring, responsibility for setting default values is moved from <code>DynamicTypeInstantiationPolicy</code> to <code>DynamicPropertiesManager</code> that establishes a <b><i>post-construct</i></b> callback that can be set to a custom <code>DynamicPropertiesInitializatonPolicy</code>. This new class also has the responsibility of building property 'slots' in the properties <code>Map</code>.
  
The dynamic model is the bulk of this feature. It provides dynamic classes to the application based on a common abstract base entity class which contains all of the EclipseLink specific support. In addition to the entity class itself there is also a simple Type-Property meta-model so that consuming applications can use the entities based on the mapping metadata.
+
Another new class <code>DynamicPropertiesManager</code> has all responsibilities associated with properties. In the base class, access to properties is managed in conjunction with a class' descriptor which is the last arbiter of whether or not a property exists and what type it is. For DBWS, properties cannot be added at runtime so a specialized sub-class <code>o.e.p.internal.xr.XRDynamicPropertiesManager</code> is used. The pattern for generated classes is:
 +
<source lang="java5">
 +
...
 +
import org.eclipse.persistence.internal.dynamic.DynamicEntityImpl;
 +
import org.eclipse.persistence.internal.dynamic.DynamicPropertiesManager;
  
==== Public Interfaces ====
+
public class Foo extends DynamicEntityImpl {
  
The package ''org.eclipse.persistence.dynamic'' will provide a set of interfaces that consumers can use when interacting with dynamic entities.
+
    public static DynamicPropertiesManager DPM = new DynamicPropertiesManager();
 +
   
 +
    public Foo() {
 +
        super();
 +
    }
 +
    public DynamicPropertiesManager fetchPropertiesManager() {
 +
        return DPM;
 +
    }
 +
}
  
* ''DynamicEntity'':
+
...
* ''EntityType'':
+
import org.eclipse.persistence.internal.xr.XRDynamicEntity;
* ''EntityProperty'':
+
import org.eclipse.persistence.internal.xr.XRDynamicPropertiesManager;
** ''EntityReferenceProperty'':
+
** ''EntityCollectionProperty'':
+
* ''DynamicHelper'':
+
* ''DynamicEntityException'':
+
  
==== Internal Implementation ====
+
public class Bar extends XRDynamicEntity {
  
The package ''org.eclipse.persistence.internal.dynamic'' includes the concrete implementation of the base entity class as well as the meta-model bridge and the byte-code weaving support for creating the concrete dynamic entity classes on the fly.
+
    public static XRDynamicPropertiesManager DPM = new XRDynamicPropertiesManager();
 +
   
 +
    public Bar() {
 +
        super();
 +
    }
  
* ''DynamicEntityImpl'':
+
    public XRDynamicPropertiesManager fetchPropertiesManager() {
* ''EntityTypeImpl'':
+
        return DPM;
* ''EntityPropertyImpl'':
+
    }
** ''EntityReferencePropertyImpl'':
+
}
** ''EntityCollectionPropertyImpl'':
+
</source>
* ''DynamicClassLoader'':
+
Later on, the <code>DPM</code> field is populated:
* ''DynamicAttributeAccessor'':
+
<source lang="java5">
 +
Field dpmField = myDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
 +
DynamicPropertiesManager dpm = (DynamicPropertiesManager)dpmField.get(null);
 +
dpm.setType(...)
  
=== Metadata Processing ===
 
  
The enhancements to the metadata processing include:
+
Field xrdpmField = xrDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
 +
XRDynamicPropertiesManager xrdpm = (XRDynamicPropertiesManager)xrdpmField.get(null);
 +
Set<String> propertyNames = new HashSet<String>();
 +
propertyNames.add("field1");
 +
propertyNames.add( ...
 +
// One-time initialization only
 +
xrdpm.setPropertyNames(propertyNames );
 +
 
 +
</source>
 +
 
 +
== Changes to public behaviour ==
 +
When default values were written to a property - say due to a <code>NullPolicy</code> in the EclipseLink meta-data or from the <code>DynamicPropertiesInitializatonPolicy</code>, the <b><i>set-edness</i></b> of the property was lost. Borrowing from work on SDO, the new behaviour is that the method <code>isSet</code> will <u>only</u> return <code>true</code> if the public <code>set(propertyName, value)</code> method is called on <code>DynamicEntity</code>
 +
 
 +
The following examples are consistent with the SDO behaviour.
 +
 
 +
<source lang="java5">
 +
DynamicEntity customerEntity = DynamicHelper.newDynamicEntity("example.Customer");
 +
// has fields:
 +
//    firstName - String
 +
//    age - int
 +
//    phoneNumbers - collection of phones (another DynamicEntity type)
 +
 
 +
customerEntity.get("firstName");  // returns null
 +
customerEntity.isSet("firstName");  // returns false
 +
customerEntity.set("firstName", null);
 +
customerEntity.isSet("firstName");  // returns true
 +
 
 +
customerEntity.get("age");  // returns 0
 +
customerEntity.isSet("age");  // returns false
 +
customerEntity.set("age", 0);
 +
customerEntity.isSet("age");  // returns true
 +
 
 +
customerEntity.get("phoneNumbers"); // returns empty list
 +
customerEntity.isSet("phoneNumbers");  // returns  false
 +
customerEntity.set("phoneNumbers", new ArrayList());
 +
customerEntity.isSet("phoneNumbers");  // returns  false (empty collections are always isSet=false)
 +
</source>
  
* Reading the attribute-type into the XML meta-model and ensuring it is processed into the JPA meta-model
+
== Class Hierarchy Diagram ==
* Handling of the 3 access types in the accessor model
+
[[Image:dynpersistRefactor.png|frameless|1050px]]
* Usage of the DynamicClassLoader wrapper when any class in a PU specifies DYNAMIC access
+
* Creation of Dynamic class when needed
+
* Avoid validation exceptions when dynamic types are used
+

Latest revision as of 10:45, 7 October 2010


EclipseLink Dynamic Persistence

This page captures the functional requirements and design of a Dynamic Persistence solution that will be added to EclipseLink from the EclipseLink Incubator. The work is being tracked by bug 200045.

Document History

Date Author Version Description & Notes
090917 Doug Clarke, Mike Norman 1.0 (most issued close)
100311 Mike Norman 2.0 (Refactoring for Sparse Merge - Phase I)


Definition

Dynamic Persistence is defined as the ability to create a dynamic persistent entity and use it within an application without a-priori the entity's Java .class being present on the classpath (or in some .jar/.war archive).

The purpose of Dynamic Persistence is to enable simplified data access where only mapping information is required and no concrete Java model is required. This allows applications with dynamic storage requirements to avoid coupling to static classes or require specialized handling of new types. The application uses standard EclipseLink APIs to read, create, modify, and remove dynamic persistent entities from their data stores based on types that are either defined in XML mapping files or are constructed within the running application.

Configuration

Static Configuration

A static configuration defines the dynamic persistent entity's structure and mappings in XML packaged with the application. This is used when the entity is generated before/during deployment and no runtime alteration of its structure is required.

Dynamic Configuration

A dynamic configuration defines the dynamic persistent entity's structure and mappings at runtime, as part of the running application.

Functional Requirements

  • JPA (ORM)
    1. Support for defining dynamic types using deployment XML (with or without sessions.xml) with EclipseLink native ORM API
    2. Support for defining dynamic types using deployment XML with sessions.xml with EclipseLink JPA using PU properties to specify sessions.xml location and session name
    3. Support for defining dynamic entities at runtime for use with EclipseLink native ORM API
    4. Support for defining dynamic entities at runtime for use with EclipseLink JPA
    5. Support for removal of a dynamic type - removing the descriptor and any entity caching
    6. Support for heterogeneous models where some entity types are persistent and some are dynamic including relationships between these
    7. Mappings Supported
      • Basic/DirectToField
      • Relationships: 1:1, 1:M, M:M, Basic/DirectCollection
      • Multi-table and composite PK
      • Single table and Joined inheritance
    8. Schema Creation
      • Support for creation of new tables - requires a non JTA JDBC connection
  • Extension/Customization of Dynamic Persistence capabilities
    1. DBWS uses dynamic entities in a 'static' configuration, i.e. once the Web service is deployed and initialized, no new properties can be introduced. In addition, the public DynamicEntity API will never be exposed. With these simpler/different requirements, ensure DBWS can function within the Dynamic Persistence framework as well as use DBWS-specific custom classes.

The targeted Java version for this functionality is Java SE 5.
NB - The build machine uses Java SE 6, but it is configured to generated version 49 byte codes (49 = Java SE 5, 50 = Java SE 6, 51 = Java SE 7 ...) which can run un-modified on all >5 JVMs.

Limitations

The following are the limitations of the initial EclipseLink Dynamic Persistence capabilities:

  1. JPA/ORM Configuration
    • Support will NOT be available for usage of JPA's orm.xml or eclipselink-orm.xml. This will require the attribute-classification to be added to the eclipselink-orm XSD and the metadata processing to support trusting this classification instead of accessing the attribute type from the entity class.
      • bug 241659: Add support to disable class validation in JPA metadata processing -
      • bug 267217: Add Named Access Type to EclipseLink ORM - enabling ValueAccessor config directly
    • There will be NO support for dynamic entity types having any static state (I.e. Hybrid classes). This means you cannot have a static class with additional dynamic attributes or a static entity superclass or mapped superclass
    • Mapping Limitations
      • EmbeddedId, Maps
      • Relationship management
  2. Dynamic Type Storage: This solution supports the usage of entity types defined at runtime but does not store or share these definitions across multiple application instances or executions of the application. It is the responsibility of the application to maintain theior definitions and create them using the provided API when required.
  3. MOXy
    • Support for defining dynamic types using deployment XML with sessions.xml to create a JAXB Context
    • Support for defining dynamic types using deployment XML with sessions.xml to create a native XML Context (future)
  4. JPA/ORM with MOXy (planned for future release)
    • Support all of the JPA/ORM configuration options above with additional support for native/JAXB MOXy mappings of the same dynamic entities
      • The MOXy mappings are limited to be a subset of the JPA/ORM mappings
  5. DBWS
    • Upgrade the DBWS runtime to use the common DynamicClassLoader and DynamicClassWriter (done)
    • If possible share a common base class and support for MOXy with JPA (done)
  6. SDO (planned for future release)
    • Upgrade the SDO runtime for dynamic data objects to use the common DynamicClassLoader and DynamicClassWriter
      • SDO will require its own base class with a pluggable value-store as well as an extended DynamicClassWriter to handle the additional interfaces necessary

Design

The design of this new public functionality of EclipseLink is based on existing solutions consumers have used. These existing customers include Oracle BPEL/ESB, EclipseLink DBWS, and EclipseLink SDO. The design of this feature is divided into several pieces each described on their own page:

Dynamic Class Creation - How EclipseLink will support creating classes
Public API for runtime JPA and Native ORM Usage
New Public API: org.eclipse.persistence.dynamic
New Private implementation: org.eclipse.persistence.internal.dynamic
Native XML Configuration - deployment/sessions XML usage
Defining/Modifying/Removing dynamic entity types at runtime

Usage Examples

The following usage examples illustrate how users of the new API can leverage dynamic persistence in their application.

Creating Types in Application

Application developers can define new dynamic entity types, configure their mappings, and add the new type to the active session.

DynamicClassLoader dcl = DynamicClassLoader.lookup(session);
Class<?> javaType = dcl.createDynamicClass("model.Simple");
 
EntityTypeBuilder typeBuilder = new JPAEntityTypeBuilder(javaType, null, "SIMPLE_TYPE");
typeBuilder.setPrimaryKeyFields("SID");
typeBuilder.addDirectMapping("id", int.class, "SID");
typeBuilder.addDirectMapping("value1", String.class, "VAL_1");
typeBuilder.addDirectMapping("value2", boolean.class, "VAL_2");
typeBuilder.addDirectMapping("value3", Calendar.class, "VAL_3");
typeBuilder.addDirectMapping("value4", Character.class, "VAL_4");
 
typeBuilder.addToSession(session, true, true);

JPA CRUD Examples

// Creating a new Employee with 1:1 to Address and 1:M to PhoneNumber
EntityType empType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Employee");
EntityType addrType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Address");
EntityType phoneType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "PhoneNumber");
 
DynamicEntity emp = empType.newInstance();
emp.set("firstName", "Sample");
emp.set("lastName", "Employee");
emp.set("gender", "Male");
emp.set("salary", 123456);
 
DynamicEntity address = addrType.newInstance();
emp.set("address", address);
 
DynamicEntity phone = phoneType.newInstance();
phone.set("type", "Mobile");
phone.set("areaCode", "613");
phone.set("number", "555-1212");
phone.set("owner", emp);
emp.<Collection<DynamicEntity>>get("phoneNumbers").add( phone);
 
em.getTransaction().begin();
em.persist(emp);
em.getTransaction().commit();

Example querying using JPQL and the generic get accessor

Query query = em.createQuery("SELECT e FROM Employee e WHERE e.manager.address.city = 'Ottawa' ORDER BY e.lastName ASC, e.firstName ASC");
List<DynamicEntity> emps = query.getResultList();
 
for (DynamicEntity emp : emps) {
    int id = emp.<Integer> get("id");
    String firstName = emp.<String> get("firstName");
    String lastName = emp.<String> get("lastName");
    int numPhones = emp.<DynamicEntity> get("manager").<Collection> get("phoneNumbers").size();
 
    System.out.println("Employee(" + lastName + ", " + firstName + " - " + numPhones + ")");
}

Dynamic Types defined in XML

Native ORM using Project.XML

Native ORM using Sessions.xml and Project.xml

JPA using Sessions.XML and Project.XML

Open Issues

The following open issues need to be addressed or documented.

  • DyanmicException needs to be properly integrated with error codes and resource bundle usage. (done)
  • The EmployeeProject tests (eclipselink.core.test.dynamic/src/org/eclipse/persistence/testing/tests/dynamic/orm/projectxml/EmployeeProject.java) fail due to configuration issues. May be around Inheritance (fixed)
  • The custom Inheritance policy that handles maintaining the class indicators as String and only converting them to classes in the convertClassNamesToClasses call and now pre-initialize needs to be carefully reviewed.
  • Need tests added for using sessions.xml with project.xml and both the SessionManager and SessionFactory approaches. (done)

Refactoring (Phase I)

The current implementation of Dynamic Persistence has a few limitations that need to be resolved in order to handle sparse merge (bug 280667). The Phase I work is specifically designed to enable future work on sparse merge (custom Fetch Groups as a write plan).

Changes to public API

To better support adding new properties at runtime, the backing store has been changed from a fixed-length Array to a Map. Accordingly, the following APIs are deprecated:

  • index-based access no longer supported:
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyName
 * @return
 */
public int getPropertyIndex(String propertyName);
 
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyIndex
 * @return
 */
public Class<?> getPropertyType(int propertyIndex);

A best-effort computation will be performed based on the position of the key (a.k.a. propertyName) in the Map. In a subsequent release, these methods will be removed.

  • new get method on DynamicEntity with an additional parameter, the Java class type to which the property is converted (bug 300398)
/**
 * Converts the value of the property identified by the specified property name
 * to the provided target class.
 * 
 * @param targetClass
 *      if the conversion is supported, the property value will be converted to this class.
 * @param propertyName
 *      the path to a valid object and property.
 * @return
 *      the converted value of the specified property.
 * @see #get(String)
 */
public <T> T get(String propertyName, Class<T> targetClass) throws DynamicException;

Separating out behaviour into policies

The current implementation incorporates setting default values for certain properties, based on information contained in the EclipseLink meta-data (default values for primitive types, initializing containers and ValueHolders, etc.) However, DBWS does not require default values and actually has some code to work around this as well as a large segment of duplicated code (certain classes could not be inherited). After the refactoring, responsibility for setting default values is moved from DynamicTypeInstantiationPolicy to DynamicPropertiesManager that establishes a post-construct callback that can be set to a custom DynamicPropertiesInitializatonPolicy. This new class also has the responsibility of building property 'slots' in the properties Map.

Another new class DynamicPropertiesManager has all responsibilities associated with properties. In the base class, access to properties is managed in conjunction with a class' descriptor which is the last arbiter of whether or not a property exists and what type it is. For DBWS, properties cannot be added at runtime so a specialized sub-class o.e.p.internal.xr.XRDynamicPropertiesManager is used. The pattern for generated classes is:

...
import org.eclipse.persistence.internal.dynamic.DynamicEntityImpl;
import org.eclipse.persistence.internal.dynamic.DynamicPropertiesManager;
 
public class Foo extends DynamicEntityImpl {
 
    public static DynamicPropertiesManager DPM = new DynamicPropertiesManager();
 
    public Foo() {
        super();
    }
    public DynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}
 
...
import org.eclipse.persistence.internal.xr.XRDynamicEntity;
import org.eclipse.persistence.internal.xr.XRDynamicPropertiesManager;
 
public class Bar extends XRDynamicEntity {
 
    public static XRDynamicPropertiesManager DPM = new XRDynamicPropertiesManager();
 
    public Bar() {
        super();
    }
 
    public XRDynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}

Later on, the DPM field is populated:

Field dpmField = myDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
DynamicPropertiesManager dpm = (DynamicPropertiesManager)dpmField.get(null);
dpm.setType(...)
 
 
Field xrdpmField = xrDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
XRDynamicPropertiesManager xrdpm = (XRDynamicPropertiesManager)xrdpmField.get(null);
Set<String> propertyNames = new HashSet<String>();
propertyNames.add("field1");
propertyNames.add( ...
// One-time initialization only
xrdpm.setPropertyNames(propertyNames );

Changes to public behaviour

When default values were written to a property - say due to a NullPolicy in the EclipseLink meta-data or from the DynamicPropertiesInitializatonPolicy, the set-edness of the property was lost. Borrowing from work on SDO, the new behaviour is that the method isSet will only return true if the public set(propertyName, value) method is called on DynamicEntity

The following examples are consistent with the SDO behaviour.

DynamicEntity customerEntity = DynamicHelper.newDynamicEntity("example.Customer");
// has fields:
//    firstName - String
//    age - int
//    phoneNumbers - collection of phones (another DynamicEntity type)
 
customerEntity.get("firstName");  // returns null
customerEntity.isSet("firstName");  // returns false
customerEntity.set("firstName", null);
customerEntity.isSet("firstName");  // returns true
 
customerEntity.get("age");  // returns 0
customerEntity.isSet("age");  // returns false
customerEntity.set("age", 0);
customerEntity.isSet("age");  // returns true
 
customerEntity.get("phoneNumbers"); // returns empty list
customerEntity.isSet("phoneNumbers");  // returns  false
customerEntity.set("phoneNumbers", new ArrayList());
customerEntity.isSet("phoneNumbers");  // returns  false (empty collections are always isSet=false)

Class Hierarchy Diagram

DynpersistRefactor.png

Back to the top