Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/Development/Dynamic"

(Internal Implementation)
 
(104 intermediate revisions by 2 users not shown)
Line 1: Line 1:
__TOC__
+
<css>
 +
  .source-sql {padding:1em;border:1px solid black; background-color: white;}
 +
  .source-java5 {padding:1em;border:1px solid black; background-color: white;}
 +
  .source-xml {padding:1em;border:1px solid black; background-color: white;}
 +
  .source-text {padding:1em;border:1px solid black; background-color: white;}
 +
</css>
 +
__NOTOC__
 
= EclipseLink Dynamic Persistence =
 
= EclipseLink Dynamic Persistence =
This page captures the functional requirements and design of a dynamic persistence solution that will allow consumers to specify their mappings. The purpose of dynamic persistence is to enable simplified data access where only mapping information is required and no concrete Java model is required.
+
This page captures the functional requirements and design of a Dynamic Persistence solution that will be added to EclipseLink from the EclipseLink Incubator. The work is being tracked by {{bug|200045}}.
  
The work is being tracked by {{bug|200045}} and is scheduled for inclusion in EclipseLink 1.2
+
== Document History ==
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Date
 +
! Author
 +
! Version Description & Notes
 +
|-
 +
| 090917
 +
| Doug Clarke, Mike Norman
 +
| 1.0 (most issued close)
 +
|-
 +
| 100311
 +
| Mike Norman
 +
| 2.0 (Refactoring for Sparse Merge - Phase I)
 +
|}
 +
<br/>
  
===Terminology===
+
__TOC__
 +
=== Definition ===
 
{{:EclipseLink/Development/Dynamic/def1}}
 
{{:EclipseLink/Development/Dynamic/def1}}
 
+
==== Configuration ====
 
{{:EclipseLink/Development/Dynamic/def2}}
 
{{:EclipseLink/Development/Dynamic/def2}}
 
 
{{:EclipseLink/Development/Dynamic/def3}}
 
{{:EclipseLink/Development/Dynamic/def3}}
  
 
== Functional Requirements ==
 
== Functional Requirements ==
 +
* JPA (ORM)
 +
*# Support for defining dynamic types using deployment XML (with or without sessions.xml) with EclipseLink native ORM API
 +
*# Support for defining dynamic types using deployment XML with sessions.xml with EclipseLink JPA using PU properties to specify sessions.xml location and session name
 +
*# Support for defining dynamic entities at runtime for use with EclipseLink native ORM API
 +
*# Support for defining dynamic entities at runtime for use with EclipseLink JPA
 +
*# Support for removal of a dynamic type - removing the descriptor and any entity caching
 +
*# Support for heterogeneous models where some entity types are persistent and some are dynamic including relationships between these
 +
*# Mappings Supported
 +
*#* Basic/DirectToField
 +
*#* Relationships: 1:1, 1:M, M:M, Basic/DirectCollection
 +
*#* Multi-table and composite PK
 +
*#* Single table and Joined inheritance
 +
*# Schema Creation
 +
*#* Support for creation of new tables - requires a non JTA JDBC connection
 +
* Extension/Customization of Dynamic Persistence capabilities
 +
*# DBWS uses dynamic entities in a 'static' configuration, i.e. once the Web service is deployed and initialized, no new properties can be introduced. In addition, the public <code>DynamicEntity</code> API will never be exposed. With these simpler/different requirements, ensure DBWS can function within the Dynamic Persistence framework as well as use DBWS-specific custom classes.
  
# Ability to use dynamic persistence through standard JPA metadata and interfaces as well as leveraging advanced EclipseLink JPA features
+
The targeted Java version for this functionality is Java SE 5. <br />NB - The build machine uses Java SE 6, but it is configured to generated version 49 byte codes (49 = Java SE 5, 50 = Java SE 6,
#* Specify mapping metadata using persistence.xml and eclipselink-orm.xml.  
+
51 = Java SE 7 ...) which can run un-modified on all >5 JVMs.
#* Create dynamic types at runtime
+
=== Limitations ===
#* Access dynamic types through standard (JPA2) metamodel + helpers
+
The following are the limitations of the initial EclipseLink Dynamic Persistence capabilities:
#* Support removal of dynamic types
+
# JPA/ORM Configuration
# Ability to use dynamic persistence through MOXy mappings
+
#* Support will NOT be available for usage of JPA's orm.xml or eclipselink-orm.xml. This will require the attribute-classification to be added to the eclipselink-orm XSD and the metadata processing to support trusting this classification instead of accessing the attribute type from the entity class.  
# Ability to use dynamic persistence within DBWS to share common infrastructure
+
#** {{bug|241659}}: Add support to disable class validation in JPA metadata processing - 
# <i>Ability to use dynamic persistence within SDO to share common infrastructure</i> - problem with mis-matched APIs:
+
#** {{bug|267217}}: Add Named Access Type to EclipseLink ORM - enabling ValueAccessor config directly
<source lang="java5">
+
#* There will be NO support for dynamic entity types having any static state (I.e. Hybrid classes). This means you cannot have a static class with additional dynamic attributes or a static entity superclass or mapped superclass
public interface DataObject {
+
#* Mapping Limitations
 +
#** EmbeddedId, Maps
 +
#** Relationship management
 +
# Dynamic Type Storage: This solution supports the usage of entity types defined at runtime but does not store or share these definitions across multiple application instances or executions of the application. It is the responsibility of the application to maintain theior definitions and create them using the provided API when required.
 +
# MOXy
 +
#* Support for defining dynamic types using deployment XML with sessions.xml to create a JAXB Context
 +
#* Support for defining dynamic types using deployment XML with sessions.xml to create a native XML Context (future)
 +
# JPA/ORM with MOXy (planned for future release)
 +
#* Support all of the JPA/ORM configuration options above with additional support for native/JAXB MOXy mappings of the same dynamic entities
 +
#** The MOXy mappings are limited to be a subset of the JPA/ORM mappings  
 +
# DBWS  
 +
#* Upgrade the DBWS runtime to use the common DynamicClassLoader and DynamicClassWriter (done)
 +
#* If possible share a common base class and support for MOXy with JPA (done)
 +
# SDO (planned for future release)
 +
#* Upgrade the SDO runtime for dynamic data objects to use the common DynamicClassLoader and DynamicClassWriter
 +
#** SDO will require its own base class with a pluggable value-store as well as an extended DynamicClassWriter to handle the additional interfaces necessary
  
    void set(String s, Object o);
+
== Design ==
 +
The design of this new public functionality of EclipseLink is based on existing solutions consumers have used. These existing customers include Oracle BPEL/ESB, EclipseLink DBWS, and EclipseLink SDO. The design of this feature is divided into several pieces each described on their own page:
 +
:[[EclipseLink/Development/Dynamic/Design_DynamicClassCreation | Dynamic Class Creation]] - How EclipseLink will support creating classes
 +
:[[EclipseLink/Development/Dynamic/Design_PublicAPI | Public API for runtime JPA and Native ORM Usage]]
 +
::New Public API: <code>org.eclipse.persistence.dynamic</code>
 +
::New Private implementation: <code>org.eclipse.persistence.internal.dynamic</code>
 +
:[[EclipseLink/Development/Dynamic/Design_NativeXML | Native XML Configuration]] - deployment/sessions XML usage
 +
:[[EclipseLink/Development/Dynamic/Design_RuntimeDefinition | Defining/Modifying/Removing dynamic entity types at runtime]]
  
}
+
== Usage Examples ==
public interface DynamicEntity {
+
  
    public DynamicEntity set(String s, Object o);
+
The following usage examples illustrate how users of the new API can leverage dynamic persistence in their application.
  
}
+
=== Creating Types in Application ===
public class BaseEntityImpl {
+
  
    public Object set(String s, Object o) {
+
Application developers can define new dynamic entity types, configure their mappings, and add the new type to the active session.
  
}
+
<source lang="java5">
public class SDOBaseEntityImpl extends BaseEntityImpl implements DataObject {
+
DynamicClassLoader dcl = DynamicClassLoader.lookup(session);
 +
Class<?> javaType = dcl.createDynamicClass("model.Simple");
  
    public void set(String s, Object o) {
+
EntityTypeBuilder typeBuilder = new JPAEntityTypeBuilder(javaType, null, "SIMPLE_TYPE");
          ^^^^^
+
typeBuilder.setPrimaryKeyFields("SID");
          return type is incompatible with BaseEntityImpl.set
+
typeBuilder.addDirectMapping("id", int.class, "SID");
}
+
typeBuilder.addDirectMapping("value1", String.class, "VAL_1");
 +
typeBuilder.addDirectMapping("value2", boolean.class, "VAL_2");
 +
typeBuilder.addDirectMapping("value3", Calendar.class, "VAL_3");
 +
typeBuilder.addDirectMapping("value4", Character.class, "VAL_4");
 +
 
 +
typeBuilder.addToSession(session, true, true);
 
</source>
 
</source>
  
=== Future Features (Current Limitations) ===
+
==== JPA CRUD Examples ====
  
In order to control the scope of this feature the following limitations have been intended and are listed here as potential future enhancements.
+
<source lang="java5">
 +
// Creating a new Employee with 1:1 to Address and 1:M to PhoneNumber
 +
EntityType empType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Employee");
 +
EntityType addrType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Address");
 +
EntityType phoneType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "PhoneNumber");
  
# '''Fixed Base Class''': This feature assumes the base entity class. In the future users may want the flexibility to specify a class of their own.
+
DynamicEntity emp = empType.newInstance();
# '''LimitedMapping Capability''': This feature does not provide support for all mapping types. The mappings currently not supported include:
+
emp.set("firstName", "Sample");
#* Inheritance - <b>Note</b> JPA2 meta-data supports inheritance, so this limitation has to go away soon!
+
emp.set("lastName", "Employee");
#* Embedded/EmbeddedId
+
emp.set("gender", "Male");
#* Composite primary keys
+
emp.set("salary", 123456);
# '''Hybrid Classes''': This feature only supports ''DynamicEntity'' persistent classes. In the future developers may want the ability to combine static java classes with one or more attributes accessed dynamically. The ''DynamicEntityImpl'' class provides support for lazy loading, fetch-groups, and attribute change-tracking. Combining this with traditional static attributes and weaving complex.
+
  
=== Incubator Exit Criteria ===
+
DynamicEntity address = addrType.newInstance();
 +
emp.set("address", address);
  
In order to exit the incubator and become part of the Core/JPA/MOXy/DBWS components we must get consensus on the usage of this functionality and it must be able to function as the base dynamic solution in all components.
+
DynamicEntity phone = phoneType.newInstance();
 +
phone.set("type", "Mobile");
 +
phone.set("areaCode", "613");
 +
phone.set("number", "555-1212");
 +
phone.set("owner", emp);
 +
emp.<Collection<DynamicEntity>>get("phoneNumbers").add( phone);
  
== Design ==
+
em.getTransaction().begin();
 +
em.persist(emp);
 +
em.getTransaction().commit();
 +
</source>
  
The design of this feature involves several enhancements to the Core of EclipseLink (foundation component) as well as to JPA. The high level design involves:
+
'''Example querying using JPQL and the generic get accessor'''
 
+
<source lang="java5">
* '''XML Configuration''': Extension of the eclipselink_orm_1_1.xsd
+
Query query = em.createQuery("SELECT e FROM Employee e WHERE e.manager.address.city = 'Ottawa' ORDER BY e.lastName ASC, e.firstName ASC");
** Add support for DYNAMIC access type in addition to FIELD and PROPERTY. ''Note: DYNAMIC can only be specified within this XML''
+
List<DynamicEntity> emps = query.getResultList();
** Add support for @attribute-type on all mapping types
+
 
+
* '''Dynamic Model'''
+
** Public interfaces: ''org.eclipse.persistence.dynamic''
+
** Private implementation: ''org.eclipse.persistence.internal.dynamic''
+
 
+
* '''Metadata Processing Enhancements'''
+
** Handling additional access type
+
** Using attribute-type instead of reflective type checking on classes
+
** Error handling for configuration limitations
+
 
+
=== XML Configuration ===
+
 
+
The change to the eclipselink_orm_1_1.xsd to enable dynamic persistence include:
+
* Adding '''DYNAMIC''' Access Type. Since this implementation only supports DYNAMIC at the class level it must be set in the persistence unit defaults or individually on each entity. If '''DYNAMIC''' is specified for an entity then no attribute can specify a different access type or an exception will be thrown during validation of the metadata.
+
* Adding of '''@access-type''' mappings to specify the Java type to be used
+
** basic, id, version: specifies the 'basic' type to use
+
** one-to-many, many-to-many, basic-collection, basic-map: In these mappings the attribute-type specifies the collection/map type to use
+
<source lang="xml">
+
<?xml version="1.0"?>
+
<entity-mappings
+
xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/orm"
+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0">
+
<persistence-unit-metadata>
+
<xml-mapping-metadata-complete />
+
<exclude-default-mappings />
+
<persistence-unit-defaults>
+
<access>DYNAMIC</access>
+
</persistence-unit-defaults>
+
</persistence-unit-metadata>
+
  
<object-type-converter name="gender" object-type="java.lang.String"
+
for (DynamicEntity emp : emps) {
data-type="java.lang.String">
+
    int id = emp.<Integer> get("id");
<conversion-value object-value="Male" data-value="M" />
+
    String firstName = emp.<String> get("firstName");
<conversion-value object-value="Female" data-value="F" />
+
    String lastName = emp.<String> get("lastName");
</object-type-converter>
+
    int numPhones = emp.<DynamicEntity> get("manager").<Collection> get("phoneNumbers").size();
  
<named-query name="Employee.findAll">
+
    System.out.println("Employee(" + lastName + ", " + firstName + " - " + numPhones + ")");
<query>SELECT e FROM Employee e ORDER BY e.id</query>
+
}
</named-query>
+
 
+
<entity class="model.Address">
+
<table name="D_ADDR" />
+
<attributes>
+
<id name="id" attribute-type="java.lang.Integer">
+
<column name="ADDRESS_ID" />
+
<generated-value />
+
</id>
+
<basic name="city" attribute-type="java.lang.String" />
+
<basic name="country" attribute-type="java.lang.String" />
+
<basic name="province" attribute-type="java.lang.String" />
+
<basic name="postalCode" attribute-type="java.lang.String">
+
<column name="P_CODE" />
+
</basic>
+
<basic name="street" attribute-type="java.lang.String" />
+
</attributes>
+
</entity>
+
<entity class="model.PhoneNumber">
+
<table name="D_PHONE" />
+
<attributes>
+
<id name="id" attribute-type="java.lang.Integer">
+
<column name="EMP_ID" updatable="false" insertable="false" />
+
</id>
+
<id name="type" attribute-type="java.lang.String">
+
<column updatable="false" />
+
</id>
+
<basic name="areaCode" attribute-type="java.lang.String">
+
<column name="AREA_CODE" />
+
</basic>
+
<basic name="number" attribute-type="java.lang.String">
+
<column name="P_NUMBER" />
+
</basic>
+
<many-to-one name="owner" target-entity="model.Employee" fetch="LAZY">
+
<join-column name="EMP_ID" />
+
</many-to-one>
+
</attributes>
+
</entity>
+
<entity class="model.Employee">
+
<table name="D_EMP" />
+
<attributes>
+
<id name="id" attribute-type="java.lang.Integer">
+
<column name="EMP_ID" />
+
<generated-value />
+
</id>
+
<basic name="firstName" attribute-type="java.lang.String">
+
<column name="F_NAME" />
+
</basic>
+
<basic name="middleInitial" attribute-type="java.lang.String">
+
<column name="INITIAL" />
+
</basic>
+
<basic name="lastName" attribute-type="java.lang.String">
+
<column name="L_NAME" />
+
</basic>
+
<basic name="startTime" attribute-type="java.sql.Time" fetch="LAZY">
+
<column name="START_TIME" />
+
</basic>
+
<basic name="endTime" attribute-type="java.sql.Time" fetch="LAZY">
+
<column name="END_TIME" />
+
</basic>
+
<basic name="gender" attribute-type="java.lang.String">
+
<column name="GENDER" />
+
<convert>gender</convert>
+
</basic>
+
<basic name="salary" attribute-type="java.lang.Double" />
+
<version name="version" attribute-type="java.lang.Long">
+
</version>
+
<many-to-one name="manager" fetch="LAZY" target-entity="model.Employee">
+
<join-column name="MANAGER_ID" />
+
</many-to-one>
+
<one-to-many name="employees" mapped-by="manager"
+
target-entity="model.Employee" attribute-type="java.util.ArrayList" />
+
<one-to-many name="phoneNumbers" target-entity="model.PhoneNumber"
+
mapped-by="owner" attribute-type="java.util.ArrayList">
+
<cascade>
+
<cascade-all />
+
</cascade>
+
<private-owned />
+
</one-to-many>
+
<one-to-one name="address" target-entity="model.Address"
+
fetch="LAZY">
+
<join-column name="ADDR_ID" />
+
<cascade>
+
<cascade-all />
+
</cascade>
+
<private-owned />
+
</one-to-one>
+
</attributes>
+
</entity>
+
</entity-mappings>
+
 
</source>
 
</source>
  
=== Dynamic Model ===
+
=== Dynamic Types defined in XML ===
[[Image:dynpersist.png|1000px]]
+
  
The dynamic model is the bulk of this feature. It provides dynamic classes to the application based on a common abstract base entity class which contains all of the EclipseLink specific support. In addition to the entity class itself there is also a simple Type-Property meta-model so that consuming applications can use the entities based on the mapping metadata.
+
==== Native ORM using Project.XML ====
  
==== Public Interfaces ====
+
==== Native ORM using Sessions.xml and Project.xml ====
  
The package ''org.eclipse.persistence.dynamic'' will provide a set of interfaces that consumers can use when interacting with dynamic entities.
+
==== JPA using Sessions.XML and Project.XML ====
  
* ''DynamicEntity''
+
== Open Issues ==
* ''DynamicEntityException''
+
* ''DynamicHelper''
+
* ''EntityType''
+
* ''EntityTypeBuilder''
+
  
==== Internal Implementation ====
+
The following open issues need to be addressed or documented.
  
The package ''org.eclipse.persistence.internal.dynamic'' includes the concrete implementation of the base entity class as well as the meta-model bridge and the byte-code weaving support for creating the concrete dynamic entity classes on the fly.
+
* DyanmicException needs to be properly integrated with error codes and resource bundle usage. (done)
 +
* The EmployeeProject tests (eclipselink.core.test.dynamic/src/org/eclipse/persistence/testing/tests/dynamic/orm/projectxml/EmployeeProject.java) fail due to configuration issues. May be around Inheritance (fixed)
 +
* The custom Inheritance policy that handles maintaining the class indicators as String and only converting them to classes in the convertClassNamesToClasses call and now pre-initialize needs to be carefully reviewed.
 +
* Need tests added for using sessions.xml with project.xml and both the SessionManager and SessionFactory approaches. (done)
  
* ''BaseEntityImpl'' - the <code>DynamicClassLoader</code> generates sub-classes of <code>BaseEntityImpl</code>. This class defines the <b>store</b> for a dynamic object, an array of Objects that take the place of member fields for persistent attributes.
+
= Refactoring (Phase I) =
 +
The current implementation of Dynamic Persistence has a few limitations that need to be resolved in order to handle sparse merge ([https://bugs.eclipse.org/bugs/show_bug.cgi?id=280667 bug 280667]). The Phase I work is specifically designed to enable future work on sparse merge (custom Fetch Groups as a <b>write</b> plan).
 +
 
 +
== Changes to public API ==
 +
To better support adding new properties at runtime, the backing store has been changed from a fixed-length <code>Array</code> to a <code>Map</code>. Accordingly, the following APIs are deprecated:
 +
* index-based access no longer supported:
 
<source lang="java5">
 
<source lang="java5">
public abstract class BaseEntityImpl implements PersistenceEntity, Cloneable {
+
/**
 +
* @deprecated
 +
* <p>
 +
* Index-based access no longer supported
 +
*
 +
* @param propertyName
 +
* @return
 +
*/
 +
public int getPropertyIndex(String propertyName);
  
    public Object[] __fields; // NB - actually implementation hides __fields
+
/**
...
+
* @deprecated
}
+
* <p>
 +
* Index-based access no longer supported
 +
*
 +
* @param propertyIndex
 +
* @return
 +
*/
 +
public Class<?> getPropertyType(int propertyIndex);
 
</source>
 
</source>
:* ''DynamicEntityImpl''
+
A best-effort computation will be performed based on the position of the key (a.k.a. propertyName) in the <code>Map</code>. In a subsequent release, these methods will be removed.
:<source lang="java5">
+
* new <code>get</code> method on <code>DynamicEntity</code> with an additional parameter, the Java class type to which the property is converted ([https://bugs.eclipse.org/bugs/show_bug.cgi?id=300398 bug 300398])
public class DynamicEntityImpl extends BaseEntityImpl {
+
<source lang="java5">
...
+
/**
}
+
* Converts the value of the property identified by the specified property name
 +
* to the provided target class.
 +
  *
 +
* @param targetClass
 +
*      if the conversion is supported, the property value will be converted to this class.
 +
* @param propertyName
 +
*      the path to a valid object and property.
 +
* @return
 +
*      the converted value of the specified property.
 +
* @see #get(String)
 +
*/
 +
public <T> T get(String propertyName, Class<T> targetClass) throws DynamicException;
 
</source>
 
</source>
* ''DynamicClassLoader''
 
* ''DynamicClassWriter''
 
* ''EntityTypeImpl''
 
* ''EntityTypeInstantiationPolicy''
 
* ''ValuesAccessor''
 
  
===== Store Mechanism =====
+
== Separating out behaviour into policies ==
EclipseLink persists an object by persisting an object's attribute; thus, EclipseLink must lookup the object's attribute. This lookup normally
+
The current implementation incorporates setting default values for certain properties, based on information contained in the EclipseLink meta-data (default values for primitive types, initializing containers and <code>ValueHolders</code>, etc.) However, DBWS does not require default values and actually has some code to work around this as well as a large segment of duplicated code (certain classes could not be inherited). After the refactoring, responsibility for setting default values is moved from <code>DynamicTypeInstantiationPolicy</code> to <code>DynamicPropertiesManager</code> that establishes a <b><i>post-construct</i></b> callback that can be set to a custom <code>DynamicPropertiesInitializatonPolicy</code>. This new class also has the responsibility of building property 'slots' in the properties <code>Map</code>.
accesses the object's member fields for each persistent attribute; the access can be handled via reflection or via callbacks to designated <code>get/set</code> methods. To accommodate these different lookup mechanisms, an attribute's mapping owns an <code>AttributeAccessor</code>:
+
<source lang="java5">
+
package org.eclipse.persistence.mappings;
+
  
public abstract class AttributeAccessor implements Cloneable, Serializable {
+
Another new class <code>DynamicPropertiesManager</code> has all responsibilities associated with properties. In the base class, access to properties is managed in conjunction with a class' descriptor which is the last arbiter of whether or not a property exists and what type it is. For DBWS, properties cannot be added at runtime so a specialized sub-class <code>o.e.p.internal.xr.XRDynamicPropertiesManager</code> is used. The pattern for generated classes is:
 +
<source lang="java5">
 
...
 
...
    /**
+
import org.eclipse.persistence.internal.dynamic.DynamicEntityImpl;
    * Return the attribute value from the object.
+
import org.eclipse.persistence.internal.dynamic.DynamicPropertiesManager;
    */
+
    public abstract Object getAttributeValueFromObject(Object object) throws DescriptorException;
+
  
     /**
+
public class Foo extends DynamicEntityImpl {
    * Set the attribute value into the object.
+
 
    */
+
     public static DynamicPropertiesManager DPM = new DynamicPropertiesManager();
     public abstract void setAttributeValueInObject(Object object, Object value) throws DescriptorException;
+
   
...
+
    public Foo() {
 +
        super();
 +
    }
 +
     public DynamicPropertiesManager fetchPropertiesManager() {
 +
        return DPM;
 +
    }
 
}
 
}
</source>
 
In the case of a dynamic object, actually member fields do not exist. Thus, the lookup must use a custom AttributeAccessor to access the <b>store</b> defined in <code>BaseEntityImpl</code>:
 
<source lang="java5">
 
public class MyAttributeAccessor extends AttributeAccessor  {
 
  
    protected int fieldIdx;
+
...
 +
import org.eclipse.persistence.internal.xr.XRDynamicEntity;
 +
import org.eclipse.persistence.internal.xr.XRDynamicPropertiesManager;
  
    public MyAttributeAccessor(int fieldIdx) {
+
public class Bar extends XRDynamicEntity {
        this.fieldIdx = fieldIdx;
+
    }
+
  
     @Override
+
     public static XRDynamicPropertiesManager DPM = new XRDynamicPropertiesManager();
     public Object getAttributeValueFromObject(Object object) {
+
   
         BaseEntityImpl baseEntity = (BaseEntityImpl)object;
+
     public Bar() {
        return baseEntity.__fields[fieldIdx];
+
         super();
 
     }
 
     }
  
    @Override
+
     public XRDynamicPropertiesManager fetchPropertiesManager() {
     public void setAttributeValueInObject(Object object, Object value) {
+
         return DPM;
         BaseEntityImpl baseEntity = (BaseEntityImpl)object;
+
        baseEntity.__fields[fieldIdx] = value;
+
 
     }
 
     }
 
}
 
}
 
</source>
 
</source>
(just an example, does not handle indirection or null/not-set semantics)
+
Later on, the <code>DPM</code> field is populated:
 +
<source lang="java5">
 +
Field dpmField = myDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
 +
DynamicPropertiesManager dpm = (DynamicPropertiesManager)dpmField.get(null);
 +
dpm.setType(...)
  
This custom accessor requires a correlation be made between an attribute and its position (<tt>fieldIdx</tt>) in the store.
 
  
<b><u>NB</u></b> TBD - should the store mechanism be abstracted to an interface, as in SDO? What about alternate store
+
Field xrdpmField = xrDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
implementations, such as a HashMap?
+
XRDynamicPropertiesManager xrdpm = (XRDynamicPropertiesManager)xrdpmField.get(null);
 +
Set<String> propertyNames = new HashSet<String>();
 +
propertyNames.add("field1");
 +
propertyNames.add( ...
 +
// One-time initialization only
 +
xrdpm.setPropertyNames(propertyNames );
  
=== Metadata Processing ===
+
</source>
  
The enhancements to the metadata processing include:
+
== Changes to public behaviour ==
 +
When default values were written to a property - say due to a <code>NullPolicy</code> in the EclipseLink meta-data or from the <code>DynamicPropertiesInitializatonPolicy</code>, the <b><i>set-edness</i></b> of the property was lost. Borrowing from work on SDO, the new behaviour is that the method <code>isSet</code> will <u>only</u> return <code>true</code> if the public <code>set(propertyName, value)</code> method is called on <code>DynamicEntity</code>
 +
 
 +
The following examples are consistent with the SDO behaviour.
 +
 
 +
<source lang="java5">
 +
DynamicEntity customerEntity = DynamicHelper.newDynamicEntity("example.Customer");
 +
// has fields:
 +
//    firstName - String
 +
//    age - int
 +
//    phoneNumbers - collection of phones (another DynamicEntity type)
 +
 
 +
customerEntity.get("firstName");  // returns null
 +
customerEntity.isSet("firstName");  // returns false
 +
customerEntity.set("firstName", null);
 +
customerEntity.isSet("firstName");  // returns true
 +
 
 +
customerEntity.get("age");  // returns 0
 +
customerEntity.isSet("age");  // returns false
 +
customerEntity.set("age", 0);
 +
customerEntity.isSet("age");  // returns true
 +
 
 +
customerEntity.get("phoneNumbers"); // returns empty list
 +
customerEntity.isSet("phoneNumbers");  // returns  false
 +
customerEntity.set("phoneNumbers", new ArrayList());
 +
customerEntity.isSet("phoneNumbers");  // returns  false (empty collections are always isSet=false)
 +
</source>
  
* Reading the attribute-type into the XML meta-model and ensuring it is processed into the JPA meta-model
+
== Class Hierarchy Diagram ==
* Handling of the 3 access types in the accessor model
+
[[Image:dynpersistRefactor.png|frameless|1050px]]
* Usage of the DynamicClassLoader wrapper when any class in a PU specifies DYNAMIC access
+
* Creation of Dynamic class when needed
+
* Avoid validation exceptions when dynamic types are used
+

Latest revision as of 10:45, 7 October 2010


EclipseLink Dynamic Persistence

This page captures the functional requirements and design of a Dynamic Persistence solution that will be added to EclipseLink from the EclipseLink Incubator. The work is being tracked by bug 200045.

Document History

Date Author Version Description & Notes
090917 Doug Clarke, Mike Norman 1.0 (most issued close)
100311 Mike Norman 2.0 (Refactoring for Sparse Merge - Phase I)


Definition

Dynamic Persistence is defined as the ability to create a dynamic persistent entity and use it within an application without a-priori the entity's Java .class being present on the classpath (or in some .jar/.war archive).

The purpose of Dynamic Persistence is to enable simplified data access where only mapping information is required and no concrete Java model is required. This allows applications with dynamic storage requirements to avoid coupling to static classes or require specialized handling of new types. The application uses standard EclipseLink APIs to read, create, modify, and remove dynamic persistent entities from their data stores based on types that are either defined in XML mapping files or are constructed within the running application.

Configuration

Static Configuration

A static configuration defines the dynamic persistent entity's structure and mappings in XML packaged with the application. This is used when the entity is generated before/during deployment and no runtime alteration of its structure is required.

Dynamic Configuration

A dynamic configuration defines the dynamic persistent entity's structure and mappings at runtime, as part of the running application.

Functional Requirements

  • JPA (ORM)
    1. Support for defining dynamic types using deployment XML (with or without sessions.xml) with EclipseLink native ORM API
    2. Support for defining dynamic types using deployment XML with sessions.xml with EclipseLink JPA using PU properties to specify sessions.xml location and session name
    3. Support for defining dynamic entities at runtime for use with EclipseLink native ORM API
    4. Support for defining dynamic entities at runtime for use with EclipseLink JPA
    5. Support for removal of a dynamic type - removing the descriptor and any entity caching
    6. Support for heterogeneous models where some entity types are persistent and some are dynamic including relationships between these
    7. Mappings Supported
      • Basic/DirectToField
      • Relationships: 1:1, 1:M, M:M, Basic/DirectCollection
      • Multi-table and composite PK
      • Single table and Joined inheritance
    8. Schema Creation
      • Support for creation of new tables - requires a non JTA JDBC connection
  • Extension/Customization of Dynamic Persistence capabilities
    1. DBWS uses dynamic entities in a 'static' configuration, i.e. once the Web service is deployed and initialized, no new properties can be introduced. In addition, the public DynamicEntity API will never be exposed. With these simpler/different requirements, ensure DBWS can function within the Dynamic Persistence framework as well as use DBWS-specific custom classes.

The targeted Java version for this functionality is Java SE 5.
NB - The build machine uses Java SE 6, but it is configured to generated version 49 byte codes (49 = Java SE 5, 50 = Java SE 6, 51 = Java SE 7 ...) which can run un-modified on all >5 JVMs.

Limitations

The following are the limitations of the initial EclipseLink Dynamic Persistence capabilities:

  1. JPA/ORM Configuration
    • Support will NOT be available for usage of JPA's orm.xml or eclipselink-orm.xml. This will require the attribute-classification to be added to the eclipselink-orm XSD and the metadata processing to support trusting this classification instead of accessing the attribute type from the entity class.
      • bug 241659: Add support to disable class validation in JPA metadata processing -
      • bug 267217: Add Named Access Type to EclipseLink ORM - enabling ValueAccessor config directly
    • There will be NO support for dynamic entity types having any static state (I.e. Hybrid classes). This means you cannot have a static class with additional dynamic attributes or a static entity superclass or mapped superclass
    • Mapping Limitations
      • EmbeddedId, Maps
      • Relationship management
  2. Dynamic Type Storage: This solution supports the usage of entity types defined at runtime but does not store or share these definitions across multiple application instances or executions of the application. It is the responsibility of the application to maintain theior definitions and create them using the provided API when required.
  3. MOXy
    • Support for defining dynamic types using deployment XML with sessions.xml to create a JAXB Context
    • Support for defining dynamic types using deployment XML with sessions.xml to create a native XML Context (future)
  4. JPA/ORM with MOXy (planned for future release)
    • Support all of the JPA/ORM configuration options above with additional support for native/JAXB MOXy mappings of the same dynamic entities
      • The MOXy mappings are limited to be a subset of the JPA/ORM mappings
  5. DBWS
    • Upgrade the DBWS runtime to use the common DynamicClassLoader and DynamicClassWriter (done)
    • If possible share a common base class and support for MOXy with JPA (done)
  6. SDO (planned for future release)
    • Upgrade the SDO runtime for dynamic data objects to use the common DynamicClassLoader and DynamicClassWriter
      • SDO will require its own base class with a pluggable value-store as well as an extended DynamicClassWriter to handle the additional interfaces necessary

Design

The design of this new public functionality of EclipseLink is based on existing solutions consumers have used. These existing customers include Oracle BPEL/ESB, EclipseLink DBWS, and EclipseLink SDO. The design of this feature is divided into several pieces each described on their own page:

Dynamic Class Creation - How EclipseLink will support creating classes
Public API for runtime JPA and Native ORM Usage
New Public API: org.eclipse.persistence.dynamic
New Private implementation: org.eclipse.persistence.internal.dynamic
Native XML Configuration - deployment/sessions XML usage
Defining/Modifying/Removing dynamic entity types at runtime

Usage Examples

The following usage examples illustrate how users of the new API can leverage dynamic persistence in their application.

Creating Types in Application

Application developers can define new dynamic entity types, configure their mappings, and add the new type to the active session.

DynamicClassLoader dcl = DynamicClassLoader.lookup(session);
Class<?> javaType = dcl.createDynamicClass("model.Simple");
 
EntityTypeBuilder typeBuilder = new JPAEntityTypeBuilder(javaType, null, "SIMPLE_TYPE");
typeBuilder.setPrimaryKeyFields("SID");
typeBuilder.addDirectMapping("id", int.class, "SID");
typeBuilder.addDirectMapping("value1", String.class, "VAL_1");
typeBuilder.addDirectMapping("value2", boolean.class, "VAL_2");
typeBuilder.addDirectMapping("value3", Calendar.class, "VAL_3");
typeBuilder.addDirectMapping("value4", Character.class, "VAL_4");
 
typeBuilder.addToSession(session, true, true);

JPA CRUD Examples

// Creating a new Employee with 1:1 to Address and 1:M to PhoneNumber
EntityType empType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Employee");
EntityType addrType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Address");
EntityType phoneType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "PhoneNumber");
 
DynamicEntity emp = empType.newInstance();
emp.set("firstName", "Sample");
emp.set("lastName", "Employee");
emp.set("gender", "Male");
emp.set("salary", 123456);
 
DynamicEntity address = addrType.newInstance();
emp.set("address", address);
 
DynamicEntity phone = phoneType.newInstance();
phone.set("type", "Mobile");
phone.set("areaCode", "613");
phone.set("number", "555-1212");
phone.set("owner", emp);
emp.<Collection<DynamicEntity>>get("phoneNumbers").add( phone);
 
em.getTransaction().begin();
em.persist(emp);
em.getTransaction().commit();

Example querying using JPQL and the generic get accessor

Query query = em.createQuery("SELECT e FROM Employee e WHERE e.manager.address.city = 'Ottawa' ORDER BY e.lastName ASC, e.firstName ASC");
List<DynamicEntity> emps = query.getResultList();
 
for (DynamicEntity emp : emps) {
    int id = emp.<Integer> get("id");
    String firstName = emp.<String> get("firstName");
    String lastName = emp.<String> get("lastName");
    int numPhones = emp.<DynamicEntity> get("manager").<Collection> get("phoneNumbers").size();
 
    System.out.println("Employee(" + lastName + ", " + firstName + " - " + numPhones + ")");
}

Dynamic Types defined in XML

Native ORM using Project.XML

Native ORM using Sessions.xml and Project.xml

JPA using Sessions.XML and Project.XML

Open Issues

The following open issues need to be addressed or documented.

  • DyanmicException needs to be properly integrated with error codes and resource bundle usage. (done)
  • The EmployeeProject tests (eclipselink.core.test.dynamic/src/org/eclipse/persistence/testing/tests/dynamic/orm/projectxml/EmployeeProject.java) fail due to configuration issues. May be around Inheritance (fixed)
  • The custom Inheritance policy that handles maintaining the class indicators as String and only converting them to classes in the convertClassNamesToClasses call and now pre-initialize needs to be carefully reviewed.
  • Need tests added for using sessions.xml with project.xml and both the SessionManager and SessionFactory approaches. (done)

Refactoring (Phase I)

The current implementation of Dynamic Persistence has a few limitations that need to be resolved in order to handle sparse merge (bug 280667). The Phase I work is specifically designed to enable future work on sparse merge (custom Fetch Groups as a write plan).

Changes to public API

To better support adding new properties at runtime, the backing store has been changed from a fixed-length Array to a Map. Accordingly, the following APIs are deprecated:

  • index-based access no longer supported:
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyName
 * @return
 */
public int getPropertyIndex(String propertyName);
 
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyIndex
 * @return
 */
public Class<?> getPropertyType(int propertyIndex);

A best-effort computation will be performed based on the position of the key (a.k.a. propertyName) in the Map. In a subsequent release, these methods will be removed.

  • new get method on DynamicEntity with an additional parameter, the Java class type to which the property is converted (bug 300398)
/**
 * Converts the value of the property identified by the specified property name
 * to the provided target class.
 * 
 * @param targetClass
 *      if the conversion is supported, the property value will be converted to this class.
 * @param propertyName
 *      the path to a valid object and property.
 * @return
 *      the converted value of the specified property.
 * @see #get(String)
 */
public <T> T get(String propertyName, Class<T> targetClass) throws DynamicException;

Separating out behaviour into policies

The current implementation incorporates setting default values for certain properties, based on information contained in the EclipseLink meta-data (default values for primitive types, initializing containers and ValueHolders, etc.) However, DBWS does not require default values and actually has some code to work around this as well as a large segment of duplicated code (certain classes could not be inherited). After the refactoring, responsibility for setting default values is moved from DynamicTypeInstantiationPolicy to DynamicPropertiesManager that establishes a post-construct callback that can be set to a custom DynamicPropertiesInitializatonPolicy. This new class also has the responsibility of building property 'slots' in the properties Map.

Another new class DynamicPropertiesManager has all responsibilities associated with properties. In the base class, access to properties is managed in conjunction with a class' descriptor which is the last arbiter of whether or not a property exists and what type it is. For DBWS, properties cannot be added at runtime so a specialized sub-class o.e.p.internal.xr.XRDynamicPropertiesManager is used. The pattern for generated classes is:

...
import org.eclipse.persistence.internal.dynamic.DynamicEntityImpl;
import org.eclipse.persistence.internal.dynamic.DynamicPropertiesManager;
 
public class Foo extends DynamicEntityImpl {
 
    public static DynamicPropertiesManager DPM = new DynamicPropertiesManager();
 
    public Foo() {
        super();
    }
    public DynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}
 
...
import org.eclipse.persistence.internal.xr.XRDynamicEntity;
import org.eclipse.persistence.internal.xr.XRDynamicPropertiesManager;
 
public class Bar extends XRDynamicEntity {
 
    public static XRDynamicPropertiesManager DPM = new XRDynamicPropertiesManager();
 
    public Bar() {
        super();
    }
 
    public XRDynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}

Later on, the DPM field is populated:

Field dpmField = myDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
DynamicPropertiesManager dpm = (DynamicPropertiesManager)dpmField.get(null);
dpm.setType(...)
 
 
Field xrdpmField = xrDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
XRDynamicPropertiesManager xrdpm = (XRDynamicPropertiesManager)xrdpmField.get(null);
Set<String> propertyNames = new HashSet<String>();
propertyNames.add("field1");
propertyNames.add( ...
// One-time initialization only
xrdpm.setPropertyNames(propertyNames );

Changes to public behaviour

When default values were written to a property - say due to a NullPolicy in the EclipseLink meta-data or from the DynamicPropertiesInitializatonPolicy, the set-edness of the property was lost. Borrowing from work on SDO, the new behaviour is that the method isSet will only return true if the public set(propertyName, value) method is called on DynamicEntity

The following examples are consistent with the SDO behaviour.

DynamicEntity customerEntity = DynamicHelper.newDynamicEntity("example.Customer");
// has fields:
//    firstName - String
//    age - int
//    phoneNumbers - collection of phones (another DynamicEntity type)
 
customerEntity.get("firstName");  // returns null
customerEntity.isSet("firstName");  // returns false
customerEntity.set("firstName", null);
customerEntity.isSet("firstName");  // returns true
 
customerEntity.get("age");  // returns 0
customerEntity.isSet("age");  // returns false
customerEntity.set("age", 0);
customerEntity.isSet("age");  // returns true
 
customerEntity.get("phoneNumbers"); // returns empty list
customerEntity.isSet("phoneNumbers");  // returns  false
customerEntity.set("phoneNumbers", new ArrayList());
customerEntity.isSet("phoneNumbers");  // returns  false (empty collections are always isSet=false)

Class Hierarchy Diagram

DynpersistRefactor.png

Back to the top