Jump to: navigation, search

EclipseLink/Development/Dynamic


EclipseLink Dynamic Persistence

This page captures the functional requirements and design of a Dynamic Persistence solution that will be added to EclipseLink from the EclipseLink Incubator. The work is being tracked by bug 200045.

Document History

Date Author Version Description & Notes
090917 Doug Clarke, Mike Norman 1.0 (most issued close)
100311 Mike Norman 2.0 (Refactoring for Sparse Merge - Phase I)


Definition

Dynamic Persistence is defined as the ability to create a dynamic persistent entity and use it within an application without a-priori the entity's Java .class being present on the classpath (or in some .jar/.war archive).

The purpose of Dynamic Persistence is to enable simplified data access where only mapping information is required and no concrete Java model is required. This allows applications with dynamic storage requirements to avoid coupling to static classes or require specialized handling of new types. The application uses standard EclipseLink APIs to read, create, modify, and remove dynamic persistent entities from their data stores based on types that are either defined in XML mapping files or are constructed within the running application.

Configuration

Static Configuration

A static configuration defines the dynamic persistent entity's structure and mappings in XML packaged with the application. This is used when the entity is generated before/during deployment and no runtime alteration of its structure is required.

Dynamic Configuration

A dynamic configuration defines the dynamic persistent entity's structure and mappings at runtime, as part of the running application.

Functional Requirements

  • JPA (ORM)
    1. Support for defining dynamic types using deployment XML (with or without sessions.xml) with EclipseLink native ORM API
    2. Support for defining dynamic types using deployment XML with sessions.xml with EclipseLink JPA using PU properties to specify sessions.xml location and session name
    3. Support for defining dynamic entities at runtime for use with EclipseLink native ORM API
    4. Support for defining dynamic entities at runtime for use with EclipseLink JPA
    5. Support for removal of a dynamic type - removing the descriptor and any entity caching
    6. Support for heterogeneous models where some entity types are persistent and some are dynamic including relationships between these
    7. Mappings Supported
      • Basic/DirectToField
      • Relationships: 1:1, 1:M, M:M, Basic/DirectCollection
      • Multi-table and composite PK
      • Single table and Joined inheritance
    8. Schema Creation
      • Support for creation of new tables - requires a non JTA JDBC connection
  • Extension/Customization of Dynamic Persistence capabilities
    1. DBWS uses dynamic entities in a 'static' configuration, i.e. once the Web service is deployed and initialized, no new properties can be introduced. In addition, the public DynamicEntity API will never be exposed. With these simpler/different requirements, ensure DBWS can function within the Dynamic Persistence framework as well as use DBWS-specific custom classes.

The targeted Java version for this functionality is Java SE 5.
NB - The build machine uses Java SE 6, but it is configured to generated version 49 byte codes (49 = Java SE 5, 50 = Java SE 6, 51 = Java SE 7 ...) which can run un-modified on all >5 JVMs.

Limitations

The following are the limitations of the initial EclipseLink Dynamic Persistence capabilities:

  1. JPA/ORM Configuration
    • Support will NOT be available for usage of JPA's orm.xml or eclipselink-orm.xml. This will require the attribute-classification to be added to the eclipselink-orm XSD and the metadata processing to support trusting this classification instead of accessing the attribute type from the entity class.
      • bug 241659: Add support to disable class validation in JPA metadata processing -
      • bug 267217: Add Named Access Type to EclipseLink ORM - enabling ValueAccessor config directly
    • There will be NO support for dynamic entity types having any static state (I.e. Hybrid classes). This means you cannot have a static class with additional dynamic attributes or a static entity superclass or mapped superclass
    • Mapping Limitations
      • EmbeddedId, Maps
      • Relationship management
  2. Dynamic Type Storage: This solution supports the usage of entity types defined at runtime but does not store or share these definitions across multiple application instances or executions of the application. It is the responsibility of the application to maintain theior definitions and create them using the provided API when required.
  3. MOXy
    • Support for defining dynamic types using deployment XML with sessions.xml to create a JAXB Context
    • Support for defining dynamic types using deployment XML with sessions.xml to create a native XML Context (future)
  4. JPA/ORM with MOXy (planned for future release)
    • Support all of the JPA/ORM configuration options above with additional support for native/JAXB MOXy mappings of the same dynamic entities
      • The MOXy mappings are limited to be a subset of the JPA/ORM mappings
  5. DBWS
    • Upgrade the DBWS runtime to use the common DynamicClassLoader and DynamicClassWriter (done)
    • If possible share a common base class and support for MOXy with JPA (done)
  6. SDO (planned for future release)
    • Upgrade the SDO runtime for dynamic data objects to use the common DynamicClassLoader and DynamicClassWriter
      • SDO will require its own base class with a pluggable value-store as well as an extended DynamicClassWriter to handle the additional interfaces necessary

Design

The design of this new public functionality of EclipseLink is based on existing solutions consumers have used. These existing customers include Oracle BPEL/ESB, EclipseLink DBWS, and EclipseLink SDO. The design of this feature is divided into several pieces each described on their own page:

Dynamic Class Creation - How EclipseLink will support creating classes
Public API for runtime JPA and Native ORM Usage
New Public API: org.eclipse.persistence.dynamic
New Private implementation: org.eclipse.persistence.internal.dynamic
Native XML Configuration - deployment/sessions XML usage
Defining/Modifying/Removing dynamic entity types at runtime

Usage Examples

The following usage examples illustrate how users of the new API can leverage dynamic persistence in their application.

Creating Types in Application

Application developers can define new dynamic entity types, configure their mappings, and add the new type to the active session.

DynamicClassLoader dcl = DynamicClassLoader.lookup(session);
Class<?> javaType = dcl.createDynamicClass("model.Simple");
 
EntityTypeBuilder typeBuilder = new JPAEntityTypeBuilder(javaType, null, "SIMPLE_TYPE");
typeBuilder.setPrimaryKeyFields("SID");
typeBuilder.addDirectMapping("id", int.class, "SID");
typeBuilder.addDirectMapping("value1", String.class, "VAL_1");
typeBuilder.addDirectMapping("value2", boolean.class, "VAL_2");
typeBuilder.addDirectMapping("value3", Calendar.class, "VAL_3");
typeBuilder.addDirectMapping("value4", Character.class, "VAL_4");
 
typeBuilder.addToSession(session, true, true);

JPA CRUD Examples

// Creating a new Employee with 1:1 to Address and 1:M to PhoneNumber
EntityType empType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Employee");
EntityType addrType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "Address");
EntityType phoneType = DynamicHelper.getType(JpaHelper.getEntityManager(em).getServerSession(), "PhoneNumber");
 
DynamicEntity emp = empType.newInstance();
emp.set("firstName", "Sample");
emp.set("lastName", "Employee");
emp.set("gender", "Male");
emp.set("salary", 123456);
 
DynamicEntity address = addrType.newInstance();
emp.set("address", address);
 
DynamicEntity phone = phoneType.newInstance();
phone.set("type", "Mobile");
phone.set("areaCode", "613");
phone.set("number", "555-1212");
phone.set("owner", emp);
emp.<Collection<DynamicEntity>>get("phoneNumbers").add( phone);
 
em.getTransaction().begin();
em.persist(emp);
em.getTransaction().commit();

Example querying using JPQL and the generic get accessor

Query query = em.createQuery("SELECT e FROM Employee e WHERE e.manager.address.city = 'Ottawa' ORDER BY e.lastName ASC, e.firstName ASC");
List<DynamicEntity> emps = query.getResultList();
 
for (DynamicEntity emp : emps) {
    int id = emp.<Integer> get("id");
    String firstName = emp.<String> get("firstName");
    String lastName = emp.<String> get("lastName");
    int numPhones = emp.<DynamicEntity> get("manager").<Collection> get("phoneNumbers").size();
 
    System.out.println("Employee(" + lastName + ", " + firstName + " - " + numPhones + ")");
}

Dynamic Types defined in XML

Native ORM using Project.XML

Native ORM using Sessions.xml and Project.xml

JPA using Sessions.XML and Project.XML

Open Issues

The following open issues need to be addressed or documented.

  • DyanmicException needs to be properly integrated with error codes and resource bundle usage. (done)
  • The EmployeeProject tests (eclipselink.core.test.dynamic/src/org/eclipse/persistence/testing/tests/dynamic/orm/projectxml/EmployeeProject.java) fail due to configuration issues. May be around Inheritance (fixed)
  • The custom Inheritance policy that handles maintaining the class indicators as String and only converting them to classes in the convertClassNamesToClasses call and now pre-initialize needs to be carefully reviewed.
  • Need tests added for using sessions.xml with project.xml and both the SessionManager and SessionFactory approaches. (done)

Refactoring (Phase I)

The current implementation of Dynamic Persistence has a few limitations that need to be resolved in order to handle sparse merge (bug 280667). The Phase I work is specifically designed to enable future work on sparse merge (custom Fetch Groups as a write plan).

Changes to public API

To better support adding new properties at runtime, the backing store has been changed from a fixed-length Array to a Map. Accordingly, the following APIs are deprecated:

  • index-based access no longer supported:
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyName
 * @return
 */
public int getPropertyIndex(String propertyName);
 
/**
 * @deprecated
 * <p>
 * Index-based access no longer supported
 * 
 * @param propertyIndex
 * @return
 */
public Class<?> getPropertyType(int propertyIndex);

A best-effort computation will be performed based on the position of the key (a.k.a. propertyName) in the Map. In a subsequent release, these methods will be removed.

  • new get method on DynamicEntity with an additional parameter, the Java class type to which the property is converted (bug 300398)
/**
 * Converts the value of the property identified by the specified property name
 * to the provided target class.
 * 
 * @param targetClass
 *      if the conversion is supported, the property value will be converted to this class.
 * @param propertyName
 *      the path to a valid object and property.
 * @return
 *      the converted value of the specified property.
 * @see #get(String)
 */
public <T> T get(String propertyName, Class<T> targetClass) throws DynamicException;

Separating out behaviour into policies

The current implementation incorporates setting default values for certain properties, based on information contained in the EclipseLink meta-data (default values for primitive types, initializing containers and ValueHolders, etc.) However, DBWS does not require default values and actually has some code to work around this as well as a large segment of duplicated code (certain classes could not be inherited). After the refactoring, responsibility for setting default values is moved from DynamicTypeInstantiationPolicy to DynamicPropertiesManager that establishes a post-construct callback that can be set to a custom DynamicPropertiesInitializatonPolicy. This new class also has the responsibility of building property 'slots' in the properties Map.

Another new class DynamicPropertiesManager has all responsibilities associated with properties. In the base class, access to properties is managed in conjunction with a class' descriptor which is the last arbiter of whether or not a property exists and what type it is. For DBWS, properties cannot be added at runtime so a specialized sub-class o.e.p.internal.xr.XRDynamicPropertiesManager is used. The pattern for generated classes is:

...
import org.eclipse.persistence.internal.dynamic.DynamicEntityImpl;
import org.eclipse.persistence.internal.dynamic.DynamicPropertiesManager;
 
public class Foo extends DynamicEntityImpl {
 
    public static DynamicPropertiesManager DPM = new DynamicPropertiesManager();
 
    public Foo() {
        super();
    }
    public DynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}
 
...
import org.eclipse.persistence.internal.xr.XRDynamicEntity;
import org.eclipse.persistence.internal.xr.XRDynamicPropertiesManager;
 
public class Bar extends XRDynamicEntity {
 
    public static XRDynamicPropertiesManager DPM = new XRDynamicPropertiesManager();
 
    public Bar() {
        super();
    }
 
    public XRDynamicPropertiesManager fetchPropertiesManager() {
        return DPM;
    }
}

Later on, the DPM field is populated:

Field dpmField = myDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
DynamicPropertiesManager dpm = (DynamicPropertiesManager)dpmField.get(null);
dpm.setType(...)
 
 
Field xrdpmField = xrDynamicClass.getField(DynamicPropertiesManager.PROPERTIES_MANAGER_FIELD);
XRDynamicPropertiesManager xrdpm = (XRDynamicPropertiesManager)xrdpmField.get(null);
Set<String> propertyNames = new HashSet<String>();
propertyNames.add("field1");
propertyNames.add( ...
// One-time initialization only
xrdpm.setPropertyNames(propertyNames );

Changes to public behaviour

When default values were written to a property - say due to a NullPolicy in the EclipseLink meta-data or from the DynamicPropertiesInitializatonPolicy, the set-edness of the property was lost. Borrowing from the SDO work, the new behaviour is that the method isSet will only return true if the public set(propertyName, value) method is called on DynamicEntity

Class Hierarchy Diagram

DynpersistRefactor.png