EclipseLink/Development/2.1/DynamicMOXy/296967

Design Specification: MOXy Support for Dynamic Persistence

ER 296967

Document History

Date	Author	Version Description & Notes
091204	Rick Barkhouse	1.0
091207	Rick Barkhouse	1.1 - Added Bootstrapping from Deployment XML
091217	Rick Barkhouse	1.2 - Added Bootstrapping from XSD
100104	Rick Barkhouse	1.3 - Expanded Overview to identify feature milestones
100107	Rick Barkhouse	1.4 - Introduced DynamicJAXBContext, reorganized Design
100113	Rick Barkhouse	1.5 - Expanding API to include lookup via XML names
100115	Rick Barkhouse	1.6 - More API reorganization, enumerating Requirements
100118	Rick Barkhouse	1.7 - Added section on Type Lookup, added Examples
100119	Rick Barkhouse	1.8 - Expanded examples demonstrating adding new mappings

Project Overview

The goal of this feature is to enable users of EclipseLink JAXB to perform typical JAXB operations without having real .class files available for their domain objects. Users will pass in some form of metadata (XML Schema, EclipseLink Deployment XML or External Metadata) to create a DynamicJAXBContext, and this metadata will be used to construct DynamicEntity objects representing the domain classes.

The ultimate purpose of this feature is to support an end-to-end JPA<->JAXB solution. EclipseLink JPA already has some support for dynamic persistence and this work will provide a path from dynamic JPA to JAXB.

Some initial groundwork was included in EclipseLink 2.0, more information is available here.

Concepts

Dynamic Persistence

The core of this feature is the dynamic persistence support that was initially added for JPA (found in org.eclipse.persistence.dynamic). To obtain a "dynamic" project (i.e. one that maps to classes that were generated in-memory), a "dry" project (one that does not have Java classes specified, only class names) is passed to the DynamicTypeBuilder, passing in an instance of DynamicClassLoader, which does the work of building the in-memory classes:

InputStream metadata = ...
Project dynamicProject = DynamicTypeBuilder.loadDynamicProject(inputStream, null, new DynamicClassLoader(Thread.currentThread().getContextClassLoader()));

When this project is used to unmarshall XML documents, the objects that are returned by EclipseLink will be subclasses of DynamicEntity. DynamicEntities offer a simple get(propertyName) / set(propertyName, propertyValue) API to manipulate their data:

DynamicEntity dynamicEmployee = (DynamicEntity) jaxbUnmarshaller().unmarshal(instanceDoc);
String firstName = dynamicEmp.get("fname");
DynamicEntity dynamicAddress = dynamicEmp.get("address");
 
...
 
dynamicEmp.set("lname", "Duggar");
address.set("street", "1001 Duggar Ranch Way");

XJC

XJC (XML-Java Compiler) is the executable compiler that is shipped with JAXB in the JDK. Normally it is run from the command line, taking an XML Schema as input and generating JAXB-annotated Java files on disk. There is a developer API for working with XJC in code as well, and we will leverage this to parse the XML Schema and create in-memory representations of the classes, stopping short of actually generating the Java code.

Requirements

1. Enable the user to use JAXB APIs to marshal and unmarshal XML data, without having Java domain classes defined.
2. Enable the user to bootstrap from various metadata sources, including EclipseLink Project XML, EclipseLink External Metadata, and XML Schema.
3. Enable the user to obtain references to DynamicEntities via either Java class names or XML complex type names.
4. Enable the user to add arbitrary properties/mappings at runtime, after initialization has been performed.
5. Ensure that the Dynamic JAXB workflow is thread-safe.

API

DynamicJAXBContext

The entry point to this feature is a new class, DynamicJAXBContext, which is a subclass of the existing EclipseLink JAXBContext (which is itself a subclass of Sun's JAXBContext). Because this functionality is proprietary and not part of the JAXB spec, we will not provide creation APIs on JAXBContextFactory.

public DynamicJAXBContext(URL metadataURL, MetadataFormat metadataFormat, ClassLoader classLoader)

• URL metadataURL - A Java URL pointing to the metadata (XML Schema, EclipseLink Deployment XML, or EclipseLink External Metadata)
• MetadataFormat metadataFormat - One of MetadataFormat.XML_SCHEMA, MetadataFormat.DEPLOYMENT_XML, or MetadataFormat.EXT_METADATA
• ClassLoader classLoader - The current class loader; a DynamicClassLoader will be created with this loader as its parent

Additionally, if the user has created an EclipseLink Project in code, and has specified only class names and not actual Class objects, a DynamicJAXBContext can be created directly from the Project:

public DynamicJAXBContext(Project project, ClassLoader classLoader)

The following methods are available to interact with DynamicJAXBContext:

public Marshaller createMarshaller()
public Unmarshaller createUnmarshaller()
// Plus all other standard JAXBContext APIs...
 
public DynamicEntity newDynamicEntity(String typeName)
public DynamicEntity newDynamicEntity(QName typeName)
public DynamicEntity newDynamicEntity(DynamicType type)
 
public DynamicType getDynamicType(String typeName)
public DynamicType getDynamicType(QName typeName)
 
public void addDirectMapping(String javaName, Class attributeClass, String xmlName, DynamicType typeToModify)
public void addCompositeObjectMapping(String javaName, DynamicType attributeClass, String xmlName, DynamicType typeToModify)
public void addCompositeCollectionMapping(String javaName, DynamicType attributeClass, String xmlName, DynamicType typeToModify)
...

Creating a DynamicJAXBContext

There are three ways to bootstrap a DynamicJAXBContext; from an EclipseLink Project, EclipseLink External Metadata, or XML Schema:

File eclipselinkProjectFile = new File("resource/eclipselink/employee-project.xml");
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(projectFile.toURI().toURL(), MetadataFormat.DEPLOYMENT_XML, loader); 
 
// OR, if you have an EclipseLink Project object that has only Java class names specified:
Project eclipselinkProject = new EmployeeProject();
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(eclipselinkProject, loader);

File metadataFile = new File("resource/eclipselink/eclipselink-oxm.xml");
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(metadataFile.toURI().toURL(), MetadataFormat.EXT_METADATA, loader);

File schemaFile = new File("resource/xsd/employee.xsd");
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(schemaFile.toURI().toURL(), MetadataFormat.XML_SCHEMA, loader);

Unmarshalling and Modifying an Object from XML

DynamicEntity employee = (DynamicEntity) dContext.createUnmarshaller().unmarshal(new File("instance.xml"));
employee.set("empId", "Smith");
employee.get("address").set("street", "1001 Riverside Dr.");

Marshalling a New Object

Since, from the user's perspective, no concrete Java classes actually exist, the user must have a way to create new instances. This is done using a DynamicHelper, and is exposed through the newDynamicEntity(String) method on DynamicJAXBContext. For example:

Using Java class/field names:

DynamicEntity employee = dContext.newDynamicEntity("org.acme.Employee");
employee.set("firstName", "Bob");
employee.set("lastName", "Barker");
dContext.createMarshaller().marshal(employee, System.out);

Using XML names:

DynamicEntity employee = dContext.newDynamicEntity(new QName("http://www.acme.org", "employee-type"));
employee.set("first-name", "Bob");
employee.set("last-name", "Barker");
dContext.createMarshaller().marshal(employee, System.out);

Type Lookup

If reference to an actual DynamicType object is desired, it can be obtained by doing a lookup on DynamicJAXBContext, or from a DynamicEntity instance:

DynamicType employeeType = dContext.getDynamicType("org.acme.Employee");
DynamicEntity employee = employeeType.newDynamicEntity();

DynamicType employeeType = dContext.getDynamicType(new QName("http://www.acme.org", "employee-type"));
DynamicEntity employee = employeeType.newDynamicEntity();

DynamicEntity employee = (DynamicEntity) dContext.createUnmarshaller().unmarshal(new File("instance.xml"));
DynamicType employeeType = employee.getDynamicType();

Adding Mappings at Runtime

New mappings can be added to an existing DynamicType by using the various 'add' methods on DynamicJAXBContext:

DynamicEntity employee = (DynamicEntity) dContext.createUnmarshaller().unmarshal(new File("instance.xml"));
 
DynamicType empType = dContext.getDynamicType("org.acme.Employee");
DynamicType addType = dContext.getDynamicType("org.acme.Address");
 
dContext.addDirectMapping("empId", String.class, "employee-id", employee.getDynamicType());
dContext.addCompositeObjectMapping("hrSupervisor", empType, "hr-supervisor", employee.getDynamicType());
dContext.addCompositeCollectionMapping("jobLocations", addType, "hr-supervisor", employee.getDynamicType());
 
employee.set("empId", "77264");
 
DynamicEntity supervisor = dContext.newDynamicEntity("org.acme.Employee");
supervisor.set("firstName", "Ritchie");
supervisor.set("lastName", "Rich");
employee.set("hrSupervisor", supervisor);
 
DynamicEntity loc1 = dContext.newDynamicEntity("org.acme.Address");
loc1.set("street", "Compound A7 Zone C");
DynamicEntity loc2 = dContext.newDynamicEntity("org.acme.Address");
loc2.set("street", "Compound E9 Zone G");
 
ArrayList<DynamicEntity> locations = new ArrayList<DynamicEntity>();
locations.add(loc1);
locations.add(loc2);
employee.set("jobLocations", locations);
 
dContext.createMarshaller().marshal(employee, System.out);

Examples

Bootstrapping from XML Schema, using XML names:

// Create a DynamicJAXBContext from an XML schema file:
File schemaFile = new File("resource/xsd/employee.xsd");
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(schemaFile.toURI().toURL(), MetadataFormat.XML_SCHEMA, loader); 
 
// Unmarshal an XML instance doc to get an Employee object:
DynamicEntity employee = (DynamicEntity) dContext.createUnmarshaller().unmarshal(new File("instance.xml"));
String currentEmployeeName = employee.get("first-name") + " " + employee.get("last-name");
 
// Add a Direct Mapping for Employee ID:
dContext.addDirectMapping("empId", String.class, "employee-id", employee.getDynamicType());
employee.set("empId", "77264");
 
// Create a new Address and set it on the Employee:
DynamicEntity newAddress = dContext.newDynamicEntity(new QName("http://www.acme.org", "address-type"));
newAddress.set("street", "773 Bank St.");
employee.set("address", newAddress);
 
// Marshal the modified Employee:
dContext.createMarshaller().marshal(employee, System.out);

Bootstrapping from EclipseLink External Metadata, using Java names:

// Create a DynamicJAXBContext from an XML schema file:
File metadataFile = new File("resource/eclipselink/eclipselink-oxm.xml");
ClassLoader loader = Thread.currentThread().getContextClassLoader();
DynamicJAXBContext dContext = new DynamicJAXBContext(metadataFile.toURI().toURL(), MetadataFormat.EXT_METADATA, loader);
 
// Unmarshal an XML instance doc to get an Employee object:
DynamicEntity employee = (DynamicEntity) dContext.createUnmarshaller().unmarshal(new File("instance.xml"));
String currentEmployeeName = employee.get("fName") + " " + employee.get("lName");
 
// Add a Direct Mapping for Employee ID:
dContext.addDirectMapping("empId", String.class, "employee-id", employee.getDynamicType());
employee.set("employee-id", "77264");
 
// Create a new Address and set it on the Employee:
DynamicEntity newAddress = dContext.newDynamicEntity("org.acme.Address");
newAddress.set("street", "773 Bank St.");
employee.set("address", newAddress);
 
// Marshal the modified Employee:
dContext.createMarshaller().marshal(employee, System.out);

Design / Functionality

Bootstrapping from Deployment XML

When constructing a DynamicJAXBContext from Deployment XML, a DynamicClassLoader is first created using the supplied ClassLoader as its parent. An InputStream is then created from the metadata URL and used to create a Project mapped to Dynamic Entities. Finally, XMLContext and DynamicHelper objects are instantiated and stored.

DynamicClassLoader dynamicClassLoader = new DynamicClassLoader(classLoader);            
InputStream inputStream = metadataURL.openStream();
Project p = DynamicTypeBuilder.loadDynamicProject(inputStream, null, dynamicClassLoader);
 
this.xmlContext = new XMLContext(p);
this.dynamicHelper = new DynamicHelper(xmlContext.getSession(0));

Bootstrapping from XML Schema

When constructing a DynamicJAXBContext from XML Schema, we can use APIs from the Java XJC compiler to parse a schema and create Java class definitions (XJC's JCodeModel) in memory, then pass these class definitions to an EclipseLink Generator (org.eclipse.persistence.jaxb.compiler.Generator) to generate an EclipseLink project. After we have created this project, we can use the DynamicTypeBuilder to create a dynamic project, generating Java classes in memory along the way.

First, we use XJC API to parse an XSD and generate a JCodeModel. Note that this code stops short of actually generating .java files, it only generates an in-memory representation of the Java files that would normally be created.

// Use XJC API to parse the schema and generate its JCodeModel
SchemaCompiler sc = XJC.createSchemaCompiler();
InputSource inputSource = new InputSource(metadataURL.toExternalForm());
sc.parseSchema(inputSource);
S2JJAXBModel model = sc.bind();
JCodeModel jCodeModel = model.generateCode(new Plugin[0], null);

We can then wrap these XJC classes in our own implentations of EclipseLink's JAXB JavaModel classes. This will allow us to use the Generator to create an EclipseLink project and mappings. The JavaModel interfaces define "wrappers" for the various Java language constructs that make up the domain model (e.g. classes, methods, constructors, annotations, packages, etc). For example:

public class XJCJavaFieldImpl implements JavaField {
 
    // XJC's definition of a Field
    protected JFieldVar xjcField;
 
    ...
 
    public int getModifiers() {
        return xjcField.mods().getValue();
    }
 
    public String getName() {
        return xjcField.name();
    }
 
    ...
}

Creating the JavaModel classes:

// Create EclipseLink JavaModel objects for each of XJC's JDefinedClasses
ArrayList<JDefinedClass> classesToProcess = new ArrayList<JDefinedClass>();
Iterator<JPackage> packages = jCodeModel.packages();
while (packages.hasNext()) {
   JPackage pkg = packages.next();
   Iterator<JDefinedClass> classes = pkg.classes();
   while (classes.hasNext()) {
      JDefinedClass cls = classes.next();
         if (!cls.name().equals("ObjectFactory")) {
            classesToProcess.add(cls);
         }
   }
}   
JavaClass[] jotClasses = createClassModelFromXJC(classesToProcess, jCodeModel);

At this point, we can instantiate a Generator and obtain a "dry" EclipseLink project, which we can then turn into a dynamic project with generated in-memory classes. Finally, we create and store an XMLContext and a DynamicHelper:

// Use the JavaModel to setup a Generator to generate an EclipseLink project 
XJCJavaModelImpl javaModel = new XJCJavaModelImpl(Thread.currentThread().getContextClassLoader(), jCodeModel);
XJCJavaModelInputImpl javaModelInput = new XJCJavaModelInputImpl(jotClasses, javaModel);
Generator g = new Generator(javaModelInput);
Project p = g.generateProject();
 
// Make a Dynamic Project from this project, because these classes do not exist on the classpath
DynamicClassLoader dynamicClassLoader = new DynamicClassLoader(classLoader);
Project dp = DynamicTypeBuilder.loadDynamicProject(p, null, dynamicClassLoader);
 
this.xmlContext = new XMLContext(dp);
this.dynamicHelper = new DynamicHelper(xmlContext.getSession(0));

Bootstrapping from EclipseLink External Metadata

TBD

Adding Properties/Mappings at Runtime

One of the main aspects of this feature is the ability to add new mappings to an existing running system (aligning with JPA's support for this scenario).

• Mappings must be fully initialized before they are added to the descriptor, for thread-safety.
• Mapping customizations will be isolated to the current XMLContext, i.e. changes made to a descriptor will only be visible in that descriptor's XMLContext, even if multiple XMLContexts were created from the same initial metadata.

GUI

No GUI is required for this feature.

Config Files

A new metadata file may be needed to specify and persist new mapping information. (See Open Issues below)

Open Issues

This section lists the open issues that are still pending that must be decided prior to fully implementing this project's requirements.

Issue #	Owner	Description / Notes
001	Rick Barkhouse	What naming conventions should we use for the new Property keys that can be passed in to JAXBContextFactory?
002	Rick Barkhouse	How will we handle concurrency / threading issues when we allow modifying mappings at runtime? Do objects created BEFORE the type was modified reflect the new mappings?
003	Rick Barkhouse	How are new mappings specified on existing descriptors? Is an additional metadata file required to hold the new mapping information?
004	Rick Barkhouse	DynamicJAXBContext constructors: What various options should be provided for metadata source (URL, Streams, DOMs, ...)? Should we autodetect metadata format by examining the header of the metadata file directly?
005	Rick Barkhouse	Do we need to pass in a ClassLoader or can we always just default it to currentContextClassloader()?
006	Rick Barkhouse	Do we want to allow DynamicEntities and normal concrete Java classes to be used together?

Decisions

This section lists decisions made. These are intended to document the resolution of open issues or constraints added to the project that are important.

Issue #	Description / Notes	Decision

Future Considerations

During the research for this project the following items were identified as out of scope but are captured here as potential future enhancements. If agreed upon during the review process these should be logged in the bug system.