Latest revision as of 12:24, 11 January 2012

Flex Columns Extension

Why use this feature?

1. You are building an application where some mappings are common to all users and some mappings are user-specific
2. You want to add mappings to your application after it is made available to a customer (even post-deployment)
3. You want to use the same EntityManagerFactory to work with your data even after the mappings have changed
4. You can provide an additional source of metadata to be used by the application ( an xml-based source is provided, or you can build your own)

An example of the type of user this is designed for is a Software-as-a-Service provider who designs a generic application that can be provided to users and allow them to customize the application to make use of data that is particular to their domain.

What basic steps do I have to take to use this feature

1. Design a standard JPA Application

2. Decide which Entities will allow flexible mappings, annotate them as such and provide facilities to store the addional data.

  @Entity
  @VirtualAccessMethods
  public class Customer{
 
...
 
    @Transient
    private Map<String, Object> extensions;
 
    public <T> T get(String name) {
        return (T) extentions.get(name);
    }
 
    public Object set(String name, Object value) {
        return extensions.put(name, value);
    }

3. When you design your schema, provide enough extra columns in your tables to accomodate the number of flexible mappings you will allow. e.g. The following table has 3 predefined columns and 3 columns designed to accomodate mappings added after design (FLEX_COL1, FLEX_COL2, FLEX_COL3)

• CUSTOMER
  • INTEGER ID
  • VARCHAR NAME
  • VARCHAR FLEX_COL1
  • VARCHAR FLEX_COL2
  • VARCHAR FLEX_CO31

4. Deploy your application

5. To provide additional mappings, provide an eclipselink-orm.xml file that contains the additional mappings.

 
   <basic name="idNumber" access="VIRTUAL" attribute-type="String">
      <column name="FLEX_1"/>
      <access-methods get-method="get" set-method="set"/>
    </basic>

6. Use persistence unit properties to get your application to use the file:

  <property name="eclipselink.metadata-source" value="XML"/>
  <property name="eclipselink.metadata-source.xml.url" value="foo://bar"/>

Requirements

• Users MUST be able to add mappings after initial deployment
• Extensions MUST be persistent. (i.e. Extensions must continue to exist if an application goes down and comes back up)
• Extensions MUST be shared amoung all EntityManagers configured to use them
• Extensions MUST be compatible with our multi-tenant features. (i.e. Extensions must be able to make use of our Multi-tenant features to be definable on a tenant by tenant basis)
• BasicMappings MUST be supported
• Extensibility MUST be configurable using traditional JPA means (annotations, eclipselink-orm.xml)
• It MUST be possible to use JPA Queries to query based on the extensions
• Extensions SHOULD be supported through the JPA metamodel
• OneToOneMappings SHOULD be supported
• It MAY be possible to add extensions without logging in the session. (See ER 341429.)

Configuration

Metadata

Extensions will be supported using our VIRTUAL access type. Virtual Access type allows properties to be set through a getter that takes an attribute name as an argument and a setter that takes an argument name and a value as an argument.

VIRTUAL access does not currently allow annotation-based configuration. As part of this feature, annotations will be added to configure the methods used by Virtual access. These annotations will have the same effect as using the eclipselink-orm.xml construct <access-methods> at a class level. They will default which methods are used by virtual mappings. EclipseLink will weave these methods if weaving is enabled. This weaving will provide equivalent functionality to weaving for PROPERTY and FIELD access. (e.g. change tracking, fetch groups, etc)

DDL generation for tables with flexible columns will be addressed in a separate feature that addresses flexible DDL generation as a complete feature.

Examples

Example 1

• Field Access for non extension fields
• Virtual Access for extension fields uses defaults (get(String), set(String, Object))
• get(String) and set(String, Object) method will be woven even if no mappings use them because of the presence of @VirtualAccessMethods
• extensions mapped in a portable way - @Transient

  @Entity
  @VirtualAccessMethods
  public class Address {
 
    @Id
    private int id;
 
    @Transient
    private Map<String, Object> extensions;
 
    public int getId(){
        return id;
    }
 
    public <T> T get(String name) {
        return (T) extentions.get(name);
    }
 
    public Object set(String name, Object value) {
        return extensions.put(name, value);
    }
 
...

Example 2

• Field Access for non extension fields
• extensions mapped in a portable way - @Transient
• @VirtualAccessMethods annotation overrides method to be used for get and for set.
• getExtension(String) and setExtension(String, Object) method will be woven even if no mappings use them because of the presence of @VirtualAccessMethods
• XML for extended mapping indicates which get and set method to use

  @Entity
  @VirtualAccessMethods(get="getExtension", set="setExtension")
  public class Address {
 
    @Id
    private int id;
 
    @Transient
    private Map<String, Object> extensions;
 
    public int getId(){
        return id;
    }
 
    public <T> T getExtension(String name) {
        return (T) extensions.get(name);
    }
 
    public Object setExtension(String name, Object value) {
        return extensions.put(name, value);
    }
 
...

 
   <basic name="name" access="VIRTUAL" attribute-type="String">
      <column name="FLEX_1"/>
      <access-methods get-method="getExtension" set-method="setExtension"/>
    </basic>

Example 3

• Property Access for non extension fields
• Virtual Access for extension fields uses defaults (get(String), set(String, Object))
• extensions mapped in a portable way - no @Transient required because of Property access
• get(String) and set(String, Object) method will be woven even if no mappings use them because of the presence of @VirtualAccessMethods

  @Entity
  @VirtualAccessMethods
  public class Address {
 
    private int id;
 
    private Map<String, Object> extensions;
 
    @Id
    public int getId(){
        return id;
    }
 
    public <T> T get(String name) {
        return (T) extensions.get(name);
    }
 
    public Object set(String name, Object value) {
        return extensions.put(name, value);
    }
 
...

EntityManagerFactory and Metadata Repository

Extensions will be added at bootstrap time through access to a metadata repository. A Metadata Repository will accessed through a class that provides methods to retrieve the metadata it holds.

The user will specifiy the class to use and any configuration information for the metadata repository through persistence unit properties. As an EntityManagerFactory bootstraps, if metadata repository information is provided, the EMF will check the metadata repository for additional mapping information and integrate it into the metadata it uses to bootstrap.

EclipseLink will initially ship with the capability of connecting to two types of metadata repository.

1. XML (high priority) - information about extensions is stored in XML
2. Database Table (medium priority) - information about extensions is stored in a database table

Additionally, the user will be able to provide an implementation of the class that access the metadata repository.

Each metadata repository access class will specify an individual set of properties to use to connect to the repository

Examples

XML File

  <property name="eclipselink.metadata-source" value="XML"/>
  <property name="eclipselink.metadata-source.xml.url" value="foo://bar"/>

User-Specified

  <property name="eclipselink.metadata-source" value="com.foo.MetadataRepository"/>
  <property name="com.foo.MetadataRepository.location" value="foo://bar"/>
  <property name="com.foo.MetadataRepository.extra-data" value="foo-bar"/>

Note: The implementer of com.foo.MetadataRepository will be free to choose the properties that their implementation requires.

Design

Configuration

Please use the examples above as a guideline configuration XML and annotations. Note that the configuration options to allow a mapping to use VIRTUAL access have been available in EclipseLink for several releases. We will be using those configuration options as they exist and any changes to those will be handled as bugs rather than through this design document.

Weaving

The intial VIRTUAL access feature did not include weaving of the get and set methods. As part of the extensions feature will will add weaving of get and set methods that use virtual access. The initial implementation will not support OneToOne mappings and throw an exception at Transformer construction time if weaving is requested for a VIRTUAL mapping that is OneToOne.

Get method

Original

    public <T> T get(String name) {
        return (T) getExtensions().get(name);
    }

Weaved

    public Object get(String name)
    {
        _persistence_checkFetched(name);
        return getExtensions().get(name);
    }

Set method

Original

    public Object set(String name, Object value)
    {
        return getExtensions().put(name, value);
    }

Weaved

    public Object set(String name, Object value)
    {
        Object obj = null;
        if(_persistence_listener != null)
        {
            obj = get(name);
        } else
        {
            _persistence_checkFetchedForSet(name);
        }
        _persistence_propertyChange(name, obj, value);
        return getExtensions().put(name, value);
    }

To allow weaving, RelationalDescriptor will have a list virtual methods added. This list will be used at transformer-construction time to allow EclipseLink to know which methods it should weave.

    /** The methods that are used by virtual attributes as getter methods and setter methods.  
     * These will be used by our weaver to properly weave those methods 
     **/
    protected List<VirtualAccessMethods> virtualMethods = null;

EntityManagerFactory

Bootstrap

EntityManagerFactory bootstrapping occurs withing EntityManagerSetupImpl. In the predeploy method, there is code that obtains the orm.xml files that contain metadata. At that point, the metadata repository will be consulted. It will provide additional metadata information in the same format as is obtained from the orm.xml file.

Refresh

A mechanism will be provided that allows a user to tell a Metadata repository to refresh. That mechanism will take two forms.

1. A direct refresh API call
2. A RemoteCommandManager command that causes all subscribed EntityManagerFactories to refresh themselves as described above.

Refresh will be supported by adding an additional proxy to our EntityManagerFactory archtecture.

Current: EntityManagerFactoryImpl -> ServerSession

New: EntityManagerFactoryWrapper implements EntityManagerFactory -> EntityManagerFactoryImpl -> ServerSession

EntityManagerFactoryWrapper will implement:

  public void refreshMetadata()

In both cases, a live EntityManager holds a reference to EntityManagerFactoryImpl

When a call is made to refreshMetadata(), EntityManagerFactoryWrapper will bootstrap a new EntityManagerFactoryImpl and use it as the basis for any new EntityManagers. The old EntityManagerFactoryImpl will continue to be available until the last EntityManager is no longer used, at which point we will rely on garbage collection to clean it up.

Metadata Source

An implementation of MetadataSoruce will access metadata for extensions. Metadata is accessed in the form of an eclipselink-orm.xml file.

package org.eclipse.persistence.jpa.metadata;
 
public interface MetadataSource{
 
    /**
     * ADVANCED:
     * In most cases, this method should not be overridden.  The implementation of
     * this method uses getEntityMappingsReader() to obtain a reader that will
     * that reads a stream in the eclipselink-orm.xml format
     * Advanced implementations of MetadataRepository have to option of overriding
     * this method.
     * @return XMLEntityMappings which are then merged in using existing metadata 
     * processing at bootstrap time when creating an entityManager
     */
    public XMLEntityMappings getMetadata(Properties properties, ClassLoader loader)

Additionally, an adapter class will be provided that implements MetadataSource containing stubbed out methods. Customers will be encouraged to implement a MetadataSource by subclasing the adapter class rather than directly implementing the interface. This strategy will allow them to tranaparently absorb any new versions of the interface in new EclipseLink versions.

XMLMetadataSource

The first implementation of MetadataSource provided by EclipseLink will access a simple XML File.

It will provide an implementation of getMetadata(properties, classlaoder) that uses the property "eclipselink.metadata-repository.xml-file.url", specified like this:

  <property name="eclipselink.metadata-source.xml.url" value="foo://bar"/>

To create an input stream on the eclipselink-orm.xml file at URL: "foo://bar"/" and build an XMLEntityMappings using our existing EclipseLink ORM parsing code.

Writing to a metadata repository

In the initial implementation writing to the metadata repository will be left up to the user.

Remote Command Manager

A Command for RemoteCommandManager will be implemented that triggers a refreshMetadata() call on all subscribed EntityManagerFactories.

Design is in progress and will be added as it becomes available

Future Enhancements

Weaving of OneToOne mappings

Implement support for weaving of non-basic VIRTUAL mappings

• Handle OneToOneMappings
• Handle indirection

Allow metadata to be updated with an in-memory structure

From GlassFish team:

1. Programmatic API to call into EclipseLink to "push" the extension definitions.
2. The data exchanged between the caller and EclipseLink via API should be in a format that just refers to extension information and not a generic data structure.
3. The API call should be on an EclipseLink artifact that does not trigger deploy. That is it should be on an artifact at EMF level.

Database Metadata Source

Provide an implementation of Metadata Source that reads from a database.

Writing to a metadata Source

Provide API to write to the XMLFile and DatabaseMetadata Sources.

Configuration of muliple getter and setter methods

• Allow annotations and xml to specify a list of methods for VIRTUAL mappings to weave

Appendices

Appendix 1: Alternatives Considered

http://wiki.eclipse.org/EclipseLink/DesignDocs/335601

Appendix 2: Annotations

/**
 * Specifies that this class contains virtual attributes.
 * This annotation is used in an EclipseLink-specific way to define
 * access methods used by mappings with accessType=VIRTUAL.
 * The xml-equivalent is the <access-methods> tag
 */
@Documented
@Target(TYPE)
@Retention(RUNTIME)
public @interface VirtualAccessMethods {
 
    /**
     * (Optional) The name of the getter method to use for the virtual property
     * This method must take a single java.lang.String parameter and return a java.lang.Object.
     * If setMethod is specified, getMethod must be specified
     */
    String get() default "get";
 
    /**
     * (Optional) The name of the setter method to use for the virtual property
     * This method must take a java.lang.String parameter and a java.lang.Object parameter.
     * If getMethod is specified, setMethod must be specified
     */
    String set() default "set";
}

Appendix 3: eclipselink-orm

No changes were made to eclipseLink-orm.xml