Revision as of 15:42, 18 March 2011

Flex Columns Extension

Schema is designed to include preallocated columns that can be used to map additional data. In this example, Customer table might look like this:

• CUSTOMER
  • INTEGER ID
  • VARCHAR NAME
  • VARCHAR FLEX_COL1
  • VARCHAR FLEX_COL2
  • VARCHAR FLEX_CO31

The columns starting with the String "FLEX_COL" are flexible and may have mappings that point to them added at any time. For instance at runtime, a user may add a mapping called "company" that maps to FLEX_COL1.

Requirements

• Users MUST be able to add extensions at Runtime
• Extensions MUST be persistent. (i.e. Extensions must continue to exist if an application goes down and comes back up)
• Extensions MUST be shared amoung all EntityManagers configured to use them
• Extensions MUST be compatible with our multi-tenant features. (i.e. Extensions must be able to make use of our Multi-tenant features to be definable on a tenant by tenant basis)
• Extensions MUST have DDL Generation support
• BasicMappings MUST be supported
• Extensibility must be configurable using tradition JPA means (annotations, eclispelink-orm.xml)
• It MUST be possible to use JPA Queries to query based on the extensions
• Extensions SHOULD be supported through the JPA metamodel
• OneToOneMappings SHOULD be supported

Configuration

Metadata

Extensions will be supported through a user-defined map that uses the extension name as the key and the value of the extension as the map value. The user will be required specify that map on their domain class. Extensions will be updated by making changes to that map.

When defining that Map, the following information is supplied

• numberOfColunms (default 10) - the number of flexible columns available
• columnNamePrefix (default "FLEX_") - the prefix for the column name. When EclipseLink autogenerates the columns a number will be provided as a suffix to complete the column name. (i.e. the first column will be called FLEX_1 and the second column will be called FLEX_2)
• defaultColumnType (default VARCHAR(255) - the default type of the columns
• extensionTableName (default FLEX_DEF) - the name of the table to store metadata about FLEX columns in. This table can be shared by extensions for multiple entities
• createNonExistingColumns - Hook for ALTER TABLE Functionality. Link to feature doc will be provided later
• columns (default is an emptyList) - Note: This functionality will be implemented in our 2nd iteration of development. This is an override column that allows you to override the names and types of certain columns and contains a list of @FlexColumn annotations.

@FlexColumn contains: (Note: as above FlexColumn will be implemented in our 2nd iteration of development)

• index - required - the index of the column to update
• name - an override for the default column name provided
• type - an override for the default type provided

API

ClassDescriptor will hold an ExtensionManager.

ExtensionManager API

• addExtension(extensionName, extensionJavaType, extensionField) - add an extension using a specific database field
• addExtension(extensionName, extensionJavaType) - add an extension and allow EclipseLink to choose a database field
• getExtensionFields() get all extension fields
• getAvailableExtensionFields() return all of the unused extension fields
• removeExtension(name)

Annotation/XML Config

  @FlexExtensions(columnNamePrefix="FLEX" 
      numberOfNumericColumns=10 numberOfStringColumns=10 
      createNonExistingColumns=false, extensionTableName="EXTS")
  private Map<String, Object> extensions = new HashMap<String, Object>();
 
  public <T> T get(String name) {
      return (T) getExtensions().get(name);
  }
 
  public Object set(String name, String value) {
      return getExtensions().put(name, value);
  }

Annotations and xml will be used to set the map as holding the extensions. A validation exception will be thrown if more than one map is configured as holding extensions on a given class.

Annotations

@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexExtensions {
    /**
     * (Optional) Specify number of columns.  This is used to automatically generate a default set of columns. 
     * Unnecessary if columns is specified
     */
    int numberOfColumns default 10;
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String columnNamePrefix default "FLEX_COL";
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String defaultColumnType default VARCHAR(255);
 
    /**
     * When set to true this will activate a feature that executes ALTER table statements to 
     * make the required columns available if they do not exist
     */
    boolean createNonExistingColumns default false;
 
    /**
     *  The name of the table that holds the extensions to use.  This table can be shared.
     */
    String extensionTableName default null;
 
    /**
     * (Optional) Define the actual columns
     */
    FlexColumn[] columns
}
 
@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexColumn {
 
    String Name;
    int index;
    String type default VARCHAR(255)
}

eclipselink-orm

  <xsd:complexType name="attributes">
    <xsd:annotation>
      <xsd:documentation>
 
        This element contains the entity field or property mappings.
        It may be sparsely populated to include only a subset of the
        fields or properties. If metadata-complete for the entity is true
        then the remainder of the attributes will be defaulted according
        to the default rules.
 
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
    ...
    <xsd:element name="flex-extensions" type="orm:flex-extensions" minOccurs="0"/>
 
<!-- **************************************************** -->
 
<xsd:complexType name="flex-extensions">
  <xsd:annotation>
    <xsd:documentation>
      ...
    </xsd:documentation>
  </xsd:annotation>
  <xsd:sequence>
    <xsd:attribute name="number-of-columns" type="xsd:integer"/>
    <xsd:element name="column" type="column" minOccurs="0" maxOccurs="unbounded"/>
    <xsd:attribute name="column-name-prefix" type="xsd:integer"/>
    <xsd:attribute name="create-non-existing-columns" type="xsd:integer"/>
  </xsd:sequence>
</xsd:complexType>

Discussion:

• To what extent does it make sense to allow the names of columns to be defined in a granular way (i.e. with an array of Columns) Based on the feedback I have had, in a real application the number of columns could be in the 100s. Defining these things in either annotations or XML could become unweildy long before you get into the 100s.
• To what extent to we need the types of the flex columns to be definable? Can they all be strings, or do we need some other types as well?
• Should we provide a config that allows something like "numberOfNumericColumns" and "numberOfVarcharColumns" to allow better granularity of columns? How many types should be supported?

API

Extension Management

Extensions are managed with an ExtensionManager held on ClassDescriptor

Extensions can be managed through either a Customizer or through the JpaHelper. In a Customizer, the ExtensionManager should be obtained through the Descriptor and API can be called on ExtensionManager.

ExtensionManager API

• addExtension(String name, Class type)
• removeExtension(String name)
• getExtensions()

JpaHelper API

• addExtension(EntityManagerFactory emf, String entity, String name, String referenceType)
• removeExtension(EntityManagerFactory emf, String entity, String name)

Design

Adding Extensions

Extensions can be added to session in any of 3 ways.

1. API call to addExtension
2. Retrieval at startup time
3. Propogation from another client

API call to addExtension

A call to addExtension on the ExtensionManager will do the following:

1. check for an existing extension with the same name and throw an exception if it exists
2. add an initialize a mapping to the descriptor for the extension
3. persist the information about the extension to the database

Persistent information about the extension will be persisted through an EclipseLink-defined entity called ExtensionProperty.

ExtensionProperty fields

• id - sequenced id field
• entityName - name of the owning entity
• name - name of the extension
• type - type of the extension

These will be stored in the table define by the extensionTable configuration option. If that is not defined, the table will be called: "EXTENSION_DEF". This table will contain the following fields:

EXTENSION_DEF table fields

• EXT_ID - NUMERIC - sequenced id field
• ENT_NAME - VARCHAR - name of the Entity that owns it
• NAME - VARCHAR - name of the extension
• TYPE - VARCHAR- string representation of the type of the extension

ExtensionManager will hold a map with key = ExtensionName and value = ExtensionProperty. EclipseLink will choose the next appropriate column for the mapping.

Retrieval at startup time

When a ClassDescriptor is intialized, the ExtensionManager will initialize by querying for existing ExtensionProperties and add mappings for each of the ExtensionProperties found.

Propogation from another client

Discussion: What are the requirements

• Does this have to be a PUSH-style notification, or could it be at EntityManager or EntityManagerFactory creation time?

Handling xToOne Mappings

DDL Generation

DDL will be generated that includes all the specified extension columns and all the extension tables.

Column Creation

See <URL for Column Creation doc here>