Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EclipseLink/DesignDocs/340192"

Line 192: Line 192:
  
 
= Design =
 
= Design =
 +
 +
== Metadata Repository ==
 +
 +
MetadataRepository will require discussion to build correct.  The following is provided for discussion
 +
 +
MetadataRepository will be an abstract class that has several concrete implementations.  It will have 3 kinds of API.
 +
 +
1. API to retreive extensions
 +
2. API to store extensions and cause them to be used
 +
3. Configuration API
 +
 +
Retrieval API
 +
 +
* URL getExtensionsAsXML() - returns a URL to an eclipselink-orm.xml document that maps the extensions.  This document will be included in the xml documents processed at bootstrap time when creating an entityManager
 +
 +
Storage API
 +
 +
* add Mapping(DatabaseMapping, Descriptor)
 +
 +
Configuration API
 +
 +
* setLocation(String location) - the location of the store, a URL.  This could represent the a file system location where the repository is, a database connection URL, or any other location
 +
 +
 +
  
 
= Old Design =
 
= Old Design =

Revision as of 13:49, 8 April 2011

Flex Columns Extension

This feature allows mappings to be added to preexisting (unmapped) columns in a table.

The Schema is designed to include preallocated columns that can be used to map additional data. In this example, Customer table might look like this:

  • CUSTOMER
    • INTEGER ID
    • VARCHAR NAME
    • VARCHAR FLEX_COL1
    • VARCHAR FLEX_COL2
    • VARCHAR FLEX_CO31

The columns starting with the String "FLEX_COL" are flexible and may have mappings that point to after initial application design. For instance at after the application has been designed, a user may add a mapping called "company" that maps to FLEX_COL1.

The use case for a scenario like this is:

  • One company develops an application with the goal of making it available to a customer. The application contains all the functionality necessary for basic use of the application
  • The application is deployed and made available to the customer. The customer can then tweak the application by adding mappings to data that is specific to their business

This kind of application works well in concert with a multi-tenant application where the initial application is developed and then each tenant can add mappings that are particular to their business.

Requirements

  • Users MUST be able to add after design time
  • Extensions MUST be persistent. (i.e. Extensions must continue to exist if an application goes down and comes back up)
  • Extensions MUST be shared amoung all EntityManagers configured to use them
  • Extensions MUST be compatible with our multi-tenant features. (i.e. Extensions must be able to make use of our Multi-tenant features to be definable on a tenant by tenant basis)
  • Extensions MUST have DDL Generation support
  • BasicMappings MUST be supported
  • Extensibility MUST be configurable using tradition JPA means (annotations, eclispelink-orm.xml)
  • It MUST be possible to use JPA Queries to query based on the extensions
  • Extensions SHOULD be supported through the JPA metamodel
  • OneToOneMappings SHOULD be supported
  • It MAY be possible to add extensions without logging in the session. (See ER 341429.)

Configuration

Metadata

Extensions will be supported using our VIRTUAL access type. Virtual Access type allows properties to be set through a getter that takes an attribute name as an argument and a setter that takes an argument name and a value as an argument.

Annotations and xml will be used to configure which mehtods will be used. Annotating a field as @Extensible will indicate to EclipseLink that the class is extensible. The value "FLEX" will be provided to the @Extensible annotation to indicate Flex Columns Extensibility. Equivalent XML will be provided.

Along with @Extensible, and @FlexExtensions annotation will be provided that allows overrides to defaults. The main purpose of this annotation will be to provide information about additional columns needed to our DDL generation feature. That annotation will allow the following data.

  • numberOfColunms (default 10) - the number of flexible columns to generate into the DDL
  • columnNamePrefix (default "FLEX_") - the prefix for the column name. When EclipseLink autogenerates the columns a number will be provided as a suffix to complete the column name. (i.e. the first column will be called FLEX_1 and the second column will be called FLEX_2)
  • defaultColumnType (default VARCHAR(255) - the default type of the columns
  • createNonExistingColumns - Hook for ALTER TABLE Functionality. Link to feature doc will be provided later
  • columns (default is an emptyList) - Note: This functionality will be implemented in our 2nd iteration of development. This is an override column that allows you to override the names and types of certain columns and contains a list of @FlexColumn annotations.

@FlexColumn contains: (Note: as above FlexColumn will be implemented in our 2nd iteration of development)

  • index - required - the index of the column to update
  • name - an override for the default column name provided
  • type - an override for the default type provided

Examples

Example 1

  • Field Access for non extension fields
  • Virtual Access for extension fields uses defaults (get(String), set(String, Object))
  • extensions mapped in a portable way - @Transient
  • @FlexExtensions optional annotation for configuring overrides for defaults - used mainly by DDL Generation
  @Entity
  @Extensible(FLEX)
  @FlexExtensions(columnNamePrefix="FLEX",
    numberOfColumns=10,
    defaultColumnType="VARCHAR(30)"     
    createNonExistingColumns=false)
  public class Address {
 
    @Id
    private int id;
 
    @Transient
    private Map<String, Object> extensions;
 
    public int getId(){
        return id;
    }
 
    public <T> T get(String name) {
        return (T) extentions.get(name);
    }
 
    public Object set(String name, String value) {
        return extensions.put(name, value);
    }
 
...

Example 2

  • Field Access for non extension fields
  • extensions mapped in a portable way - @Transient
  • Preexisting table assumed. @FlexExtensions annotation omitted
  • Virtual Access for extension fields uses XML to override default methods.


  @Entity
  @Extensible(FLEX)
  public class Address {
 
    @Id
    private int id;
 
    @Transient
    private Map<String, Object> extensions;
 
    public int getId(){
        return id;
    }
 
    public <T> T getExtension(String name) {
        return (T) extensions.get(name);
    }
 
    public Object setExtension(String name, String value) {
        return extensions.put(name, value);
    }
 
...
 
   <basic name="name" attribute-type="String">
      <column name="FLEX_1"/>
      <access-methods get-method="getExtension" set-method="setExtension"/>
    </basic>

Example 3

  • Property Access for non extension fields
  • Virtual Access for extension fields uses defaults (get(String), set(String, Object))
  • extensions mapped in a portable way - no @Transient required because of Property access
  • Preexisting table assumed. @FlexExtensions annotation omitted
  @Entity
  @Extensible(FLEX)
  public class Address {
 
    private int id;
 
    private Map<String, Object> extensions;
 
    @Id
    public int getId(){
        return id;
    }
 
    public <T> T get(String name) {
        return (T) extensions.get(name);
    }
 
    public Object set(String name, String value) {
        return extensions.put(name, value);
    }
 
...


Metadata Repository

Extensions will be stored and retrieved from a metadata repository. The metadata repository will be a pluggable part of the EntityManagerFactory. Implementations for several types of metadata repostory will be provided.

  • Database Table - information about extensions is stored in a database table
  • XML - information about extensions is stored in XML
  • User-specified - The user will be able to provide an implementation of the class that access the metadata repository

At startup, an EntityManagerFactory retrieves additional mappings from the metadata factory and applies them to its metadata.

EntityManagerFactory

It will be possible to notify an EntityManagerFactory that it should refresh its mappings. On notificatin, it will rebootstrap a new session that includes the new mappings. The existing session will be kept as along as existing entity managers reference it. New EntityManagers will use the new session.

Notification will be availabe through direct API or through our RemoteCommandManager.

Design

Metadata Repository

MetadataRepository will require discussion to build correct. The following is provided for discussion

MetadataRepository will be an abstract class that has several concrete implementations. It will have 3 kinds of API.

1. API to retreive extensions 2. API to store extensions and cause them to be used 3. Configuration API

Retrieval API

  • URL getExtensionsAsXML() - returns a URL to an eclipselink-orm.xml document that maps the extensions. This document will be included in the xml documents processed at bootstrap time when creating an entityManager

Storage API

  • add Mapping(DatabaseMapping, Descriptor)

Configuration API

  • setLocation(String location) - the location of the store, a URL. This could represent the a file system location where the repository is, a database connection URL, or any other location



Old Design

Please ignore the design below. It has been kept around until any useful parts of it can be incorporated in the update design.

FlexExtensionManager

Each ClassDescriptor may have an ExtensionManager. For extensible types, the ExtensionManager will be non-null and will be set either by annotation, xml, or explicitly on the descriptor. Different types of ExtensionManager are possible. The flex column behavior is managed by a FlexExtensionManager.

FlexExtensionManager holds the following state:

  • extensionsAccessor - an Accessor that gives it access to the Map on the domain class that holds the extensions. e.g. the Map annotated as @FlexExtensions
  • availableFields - the list of DatabaseField that it believes can be used for new extensions
  • usedFields - the list of DatabaseField that it believes is used for existing extensions
  • shouldCreateNonExistingColumns - a boolean used to enable the additional feature that will allow ALTER schema language to create new columns (more to come)


Extension Data

Extension data will be persisted to the database. Each extension will be represented by an ExtensionProperty holding the following data:

  • String - entityType - the name of the Entity it represents
  • DatabaseField - field - a DatabaseField representing the column used by this extension
  • Class - type - the target class of the mapping
  • String - name - the name of the extension
  • mapping - a reference to the mapping for this extension

An project containing an Extensible type will have a descriptor added to it for ExtensionProperty. Any time an Extension is added, its ExtensionProperty will be persisted. The table will contain the following information:

  • ENTITY_TYPE (PK) - Directly mapped from entityType
  • COLUMN_NAME (PK) - Name of the field stored in field
  • TARGET_TYPE - Class name of the class represented by type
  • NAME - Directly mapped from name

At descriptor initialization time, a ReadAllQuery will be issued for all ExtensionProperties with an ENTITY_TYPE. These will be stored by the ExtensionManager and used to build the appropritate mappings.

Adding Extensions

Extensions are added through a call to extensionManager.addExtension(name, type). When addExtension is called, the following will occur:

  1. a check will be made to ensure the extension does not already exist in the descriptor and an exception will be thrown if it does
  2. the list of available fields will be checked for a field that can represent the type that is passed in
  3. If no field is available either the Column Creation feature will be used, or an exception will be thrown.
  4. the availableFields and usedFields lists will be updated
  5. An instance of ExtensionProperty will be created and persisted, if persisting fails for any reason, an exception will be thrown and the availableFields and usedFields lists will be reset
  6. A mapping will be created to represent the ExtensionProperty and it will be added to the descriptor

Mapping Types

Extensions may only be the following types of mappings

  • Basic
  • OneToOne (single foreign key)

MapValueAttributeAccessor

Since all extensions will be stored in the same map, mappings cooresponding with those extensions will use a MapValueAttributeAccessor.

MapValueAttributeAccessor will have knowledge of the map holding the extensions and a call to getAttributeValueFromObject will retreive the value from the map based on the extension's property name and a call to setAttributeValueInObject will put a value in the map based on the property name.

Basic Mappings

The initial iteration will allow Basic mappings. Basic mappings are fairly easy to implement as they are a simple coorespondance between a field and value in the map. A basic mapping will be constructed with the appropriate data and added to the descriptor

OneToOne Mappings

OneToOne mappings to Objects with a single primary key will be supported. The flex column will be used as the foreign key.

Extension Propogation

Extensions will be refreshed from the database each time an EntityManagerFactory or EntityManager is created.

No Push style notification via our RemoteCommandManager or other mechanism is currently planned.

DDL Generation

DDL generation will allow the creation of the extension table and also include any known extension columns to domain tables

Column Creation

See <URL for Column Creation doc here>

Future Enhancemts

Virtual Access access the extensions field directly

  • Field Access for non extension fields
  • Virtual Access for extension fields uses @Extensions annotation on attribute that will hold the extensions. This assumes an enhancement to allow Virtual access to operate on a field
  • extensions mapped in a non portable way - @Extensions
  • Preexisting table assumed. @FlexExtensions annotation omitted
  @Entity
  @Extensible(FLEX)
  public class Address {
 
    @Id
    private int id;
 
    @Extensions
    private Map<String, Object> extensions;
 
    public int getId(){
        return id;
    }
 
...

Appendices

Apendix 1: Annotations

@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexExtensions {
    /**
     * (Optional) Specify number of columns.  This is used to automatically generate a default set of columns. 
     * Unnecessary if columns is specified
     */
    int numberOfColumns default 10;
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String columnNamePrefix default "FLEX_COL";
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String defaultColumnType default VARCHAR(255);
 
    /**
     * When set to true this will activate a feature that executes ALTER table statements to 
     * make the required columns available if they do not exist
     */
    boolean createNonExistingColumns default false;
 
    /**
     *  (Optional) The table that holds the extensions to use.  This table can be shared.
     */
    Table table;
 
    /**
     * (Optional) Define column overrides
     */
    FlexColumn[] columns
}
 
@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexColumn {
 
    /**
     * Mandatory - the index of the column to override.  Between 1 and numberOfColumns.
     * Exception will be thrown for numbers < 1 or > numberOfColumns
     */
    int index;
 
    /**
     * Column name - used to override default column names
     */
    String name;
 
    /**
     * Column type - used to override default column type
     */
    String type default VARCHAR(255)
}

Appendix 2: eclipselink-orm

  <xsd:complexType name="attributes">
    <xsd:annotation>
      <xsd:documentation>
 
        This element contains the entity field or property mappings.
        It may be sparsely populated to include only a subset of the
        fields or properties. If metadata-complete for the entity is true
        then the remainder of the attributes will be defaulted according
        to the default rules.
 
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
    ...
    <xsd:element name="flex-extensions" type="orm:flex-extensions" minOccurs="0"/>
 
<!-- **************************************************** -->
 
<xsd:complexType name="flex-extensions">
  <xsd:annotation>
    <xsd:documentation>
      ...
    </xsd:documentation>
  </xsd:annotation>
  <xsd:sequence>
    <xsd:attribute name="number-of-columns" type="xsd:integer"/>
    <xsd:element name="column-name-prefix" type="xsd:string"/>
    <xsd:attribute name="default-column-type" type="xsd:string"/>
    <xsd:attribute name="create-non-existing-columns" type="xsd:boolean"/>
    <xsd:attribute name="extension-table" type="eclipselink-orm:table"/>
    <xsd:element name="flex-columns" type="eclipselink-orm:flex-column" minOccurs="0" maxOccurs="unbounded"/>
  </xsd:sequence>
</xsd:complexType>
 
<xsd:complexType name="flex-column">
  <xsd:annotation>
    <xsd:documentation>
      ...
    </xsd:documentation>
  </xsd:annotation>
  <xsd:sequence>
    <xsd:attribute name="index" type="xsd:integer"/>
    <xsd:attribute name="name" type="xsd:string"/>
    <xsd:attribute name="type" type="xsd:string"/>
  </xsd:sequence>
</xsd:complexType>

Back to the top