Latest revision as of 11:02, 16 June 2011

EclipseLink 2.1: Enhanced FetchGroup Support

Requirements

This feature will enhance FetchGroup to address the following requirements:

The enhancement to FetchGroup support will include addressing the following requirements:

Configuration

1. Default FetchGroup will be defined when one or more basic mappings are configured LAZY and the entity class implements FetchGroupTracker (typically introduced through weaving).
   • The default FetchGroup defined by mappings will not involve relationships
   • The default FetchGroup can be customized through API
2. Named FetchGroup can be defined using annotations, eclipselink-orm.xml, or API
3. A FetchGroup is assumed to include all required attributes even if not specified. These required attributes will be added to the FetchGroup when it is initialzied
   • Initialization of default and named FetchGroup occurs when the FetchGroup manager is initialized
   • Dynamic FetchGroup are initialized when first used.

Query Usage

1. The default FetchGroup is used on any query/find operation when no FetchGroup is specified
   • A query/find can be customized to not use the default FetchGroup.
2. A named FetchGroup can be specified that will be used if it exists
   • Can be specified using JPA query hint or directly on the native query object
   • If the named FetchGroup cannot be found in the descriptor hierarchy none will be used a warning message logged

Triggering Attempt to access an attribute that's not part of the fetch group triggers reading of the rest of the object:

1. Outside transaction entire object (fetch group members included) is refreshed in the shared cache.
2. Inside transaction the previously fetched attributes are NOT refreshed. That means that:
   1. Changes that were potentially made to the fetched attributes by the user won't be overridden.
   2. Optimistic lock exception will be thrown (on commit or flush) if the object changed in the data base since read originally (version field is always part of any fetch group).

Cached Entities

1. If an entity is already cached (shared, isolated, UOW) then the resulting entity must have all of the items specified in the FetchGroup loaded.
   1. If the cached object has FetchGroup then resulting object will have a FetchGroup that is the union of the original one and the new one.
      1. That allows individual triggering of the attributes originally excluded from the fetch group (as opposed to all at once):
         1. Before accessing the not fetched attribute re-select the object with a new fetch group that contains this attribute only.
   • When returning an entity that is partial the FetchGroup MUST always include all items that are populated

Detaching Partial Entities

1. Entities that are detached when they are partially populated based on a FetchGroup must maintain a FetchGroup in their state so that what has been loaded versus what has not been is known.
2. Attempts to access unfetched attributes will cause an exception to be thrown
3. Modifying the state of a partial entity will cause its detached FetchGroup to be enhanced to include additional attributes that are set
   • in the case of collections the full collection must be set
4. When a partial entity with a FetchGroup is merged into a persistence context only those items defined in the FetchGroup will be merged
   • The cascade MERGE configurations on the mappings will be used and thus items in the FetchGroup without cascade MERGE configured will be ignored.

Usage Examples

The following usage examples are provided to assist in the understanding of this new functionality. The complete implementation details are cobvered in the design section of this specification.

Configuration

In order to use a FetchGroup developers must configure default, named, or dynamic FetchGroup instances for use in queries, copying, and merging of entities.

Default FetchGroup

The default FetchGroup is determined through the use of fetch=LAZY on basic mappings. There is no support for relationships in default the default FetchGroup unless the default FetchGroup is manually configured on an entity type's descriptor using API (DescritporCustomizer).

DescriptorCustomizer Example

FetchGroup phoneFG = new FetchGroup();
phoneFG.addAttribute("number");
ClassDescriptor phoneDescriptor = session.getClassDescriptor(PhoneNumber.class);
phoneDescriptor.getFetchGroupManager().setDefaultFetchGroup(phoneFG);

Named FetchGroup

A named FetchGroup can be configured through annotations or API (DecsriptorCustomizer). This feature includes extensions to the @FetchGroup annotation and eclipselink-orm.xml to support defining FetchGroup items for relationships.

Simple Annotation Example

A defined named FetchGroup works the same as a dynamic FetchGroup where initially no attributes exist in the FetchGroup and only the minimal required attributes for identity and version will be added.

@Entity
@FetchGroup(name="named-example", attributes={
        @FetchAttribute(name="id"), 
        @FetchAttribute(name="version"), 
        @FetchAttribute(name="name"), 
})
public class MyEntity {
 
    @Id
    private int id;
 
    @Version
    private long version;
 
    private String name;
 
    private int size;

This same FetchGroup can be specified minimally by exclsuing the required identity and version attributes that will be added automatically as:

@Entity
@FetchGroup(name="named-example", attributes={
        @FetchAttribute(name="name"), 
})

Relationships Annotation Example

When specifying a relationship it is assumed (in 2.1) that the target of the relationship should be minimally loaded (id and version attributes).

@Entity
@FetchGroup(name="named-example", attributes={
        @FetchAttribute(name="id"), 
        @FetchAttribute(name="version"), 
        @FetchAttribute(name="firstName"), 
        @FetchAttribute(name="lastName"), 
        @FetchAttribute(name="address") 
})
public class Employee{

In this example when the address attribute is loaded based on the result of a query using this named FetchGroup it will only have its minimal attributes loaded.

@Entity
@FetchGroup(name="named-example", attributes={
        @FetchAttribute(name="id"), 
        @FetchAttribute(name="version"), 
        @FetchAttribute(name="firstName"), 
        @FetchAttribute(name="lastName"), 
        @FetchAttribute(name="address.city")
})
public class Employee{

In this example above the address.city attribute is specified meaning that when the address is loaded its minimal attributes (identity and version) plus the city attribute will be loaded.

EclipseLink ORM XML Example

TODO: Add example of how a FetchGroup with relationships is defined in XML

Descriptor Customizer Example

public class EmployeeFetchGroupCustomizer implements DescriptorCustomizer {
    public void customize(ClassDescriptor descriptor) throws Exception {
        FetchGroup<Employee> fg = new FetchGroup<Employee>("Employee.fg");
        fg.addAttribute("id");
        fg.addAttribute("version");
        fg.addAttribute("firstName");
        fg.addAttribute("lastName");
        fg.addAttribute("address.city");
        descriptor.getFetchGroupManager().addFetchGroup(fg);
    }
}

Querying

A FetchGroup is used in the processing of a query when a default FetchGroup exists on the entity type's descriptor or one is specified on the query.

Named FetchGroup Example

Query query = em.createQuery("SELECT e FROM Employee e WHERE e.id = :ID");
query.setParameter("ID", Queries.minimumEmployeeId(em));
query.setHint(QueryHints.FETCH_GROUP_NAME, "test");

Dynamic FetchGroup Example

Query query = em.createQuery("SELECT e FROM Employee e WHERE e.gender = :GENDER");
query.setParameter("GENDER", Gender.Male);
 
// Define the fields to be fetched on Employee
FetchGroup fg = new FetchGroup();
fg.addAttribute("id");
fg.addAttribute("version");
fg.addAttribute("firstName");
fg.addAttribute("lastName");
fg.addAttribute("address.city");
fg.addAttribute("address.postalCode");
 
// Configure the dynamic FetchGroup
query.setHint(QueryHints.FETCH_GROUP, fg);
 
List<Employee> emps = query.getResultList();

Overlapping FetchGroup Queries

Two FetchGroup can be used on different queries for an entity but each only loads one LOB without causing the N additional LOB columns from being loaded.

// Employee has several lob attributes that are very expensive to read.
// To avoid reading the lobs a fetch group is used.
Query query = em1.createQuery("SELECT e FROM Employee e WHERE e.id = 1");
// Employee read into em cache with only firstName and lastName attributes
// (+ primary key and version that are automatically added to any fetch group).
FetchGroup<Employee> namesFG = new FetchGroup<Employee>("namesFG");
namesFG.addAttribute("firstName");
namesFG.addAttribute("lastName");
query.setHint(QueryHints.FETCH_GROUP_NAME, namesFG);
emp = query.getSingleResult();
em1.close();
 
// After em1 is closed the Employee object remains in the shared cache.
// The Employee object has EntityFetchGroup that indicates which attributes has been fetched: {id, version, firstName, lastName}
 
// The user needs to work with lob1 attribute, however accessing
// not fetched attribute would trigger reading of the whole object, including not required attributes lob2, ...lob10.
// Instead the Employee object is read with a new fetch group that consists only of lob1:
FetchGroup<Employee> lob1FG = new FetchGroup<Employee>("lob1FG");
lob1FG.addAttribute("lob1");
Query query = em2.createQuery("SELECT e FROM Employee e WHERE e.id = 1");
query.setHint(QueryHints.FETCH_GROUP_NAME, lob1FG);
// the Employee object is found in the shared cache,
// the attributes that are not found in the cached Employee's EntityFetchGroup (lob1) are read from the db.
// The resulting object fetched attributes is the union of the two fetch groups: {id, version, firstName, lastName, lob1}
emp = query.getSingleResult();
// lob1 has been already fetched - no reading from the db is required.
emp.lob1();

Detached Entities

The following usage examples illustrate how a FetchGroup can be used with detached entities.

Copy

Copy using custom policy

ObjectCopyingPolicy policy = new ObjectCopyingPolicy();
policy.addAttribute("id");
 
JpaEntityManager elem = em.unwrap(JpaEntityManager.class);
Employee empCopy = elem.copy(emp, policy);

Copy using LoadGroup

ObjectCopyingPolicy policy = loadGroup.toCopyPolicy();
 
JpaEntityManager elem = em.unwrap(JpaEntityManager.class);
Employee empCopy = elem.copy(emp, policy);

Load

Load using LoadGroup

Load the specified in LoadGroup relationships for either a single entity or a collection.

LoadGroup loadGroup = new LoadGroup();
loadGroup.addAttribute("address");
loadGroup.addAttribute("phoneNumbers");
loadGroup.addAttribute("manager.projects.projectLeader");
 
JpaEntityManager elem = em.unwrap(JpaEntityManager.class);
elem.load(employeeObject, loadGroup);
//or
elem.load(employeeCollection, loadGroup);

FetchGroups and LoadGroups When a FetchGroup is applied to a query by default the FetchGroup also loads its relational attributes. The qauery for Employees executed with rhe following FetchGroup

FetchGroup fetchGroup = new FetchGroup();
fetchGroup.addAttribute("id");
fetchGroup.addAttribute("version");
fetchGroup.addAttribute("firstName");
fetchGroup.addAttribute("lastName");
fetchGroup.addAttribute("address");
fetchGroup.addAttribute("phoneNumbers.owner");
fetchGroup.addAttribute("phoneNumbers.type");
fetchGroup.addAttribute("phoneNumbers.areaCode");
fetchGroup.addAttribute("manager.id");
fetchGroup.addAttribute("manager.version");
fetchGroup.addAttribute("manager.firstName");
fetchGroup.addAttribute("manager.lastName");
fetchGroup.addAttribute("manager.projects.projectLeader.id");
fetchGroup.addAttribute("manager.projects.projectLeader.version");
fetchGroup.addAttribute("manager.projects.projectLeader.firstName");
fetchGroup.addAttribute("manager.projects.projectLeader.lastName");

not only limit the results to the specified attributes, but also ensures that all specified relations are loaded. Before returning query result Eclipselink creates a LoadGroup from the FetchGroup and uses it to load the objects.

Hovewer these two functions can be separated.

fetchGroup.setShouldLoad(false);

ensures that only eager relationships are loaded.

A LoadGroup could be created from a FetchGroup.

LoadGroup loadGroup = fetchGroup.toLoadGroup();

All non-relational attributes present in the LoadPlan will be ignored, it will be functionally equivalent to:

LoadGroup loadGroup = new LoadGroup();
loadGroup.addAttribute("address");
loadGroup.addAttribute("phoneNumbers");
loadGroup.addAttribute("manager.projects.projectLeader");

Copy using LoadGroup

ObjectCopyingPolicy policy = loadGroup.toCopyPolicy();
 
JpaEntityManager elem = em.unwrap(JpaEntityManager.class);
Employee empCopy = elem.copy(emp, policy);

FetchGroup "Sparse" Merge

A partial entity with a FetchGroup attached will be merged based on that FetchGroup in combination with cascade merge setting on the entity's mapping.

Design

The following sections describe the high level design of the changes that are proposed to address the requirements and usage examples above.

AttributeGroup Hierarchy

TODO: Explain proposed structure of a FetchGroup and its map of FetchAttributes

Query Execution

When a query is executed a FetchGroup is determined for the query based on the following rules

1. If a FetchGroup is specified on the query through direct API or QueryHintHandler then that FetchGroup will be used
2. If a FetchGroup name is specified on the query through direct API or QueryHintHandler then that name will be used to lookup a query in the reference descriptor's hierarchy using standard inheritance rules.
   1. If the query is not found for the name ... TODO
3. If a default FetchGroup is specified on the descriptor or any of its parent descriptors that FetchGroup will be used

Note: the FetchGroup used in a query execution should not be cached in the query.

EntityFetchGroup

An EntityFetchGroup instance is set on each entity resulting from a FetchGroup query execution. The EntityFetchGroup initially only has the shared execution FetchGroup as its state. The EntityFetchGroup is used to represent the exact state of a partial entity and must be unique per entity since set operations as well as merged FetchGroup queries will result in entities having a unique set of attributes loaded.

A set performed on unfecthed attribute triggers reading of the entire object, setting the attribute and removal of object's EntityFetchGroup.

"Overlapping FetchGroup Queries" paragraph above explains on a simple example how the two merging fetch groups form a union. Note that executing a primary key query (find) with an empty fetch group (that will contain only the mandatory primary key and version attributes) will guarantee that the object (if cached) will be broughyt from the cache without hitting the db.

Detached Entities

When an entity is copied (JpaHelper.copy(FetchGroup fg, Object entity)) or serialized only the attributes specified in the attached or provided FetchGroup are included. All other attributes are left as the default value initialized by the constructor.

When a set is called on a partial entity (with a FetchGroup) the entity's state is enhanced to include the provided value and the EntityFetchGroup has this attribute added.

When a get method id called on a partial entity for an attribute that is not included in its FetchGroup an exception is thrown.

The get/set attribute checking in a detached entity is done using woven FetchGroupTracker code in the entity in combination with the specific FetchGroup type attached to the detached entity.

Open Issues

1. Merging FetchGroup query results into the working copy of an entity
2. Modification of a shared FetchGroup after it has been used in a query result
3. Merging a cached object without a fetch group with a fetch group query (or another way around)
   1. Fetch group that contains a lazy relationship triggers that relationship, the object without fetch group does not;
      1. Does it mean it would make sense to have a fetch group that contains ALL the attributes?