EclipseLink/Development/JPA 2.0/jpql

From Eclipsepedia

Jump to: navigation, search

Contents

JPQL Updates

JPA 2.0 Root | Enhancement Request

Issue Summary

JPA 2.0 introduces a number of JPQL updates. Each upate will implemented as either one or two of small features. For features where there is no current EclipseLink Expression framework support, Expression support will be first added. When Expression Framework support is available, our JPQL parser will be updated to make use of that support.

The following is the list of changes to JPQL we are currently aware of:

  • Navigation across embeddeds (e.g. select e from Entity e where e.embedded.value = :value)
  • Key/Value/Entry access for maps (e.g. select KEY(m) from Entity e join e.map m where VALUE(m) = :value) - depends on Extended Maps <Complete>
  • Add an "AS" keyword to Select (e.g. select e as y from Entity e)
  • Added alloable arguments to constructor expressions (e.g. select new MyObject(e, 11) from Entity e)
  • Subselects can now collection information (e.g. select e from Entity e where e.id in (select a.id from AEntity a join a.collection c where .....)
  • IN can now take a collection as a parameter (e.g. select e from Entity e where e.id in :value)
  • Expressions referring to collection members now use entity_or_value_expression
  • Entity type Expressions have been added (e.g. select e from EntitySuperClass e where TYPE(e) = EntitySubclass.class)
  • Case Expression have now been added (e.g. updateEntity e SET e.value = CASE WHEN e.value2 = 1 THEN 11 WHEN e.value2 = 2 THEN 22 ELSE 0

END

  • A list Index operator has now been added for ordered lists - depends on Ordered lists
  • Concat can now take an additional argument
  • Nested Fetch Joins
  • Support JDBC escape syntax for date, time and timestamp

Feature design

Maps

JPQL now allows keywords KEY, VALUE and ENTRY for navigation into mappings that are Maps. KEY and VALUE can be used either in the SELECT clause or the WHERE clause. ENTRY can only be used in the FROM clause.

Examples:

  • select KEY(m) from Entity e join e.map m where KEY(m) = :value
  • select VALUE(m) from Entity e join e.map m where VALUE(m).id = :id
  • select ENTRY(m) from Entity e join e.map m where e.id = :id

Expression Design

By default, Expressions that navigate over a Map in EclipseLink navigate to the Map value. As a result, no Expression changes are required to support VALUE().

Examples

We will add mayKey() and mapEntry() to Expression. Here are some same expressions. Assume map is an attribute of type Map:

In the SELECT

  • expressionBuilder.get("map").mapKey()
  • expressionBuilder.get("map").mapKey().get("id")
  • expressionBuilder.get("map").mapEntry()

In the WHERE

  • expressionBuilder.get("map").mapKey().equal(1);
  • expressionBuilder.get("map").mapKey().equal(myEntity);
  • expressionBuilder.get("map").mapKey().get("myAggregate").get("string").equal("string1");

Code Updates

The following is a summary of the main changes that will be required:

  • The internal code to support this will be in a new subclass of QueryKeyExpression called: MapEntryExpression. MapEntryExpression will hold a variable called returnMapEntry which will be set to true if the ENTRY should be returned and false if the KEY should be returned.
  • ReportQueryResult will be updated to properly build results that are KEY or ENTRY
  • MapContainerPolicy, MappedKeyMapContainerPolicy and DirectMapContainerPolicy will all be updated to be able to calculate the Java type of their key

Testing

Expression based testing will be added for the following scenarios

  • DirectCollectionMapping with mapKey accessed in WHERE clause for Direct, Aggregate and Entity keys
  • AggregateCollectionMapping with mapKey accessed in WHERE clause for Direct, Aggregate and Entity keys
  • ManyToManyMapping with mapKey accessed in WHERE clause for Direct, Aggregate and Entity keys
  • UnidirectionalOneToManyMapping with mapKey accessed in WHERE clause for Direct, Aggregate and Entity keys
  • OneToManyMapping with mapKey accessed in WHERE clause for Direct, Aggregate and Entity keys
  • OneToManyMapping with mapKey accessed in SELECT clause for Direct, Aggregate and Entity keys
  • OneToManyMapping with mapEntry accessed in SELECT clause
  • In Memory OneToManyMapping with mapKey accessed in WHERE clause
  • Expression that navigate through map keys. e.g. expressionBuilder.get("map").mapKey().get("myAggregate").get("string").equal("string1");

JPQL

Parser Updates

  comparison_expression ::= …|  entity_type_expression}
  entity_type_expression ::= type_discriminator | entity_type_literal | input_parameter
  type_discriminator ::= TYPE(identification_variable | single_valued_object_path_expression | input_parameter)
  qualified_identification_variable ::= KEY(identification_variable) | VALUE(identification_variable) | ENTRY(identification_variable)
  general_identification_variable ::= identification_variable | KEY(identification_variable) |VALUE(identification_variable)


  • The JPQL grammar file will be updated with the new KEY, VALUE and ENTRY keywords.
  • VALUE will be treated as a no-op as the functionality necessary to make value work is already the default behavior.
  • A MapKeyNode and a MapEntryNode will be added to the collection of nodes that can compose the parse tree. They will be designed to create the appropriate Expression instance

Validation

Validation will be done in several places

  • At Expression validation time, MapEntryExpression will ensure the mapping exists, is a collection mapping and contains a container policy that is appropriate to a map mapping
  • ParameterExpression will have added functionality to match the type of they Parameter against the type of the map key
  • MapKeyExpression will calculate the type of the MapKey for type Validation
  • MapEntryExpression will set it's type to Map.Entry for type Validation
  • In cases where EclipseLink does not have the type of the Map Key in it's metadata we will avoid doing type checking and assume expressions are valid

Testing

A selection of tests will be added to cover various possibilities

  • Selecting a MapEntry
  • Selecting a MapKey
  • Selecting a MapValue
  • MapKey in WHERE clause
  • MapValue in WHERE clause
  • navigating through a MapKey in the WHERE clause
  • navigating through a MapValue in the WHERE clause



Collection Parameters for IN

Expression Design

This is already supported by the EclipseLink Expression framework

JPQL

  {state_field_path_expression | type_discriminator} [NOT] IN
  { ( in_item {, in_item}* ) | (subquery) | collection_valued_input_parameter }
  
  in_item ::= literal | single_valued_input_parameter

Example

  • select e from employee e where e.id in :param

Parser Only two changes are required for JPQL

  1. The parser will be updated to accept a list parameter for inExpression
  2. The validation will be updated to treat a list parameter as type Collection which will allow validation to proceed normally

Testing

A test will be added to the JUnitJPQLSimpleTestSuite




TYPE Expressions

Expression Design

http://wiki.eclipse.org/EclipseLink/Development/JPA_2.0/entity_type_expressions

JPQL Design

  comparison_expression ::= ... |  entity_type_expression}

  entity_type_expression ::= type_discriminator | entity_type_literal |input_parameter

  type_discriminator ::= TYPE(identification_variable | single_valued_object_path_expression |input_parameter)

Examples

  • select p from Project p where type(p) = LargeProject // LargeProject is an entity_type_literal that refers to an entity name
  • select p from Project p where type(p) = :param
  • select p from Project p where type(p) in (LargeProject, SmallProject)

Parser Updates

The BNF for types makes is somewhat non-deterministic in certain cases. The following rule can make it confusing for the parser to determine the user's intent:

entity_type_expression ::= type_discriminator | entity_type_literal | input_parameter

entity_type_literal in indistinguishable from a variable reference

As a result it is impossible to tell if a query like 'select e from Employee e where SmallProject = LargeProject' is an incorrectly written query or a TYPE query that evaluates constants. We have decided that any string that evaluates to the name of an Entity will be treated as an EntityTypeLiteral and not a variable.

  • The new TYPE token will be added to the Lexer
  • inItem will be updated to accept an abstract schema name - for a entity_type_literal
  • arithmeticPrimary will be update to accent entityTypeExpression
  • an entityTypeExpression rule will be added that maps to a new parse tree node - ClassForInheritanceNode
  • ClassForInheritanceNode will implement logic to build a type expression
  • VariableNode has been updated to allow it to represent an entity_type_literal

Validation

  • ClassForInheritanceNode will set its type to Class.class

Testing

We will add the following tests:

  • TYPE with = and a entity_type_literal
  • TYPE with = and a parameter
  • TYPE with IN and a set of entity_type_literals
  • TYPE with IN and a parameter

Concat

JQPL now allows more than two arguments to CONCAT

Expression Design

We will reuse the current concat expression and call it multiple times

JPQL Design

  functions_returning_strings ::= CONCAT(string_primary, string_primary {, string_primary}*) 

Examples

  • select e from employee e where concat(e.lastName, e.ssn, e.id) = :param

Parser Updates

The parser currently creates an expression for the first argument and does an Expresson.concat() on it with the second argument. We will simply update this code to repeatedly call Expression.concat on arguments 2 through n.

Vaidation

Validation is unchanged except that each arguement of the concat is validated.

Testing

  • A simple 3 argument concat test will be added.

Substring

The third argument to substring is now optional

Expression Design

In most cases, a substring expression can be written to the database as a simple expression.

TopLink: employee.get("firstName").substring(1)
SQL: SUBSTR(FIRST_NAME, 1)

Several databases do not support a 2-argument substring. In that case, we will use a length function as the third argument to the 3-argument function

  • SQLServer - SUBSTR(FIRST_NAME, 1, LEN(FIRST_NAME))
  • Sybase - SUBSTR(FIRST_NAME, 1, CHAR_LENGTH(FIRST_NAME))

Parser Design

  SUBSTRING(string_primary, simple_arithmetic_expression [, simple_arithmetic_expression]) 

Parser Updates

The parser will be modified to make the 3rd argument to substring optional and call the appropriate Expression method depending on if it is specified.

Validation

Validation will be unchanged

Testing

  • Simple select with substring in the Where clause

Case

The spec defines 4 types of case statements:

  1. A general case expression that uses conditional expressions to determine the result
  2. A simple case expression that uses comparison to a set of values to determine the result
  3. A coalesce expression that returns the first non-null argument
  4. A nullif expression that tests if arguments are equal and returns null if they are

Expression Design

http://wiki.eclipse.org/EclipseLink/Development/JPA_2.0/case_expressions

Parser Design

Parser Updates

  case_expression ::= general_case_expression | simple_case_expression |coalesce_expression | nullif_expression

  general_case_expression::= CASE when_clause {when_clause}* ELSE scalar_expression END

  when_clause::= WHEN conditional_expression THEN scalar_expression

  simple_case_expression::= CASE case_operand simple_when_clause {simple_when_clause}* ELSE scalar_expression END

  case_operand::= state_field_path_expression | type_discriminator

  simple_when_clause::= WHEN scalar_expression THEN scalar_expression

  coalesce_expression::= COALESCE(scalar_expression {, scalar_expression}+)

  nullif_expression::= NULLIF(scalar_expression, scalar_expression)

The current grammar handles these rules virtually unaltered

NullIf

  • NullIf functions like any other two argument expression. The left and right elements are used to store the two arguments and the expression is built from those arguments.

Coalesce

  • CoalesceNode has a list of clauses
  • When the expression is built, clause is asked to build itself and they are built into a list used to create the expression

Case

  • CaseNode is used to build both Simple Case expressions and general case expressions
  • WhenThenNode has been implemented to allow the construction of WHEN-THEN clauses in case node. It handles both simple and general types of WHEN-THEN clauses
  • WhenThenNode is a simple Node with the When as the left child and the Then as the right child.
  • The type of WhenThenNode is set to the type of the right child
  • CaseNode has a list of WhenThen clauses, a right child for the Else clause and optionally a left child for simple case statements
  • For simple case statements, the left child is use for the caseOperand
  • CaseNode builds a map of from the expressions built from the WhenThen nodes with the When as the key and the Then as the value

Validation

  • For Case and Coalesce if all the then nodes have the same Type, that type is used for the case statement, else Object is used for the type
  • For NullIf node, the type of the right argument is used

Testing

  • Each of simple case, general case, coalesce, and null iff
    • In the select clause
    • In the where clause
  • Update with Case



Index

Allows to query on the index field of a list with an Order column

Expression Design

http://wiki.eclipse.org/EclipseLink/Development/JPA_2.0/ordered_lists

Parser Design

Parser Updates

  functions_returning_numerics::= ... | INDEX(identification_variable)

We will add handling of index to functionReturningNumerics exactly as it is describe in the specification and new IndexNode will be added to build expressions and connect to the query.

Validation

  • Since the BNF only allows index to be called on a variable, and exception will be thrown if Index is specified somewhere other than a variable.
  • The type returned from index will be set as Integer as described in the spec

Testing

  • Index in the select clause
  • Index in the where clause

Scalar Expression

The new Grammar specifies which values are scalar in a more specific way

Expression Design

This does not require expression implementation. If is a new holder for already implemented expressions.

Parser Design

scalar_expression ::= simple_arithmetic_expression | string_primary | enum_primary | datetime_primary |
  boolean_primary | case_expression | entity_type_expression

Parser Updates

Due to the way the grammar is currently designed we will construct a scalar as two types. The first type will include all the types listed above. The second will not contain simple_arithmetic_expression. This will be used to make grammars that require scalar expression, but do also have rules that resolve to simple_arithmetic_expression deterministic.

Updates will be made all have all elements that now refer to scalar expression to refer to this expression:

  • newValue for update
  • selectExpression
  • simpleConditionalExpression (uses the version without simple_arithmetic_expression)
  • comparisonExpressionRightOperand (uses the version without simple_arithmetic_expression)
  • multple parts of case expressions


Validation

Since this is a holder for existing expressions, the validation is already implemented

Testing

There is no new testing. The existing tests and the tests written for new JPQL will exercise this functionality


Extra join navigation

The parser now allows multi-level-dotted-expressions in joins.

Expression Design

EclipseLink already supports this in our expression framework. No changes are required.

Parser Design

Parser Updates

  join_collection_valued_path_expression::=   identification_variable. {single_valued_embeddable_object_field.}* collection_valued_field
  join_single_valued_path_expression::= variable.{single_valued_embeddable_object_field.}*single_valued_object_field

No changes are required to the parser since the grammar already accepts these types of expressions.

Examples

  • select e from Holder e join e.singleAttribute.attribute a

Validation

This feature simply removes validation that disallowed this in previous JPQL strings.

Testing

  • joining through a 1-1 to a 1-1 (e.manager.address)
  • joining through a 1-1 to a 1-M (e.manager.phoneNumbers)

AS in Select Clause

Items in the select clause can have an identifier associated with them for use in an order by clause

Expression Design

This function does not require any expression work. It simply adds an alias that points to an expression in the JPQL parsing framework.

Parser Design

Parser Updates

  select_clause ::= SELECT [DISTINCT] select_item {, select_item}*
  select_item ::= select_expression [[AS] result_variable]
  orderby_item ::= state_field_path_expression | result_variable [ ASC | DESC ]

The parser currently holds a list of expressions that are held in the select clause. It will be updated to also include a list of identifiers. Those lists will be kept parallel. When an expression is added, an identifier will also be added. If no identifier is provided, null will be added to this list. When the select clause is processed, these two lists will be examined and for each alias in the list, the select expression's list of declared variables will be updated to include a reference from the alias to the appropriate DotNode.

Additionally, when the query is building the expression related to the alias will be cached in the GenerationContext and reused any time it an Order by uses it.

OrderByItemNode will be updated to hold either a DotNode (as it did before) or a String. If it holds a String, the string will be used to look up the appropriate DotNode cached by the SelectClause and the expression cached in the GenerationContext.

Examples

  • select e.firstName as ename from Employee e where e.lastName = :param order by ename

Validation

An exception will be thrown if the OrderBy clause references an alias that has not been defined.

Testing

  • order by an alias.
  • validation test for order by non-existant alias

New Arguments to Constructor Expressions

Constructor expressions have been expanded to accept all scalar expressions (see above)

Expression Design

This is already handled by the Expression framework, but a small modification is required to ConstructorReportItem to allow computation of the type of constructor items that use KEY() and ENTRY() and constants. The initialization code will be updated to get the appropriate type for those types of expressions.

Parser Design

Parser Updates

  constructor_item ::= ...  | scalar_expression | ... | identification_variable

The parser now accepts a scalarExpression as part of a constructorItem.

Examples

  • select new Foo(e.name, KEY(e.map)) from Employee e
  • select new Foo(true, 'tiger' e.lastName) from Employee e

Validation

This feature is validated by the parser and by the already-existing validation of the constructor items.

Testing

  • constructor query that uses a constant
  • consturctor query that uses a KEY

More Flexible Subselects

Subselects have been expanded to allow joined expressions

Expression Design

This functionality is already supported by the expression framework.

Parser Design

Parser Updates

  subquery_from_clause ::= FROM subselect_identification_variable_declaration {, subselect_identification_variable_declaration |  collection_member_declaration}*

  subselect_identification_variable_declaration ::= identification_variable_declaration | derived_path_expression [AS] identification_variable {join}*| derived_collection_member_declaration

  derived_path_expression ::=   superquery_identification_variable.{single_valued_object_field.}*collection_valued_field     |identification_variable.{single_valued_object_field.}*single_valued_object_field

  derived_collection_member_declaration ::=   IN superquery_identification_variable.{single_valued_object_field.}*collection_valued_field

subqueryFromClause has been expanded to allow collectionMemberDeclarations (IN syntax) and subselectIdentificationVariableDeclaration has been expanded to allow a list of joins

Examples

  • select e from Employee e where e.name in (select mEmp.name from Manager m, in(m.emps) mEmp)
  • select e from Employee e where e.name in (select mEmp.name from Manager m join m.emps mEmp)

Validation

This feature removes validation that disabled this functionality.

Testing

  • subselect with IN syntax
  • subselect with JOIN syntax

Additions to Collection Member Expressions

Collection member expressions now allow literals and path expressions that resolve to entities (i.e. paths that end in xToOne mappings)

Expression Design

This is already support by the expression framework.

Parser Design

Parser Updates

  collection_member_expression ::= entity_or_value_expression [NOT] MEMBER [OF] collection_valued_path_expression

  entity_or_value_expression ::= single_valued_assocation_path_expression | state_field_path_expression | simple_entity_or_value_expression

  simple_entity_or_value_expression ::= identification_variable | input_parameter | literal

No parsing changes are required for this change. All the code necessary to make this work is held in the generated parse tree nodes.

Examples

  • Select p from Person p where 'Joe' member of p.nicknames
  • Select p from Person p where p.favouriteDog member of p.dogs

Validation

No validation changes required.

Testing

  • member of with a constant (10 member of e.list)
  • member of with a navigation (e.name member of f.list)

JDBC Escape Syntax for Date, Time and Timestamp

The JDBC escape syntax may be used for the specification of date, time, and timestamp literals. For example:

SELECT o FROM Customer c JOIN c.orders o WHERE c.name = 'Smith' AND o.submissionDate < {d '2008-12-31'}

The portability of this syntax for date, time, and timestamp literals is dependent upon the JDBC driver in use. Persistence providers are not required to translate from this syntax into the native syntax of the database or driver."

The syntax is not explicitly provided in the spec. The following will be used.


literalTemporal = DATE_LITERAL | TIME_LITERAL | TIMESTAMP_LITERAL 
DATE_LITERAL
    : LEFT_CURLY_BRACKET ('d') (' ' | '\t')+ '\'' DATE_STRING '\'' (' ' | '\t')* RIGHT_CURLY_BRACKET
    ;

TIME_LITERAL
    : LEFT_CURLY_BRACKET ('t') (' ' | '\t')+ '\'' TIME_STRING '\'' (' ' | '\t')* RIGHT_CURLY_BRACKET
    ;
    
TIMESTAMP_LITERAL
    : LEFT_CURLY_BRACKET ('ts') (' ' | '\t')+ '\'' DATE_STRING ' ' TIME_STRING '\'' (' ' | '\t')* RIGHT_CURLY_BRACKET
    ;

DATE_STRING
    : '0'..'9' '0'..'9' '0'..'9' '0'..'9' '-' '0'..'9' '0'..'9' '-' '0'..'9' '0'..'9'
    ;
    
TIME_STRING
    : '0'..'9' ('0'..'9')? ':' '0'..'9' '0'..'9' ':' '0'..'9' '0'..'9' '.' '0'..'9'*
    ;

Expression Design

At the moment, String literals are appended to SQL with quotes. As a result, we cannot use a string literal for this element. We will add a DateConstantExpression subclass of ConstantExpression. It will ensure no conversion occurs on the string that has been passed from JPQL and that it is written to the SQL as is.

Parser Design

Parser Updates

The rules for the syntax are not described in the JPA specification. The rules listed above are directly from the implemenation. literalTemporal will be added to the scalarExpression syntax and a TemporalLiteralNode will be added to the parse tree.

Examples

  • SELECT e FROM Employee e where e.period.startDate > {d '1901-01-01'}

Validation

  • The JPA specification defers validation to the JDBC driver and we will do the same

Testing

  • Select with a date in the where clause