EMFIncQuery/UserDocumentation/Query Based Features

From Eclipsepedia

Jump to: navigation, search

EMF-IncQuery supports the definition of efficient, incrementally maintained, well-behaving derived features in EMF by using advanced model queries and incremental evaluation for calculating the value of derived features and providing automated code generation for integrating into existing applications.

Contents

Main scope of query-based features

  • Integrate model query results into EMF applications as structural features
  • Replace low performance derived feature implementations with incrementally evaluated model queries
  • Provide a flexible interlinking method for fragmented models
  • Support declarative definition of high-performance computed features with automatic code generation and validation

Overview

Derived features in EMF models represent information (attribute values, references) computed from the rest of the model, such as the number of elements in a given collection or the set of elements satisfying some additional conditions. Such derived features can ease the handling of models significantly, as they appear in the same way as regular features. However, in order to achieve complete transparency for derived features, the developer must ensure that proper change notifications are sent when model modifications cause changes in the value of the derived feature as well. Finally, since the value of the derived feature might be retrieved often, complete recalculation of the value may impact application performance. Therefore, it is better to keep a cached version of the value and update it incrementally based on changes in the model.

Usually, developers who use derived features in EMF have to manually solve each of these challenges for each derived feature they introduce into their model. Furthermore, although the derived features almost always represent the result of a model query (including type constraints, navigation, aggregation), they are implemented as imperative Java code.

In order to help developers in using derived features, EMF-IncQuery supports the definition of model queries that provide the results for the derived feature value calculation and includes out-of-the-box change notification and incremental maintenance of results. Additionally, the automatic generation of the glue code between the EMF model code and EMF-IncQuery offers easy integration into any existing EMF application.

Well-behaving structural features

The incremental approach of EMF-IncQuery relies on change notifications from every object and every feature in the model that is used in the query definitions. Therefore, a regular volatile feature that has no field, therefore there it does not store the current value of the feature and usually does not send proper change notifications (e.g. SET oldValue to newValue ). Such features are ignored by EMF-IncQuery, unless there is an explicit declaration, that the feature implementation sends proper change notifications at all times. These are called well-behaving structural features.

If your application uses volatile (and often derived) features, you provide proper notifications for them and would like to include them in query definitions, you can explicitly tell EMF-IncQuery that the feature is well-behaving. There is two ways to do this:

  1. extend the org.eclipse.incquery.runtime.base.wellbehaving.derived.features extension point as described here
  2. register your feature directly into the org.eclipse.incquery.runtime.base.comprehension.WellbehavingDerivedFeatureRegistry using the various registerX methods. Warning: you must call this method before executing any queries (i.e. before the first getMatcher() or getEngine() call), since EMF-IncQuery checks the registry when it traverses the model.

Examples

For demonstration, we will use the school metamodel from the introductory example:

School-metamodel.png

Example derived features in this metamodel could be the following:

  • Students enrolled in a given course: this feature would return the list of Students reached with the students reference from the SchoolClass that is connected by the schoolClass reference to the Course.
  • Number of specialization courses: this feature would count the number of SpecializationCourse objects in the courses reference of a given School.

Other examples

You can find examples using the EMF-IncQuery based derived features in the following locations:

Furthermore, we use such derived features in the snapshot models that are used for serializing result sets of EMF-IncQuery matchers.

How to use

EMF-IncQuery only provides the back-end for derived features, the developer must define the feature itself in the metamodel first. Once that is complete, the developer creates the query in a regular EMF-IncQuery project in a query definition file and adds a specific annotation with the correct parameters to invoke the code generation. These steps are detailed in the following:

Definition of the derived feature

  1. In the Ecore model (.ecore file), create the desired EAttribute or EReference in the selected EClass and set the name, type and multiplicity information correctly.
  2. Use the following configuration for the other attributes of the created EStructuralFeature:
    • derived = true (to indicate that the value of the feature is computed from the model)
    • changeable = false (to remove setter methods)
    • transient = true (to avoid persisting the value into file)
    • volatile = true (to remove the field declaration in the object)
  3. In the Generator model (.genmodel), right-click on the top-level element and select Reload, click Next, Load, and Finish to update the Generator model with the changes done in the Ecore model.
  4. Right-click on the top-level element and select Generate Model Code to ensure that the getters are properly generated into the EMF model code. You can regenerate the Edit and Editor code as well, though those are not necessary here.

Definition of the model query

  1. Create an EMF-IncQuery project and query definition (.eiq) file as described in the cheat sheet or this tutorial.
  2. Make sure that you imported your metamodel into the query definition. Create the EMF-IncQuery generator model, if necessary (.eiqgen file).
  3. Make sure that the project containing the generated code is in the same workspace as the EMF-IncQuery project.
  4. Create the query corresponding to your derived feature. For example, the students enrolled in a given course feature would look like this:
package school.queries;
 
import "http:///school.ecore"
 
@QueryBasedFeature
pattern studentsOfCourse(This : Course, Target : Student) = {
   Course.schoolClass.students(This,Target);
}

Note that the first parameter of the pattern is the source of the derived feature and the second is the target. Although not mandatory, is is good practice to use the (This : EClass, Target) format to ease understanding. The @QueryBasedFeature annotation indicates to the code generator that the glue code has to be generated in the model code.

Save the query definition, which initiates the code generation. After it completes, you can open the implementation code to ensure that the new getter method is correctly created. Note that a well-behaving derived feature extension is also generated into the plugin.xml of the EMF-IncQuery project to indicate that the given derived feature correctly sends change notifications if the correct project is loaded.

Running the application

Once the glue code is generated, you can use the derived features by including the EMF-IncQuery project into your runtime together with the model project.

Annotation parameters

The @QueryBasedFeatureannotation uses defaults for each possible parameters, which allows developers to avoid using any parameters if the query is correctly written.

In short, parameters are not needed, if the following conditions are satisfied:

  • The name of the pattern is the same as the name of the derived feature (comparison uses String.equals())
  • The first parameter is the defining EClass and its type is correctly given (e.g. This : Course)
  • The second parameter is the target of the derived feature
  • The derived feature value is a single EObject or a collection of EObjects

If the derived feature and its query does not satisfy the above conditions, the following parameters can be used in the annotation:

  • feature ="featureName" (default: pattern name) - indicates which derived feature is defined by the pattern
  • source ="Src" (default: first parameter) - indicates which query parameter (using its name) is the source EObject, the inferred type of this parameter indicates which EClass generated code has to be modified
  • target ="Trg" (default: second parameter) - indicates which query parameter (using its name) is the target of the derived feature
  • kind ="single/many/counter/sum/iteration" (default: feature.isMany?many:single) - indicates what kind of calculation should be done on the query results to map them to derived feature values
  • keepCache ="true/false" (default: true) - indicates whether a separate cache should be kept with the current value. Single and Many kind derived features can work without keeping an additional cache, as the EMF-IncQuery RETE network already keeps a cache of the current values.

Developer documentation

The JavaDoc can be found here.

Overview of the implementation

To support query-backed features captured as derived features, the outputs of the EMF-IncQuery engine need to be integrated into the EMF model access layer at two points: (1) query results are provided in the getter functions of derived features, and (2) query result deltas are processed to generate EMF Notification objects that are passed through the standard EMF API so that application code can process them transparently.

Querybased-features-approach.png

The application accesses both the model and the query results through the standard EMF model access layer -- hence, no modification of application source code is necessary. In the background, our novel derived feature handlers are attached to the EMF model plugin that integrate the generated query components (pattern matchers). When an EMF application intends to read a soft link (B1), the current value is provided by the corresponding handler (B2) by simply retrieving the value from the cache of the related query. When the application modifies the EMF model (A1), this change is propagated to the generated query components of EMF-IncQuery along notifications (A2), which may update the delta monitors of the handlers (A3). Changes of derived features may in turn trigger further changes in the results sets of other derived features (A4).

Instantiating query-based feature handlers

The easiest way is to create a simple query-based feature and look at the generated code in the getter function.

If you need to create a handler for some reason, use the static getQueryBasedFeatureHandler() methods of the QueryBasedFeatureHelper class.

Example codes that were generated for the school example:

/**
 * EMF-IncQuery handler for query-based feature numberOfTeachers
 */
private QueryBasedFeatureHandler numberOfTeachersHandler;
 
/**
 * <!-- begin-user-doc --> <!-- end-user-doc -->
 * @derived getter created by EMF-IncQuery for query-based feature numberOfTeachers
 */
public int getNumberOfTeachers() {
 if (numberOfTeachersHandler == null) {
  numberOfTeachersHandler = QueryBasedFeatureHelper.getQueryBasedFeatureHandler(
    this, SchoolPackageImpl.Literals.SCHOOL__NUMBER_OF_TEACHERS,
    "teachers", "School", null, FeatureKind.COUNTER,
    true, false);
 }
 return numberOfTeachersHandler.getIntValue(this);
}
 
/**
 * EMF-IncQuery handler for query-based feature teachersWithMostCourses
 */
private QueryBasedFeatureHandler teachersWithMostCoursesHandler;
 
/**
 * <!-- begin-user-doc --> <!-- end-user-doc -->
 * @derived getter created by EMF-IncQuery for query-based feature teachersWithMostCourses
 */
public EList<Teacher> getTeachersWithMostCourses() {
 if (teachersWithMostCoursesHandler == null) {
  teachersWithMostCoursesHandler = QueryBasedFeatureHelper.getQueryBasedFeatureHandler(
    this, SchoolPackageImpl.Literals.SCHOOL__TEACHERS_WITH_MOST_COURSES,
    "teachersWithMostCourses", "School", "Teacher", FeatureKind.MANY_REFERENCE,
    true, false);
 }
 return teachersWithMostCoursesHandler.getManyReferenceValueAsEList(this);
}
 
/**
 * EMF-IncQuery handler for query-based feature lastYear
 */
private QueryBasedFeatureHandler lastYearHandler;
 
/**
 * <!-- begin-user-doc --> <!-- end-user-doc -->
 * @derived getter created by EMF-IncQuery for query-based feature lastYear
 */
public Year basicGetLastYear() {
 if (lastYearHandler == null) {
  lastYearHandler = QueryBasedFeatureHelper.getQueryBasedFeatureHandler(
   this, SchoolPackageImpl.Literals.SCHOOL__LAST_YEAR,
   "lastYear", "School", "Year", FeatureKind.SINGLE_REFERENCE,
   true, false);
 }
 return (school.Year) lastYearHandler.getSingleReferenceValue(this);
}

Accessing the current value of query-based features

The most straightforward way is to call the getter method of the feature itself. However, if for some reason that is not possible, you can access the values using the getter methods of the QueryBasedFeatureHandler object. Apart from the generic getValue, there are specific methods (getIntValue, getSingleReferenceValue etc.), each returning a properly typed target for a given source element.

Development plans

Tracing

In order to support debugging query-based features, we should add a "querybasedfeature" option for the (not-yet-existing) global tracing of EMF-IncQuery.

The tracing should cover all core functionality:

  • initializing query-based features
  • computing the initial value of the feature for a given source
  • returning results to getter calls
  • handling incremental updates (both match set changes and life-cycle notifications)

Using setting delegates

Instead of overwriting the generated model code, it is possible to use setting delegates for wrapping query-based features. Setting delegates are the currently recommended way of integrating derived feature computation into EMF models.

In EMF-IncQuery 0.8, the current generator will be replaced with creating appropriate annotations for using setting delegates. This means that only the Ecore file is modified, but also that the code generation from the genmodel will have to be invoked as well.

To set up setting delegates, you have to put annotations on the EPackage and EStructuralFeatures:

  • on the EPackage, you declare which setting delegates to use:
<eAnnotations source="http://www.eclipse.org/emf/2002/Ecore">
  <details key="settingDelegates" value="org.eclipse.incquery.querybasedfeature"/>
</eAnnotations>
  • on the EStructuralFeature which is a query-based feature:
<eAnnotations source="org.eclipse.incquery.querybasedfeature">
  <details key="patternFQN" value="querypackage.patternName"/>
</eAnnotations>

The setting delegate factory is registered by the query-based feature runtime plug-in and EMF will use the factory to create the setting delegate for query-based derived features.

Remaining issues
  • The setting delegate factory should use the pattern registry to find the required Query Specification to work with. This way, queries defined dynamically can be registered by their creators independently of the query-based feature runtime.
  • The "scope" of a feature is important, whether it's the parent EObject, its resorce or the whole resource set. Since setting delegates have to find the proper handler in case of getters, the scope must be taken into consideration.
    • The scope should be part of the @QueryBasedFeature annotation, with the default value "resource set".
    • We need to handle incorrect scopes (e.g. the parent is not in a resource or resource set): we can either (1) not initialize the handler at all, (2) initialize it with a different scope and recognise when the scope should be corrected, (3) delay initialization, listen to model changes and initialize the handler when the scope becomes correct.
    • The scope operations should be logged when tracing is enabled.
  • The query-based feature needs an IncQueryEngine to access the match set of the query that drives the feature. Since the engine must be created with dynamic mode enabled to use it in certain scenarios, the delegate factory needs a way to accept specific engines to use for given scopes.
    • One possibility is to add a putEngineIntoScopeMap method that can be called by clients.

Extending query-based features for subclasses

It is possible that a subclass (in a different Ecore model most likely) would want to override the behaviour of a query-based feature. It would be nice to have a way to extend/override the query that is used for computing the query.

  1. Using query libraries, we could have a query that composes the original pattern with the additional constraints (either with an additional pattern body or just as part of the new body).
  2. We need to ensure that multiple extensions are possible and they don't cancel each other out. In the case of setting delegates, this is possible, as long as the setting delegate knows about all possible extensions (seems entirely possible).
  3. The @QueryBasedFeature annotation must include an optional parameter that specifies the extension. This is required to identify cases when a feature would be accidentally overridden.

Advanced issues

Creating an iteration query-based feature

It is possible to create a query-based feature that is not simply the result of the model query, but the value calculated by an iteration algorithm on the results of the query.

Important: the iteration algorithm must be able to compute the new value based on it's current value and the new or lost match of the used query.

In order to create your own iteration feature, you need to subclass QueryBasedFeature and implement the following methods:

  • newMatchIteration(IPatternMatch): based on the match (that just appeared) passed to the method, return a notification that represents the changes in the value of the feature. Note that you should not send out this notification, that is the responsibility of the feature handler.
  • lostMatchIteration(IPatternMatch): based on the match (that just disappeared) passed to the method, return a notification that represents the changes in the value of the feature. Note that you should not send out this notification, that is the responsibility of the feature handler.
  • getValueIteration(Object): based on the source element passed to the method, return the value for the feature.

Pitfalls

Code generation fails for derived feature query

Ensure that both the .genmodel file and the model project with the generated EMF model code is available in the same workspace as the EMF-IncQuery project with the query definitions.

Multiple results for a query used in a single (upper bound = 1) feature

If you define a query for a single feature that returns multiple results for a given source model element, the value of the derived feature will in most cases be the value from the last match that appeared. However, it is possible to change the values in a way that the feature will have no value, even though it might have exactly one. Therefore, it is important to define the queries for the feature in a way that only one result is possible. You can either make assumptions on your models and use other ways to ensure that there is only one match, or you can explicitly declare in the pattern, that it should only match once for a given source element. Additionally, you can use the Validation framework of EMF-IncQuery to create feedback for the user when the query would have multiple results indicating that the model is invalid.

The following is an example for a validated, ensured single feature:

@QueryBasedFeature
pattern singleFeature(This : SourceType, Target : TargetType){
 find internalQuery(This, Target);
 1 == count find internalQuery(This, Target);
}
 
private pattern internalQuery(This : SourceType, Target : TargetType){
 // actual query definition
}
 
@Constraint(location = "This", severity = "error",
   message="Multiple values for $This.name$.singleFeature!")
pattern singleFeatureInvalid(This : SourceType){
 1 < count find internalQuery(This, _Target);
}

UnsupportedOperationException during model editing, even after successful generation

If you have multiple inheritance in your metamodel, it is possible that the getter for a feature will be implemented in more than one place. The easy way to avoid this is to ensure, that query-based features are only inherited from one supertype and that supertype is used as the extension and not only as interface (i.e. that type must be the first in the values of the supertypes feature).

In the unfortunate case when you have query-based features in multiple supertypes, the generator will only override the getter in the implementation class of the defining EClass, so you will have to copy-paste the generated getter code and the handler into the subclass implementation as well.

Future versions of EMF-IncQuery may support the automatic generation into multiple implementation classes.

Alternatively, you can try the setting delegate based implementation, which avoids this problem.