Difference between revisions of "VIATRA/Query/UserDocumentation/API"

Revision as of 10:24, 12 September 2014

Overview

This page presents the basics of EMF-IncQuery's Java API. It supersedes the following contents of the original documentation on IncQuery.net:

• http://incquery.net/incquery/new/examples/application (Headless example)
• http://incquery.net/incquery/documentation/api (Pattern Matcher API documentation)
• http://incquery.net/incquery/new/examples/school#Generated_code_overview (The latter part of the School example)

The contents only over the basic use-cases. For advanced features, see Advanced API features.

Running example

All the code examples and explanations will be given in the context of the Headless example. The up-to-date sample source code to this page is found in Git here: http://git.eclipse.org/c/incquery/org.eclipse.incquery.examples.git/tree/headless Most notably,

• the patterns are found in headlessQueries.eiq
• and the API usage samples are found in IncQueryHeadless.java

Javadoc

The most up-to-date Javadocs for the EMF-IncQuery API can be found online at http://eclipse.org/incquery/javadoc/

EMF-IncQuery Java API

The most typical way of using the EMF-IncQuery API is to make use of the generated code that is found in the "src-gen" folder of your EMF-IncQuery project. This generated code provides easy and typesafe access to most of EMF-IncQuery's features from Java code. The only important thing to keep in mind is that the generated code is available only when your EMF-IncQuery project is loaded into the Eclipse runtime, i.e. for working with the generated code for development purposes you'll need to use Eclipse Application launch configurations.

EMF-IncQuery also supports a "dynamic", generic API that allows to make use of patterns without relying on the generated code. The generic API shares functionality with the base classes of the "generated" API, and for most scenarios there is no performance difference between the two. A notable exception for this rule of thumb are check() expressions, where the generated code that is invoked through the generated API will execute the Java code instead of interpreting Xbase.

In this wiki, we will present a simple introduction to the basics of the "generated" API, to introduce features that will help you to integrate EMF-IncQuery queries into your Java application. The Advanced API features page discusses the usage of the dynamic and generic APIs.

Most important classes and their relationships

For every pattern definition, the EMF-IncQuery tooling generates a Match, a Matcher, into a Java package that corresponds to the package of the pattern definition, constituting the public API that is intended for client code usage. In addition, some helper classes are generated into the ".uitl" subpackage:

• if check expressions are present in your pattern definition, several Evaluator classes are also generated.
• a MatchProcessor class is generated to aid match set traversal.
• Finally, a QuerySpecification helper class is also generated that supports advanced lifecycle management (which is not necessary in most cases).

In order to fully understand the capabilities of these classes, it is a good idea to read through the Generic API section of the Advanced API features page.

Match

A Match object represents a single match of the pattern, i.e. a tuple of objects whose members point to corresponding elements of the instance model (or scalar values) that the pattern is matched against. It is essentially a Data Transfer Object that is used to extract query result information from EMF-IncQuery, with an SQL analogy you could think of it as one "row" of the result set of a query. The generated fields correspond to the pattern header parameters.

You can also use Match objects to specify fixed input parameters to a query (while other fields can be left unspecified) - analogously to a "prepared" SQL statement that accepts input parameter bindings. In this case, the input Match will act as a filter (mask) and the results of you queries will also be instances of this class (where parameters already have the values given in the input). See TODO below for further details.

The code example below shows the EClassNamesMatch class generated for the eClassNames pattern of the running example. The generated class implements the interface IPatternMatch and extends the BasePatternMatch internal implementation class.

public abstract class EClassNamesMatch implements IPatternMatch {
  /**
   * members and constructor
   */
  private EClass fC;
  private String fN;
  private static String[] parameterNames = {"C", "N"};
  private EClassNamesMatch(EClass pC, String pN) {}
  /** getters and setters **/
  public Object get(String parameterName) { }
  public EClass getC() {}
  public String getN() {}
  public boolean set(String parameterName, Object newValue) {}
  public void setC(EClass pC) {}
  public void setN(String pN) {}
  /** utility functions **/
  public String patternName() {}
  public String[] parameterNames() {}
  public Object[] toArray() {}
  public String prettyPrint() {}
 
  public int hashCode() {}
  public boolean equals(Object obj) {}
  /** access to the internal EMF-based representation, as a sort of "reflection" **/
  public Pattern pattern() {}
  /** Mutable inner subclass so that matches can be "setted" - for parameter passing **/
  static final class Mutable extends EClassNamesMatch {}
}

Matcher

The Matcher is the main entry point of the EMF-IncQuery API, with pattern-specific query methods. It provides access to the three key features of EMF-IncQuery:

• First of all it provides means to initialize a pattern matcher for a given EMF instance model which can either be a Resource, a ResourceSet, or an EObject (in this latter case, the scope of the matching will be the containment tree under the passed EObject). We recommend the use of ResourceSets if possible to avoid cross-reference related issues.
• After the initialization of the engine, the Matcher provides getter methods to retrieve the contents of the match set. For easy iteration over the match set it provides a convenience method (forEachMatch) as well, as this is the most frequent use case in our observation. Of course it contains other handy features (e.g.: countMatches, hasMatch) to help integration.
• Finally, it provides means to efficiently track the changes in the match set in an event-driven fashion.

public class EClassNamesMatcher implements IncQueryMatcher<EClassNamesMatch> {
  /** initializer methods **/
  public EClassNamesMatcher(IncQueryEngine engine) throws IncQueryException {}
 
  /** access to match set **/
  public Collection<EClassNamesMatch> getAllMatches(); // inherited
  public Collection<EClassNamesMatch> getAllMatches(EClass pC, final String pN) {}
  public EClassNamesMatch getOneArbitraryMatch(); // inherited
  public EClassNamesMatch getOneArbitraryMatch(EClass pC, String pN) {}
  public boolean hasMatch(); // inherited
  public boolean hasMatch(EClass pC, String pN) {}
  public int countMatches(); // inherited
  public int countMatches(EClass pC, String pN) {}
 
  /** Retrieve the set of values that occur in matches for C or N.**/
  public Set<EClass> getAllValuesOfC() {}
  public Set<EClass> getAllValuesOfC(EClassNamesMatch partialMatch) {}
  public Set<EClass> getAllValuesOfC(String pN) {}
  public Set<String> getAllValuesOfN() {}
  public Set<String> getAllValuesOfN(EClassNamesMatch partialMatch) {}
  public Set<String> getAllValuesOfN(EClass pC) {}
 
  /** iterate over matches using a lambda **/
  public void forEachMatch(IMatchProcessor<? super EClassNamesMatch> processor); // inherited
  public void forEachMatch(EClass pC, String pN, IMatchProcessor<? super EClassNamesMatch> processor) {}
  public void forOneArbitraryMatch(IMatchProcessor<? super EClassNamesMatch> processor); // inherited
  public boolean forOneArbitraryMatch(EClass pC, String pN, IMatchProcessor<? super EClassNamesMatch> processor) {}
 
  /** Returns a new (partial) Match object for the matcher. 
   *  This can be used e.g. to call the matcher with a partial match. **/
  public EClassNamesMatch newMatch(EClass pC, String pN) {}
}

Helper classes

• MatchProcessor

The Matcher provides a function to iterate over the match set and invoke the process() method of the IMatchProcessor interface with every match. You can think of this as a "lambda" to ease typical query result processing tasks. To this end, an abstract processor class is generated, which you can override to implement the logic you would like to use. The abstract class unpacks the match variables so it can be used directly in the process() method.

/**
 * A match processor tailored for the headless.eClassNames pattern.
 */
public abstract class EClassNamesProcessor implements IMatchProcessor<EClassNamesMatch> {
  /**
   * Defines the action that is to be executed on each match.
   */
  public abstract void process(final EClass C, final String N);
}

• Evaluator: If your pattern contains check expressions an evaluator java code is generated from it. It is used by the engine during a query to evaluate the expression’s result. In most cases you don’t need to deal with these classes.
• QuerySpecification: A pattern-specific specification that can instantiate a Matcher class in a type-safe way. You can get an instance of it via the Matcher class’s specification() method. The recommended way to instantiate a Matcher is with an IncQueryEngine. In both cases if the pattern is already registered (with the same root in the case of the Notifier method) then only a lightweight reference is created which points to the existing engine.

/**
 * A pattern-specific query specification that can instantiate EClassNamesMatcher in a type-safe way.
 */
public final class EClassNamesQuerySpecification extends BaseGeneratedQuerySpecification<EClassNamesMatcher> {
  /** @return the singleton instance of the query specification **/
  public static EClassNamesQuerySpecification instance() throws IncQueryException {}
}

Lifecycle management

In EMF-IncQuery, all pattern matching (query evaluation) is carried out in IncQueryEngine instances that are accessed through the user-friendly generated classes of the public API. The IncQueryEngine associated to your patterns can be accessed and managed through the EngineManager singleton class, to track and manipulate their lifecycles. By default, for each instance model root (Resource, ResourceSet or EObject tree) a single, managed IncQueryEngine is created, which is shared by all objects that access EMF-IncQuery's features through the generated API. The IncQueryEngine is attached to an EMF model root (Notifier of concrete type Resource, ResourceSet or EObject) and it is retained on the heap as long as the model itself is there. It will listen on EMF update notifications stemming from the given model in order to maintain live results. If you release all references to the model (e.g. unload the resource), the IncQueryEngine can also be garbage collected (as long as there are no other inbound references on it).

In all, for most (basic) scenarios, the following workflow should be followed:

• initialize/load the model
• initialize your IncQueryEngine instance
• initialize pattern matchers, or groups of pattern matchers and use them
• if you release the model and your IncQueryEngine instance, all resources will be freed by the garbage collector.

For advanced scenarios (if you wish to manage lifecycles at a more finegrained level), you have the option of creating unmanaged IncQueryEngines and dispose of them independently of your instance model. For most use-cases though, we recommend the use of managed engines, this is the default and optimized behavior, as these engines can share common indices and caches to save memory and CPU time. The EngineManager ensures that there will be no duplicated engine for the same model root (Notifier) object. Creating an unmanaged engine will give you certain additional benefits, however additional considerations should be applied. See Advanced lifecycle management for details.

Typical programming patterns

In the followings, we provide short source code samples (with some explanations) that cover the most important use-cases supported by the EMF-IncQuery API.

Loading an instance model and executing a query

// get all matches of the pattern
// initialization
// phase 1: (managed) IncQueryEngine
IncQueryEngine engine = IncQueryEngine.on(resource /* or resourceSet */); 
// phase 2: the matcher itself
EObjectMatcher matcher = EObjectMatcher.on(engine);
// get all matches of the pattern
Collection<EObjectMatch> matches = matcher.getAllMatches();
// process matches, produce some output
StringBuilder results = new StringBuilder();
prettyPrintMatches(results, matches);

Using the MatchProcessor

matcher2.forEachMatch(new EClassNamesProcessor() {
 @Override
 public void process(EClass c, String n) {
  results.append("\tEClass: " + c.toString() + "\n");
 }
});

Matching with partially bound input parameters and using result set projections

matcher2.forEachMatch( matcher2.newMatch(null, "A") , new EClassNamesProcessor() {
 @Override
 public void process(EClass c, String n) {
  results.append("\tEClass with name A: " + c.toString() + "\n");
 }
});	
 
// alternatively:
matcher2.forEachMatch(null, "A" , new EClassNamesProcessor() {
 @Override
 public void process(EClass c, String n) {
  results.append("\tEClass with name A: " + c.toString() + "\n");
 }
});

// projections
for (EClass ec: matcher2.getAllValuesOfc(matcher2.newMatch(null,"A")))
{
 results.append("\tEClass with name A: " + ec.toString() + "\n");
}

Initialization of pattern groups

Using pattern groups is important for performance. By default, EMF-IncQuery performs a traversal of the instance model when a matcher is accessed through the IncQueryEngine for the first time. If you wish to use several pattern matchers, it is a good idea to make use of the generated pattern group class and prepare the IncQueryEngine to perform a combined traversal (with minimal additional overhead) so that any additional Matcher initializations avoid re-traversals.

// phase 1: (managed) IncQueryEngine
IncQueryEngine engine = IncQueryEngine.on(resource);
// phase 2: the group of pattern matchers
HeadlessQueries patternGroup = HeadlessQueries.instance();
patternGroup.prepare(engine);
// from here on everything is the same
EObjectMatcher matcher = EObjectMatcher.on(engine);
// get all matches of the pattern
Collection<EObjectMatch> matches = matcher.getAllMatches();
prettyPrintMatches(results, matches);
// ... //
// matching with partially bound input parameters
// because EClassNamesMatcher is included in the patterngroup, *no new traversal* will be done here
EClassNamesMatcher matcher2 = EClassNamesMatcher.on(engine);

Tracking changes in match sets efficiently

There are some usecases where you don’t want to follow every change of a pattern’s match, just gather them together and process them when you’re ready. EMF-IncQuery provides several means of doing this (see the Advanced query result set change processing page for details), but we recommend using JFace databinding for basic purposes. To this end, the IncQueryObservables utility class can transform the result set of your matcher into an observable list or set that can be tracked and even data bound easily.

For headless (or non-UI thread) execution, please use the simple DefaultRealm implementation provided in the example (and invoke it on the appropriate thread).

// (+) changes can also be tracked using JFace Databinding
// this approach provides good performance, as the observable callbacks are guaranteed to be called
//   in a consistent state, and only when there is a relevant change; anything
//   can be written into the callback method
// (-) * the databinding API introduces additional dependencies
//     * is does not support generics, hence typesafe programming is not possible
//     * a "Realm" needs to be set up for headless execution
DefaultRealm realm = new DefaultRealm(); // this is necessary for headless execution (or when you
// wish to execute outside of the UI thread. make sure to invoke it on the appropriate thread!
IObservableSet set = IncQueryObservables.observeMatchesAsSet(matcher);
set.addSetChangeListener(new ISetChangeListener() {
 @Override
 public void handleSetChange(SetChangeEvent event) {
  for (Object _o : event.diff.getAdditions()) {
   if (_o instanceof EPackageMatch) {
    results.append("\tNew EPackage found by changeset databinding: " + ((EPackageMatch)_o).getP().getName()+"\n");
   }
  }
});

With this facility, it is easy to build an Eclipse UI on top of abstract views of your model. Additionally, two further features you may find useful are:

• IncQuery Databinding provides means to define observable values on pattern matches that can be used for displaying individual results in JFace viewers
• A complementary feature is provided by IncQuery Viewers, which provides additional, higher level and easy-to-use generative features to further ease this task (and also add very powerful visualization capabilities to developing queries).

Advanced capabilities

In addition to the basic features discussed above, EMF-IncQuery provides several additional advanced features of interest:

• the Generic API to dynamically instantiate and execute patterns from code, without code generation
• advanced query result change processing and lifecycle management features for performance-critical and/or resource constrained applications
• IncQuery Base, a light-weight standalone base indexer library that provides incremental evaluation for core queries such as "get all instances", "get inverse reference targets", as well as incremental transitive closure.

These topics are discussed in detail on the Advanced API topics page.