Jump to: navigation, search

EMFIncQuery/UserDocumentation/API/Advanced

Overview

This page overviews advanced use-cases for EMF-IncQuery. The topics cover

  • the Generic API, to be used when the features supported by the generated API do not suffice (e.g. when working with dynamically defined patterns, or patterns whose handles are not known at compile time)
  • Advanced change processing APIs, and advanced lifecycle management techniques to be used for performance-critical and/or resource-constrained applications
  • IncQuery Base, the low-level query and indexer layer underneath EMF-IncQuery pattern matchers

Running example

All the code examples and explanations will be given in the context of the Headless example. The up-to-date sample source code to this page is found in Git here: http://git.eclipse.org/c/incquery/org.eclipse.incquery.examples.git/tree/headless Most notably,

Javadoc

The most up-to-date Javadocs for the EMF-IncQuery API can be found online at http://eclipse.org/incquery/javadoc/

The IncQuery Generic API

The "generic" API differs from the generated one in two key aspects:

  • it can be used to apply queries and use other IncQuery features without generating code and loading the resulting bundles into the running configuration. In other words, you just need to supply the EMF-based in-memory representation (an instance of the Pattern class)
  • the generic API is not "type safe" in the sense that the Java types of your pattern variables is not known and needs to be handled dynamically (e.g. by instanceof - typecase combos).

Initializing matchers and accessing results

Sample code

Using the Generic API:

public String executeDemo_GenericAPI(String modelPath, String patternFQN) {
 final StringBuilder results = new StringBuilder();
 Resource resource = loadModel(modelPath);
 if (resource != null) {
  try {
   // get all matches of the pattern
   // create an *unmanaged* engine to ensure that noone else is going
   // to use our engine
   AdvancedIncQueryEngine engine = AdvancedIncQueryEngine.createUnmanagedEngine(resource);
   // instantiate a pattern matcher through the registry, by only knowing its FQN
   // assuming that there is a pattern definition registered matching 'patternFQN'
   Pattern p = null;
   IQuerySpecification<?> querySpecification = null;
 
   // use a trick to load Pattern models from a file
   ResourceSet resourceSet = new ResourceSetImpl();
   // here, we make use of the (undocumented) fact that the Pattern model 
   // is stored inside the hidden "queries" directory inside an EMF-IncQuery project
   URI fileURI = URI.createPlatformPluginURI("headlessQueries.incquery/queries/globalEiqModel.xmi", false);
   Resource patternResource = resourceSet.getResource(fileURI, true);
   // navigate to the pattern definition that we want
   if (patternResource != null) {
     if (patternResource.getErrors().size() == 0 && patternResource.getContents().size() >= 1) {
       EObject topElement = patternResource.getContents().get(0);
          if (topElement instanceof PatternModel) {
        	for (Pattern _p  : ((PatternModel) topElement).getPatterns()) {
             		if (_p.getName().equals(patternFQN)) {
              			p = _p; break;
               		}
               	}
          }
     }
  }
 
   if (p!=null) {
    querySpecification = QuerySpecificationRegistry.getQuerySpecification(p);
   }
   else {
    // fall back to the registry in case the pattern model extraction didn't work
    querySpecification = QuerySpecificationRegistry.getQuerySpecification(patternFQN);
   }
 
   if (querySpecification!=null) {
    IncQueryMatcher<? extends IPatternMatch> matcher = querySpecification.getMatcher(engine);
    Collection<? extends IPatternMatch> matches = matcher.getAllMatches();
    prettyPrintMatches(results, matches);
   }
   // wipe the engine
   engine.wipe();
   // after a wipe, new patterns can be rebuilt with much less overhead than 
   // complete traversal (as the base indexes will be kept)
   // completely dispose of the engine once's it is not needed
   engine.dispose();
   resource.unload();
  } catch (IncQueryException e) {
   e.printStackTrace();
   results.append(e.getMessage());
  } 
 } else {
  results.append("Resource not found");
 }
 return results.toString();
}

Loading EIQ resources programmatically

/**
 * Returns the match set for patternFQN over the model in modelPath in pretty printed form
 * 
 * @param modelPath
 * @param patternFQN
 * @return
 */
public String executeDemo_GenericAPI_LoadFromEIQ(String modelPath, String patternFQN) {
 final StringBuilder results = new StringBuilder(); 
 Resource resource = loadModel(modelPath);
 if (resource != null) {
  try {
    // get all matches of the pattern
    // create an *unmanaged* engine to ensure that noone else is going
    // to use our engine
    AdvancedIncQueryEngine engine = AdvancedIncQueryEngine.createUnmanagedEngine(resource);
    // instantiate a pattern matcher through the registry, by only knowing its FQN
    // assuming that there is a pattern definition registered matching 'patternFQN'
    Pattern p = null;
    // Xtext resource magic -- this is needed for EIQ resources;
    new EMFPatternLanguageStandaloneSetup()
    {
     @Override
     public Injector createInjector() { return Guice.createInjector(new GeneratorModule()); }
    }
   .createInjectorAndDoEMFRegistration();
    // use a trick to load Pattern models from a file
    ResourceSet resourceSet = new ResourceSetImpl();
    URI fileURI = URI.createPlatformResourceURI("test/src/test/test.eiq", false);
    Resource patternResource = resourceSet.getResource(fileURI, true);
    // navigate to the pattern definition that we want
    if (patternResource != null) {
     if (patternResource.getErrors().size() == 0 && patternResource.getContents().size() >= 1) {
      EObject topElement = patternResource.getContents().get(0);
       if (topElement instanceof PatternModel) {
        for (Pattern _p  : ((PatternModel) topElement).getPatterns()) {
         if (_p.getName().equals(patternFQN)) {
          p = _p; break;
         }
        }
       }
      }
     }
     IncQueryMatcher<? extends IPatternMatch> matcher = engine.getMatcher(p);
     if (matcher!=null) {
      Collection<? extends IPatternMatch> matches = matcher.getAllMatches();
       prettyPrintMatches(results, matches);
      }
     // wipe the engine
     engine.wipe();
     // after a wipe, new patterns can be rebuilt with much less overhead than 
     // complete traversal (as the base indexes will be kept)
     // completely dispose of the engine once's it is not needed
     engine.dispose();
     resource.unload();
    } catch (IncQueryException e) {
      e.printStackTrace();
     results.append(e.getMessage());
    }
   } else {
    results.append("Resource not found");
  }
 return results.toString();
}

API interfaces

  • IPatternMatch Javadoc
    • reflection: pattern(), patternName()
    • getters and setters
    • utility functions (toArray, prettyPrint)
  • IncQueryMatcher Javadoc
    • reflection
    • get all matches
    • get single/arbitrary match
    • check for a match
    • number of matches
    • process matches
    • access change processing features
    • create a new Match for input binding
    • access projected value sets

The Pattern Registry and Matcher Factories

TODO The IMatcherFactory interface Javadoc

  • reflection
  • initialize the matcher


Advanced query result set change processing

Match update callbacks

private void changeProcessing_lowlevel(final StringBuilder results, IncQueryMatcher<? extends IPatternMatch> matcher) {
 // (+) these update callbacks are called whenever there is an actual change in the
 // result set of the pattern you are interested in. Hence, they are called fewer times
 // than the "afterUpdates" option, giving better performance.
 // (-)  the downside is that the callbacks are *not* guaranteed to be called in a consistent
 // state (i.e. when the update propagation is settled), hence
 //  * you must not invoke pattern matching and model manipulation _inside_ the callback method
 //  * the callbacks might encounter "hazards", i.e. when an appearance is followed immediately by a disappearance.
 engine.addMatchUpdateListener(matcher, new IMatchUpdateListener<IPatternMatch>() {
  @Override
  public void notifyDisappearance(IPatternMatch match) {
  // left empty
  }
  @Override
  public void notifyAppearance(IPatternMatch match) {
   results.append("\tNew match found by changeset low level callback: " + match.prettyPrint()+"\n");
  }
 }, false);
}

Using the EVM

The Event-driven VM can also be used for this purpose. TODO how

Advanced IncQuery Lifecycle management

Managed vs. unmanaged IncQueryEngines

Disposing

If you want to remove the matchers from the engine you can call the wipe() method on it. It discards any pattern matcher caches and forgets the known patterns. The base index built directly on the underlying EMF model, however, is kept in memory to allow reuse when new pattern matchers are built. If you don’t want to use it anymore call the dispose() instead, to completely disconnect and dismantle the engine.

  • wipe
  • dispose

IncQuery Base

EMF-IncQuery provides a light-weight indexer library called Base that aims to provide several useful (some would even argue critical) features for querying EMF models:

  • inverse navigation along EReferences
  • finding and incrementally tracking all model elements by attribute value/type (i.e. inverse navigation along EAttributes)
  • incrementally computing transitive reachability along given reference types (i.e. transitive closure of an EMF model)
  • getting and tracking all the (direct) instances of a given EClass

The point of IncQuery Base is to provide all of these in an incremental way, which means that once the query evaluator is attached to an EMF model, as long as it stays attached, the query results can be retrieved instantly (as the query result cache is automatically updated). IncQuery Base is a lightweight, small Java library that can be integrated easily to any EMF-based tool as it can be used in a stand-alone way, without the rest of EMF-IncQuery.

We are aware that some of the functionality can be found in some Ecore utility classes (for example ECrossReferenceAdapter). These standard implementations are non-incremental, and are thus do not scale well in scenarios where high query evaluation performance is necessary (such as e.g. on-the-fly well-formedness validation or live view maintenance). IncQuery Base has an additional important feature that is not present elsewhere: it contains very efficient implementations of transitive closure that can be used e.g. to maintain reachability regions incrementally, in very large EMF instance models.

The detailed documentation is covered in EMFIncQuery/UserDocumentation/API/BaseIndexer

Extracting reachability paths from transitive closure

TODO