EMFIncQuery/DeveloperDocumentation/IncQueryXcore

From Eclipsepedia

Jump to: navigation, search

Contents

Aim of the project

The Xcore [4] project aims to provide a textual syntax for the definition of EMF metamodels. You can use it not only to specify the structure of your model, but also the behavior of your operations and derived features as well as the conversion logic of your data types. It eliminates the dividing line between modeling and programming, combining the advantages of each.

Viewing the scope of the EMF-IncQuery [1][3] project an idea may immediately arise; it would be really nice to use the powerful pattern language of EMF-IncQuery to define derived features in your Xcore metamodel. This project aims to support exactly this functionality, that is, beyond the standard Xcore way to define derived features (basically with Xbase expressions), it is now possible to refer to pattern definitions in the Xcore files. One can specify which pattern should serve as the underlying logic for the evaluation of the given derived feature in runtime.

Another key benefit of using EMF-IncQuery to define derived features is that these features are well-behaving features [6] as opposed to most of the standard derived features in EMF. A derived feature is well-behaving if proper change notifications are sent upon modifications of the underlying model elements. However, this does not hold for regular derived features which usually have no field (meaning that they do not store the current value) therefore they are unable to send proper notifications (e.g. SET oldValud to newValue).

The integration was developed as seamlessly as possible; all the original Xcore semantics and toolset are still available, however, the IncQuery Xcore editor now supports the definition of the above mentioned features with additional features like validation, proposal providers, etc.

Requirements

  1. Java JRE or JDK (1.6 or above)
  2. Eclipse Kepler (4.3)
  3. Xtext 2.5 M2 SDK http://download.eclipse.org/modeling/tmf/xtext/updates/composite/milestones/
  4. EMF all in one SDK 2.10 milestone & Xcore http://download.eclipse.org/modeling/emf/emf/updates/milestones
  5. EMF Transaction http://download.eclipse.org/modeling/emf/transaction/updates/milestones
  6. GEF4 Zest Visualization Toolkit SDK integration http://download.eclipse.org/tools/gef/gef4/updates/integration (optional - install it if you want to use the GEF4 feature of EMF-IncQuery)
  7. GMF Runtime SDK, Graphiti SDK from Kepler update site (optional - install it if you want to use the GMF / Graphiti feature of EMF-IncQuery)
  8. EMF-IncQuery with Xcore integration http://download.eclipse.org/incquery/updates-xcoreiq

Overview and example

You can get a really good overview about the Xcore project if you browse through the official tutorial http://wiki.eclipse.org/Xcore. In this section we will use the same metamodel but with additional derived feature definitions. In order to start trying out the tool, create a new Xcore project and an IncQuery & Xcore file with the file extension .xcoreiq.

Note that, at the moment you can only use the IncQuery support for Xcore files if the metamodel and the patterns are defined in the same project, that is, the project must have both Xcore (Xtext) and IncQuery natures.

The metamodel is originated from the well-known library example from the EMF examples. We will model some simple entities like Library, Books and Writers. We also want to store the authors of these books and various properties like citations between the books, the own books of an author, etc. You can obtain the library examples from the EMF-IncQuery Examples git repository [2] (library folder). The metamodel in the library.domain.base project contains some basic derived features while the metamodel in the library.domain project contains other, more complex derived feature definitions.

Two kinds of IncQuery-backed derived features can be defined:

  • attributes (EAttribute) with the 'incquery-derived' keyword. Naturally, the type of these features can only be primitive types
  • references (EReference) with the 'incquery-derived refers' keyword. The type of these features can be any reference type (user-defined classes for example)

Metamodel (Library.xcoreiq)

class Library {  
	String name 
	contains Writer[] writers opposite library
	contains Book[] books opposite library
	
	// derived features
	incquery-derived refers Book[] suspiciousBooks spec validation.suspiciousBook
	incquery-derived BookCategory mostPopularBookCategory spec mostPopularBookCategory
}

class Writer {
	String firstName   
	String lastName
	container Library library opposite writers
	refers Book[] books opposite writers
	
	// derived features
	incquery-derived String name spec writerName
	incquery-derived Double averageNumberOfCoauthorsPerBook spec averageNumberOfCoauthorsPerBook
	incquery-derived refers Writer[] coAuthors spec coAuthorsOfWriter
	incquery-derived refers Book[] ownBooks spec ownBooksOfWriter
}

class Book {
	String title = "" // set a default value
	Integer pages
	BookCategory bookCategory
	refers Book[] citations
	refers Writer[] writers opposite books
	container Library library opposite books

	// derived features
	incquery-derived Integer numberOfCitations spec numberOfCitations 
	incquery-derived refers Book[] allCitations spec allBookCitations 
	incquery-derived Integer numberOfWriters spec numberOfWriters
}

enum BookCategory {
	Mystery = 0,
	ScienceFiction = 1,
	Biography = 2
}

Here we have defined 3 classes and the BookCategory enumeration for specifying the category of a book. The formal definitions of the derived features:

  • Library.suspiciousBooks (reference): the list of those books which (1) has an empty title or (2) transitively cites itself
  • Library.mostPopularBookCategory (attribute): the most popular book category of the library (no other category has more books)


  • Writer.ownBooks (reference): the feature should return only those books that have the writer as the only author (no co-authors for the given book)
  • Writer.writerName (attribute): the name of the writer is simply obtained by concatenating the first name and the last name together
  • Writer.coAuthorsOfWriter (reference): the feature references all other writers who is a co-author of this writer (they share a Book)
  • Writer.averageNumberOfCoauthorsPerBook (attribute): the number of co-authors divided by the number of books of the writer


  • Book.allCitations (reference): the feature should return all citations for the given book. This will be computed transitively, that is, if the citations are B1 -> B2 and B2 -> B3 then the set of all citations for the B1 book will contain both B2 and B3.
  • Book.numberOfCitations (attribute): the size of the Book.allCitations feature's value (the number of all citations)
  • Book.numberOfWriters (attribute): the number of the co-authors for the given book

Pattern definitions (LibraryPatterns.eiq)

// ***** relationships ********
/*
 * CW is the coauthor of W (they share the same book and CW != W) 
 */
pattern coAuthorsOfWriter(W : Writer, CW : Writer) {
	Book(B);
	Writer.books(W, B);
	Writer.books(CW, B);
	W != CW;
}

/*
 * The name of the writer W is N (first name and last name are concatenated)
 */
pattern writerName(W : Writer, N) {
	Writer.firstName(W, FN);
	Writer.lastName(W, LN);
	N == eval(FN + " " + LN);
}

/*
 * N represents the most popular book category of Library L, if that can be defined uniquely:
 * - if there is only one most popular book then that one is returned
 * - if there are two or more categories with the same number of books then 'None' is returned
 */
pattern mostPopularBookCategory(L : Library, N) {
	V == count find mostPopularBookCategory0(L, _CX);
	check(V == 1);
	find mostPopularBookCategory0(L, C);
	N == eval(C.toString);
} or {
	V == count find mostPopularBookCategory0(L, _CX);
	check(V > 1 || V == 0);
	N == "None";
}

/*
 * Category C has the most books in the Library L.
 */
pattern mostPopularBookCategory0(L : Library, C) = {
	Library.books.bookCategory(L, C);
	neg find moreBooksInCategoryThan(_Cx, C);
}
 
/*
 * There are more books with category C1 than books with category C2. 
 */
pattern moreBooksInCategoryThan(C1 : BookCategory, C2 : BookCategory) {
	N == count find categoryOfBook(C1, _B1);
	M == count find categoryOfBook(C2, _B2);
	check(N > M);
	check(C1 != C2);
} 

/*
 * The book category of Book B is BookCategory C
 */
pattern categoryOfBook(C : BookCategory, B : Book) {
	Book.bookCategory(B, C);
}

/*
 * The "own" books of writer, i.e. the ones that have only w as the author.
 */
pattern ownBooksOfWriter(W : Writer, B : Book) {
	Writer.books(W, B);
	find numberOfWritersOfBook(B, 1);
} 

/*
 * All of the (transitive) citations of Book b1 contain Book b2.
 */
pattern allBookCitations(B1 : Book, B2 : Book) {
	find bookCitation+(B1, B2);
}

/*
 * Book ref is contained in the set of books that (transitively) reference Book b. 
 */
pattern allReferencesTo(B: Book, Ref: Book){
	find bookCitation+(Ref, B);
}

/*
 * Book b1 cites Book b2.
 */
pattern bookCitation(B1 : Book, B2 : Book) {
	Book.citations(B1, B2);
}

/*
 * Writer W is the writer of Book B.
 */
pattern bookOfWriter(W : Writer, B : Book) {
	Writer.books(W, B);
}

// ************* numbers **********

/*
 * The average number of coauthors per book of a writer is 
 * the number of coauthors divided by the number of books.  
 */
pattern averageNumberOfCoauthorsPerBook(W : Writer, A) {
	find numberOfBooksOfWriter(W, NumOfBooks);
	NumOfCoAuthors == count find coAuthorsOfWriter(W, _CW);
	A == eval(if (NumOfBooks == 0) 0 else NumOfCoAuthors.doubleValue / NumOfBooks.doubleValue);
}

/*
 * The number of (transitive) citations of Book B1 is N.
 */
pattern numberOfCitations(B1 : Book, N) {
	N == count find allBookCitations(B1, _B2);
}

/*
 * The number of the books of Writer W is N.  
 */
pattern numberOfBooksOfWriter(W : Writer, N) {
	N == count find bookOfWriter(W, _B);
}

/*
 * The number of writers of Book B is N. 
 */
pattern numberOfWritersOfBook(B : Book, N) {
	N == count find bookOfWriter(B, _W);
}

Details of some pattern definitions:

  • numberOfWritersOfBook: returns the number of writers for a given book. It uses the count construct with the bookOfWriter pattern.
  • numberOfBooksOfWriter: returns the number of books for a given writer. Similarly, it uses the count construct with the bookOfWriter pattern, but now the W variable is quantified.
  • allBookCitations: returns all of the citations for the given book. It uses the transitive closure of the bookCitation pattern.
  • ownBooksOfWriter: returns the own books of a given writer. It uses the numberOfWritersOfBook pattern, where the value of parameter N must be exactly 1.
  • mostPopularBookCategory: the pattern uses a helper mostPopularBookCategory0 pattern. Basically, the latter one returns the most popular book(s) of the library and the former one is just a wrapper around the result. If only one category is the result then we return that one, in any other case we return 'None'.

Referring to the generated EPackage in the eiq file

An interesting problem arises when we want to develop the xcore and eiq files in parallel: in the metamodel we would like to specify derived features which would use pattern definitions which rely on the metamodel elements. To overcome this cyclic dependency problem, the eiq file editor uses the Xtext index and scoping when proposing EPackage imports. During the editing of the xcoreiq file, the corresponding EPackage is generated in the runtime, but the appropriate entry will only be inserted into the plugin.xml file if the xcoreiq file is error-free.

Generated Artifacts

If you edit and save the xcoreiq file the genmodel for the Ecore metamodel will be regenerated and the EMF model code will also be generated under the src-gen folder. You can customize further the various aspects of the code generation (name of the edit / editor plugins, source folders, etc. basically everything that you are used to while editing the properties of a standard Ecore metamodel). I just want to point out two annotations that you can use in the xcoreiq file:

  • @GenModel(editDirectory="/incquery.xcore.library.example.edit/src"): specifies the directory of the sources for the edit plug-in
  • @GenModel(editorDirectory="/incquery.xcore.test.editor/src"): specifies the directory of the sources for the editor plug-in

If the given projects do not exist, then those will be created and the appropriate code will be (re)generated.

The generated metamodel artifacts can be used in two different use cases:

  1. Dynamic Instance Mode: debugging of metamodels and queries are made easy with this functionality. One can develop the metamodel and queries at the same time, in the same workspace. You just need to create a dynamic instance model and you can modify your model on the fly. All the values of the features can be observed through the Properties View of Eclipse. Note that, this mode does not require the code to be generated, it infers everything from the xcoreiq file itself. The creation of a Dynamic Instance Model is described in the Xcore tutorial briefly.
  2. Using the Generated Code: this is the standard way of using the generated EMF plug-ins (model/edit/editor/test). Note that, if you have created your metamodel with Xcore there will not be any genmodel file present, the same functionality is provided by the xcoreiq file itself. Actually, the plugin.xml entry about the org.eclipse.emf.ecore.generated_package extension refers to the full path of the xcore

In either case, the IncQuery engine will be automatically initialized and maintained after model changes without any additional user interaction. The IncQuery backed derived features are well behaving features and they provide proper EMF notifications after model manipulations.

Advanced issues

This section describes some interesting aspects of the implementation. 

The Xtext language and language artifacts for the project

The language extends the Xcore language and adds some additional metamodel elements for the definition of the IncQuery based derived features. The Ecore metamodel of the language is created explicitly (not generated by the Xtext generator). The mwe2 workflow was customized to generate only those language artifacts which cannot be reused from Xcore (own proposal provider, additional validators, etc. were implemented).

SettingDelegates

The value of the derived features is computed by SettingDelegates [5] in both dynamic and generated cases. The org.eclipse.incquery.querybasedfeatures.runtime.QueryBasedFeatureSettingDelegateFactory class is used to create the delegates in runtime. This class is registered in an org.eclipse.emf.ecore.setting_delegate extension point.

During the Xcore -> Ecore building in runtime and code generation phases (for the generated case) we use some special annotations on the EPackage itself and on the derived features (the corresponding EStructuralFeature):

  • The package is annotated with the <settingDelegates, org.eclipse.incquery.querybasedfeature> key-value pair with "http://www.eclipse.org/emf/2002/Ecore" as the source of the annotation. This definition will tell EMF that there are features defined in the EPackage which use setting delegates to provide their value and the corresponding factory must be used to create these delegates.
  • The features are annotated with the <patternFQN, "fully qualified name of the pattern"> key-value pair with "org.eclipse.incquery.querybasedfeature" being the source of the annotation. This annotation will be used by the QueryBasedFeatureSettingDelegateFactory to obtain the fully qualified name of the pattern that should be used for the evaluation of the setting delegate. The factory will search for the corresponding query specification in the QuerySpecificationRegistry. Note that, in the dynamic case these specifications must be registered by hand, because the QueryBasedFeatureSettingDelegateFactory uses the target platform to collect the available query specifications (coming from plugin.xml entries). Normally, these specifications would be available only one Eclipse instance level higher.

For more details take a look at the org.eclipse.incquery.xcore.generator.IncQueryXcoreGenerator.xtend and org.eclipse.incquery.xcore.util.IncQueryXcoreEcoreBuilder.java classes.

Editing Dynamic Instance Model from the Model Editors

Dynamic instance models must be edited with the Reflective Ecore Model Editor for IncQuery-Xcore, because this editor initializes the QuerySpecificationRegistry before the model can be edited. Without this step the derived features will not work and all setting delegates return an empty result as the underlying query specifications cannot be found.

An other problem with the standard Sample Reflective Editor is that it loads an unnecessarily big amount of TypeResource instances into the ResourceSet. The problem gets even worse if you want to edit some references in the Properties View. After clicking into the View you need to wait 1-2 mins until all additional (and unnecessary) TypeResources are loaded into the ResourceSet (this is because of the poor implementation of the underlying internal logic). This would also affect the performance of the EMF-IncQuery engine because a huge ResourceSet needs to be traversed. In order to avoid this problem the Reflective Ecore Model Editor for IncQuery-Xcore is optimized to avoid this excessive loading. To achieve even better performance, the scope of pattern matching is restricted to Resource scope for the Xcoreiq ResourceSets and the EMF-IncQuery Base Index is working in Dynamic EMF Mode.

Note that, this issue does not affect the generated case in any way.

Resources and useful links

  1. EMF-IncQuery git repository: http://git.eclipse.org/c/incquery/org.eclipse.incquery.git
  2. EMF-IncQuery Examples git repository: http://git.eclipse.org/c/incquery/org.eclipse.incquery.examples.git/
  3. EMF-IncQuery homepage: http://incquery.net
  4. Xcore tutorial: http://wiki.eclipse.org/Xcore
  5. An overview about the EStructuralFeature SettingDelegates: http://wiki.eclipse.org/EMF/New_and_Noteworthy/Helios#Support_for_Feature_Setting_Delegates
  6. An overview about the EMF-IncQuery backed derived features: http://incquery.net/incquery/documentation/derivedfeatures