View source for EMF/Query2/DevGuide

EMF Query2 Developer Guide 

== Installation  ==
Query2 can be installed from following update-site:
https://hudson.eclipse.org/hudson/job/tycho-query2-nightly/lastSuccessfulBuild/artifact/targetPlatform/

If you want to use string syntax of query2, you need to install Xtext. The update-sites for Xtext is located here: http://www.eclipse.org/Xtext/download/

== Overview of Query2  ==

Query2 provides easy mechanism to query emf metadata with minimal loading of resources. Query2 has an in-built indexing mechanism which indexes types, resources and references. This data is used during query execution and hence avoiding loading of resources wherever needed. Hence, if you want to use query2, you are mandatorily required to setup indexing. After that you can use it execute queries. We will see how to set it up in later sections. Query2 also provides you different ways to specify queries. We will see them in coming sections. 

= Using Query2  =

== Setting up Indexes  ==

'''Step 1 ''' Create dependency to index plugin: Add dependency to the plugins 

*org.eclipse.emf.query.index.ui 
*org.eclipse.emf.query.index

'''Step 2 ''' Using Indexing api: To index the resources, you can use IndexFactory as the starting point. It is a singleton class, which can keep the index for your complete workspace. 

*Following is code sample to index new resource:
<pre>IndexFactory.getInstance().executeUpdateCommand(
new UpdateCommandAdapter() {

@Override
public void execute(final IndexUpdater updater) {
      final ResourceIndexer indexer = new ResourceIndexer();
      final ResourceSet rs = new ResourceSetImpl();
	try {
		indexer.resourceChanged(updater, rs.getResource(URI.createPlatformResourceURI(resource.getFullPath().toString(),	true), true));
	} catch (Exception e) {
	// Put your logging here					
	}
	}
});
</pre> 
As mentioned in the example, use UpdateCommand on Index instead of directly using ResourceIndexer.resourceChanged() to ensure locking on the Index. 

*Following is a code sample to remove a resource from index:
<pre>IndexFactory.getInstance().executeUpdateCommand(
new UpdateCommandAdapter() {

	@Override
	public void execute(IndexUpdater updater) {
         updater.deleteResource(URI.createPlatformResourceURI(
               resource.getFullPath().toString(), true));
	}
});
</pre> 
*To query the indexes, there is factory class: IndexQueryFactory which gives different queries possible in Index.

Following is a code sample to retrieve EObject of a type from Index: 
<pre>final EObjectQuery&lt;EObjectDescriptor&gt; eObjectQuery = IndexQueryFactory.createEObjectQuery();
	eObjectQuery.eClassURI(eClassUris[i]);			index.executeQueryCommand(new QueryCommand() {

	@Override
	public void execute(QueryExecutor queryExecutor) {
	QueryResult&lt;EObjectDescriptor&gt; execute =  
                    queryExecutor.execute(eObjectQuery);
	for (EObjectDescriptor objDesc&nbsp;: execute) {
		URI candidateResourceUri = objDesc.getResourceURI();
		boolean isInScope = scope.contains(candidateResourceUri);
		if (isInclusiveScope == isInScope) {
		// add uri of instance to result
                 result.add(candidateResourceUri.
                 appendFragment(objDesc.getFragment()));
		}
	}
}
});
</pre> 
Following is a code sample to retrieve EReference from Index: 
<pre>final EReferenceQuery&lt;EReferenceDescriptor&gt; eReferenceQuery = IndexQueryFactory.createEReferenceQuery();
eReferenceQuery.sourceEObject().resource().uri(fromResource.toString());
	eReferenceQuery.eReferenceURI(referenceUri);

	index.executeQueryCommand(new QueryCommand() {

	@Override
	public void execute(QueryExecutor queryExecutor) {
		QueryResult&lt;EReferenceDescriptor&gt; execute =
                             queryExecutor.execute(eReferenceQuery);
		for (EReferenceDescriptor eReferenceDescriptor&nbsp;: execute) {
		URI targetResourceUri =
                             eReferenceDescriptor.getTargetResourceURI();
		if (targetResourceUri&nbsp;!= null) {
				result.add(targetResourceUri);
			}
		}
	}
});
</pre> 
Similarly you can IndexQueryFactory .createResourceQuery() to create queries on Resources on the common index. 

'''Step 3 ''' Using QueryIndexBuilder: In eclipse IDE, we work on eclipse resources. Query2 indexing component provides an utility to index all the resources in your project using a custom builder. To enable it, you have to execute following steps: 

1. Add the project nature: “org.eclipse.emf.query2.index.ui.queryIndexNature” Open the .project file for your project and add this nature. 

2. Add Builder: “org.eclipse.emf.query2.index.ui.queryIndexBuilder” Open the .project file and add this builder. 

3. The Builder will index all the files with extensions registered in EMF extensionToFactoryMap. You can add your file extension to factory using plugin.xml e.g as in following snippet: 
<pre>&lt;extension point="org.eclipse.emf.ecore.extension_parser"&gt;
      &lt;parser class="com.demo.ResourceFactoryImpl"
       type="xml"&gt;
      &lt;/parser&gt;
&lt;/extension&gt;
</pre>

== Writing Queries in SQL like syntax  ==

The support to write string based queries is present if you install query2.syntax feature. If you have these features execute following steps to have the setup ready for execution:

1. In your plugin create a new file by some name and file extension “query” e.g searchQueries.query This file will contain all the queries you want to execute 

2. Specify query in the file: 

a. Specify import statement to the URI which will be used to identify types in the Package at runtime (more info [http://www.eclipse.org/Xtext/documentation/latest/xtext.html#syntax here])e.g. 
<pre> import "http://www.sap.com/sap.ui.library.xsd" </pre> 
b. Specify title for the query 

c. Specify the query 

An example for b and c is below: 
<pre>SearchAllLibraries:
from Library withoutsubtypes as lib select lib

SearchAllTypes:
from SimpleType withoutsubtypes as type select type
</pre> 
For execution of query, the queries in the .query file are loaded and used for execution. A sample code for execution is following: 
<pre>new StandaloneSetup().setPlatformUri("..");
Injector injector = new QueryStandaloneSetup().createInjectorAndDoEMFRegistration();

XtextResourceSet set = injector.getInstance(XtextResourceSet.class);
set.addLoadOption(XtextResource.OPTION_RESOLVE_ALL, Boolean.TRUE);
URI resourceURI = URI.createURI(
         "platform:/plugin/&lt;plugin-name&gt;/queries/searchQueries.query");
URI normalized = set.getURIConverter().normalize(resourceURI);
LazyLinkingResource xtextResource = (LazyLinkingResource) set.getResource(normalized, true);
Model	model = (Model) xtextResource.getContents().get(0);
MQLquery query = findQuery("SearchAllLibraries");
Query internalQuery = QueryTransformer.transform(query);
Index indexFactory = IndexFactory.getInstance();
QueryProcessor queryProcessor = QueryProcessorFactory.getDefault().createQueryProcessor(indexFactory);
QueryContext queryContext = getQueryContext(rs);
final ResultSet result = queryProcessor.execute(internalQuery, queryContext);
</pre> 

==== Samples Queries  ====

'''Note:''' The model used in the queries is mentioned in Appendix: Sample data 1. Select all books borrowed by an author: 
<pre>from Book as b 
	select b 
	where b.borrowedBy in (
		from Manuscript as m, Person as p  select p where m.author = p
	)
</pre> 
2. Select pages with title “Linux made easy” 
<pre>from Manuscript as m select m.pages where m.title = 'Linux Made Easy'
</pre> 
3. Select all Manuscripts only In Berlin.xml or Hamburg.xmi 
<pre>from Library as lib 
        in resources {"platform:/resource/org.eclipse.emf.query2.librarytest/data/library/Hamburg.xmi","platform:/resource/org.eclipse.emf.query2.librarytest/data/library/Berlin.xmi"},
	Book as b, 
	Manuscript as m 
	select m 
	where lib.books = b
		and b.instanceOf = m
</pre> 
For accessing sample data, refer to Appendix: Sample data

== Writing Queries in Object format  ==

The main class to create Query object is: org.eclipse.emf.query2.Query. You can use the constructors in Query class to create instance of Query. A Query requires mandatorily define following: 

#For specifying select clause: One or more SelectAttrs or SelectAlias 
#For specifying From clause: One or more FromType\FromFixedSet

You can also specify where clauses to specify conditions by specifying following: 

#For normal conditions: Use following:

'''LocalWhereEntry:''' A local where-entry allows the definition of a boolean expression, which compares the primitive typed attributes of one alias. 

#For Join conditions: Use any of following:

a. '''WhereComparisonAliases''': A where-comparison aliases compares two aliases to each other for equality, where both aliases have to be of the same type. 

b. '''WhereComparisonAttrs''': A where-comparison attributes compares two attributes of two aliases to each other for equality, where both attributes have to be of the same primitive type. 

c. '''WhereNestedReference''': A nested where-entry defines a where-entry which constrains an alias by connecting it via an association to the results of a nested query. Nested where-entries can be negated. 

d. '''WhereRelationReference''': A where-relation defines a where-entry which constrains two aliases to be involved in an association relation 

=== Sample Queries: ===
''' Note: ''' The model used in the queries is mentioned in Appendix: Sample data

1. Select all books borrowed by an author
<pre>
SelectEntry selectPersonEntry = new SelectAlias("p");

FromType fromManuscriptType = new FromType("m", "http://eclipse.org/modeling/emf/query/1.0.0#//Manuscript", false ,
new TypeScopeProvider() {

public boolean isInclusiveScope() {
	return true;
}

public URI[] getResourceScope() {
	return new URI[0];
}
});

FromType fromPersonType = new FromType("p", "http://eclipse.org/modeling/emf/query/1.0.0#//Person", false ,
new TypeScopeProvider() {
	public boolean isInclusiveScope() {
		return true;
	}
	public URI[] getResourceScope() {
		return new URI[0];
	}
});

WhereRelationReference compareAuthorToPerson = new WhereRelationReference("m", "author", "p");

Query nestedQuery = new Query(new SelectAttrs[] {selectPersonEntry}, new FromType[] { fromManuscriptType, fromPersonType }, new WhereEntry[] { compareAuthorToPerson });

WhereNestedReference nestedQueryReference = new WhereNestedReference(false, "b", "borrowedBy", nestedQuery);

Query query = new Query(new SelectAttrs[] {selectEntry}, new FromType[] { fromType }, new WhereEntry[] { nestedQueryReference });

Index indexFactory = IndexFactory.getInstance();
QueryProcessor queryProcessor = QueryProcessorFactory.getDefault().createQueryProcessor(indexFactory);
QueryContext queryContext = getQueryContext(rs);
final ResultSet result = queryProcessor.execute(query, queryContext);
</pre>

2. Select pages with title “Linux made easy”
<pre>
SelectAttrs selectAttrs = new SelectAttrs("m", new String[] { "pages" });

FromType fromType = new FromType("m", "http://eclipse.org/modeling/emf/query/1.0.0#//Manuscript", false, 
new  TypeScopeProvider() {

	public boolean isInclusiveScope() {
		return true;
	}
               public URI[] getResourceScope() {
		return new URI[0];
	}
);
						
WhereString whereClause =  new WhereString("title", Operation.EQUAL, "Linus Made Easy");
LocalWhereEntry localWhereEntry = new LocalWhereEntry("m", whereClause);

Query query = new Query(new SelectAttrs[] {selectAttrs}, new FromType[] { fromType }, new WhereEntry[] { localWhereEntry });

Index indexFactory = IndexFactory.getInstance();
QueryProcessor queryProcessor = QueryProcessorFactory.getDefault().createQueryProcessor(indexFactory);
QueryContext queryContext = getQueryContext(rs);
final ResultSet result = queryProcessor.execute(query, queryContext);
</pre>
For accessing sample data, refer to Appendix: Sample data

<br>

== Retrieving Query Results  ==

After execution, the query processor returns a ResultSet. It can used to retrieve the results of the query. Main methods of ResultSet are: 

<br> 

{| width="100%" border="1" align="left" cellpadding="1" cellspacing="1"
|-
| &nbsp;Metohd 
| &nbsp;Description
|-
| getQueryColumnTypes() 
| Returns ColumnType for each select entry in the query
|-
| getUris(String alias) 
| Returns all the URIs in the result for a given alias
|-
| URI getUri(int position, String alias) 
| Returns URI at particular index in the results for a given alias
|-
| getSize() 
| Returns the size of the results
|}

<br> <br> 

----

----

= Appendix  =

=== String based Syntax  ===

'''from''' ''&lt;EClass&gt;'' '''withoutsubtypes as''' ''&lt;alias&gt;'' '''in resources''' { uri1, uri2 … } '''select''' ''&lt;alias&gt;'' '''where''' ''&lt;alias&gt;.&lt;attribute&gt; = &lt;alias or alias-attribute&gt;'' '''and\or''' ''&lt;alias or alias attribute&gt;'' '''in''' ''( &lt;nested-query&gt; )'' 

*Optional keywords: withoutsubtypes, and\or, in resources, in 
*Mandatory keywords: from, as, select, where

=== Sample Data  ===

The samples data exists in eclipse CVS at following location: “/cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.query/org.eclipse.emf.query2/tests/org.eclipse.emf.query2.librarytest” For accessing eclipse CVS you can refer to following page: [http://wiki.eclipse.org/CVS_Howto CVS_HowTo]. 

Use “''':pserver:anonymous@dev.eclipse.org:/cvsroot/modeling'''” to access projects related to Modeling. The model used in the samples data is following: 

[[Image:LibraryModeForQuery2Demo.png]]

=== Date and Big Integer Support  ===

Query 2 supports datatype like Big Integer and Date. 

Now if any attribute in the model is expressed as a type of BigInteger.The query core now understands the BigInteger data type
and evaluate the queries on the basis of that.

Example of Date Queries, in the Query Editor is as follows:- 

<pre>
SelectPagesandDate: from Manuscript as m select m.title , m.date

PagesByDate: from Manuscript as m select m.title where m.date &lt; 2008-10-01 

PagesBetweenDate: from Manuscript as m select m.title where m.date &gt; 2008-10-01 and m.date &lt; 2010-10-01
</pre>

The Limitation with the date datatype in String based syntax is that, only following formats are supported in it:- 

{| width="600" border="1" cellpadding="1" cellspacing="1"
|+ '''Date Formats Supported in Query Editor and their Examples''' 
|-
| '''Format''' 
| '''Example'''
|-
| yyyy-MM-dd HH:mm:ss.SSSZ 
| 2010-10-12 10:12:39.129 PDT
|-
| yyyy-MM-dd HH:mm:ss.SSS 
| 2010-10-12 10:12:39.129
|-
| yyyy-MM-dd HH:mm:ss 
| 2010-10-12 10:12:39
|-
| yyyy-MM-dd HH:mm 
| 2010-10-12 10:12
|-
| yyyy-MM-dd 
| 2010-10-12
|}
<br>
However, using the AST format (Object Based Format) does not have this limitation.

<pre>
		URI manuscriptUri = EcoreUtil.getURI(LibraryPackage.Literals.MANUSCRIPT);

		// from clause
		FromType manuscripts = new FromType("m", manuscriptUri, true); //$NON-NLS-1$
		FromEntry[] fromEntries = new FromEntry[] { manuscripts };

		// select clause
		SelectAttrs selectManuscriptName = new SelectAttrs(
				"m", new String[] { "name", "date" }); //$NON-NLS-1$ //$NON-NLS-2$
		SelectAlias selectManuscript = new SelectAlias("m"); //$NON-NLS-1$
		SelectEntry[] selectEntries = new SelectEntry[] { selectManuscript,
				selectManuscriptName };

		// Create an element of Date Type
                //You can create a date of any format you want.
		SimpleDateFormat date = new SimpleDateFormat("dd-MM-yyyy");
		Date dt = null;
		try {
			dt = date.parse("11-11-1999");
		} catch (ParseException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}

		// where entries
		WhereDate whereDateGreaterThan = new WhereDate(
				"date", Operation.GREATER, dt); //$NON-NLS-1$
		WhereEntry whereDateEntry = new LocalWhereEntry(
				"m", whereDateGreaterThan); //$NON-NLS-1$
		WhereEntry[] whereEntries = new WhereEntry[] { whereDateEntry };

		// the actual query
		Query query = new Query(selectEntries, fromEntries, whereEntries);

		QueryContext queryContext = this
				.getQueryContext();
		ResultSet resultSet = this.getMQLProcessor().execute(query,
				queryContext);
		Object objectName = resultSet.getAttribute(0, "m", "name");
		assertEquals("EMF", objectName.toString());
		Object date = resultSet.getAttribute(0, "m", "date");
		try {
			Date expectedDate = date.parse("20-11-2000");
			assertEquals(expectedDate, travelDate);
		} catch (ParseException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}
</pre>
<br>

=== Diagnosing Indices  ===

To facilitate the users to view the resources which have been indexed, Query 2 provides a dedicated view called '''Index View'''. 

Index View displays the indices in two broad categories:- '''Resource Index''' and '''Type Index'''. 

'''Resource Index''' basically lists all the indexed resources. The indexed resources are represented by the Resource URIs in the Index View. Each Resource URI node consists of three child nodes:- '''EObject Tables''', '''Incoming Link Tables''' and '''Outgoing Links Table'''. EObject Tables consists of a list of EObject URIs which represent the EObjects inside that particular resource. The Incoming and and Outgoing Link Tables are group of Reference URIs which represent incoming links to the resource and outgoing links from the resource respectively. 

{| class="FCK__ShowTableBorders"
|-
| width="20%" | 
*In the Sample Data given,&nbsp;there are two resources, '''resERefQuery_1.xmi''' and '''resERefQuery_2.xmi'''.
*Resource resERefQuery_2.xmi contains two local EObjects '''EClass2_2''' and '''EClass2'''. Whereas resERefQuery_1.xmi has two local EObjects '''EClass1''' which inherits from EClass2 in resERefQuery_2.xmi and '''EClass1_2'''. 
*Each resource is represented by their resource uri, for Eg. '''platform:/resource/TestData/data/resourcesample/resERefQuery_1.xmi''' for resERefQuery_1.xmi. 
*The EObject tables represent all the EObjects present inside the resource, hence, in the given sample data, resERefQuery_2.xmi will contain EClass2_2 and EClass2 whereas, resERefQuery_1.xmi will contain EClass1, EClass1_2 as well as another entry which represents a pointer to the super type. 
*In the Incoming Link Table, we can see the links coming to the given resource. For Eg in the case of resERefQuery_2.xmi there are incoming references towards the resource, as it is the container resource of EClass2 which is the superclass of EClass1. Since it has no references from the resource, Outgoing links table is empty.

| width="80%" align="center" | 
[[Image:Index Example.jpg]]

|}

'''Type Index''' lists the indexed resources in a grouped format based on the type of the resources. For Eg. EPackage, EClass etc. The type is represented by the URI of the type. 

[[Image:Index View.png]] 

Other than just displaying the Indices in the Index View, you can also rebuild the Indices using '''Re-Create Indices''' toolbar item. 

You can also open the editor for the indexed resource using '''Open in Editor''' Context menu item. If a resource URI is selected, editor for that resource is opened. But if an EObject URI is selected, editor for the resource containing the EObject is opened whereas for Reference URIs, source and target resources are opened in the editor. Index View also enables the user to copy the qualified name of any physical node using the '''Copy Qualified Name''' context menu item. 

Index View is also facilitated with '''Filters''', so that the users can filter out Local tables(EObject table, Incoming Links Table, Outgoing Links Table) if not required. 

Index View also has '''Legends''' for the users to understand the representation of each icon in the Index View. 

[[Image:Filters Index View.png]]