Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "EmfIndex Comparison"

m
 
(52 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page is intended to compare the two index implementations form different viewpoints such as
+
This page is intended to compare the two index implementations form different viewpoints.<br>
  
*API
+
= Performance  =
*Performence
+
Performance measurement was done on a T61 laptop.
*Memory Consumption
+
*How to build a convinient Query API on top of the low level API<br>
+
  
= API =
+
== Indexing ==
  
== Indexing API ==
+
=== Indexing time ===
 +
 
 +
This test measured the time which is needed to index x-times the content of Ecore.ecore. (Containing 393 instances of EObject and 520 references)<br>
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex indexingTime.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1
 +
| 2
 +
| 4
 +
| 8
 +
| 16
 +
| 32
 +
| 64
 +
| 128
 +
| 256
 +
| 512
 +
| 1024
 +
| 2048
 +
|-
 +
| SAP
 +
| 31 ms
 +
| 33 ms
 +
| 32 ms
 +
| 62 ms
 +
| 141 ms
 +
| 249 ms
 +
| 468 ms
 +
| 1061 ms
 +
| 1950 ms
 +
| 3994 ms
 +
| 7816 ms
 +
| 16302 ms
 +
|-
 +
| Itemis
 +
| 31 ms
 +
| 32 ms
 +
| 31 ms
 +
| 78 ms
 +
| 125 ms
 +
| 250 ms
 +
| 640 ms
 +
| 983 ms
 +
| 2153 ms
 +
| 4446 ms
 +
| 8408 ms
 +
| 16349 ms
 +
|}
 +
 
 +
|}
 +
 
 +
=== Index save time&nbsp;(logarithmic scale on x and y axis) ===
 +
 
 +
This test measured the time which is needed to dump the index to file system.<br>
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex indexSaveTime.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1
 +
| 2
 +
| 4
 +
| 8
 +
| 16
 +
| 32
 +
| 64
 +
| 128
 +
| 256
 +
| 512
 +
| 1024
 +
| 2048
 +
|-
 +
| SAP
 +
| 12 ms
 +
| 13 ms
 +
| 15 ms
 +
| 29 ms
 +
| 63 ms
 +
| 94 ms
 +
| 218 ms
 +
| 390 ms
 +
| 1139 ms
 +
| 1809 ms
 +
| 3697 ms
 +
| 7519 ms
 +
|-
 +
| Itemis
 +
| 15 ms
 +
| 15 ms
 +
| 16 ms
 +
| 32 ms
 +
| 62 ms
 +
| 109 ms
 +
| 203 ms
 +
| 521 ms
 +
| 843 ms
 +
| 1670 ms
 +
| 3808 ms
 +
| 7877 ms
 +
|}
 +
 
 +
|}
 +
 
 +
=== Memory consumption  ===
 +
 
 +
This test measured the in memory size of the index by use of the Memory Analyzer Tool. In the SAP case, paging was disabled<br>
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex AllInMemory.png|thumb]]
 +
|
 +
<br>
 +
 
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1000
 +
| 2000
 +
| 3000
 +
|-
 +
| SAP
 +
| 101 million bytes
 +
| 203 million bytes
 +
| 305 million bytes
 +
|-
 +
| Itemis
 +
| 141 million bytes
 +
| 281 million bytes
 +
| 422 million bytes
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
=== Required Memory on Disc<br> ===
 +
 
 +
These values show the amount of kilobytes per Ecore content.
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:SizeOnDisc.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| SAP
 +
| 92 kB
 +
|-
 +
| Itemis
 +
| 30 kB
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
 
 +
== Query response time  ==
 +
 
 +
=== Query All Resources, EObject and all EReferences<br> ===
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex ResponseTimeAllEObjects.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1
 +
| 2
 +
| 4
 +
| 8
 +
| 16
 +
| 32
 +
| 64
 +
| 128
 +
| 256
 +
| 512
 +
| 1024
 +
| 2048
 +
|-
 +
| SAP
 +
| 0,4 ms
 +
| 0,5 ms
 +
| 1,2 ms
 +
| 2 ms
 +
| 5 ms
 +
| 8 ms
 +
| 15 ms
 +
| 28 ms
 +
| 58 ms
 +
| 119 ms
 +
| 227 ms
 +
| 453 ms
 +
|-
 +
| Itemis
 +
| 2 ms
 +
| 3 ms
 +
| 5 ms
 +
| 8 ms
 +
| 16 ms
 +
| 33 ms
 +
| 64 ms
 +
| 137 ms
 +
| 262 ms
 +
| 510 ms
 +
| 1030 ms
 +
| 2049 ms
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
<br>
 +
 
 +
=== Query all references targeting a certain resource (logarithmic scale on x and y axis)<br> ===
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex ResponseTimeBackwardNavi.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1
 +
| 2
 +
| 4
 +
| 8
 +
| 16
 +
| 32
 +
| 64
 +
| 128
 +
| 256
 +
| 512
 +
| 1024
 +
| 2048
 +
|-
 +
| SAP
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
| 0,4 ms
 +
|-
 +
| Itemis
 +
| 1,2 ms
 +
| 2,3 ms
 +
| 4,8 ms
 +
| 9,3 ms
 +
| 17,4 ms
 +
| 35 ms
 +
| 71 ms
 +
| 177 ms
 +
| 279 ms
 +
| 553 ms
 +
| 1108 ms
 +
| 2222 ms
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
<br>
 +
 
 +
=== Query all instances of "EClass" (logarithmic scale on x and y axis)<br> ===
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex ResponseTimeTypeQuery.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| <br>
 +
| 1
 +
| 2
 +
| 4
 +
| 8
 +
| 16
 +
| 32
 +
| 64
 +
| 128
 +
| 256
 +
| 512
 +
| 1024
 +
| 2048
 +
|-
 +
| SAP
 +
| 0,03 ms
 +
| 0,03 ms
 +
| 0,04 ms
 +
| 0,05 ms
 +
| 0,06 ms
 +
| 0,1 ms
 +
| 0,16 ms
 +
| 0,3 ms
 +
| 0,6 ms
 +
| 1,1 ms
 +
| 2,9 ms
 +
| 6 ms
 +
|-
 +
| Itemis
 +
| 0,3 ms
 +
| 0,5 ms
 +
| 1,1 ms
 +
| 2,2 ms
 +
| 4,3 ms
 +
| 8,8 ms
 +
| 18,2 ms
 +
| 37,5 ms
 +
| 70 ms
 +
| 141 ms
 +
| 282 ms
 +
| 568 ms
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
<br>
 +
 
 +
=== Navigation Query required for convenient API<br> ===
 +
 
 +
These values show how fast the queries respond that are required to simulate interconnected descriptors.
 +
 
 +
{| class="FCK__ShowTableBorders" cellspacing="1" cellpadding="1"
 +
|-
 +
| [[Image:EmfIndex ResponseTimeNavigationQueries.png|thumb]]
 +
|
 +
{| border="1"
 +
|-
 +
| eObject.getResourceDescriptor()
 +
| 7 µs
 +
|-
 +
| resource.getEObjects()
 +
| 104 µs
 +
|-
 +
| resource.getReferences()
 +
| 220 µs
 +
|-
 +
| reference.getTargetObject()
 +
| 12 µs
 +
|}
 +
 
 +
<br>
 +
 
 +
|}
 +
 
 +
<br>
 +
 
 +
<br><br>
 +
 
 +
= Convenient query API  =
 +
 
 +
The following proposal can be implemented on top of the low level API to provide a more convenient query API as proposed by Itemis. The basic idea would be to provide a specialized query factory for the convenient queries, which returns wrappers around the low level query objects. The query wrapper would also return a wrapped QueryResult object which in turn wraps the low level descriptors. The following snippet shows an example of the query factory.
  
The current proposal form itemis introduces the concept of Indexers. An Indexer extracts the data from a Model and stores it in the index:
 
 
<source lang="java">
 
<source lang="java">
final Resource r = ...;
+
public class ConvenientIndexQueryFactory {
final ResourceIndexerImpl resourceIndexer = new ResourceIndexerImpl();
+
index.executeUpdateCommand(new UpdateCommand<Boolean>() {
+
public Boolean execute(IndexUpdater indexUpdater, QueryExecutor queryExecutor, Index indexStore) {
+
resourceIndexer.resourceChanged(r, indexUpdater);
+
return true;
+
}
+
});
+
</source>
+
+
== Query API  ==
+
  
== Descriptors ==
+
public ConvenientEObjectQuery<?> createEObjectQuery() {
 +
...
 +
}
 +
 
 +
public ConvenientEReferenceQuery<?> createEReferenceQuery() {
 +
...
 +
}
 +
 
 +
public ConvenientResourceQuery<ConvenientResourceDescriptor> createResourceQuery() {
 +
return new ConvenientResourceQueryImpl<ConvenientResourceDescriptor>();
 +
}
 +
</source>
 +
 
 +
The returned resource query needs to subclass our ResourceQueryImpl class and override the createQueryResult() method. This method gets an iterable with low level descriptors. The QueryResult only implements the Iterable interface and has no further methods.
 +
 
 +
<source lang="java">
 +
public class ConvenientResourceQueryImpl<T> extends ResourceQueryImpl<T> implements ConvenientResourceQuery<T> {
 +
 
 +
@Override
 +
public QueryResult<T> createQueryResult(QueryExecutorInternal queryExecutor, Iterable<ResourceDescriptor> result) {
 +
return ...; // new QueryResult implementation
 +
}
 +
 
 +
}
 +
</source>
 +
 
 +
A convenient user may use this API as like as the low level API:
 +
 
 +
<source lang="java">
 +
public class ConvenientUser {
 +
 
 +
public void test() {
 +
final ConvenientResourceQuery<ConvenientResourceDescriptor> query = new ConvenientIndexQueryFactory().createResourceQuery();
 +
query.uri("hallo");
 +
 
 +
Index index = ...;
 +
index.executeQueryCommand(new QueryCommand() {
 +
 
 +
@Override
 +
public void execute(QueryExecutor queryExecutor) {
 +
QueryResult<ConvenientResourceDescriptor> execute = queryExecutor.execute(query);
 +
}
 +
 
 +
});
 +
}
 +
}
 +
</source>

Latest revision as of 10:00, 15 September 2009

This page is intended to compare the two index implementations form different viewpoints.

Performance

Performance measurement was done on a T61 laptop.

Indexing

Indexing time

This test measured the time which is needed to index x-times the content of Ecore.ecore. (Containing 393 instances of EObject and 520 references)

EmfIndex indexingTime.png

1 2 4 8 16 32 64 128 256 512 1024 2048
SAP 31 ms 33 ms 32 ms 62 ms 141 ms 249 ms 468 ms 1061 ms 1950 ms 3994 ms 7816 ms 16302 ms
Itemis 31 ms 32 ms 31 ms 78 ms 125 ms 250 ms 640 ms 983 ms 2153 ms 4446 ms 8408 ms 16349 ms

Index save time (logarithmic scale on x and y axis)

This test measured the time which is needed to dump the index to file system.

EmfIndex indexSaveTime.png

1 2 4 8 16 32 64 128 256 512 1024 2048
SAP 12 ms 13 ms 15 ms 29 ms 63 ms 94 ms 218 ms 390 ms 1139 ms 1809 ms 3697 ms 7519 ms
Itemis 15 ms 15 ms 16 ms 32 ms 62 ms 109 ms 203 ms 521 ms 843 ms 1670 ms 3808 ms 7877 ms

Memory consumption

This test measured the in memory size of the index by use of the Memory Analyzer Tool. In the SAP case, paging was disabled

EmfIndex AllInMemory.png



1000 2000 3000
SAP 101 million bytes 203 million bytes 305 million bytes
Itemis 141 million bytes 281 million bytes 422 million bytes


Required Memory on Disc

These values show the amount of kilobytes per Ecore content.

SizeOnDisc.png
SAP 92 kB
Itemis 30 kB



Query response time

Query All Resources, EObject and all EReferences

EmfIndex ResponseTimeAllEObjects.png

1 2 4 8 16 32 64 128 256 512 1024 2048
SAP 0,4 ms 0,5 ms 1,2 ms 2 ms 5 ms 8 ms 15 ms 28 ms 58 ms 119 ms 227 ms 453 ms
Itemis 2 ms 3 ms 5 ms 8 ms 16 ms 33 ms 64 ms 137 ms 262 ms 510 ms 1030 ms 2049 ms



Query all references targeting a certain resource (logarithmic scale on x and y axis)

EmfIndex ResponseTimeBackwardNavi.png

1 2 4 8 16 32 64 128 256 512 1024 2048
SAP 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms 0,4 ms
Itemis 1,2 ms 2,3 ms 4,8 ms 9,3 ms 17,4 ms 35 ms 71 ms 177 ms 279 ms 553 ms 1108 ms 2222 ms



Query all instances of "EClass" (logarithmic scale on x and y axis)

EmfIndex ResponseTimeTypeQuery.png

1 2 4 8 16 32 64 128 256 512 1024 2048
SAP 0,03 ms 0,03 ms 0,04 ms 0,05 ms 0,06 ms 0,1 ms 0,16 ms 0,3 ms 0,6 ms 1,1 ms 2,9 ms 6 ms
Itemis 0,3 ms 0,5 ms 1,1 ms 2,2 ms 4,3 ms 8,8 ms 18,2 ms 37,5 ms 70 ms 141 ms 282 ms 568 ms



Navigation Query required for convenient API

These values show how fast the queries respond that are required to simulate interconnected descriptors.

EmfIndex ResponseTimeNavigationQueries.png
eObject.getResourceDescriptor() 7 µs
resource.getEObjects() 104 µs
resource.getReferences() 220 µs
reference.getTargetObject() 12 µs





Convenient query API

The following proposal can be implemented on top of the low level API to provide a more convenient query API as proposed by Itemis. The basic idea would be to provide a specialized query factory for the convenient queries, which returns wrappers around the low level query objects. The query wrapper would also return a wrapped QueryResult object which in turn wraps the low level descriptors. The following snippet shows an example of the query factory.

public class ConvenientIndexQueryFactory {
 
 public ConvenientEObjectQuery<?> createEObjectQuery() {
 ...
 }
 
 public ConvenientEReferenceQuery<?> createEReferenceQuery() {
 ...
 }
 
 public ConvenientResourceQuery<ConvenientResourceDescriptor> createResourceQuery() {
 return new ConvenientResourceQueryImpl<ConvenientResourceDescriptor>();
 }

The returned resource query needs to subclass our ResourceQueryImpl class and override the createQueryResult() method. This method gets an iterable with low level descriptors. The QueryResult only implements the Iterable interface and has no further methods.

public class ConvenientResourceQueryImpl<T> extends ResourceQueryImpl<T> implements ConvenientResourceQuery<T> {
 
 @Override
 public QueryResult<T> createQueryResult(QueryExecutorInternal queryExecutor, Iterable<ResourceDescriptor> result) {
 return ...; // new QueryResult implementation
 }
 
}

A convenient user may use this API as like as the low level API:

public class ConvenientUser {
 
 public void test() {
 final ConvenientResourceQuery<ConvenientResourceDescriptor> query = new ConvenientIndexQueryFactory().createResourceQuery();
 query.uri("hallo");
 
 Index index = ...;
 index.executeQueryCommand(new QueryCommand() {
 
 @Override
 public void execute(QueryExecutor queryExecutor) {
 QueryResult<ConvenientResourceDescriptor> execute = queryExecutor.execute(query);
 }
 
 });
 }
}

Back to the top