Difference between revisions of "EMFIndex Meeting Minutes"

Latest revision as of 09:49, 17 December 2009

17th of December 2009

Participants

• Sven Efftinge(itemis)
• Jan Köhnlein (itemis)
• Christian Mohr (SAP)
• Martin Strenge (SAP)
• Sebastian Zarnekow (itemis)

Sven presents the latest changes in Xtext regarding the index.

• ResourceDecriptions
• Used in builder
  • PersistableResourceDescriptions are updated
• Atomic update, immutable state
• No more direct dependency to EMF Index
• Eventing: When index state is updated events with deltas are fired
• Query by type and query by type and name
• Optimization: Fast querying for named objects in resource descriptions
• Synchronization of queries, asynchronous index updates

26th of August 2009

Participants

• Sven Efftinge(itemis)
• Boris Gruschko (SAP)
• Jan Köhnlein (itemis)
• Ed Merks
• Christian Mohr (SAP)
• Martin Strenge (SAP)
• Jos Warmer (Ordina)
• Sebastian Zarnekow (itemis)

Jan summarizes Jos's latest email, in which he wrote that he needs nested descriptors but could live with wildcard queries. Jos agrees.

Discussion on object-oriented (itemis) vs string-oriented (SAP) API

• Christian/Bernd cannot understand why we need a high-level (OO) API.
• Sven/Jan/Sebastian want to provide an API that is convenient for the users. Handling strings would be very uncomfortable. We should do the heavy lifting for the user, i.e. provide nice APIs.

Discussion on layering APIs

• Boris proposes to build the OO API on the String API.
• Sven says that is not an option as the OO implementation in progress is based on EMF. Proposes other way around.
• Bernd says that is not acceptable for performance reasons.

Discussion on how many indices should be there at runtime

• Christian/Bernd/Boris are concerned about memory consumption if multiple indices are running. Don't see that how it is possible to switch the actual index implementation in use.
• Jan/Sven say it is feasible to switch implementation. Default should be the one that ships with EMFIndex. Runtime config can be achieved in a variety of ways: Workspace setting, preferences, extension points...

How to proceed

• SAP is continuing their implementation based on their API as their deadline is approaching.
• itemis is implementing their EMF based implementation as they need it now to advance Xtext.

12th of August 2009

Participants

• Sven Efftinge(itemis)
• Boris Gruschko (SAP)
• Jan Köhnlein (itemis)
• Christian Mohr (SAP)
• Martin Strenge (SAP)
• Sebastian Zarnekow (itemis)

Jan presented the new API proposal from itemis (see [1]). Main purpose: Make API leaner and extensible at the right places.

Descriptors

• Base descriptor interfaces can be extended to add domain specific extensions. We'll provide support for the generic userData in the base implementation, but encourage DSL developers to define their own descriptor types instead.
• Descriptor API focusses on object graphs rather than references by Strings.
• Introduction of new ContainerDescriptor that reflects something containing resources (projects, folders, etc). ContainerDescriptors can be nested.
• Default implementation will try to use EMF for descriptors.

Query

• Improved support for transactions: Queries can only be executed inside a QueryCommand.
• Queries no longer contain execution code, but are mere datastructures.
• QueryExecutor can be filled with QueryExecutor strategies that now how to execute a given query against a given backend. Registration needs some kind of priority.
• EMF Index will define a secondary API with generic queries (against the base descriptor interfaces). Backend implementors must at least implement QueryExecutionStrategies for these base queries.
• DSL / backend developers can provide their own Queries and QueryExecutorStrategies for their descriptor implementations.

Store

• DAO abstractions were used in the store API only, and were neither convenient for file nor for DB persistence. They have been removed.
• Primary storage API has been made optional, as in the DB case we might not have write operations on the index.
• IndexUpdater updates the index data given a new subtree of descriptors.
• ResourceIndexer may be part of secondary API.

Discussions

• It's unclear whether QueryExecutorStrategies represent backend- or domain-specific stuff. Answer: Maybe both. Strategy decides itself whether it can execute a query against a given Index.
• Are events fired in a CDO scenario? Answer: yes, but that has to be taken care of by another API maybe based on adapters. Removed update methods for EObjectDescriptors and EReferenceDescriptors accordingly.
• What is the update strategy for ContainerDescriptors? Answer: Still somewhat open.
• Store API should look similar to query API with respect to transaction. Proposal has been updated.
• IndexUpdater (formerly IndexFeeder) interface needs descriptors now. Might be a performance penalty in a non-EMF implementation.

6th of August 2009

Participants

• Sven Efftinge(itemis)
• Boris Gruschko (SAP)
• Jan Köhnlein (itemis)
• Bernd Kolb (SAP)
• Christian Mohr (SAP)
• Martin Strenge (SAP)
• Knut Wannheden (paranor)
• Jos Warmer(Ordina)

SAP (Martin?) presented their new proposed API patch (presented diagram).

• New concept ContentFilter: Reusable generic extensions/strategies that contribute to the userData of an element that is indexed.

General discussion on the "name" of an EObject

Sven:

• It's a first class artifact for Xtext and should be reflected in the API.
• Do we need a ContentFilter for that? What if two ContentFilters define the name differently?
• Deciding on what information is to be stored in the index is responsibility of language designer.

Bernd:

• "name" is still in the API of the EObjectResult.
• ContentFilter needed to make sure that some generic information needed by EMF Query is available in the index.

Discussion on defining the "universe": How to I specify what's in the scope of a query, e.g. when names are no longer unique.

Jan: Proposed an additional descriptor for grouping ResourceDescriptors or a "predefined query" to limit the amount of descriptors to be matched in a query.

Bernd: Nearly impossible to implement efficiently.

Jos: Better define the universe outside of EMF index, analogous to the classpath in Java. Have separate index instances for each universe.

Jan: That would either double information if models are used in multiple different contexts or require linking between indices.

Conclusion

Sven: Agrees with many changes, e.g. the elimination of M2 Ecore descriptors.

Jan: Number of commonalities has increased. Still wants EMF Index to be domain-specifically extensible and usable without EMF Query. Big question is if tools should build on top of EMF Index or EMF Query.

Bernd: Proposes a separate meeting to demonstrate the new EMF Query implementation.

Next steps:

• itemis will revise the API change and come up with a new proposal.

31th of July 2009

Participants

• Boris Gruschko (SAP)
• Jan Köhnlein (itemis)
• Bernd Kolb (SAP)
• Christian Mohr (SAP)
• Martin Strenge (SAP)

SAP presented their latest API proposal.

• XDescriptors are replaced by XResults as they are only relevant as query results.
• XResults reference each other only by URI.
• M2 information is treated the same way as M1 information, i.e. no more explicit EClass/EReference/EPackage entries
• Indexing has been generified: User data has been removed from Resource and EObject entries as well as displayName.
  

Discussion on user data

Jan:

• We need domain-specific information in the index for efficient querying in domain-specific tools, e.g. for fully qualified names.
  
• We should not have to load resources to match such queries.
  

SAP:

• Any additional information is consuming much memory.
  
• Only scopes offered by the index are resource and eclass scopes, as the experience shows these are sufficient to limit the amount of resources to be loaded for matching the query.
  
• Not clear how index is kept in sync when to user info is derived information.
  
• Not clear how two applications who's responsible for indexing when two apps require different user data on the same metamodel.
  
• Higher level queries - such as the domain specific ones - are treated on the EMF Query level.

SAP shows their ideas on EMF Query, based on their  MQL (MOIN Query Language)

There seems to be a different understanding on the architecture of a domain-specific tool

itemis:

• Domain-specifically configurable EMF Index is used directly.
• Different backends for different model storage technologies (DB, filesystem, ...)

SAP:

• EMF Index is generic
• EMF Index mainly used to make EMF Queries more efficient in the absence of a DB

Next steps:

• Inform all stakeholders, collect requirements and decide on the API based on that.
• Propose Christian and Martin as committers