Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: for the plan.

Jump to: navigation, search

Xtext/planning 0.8.0

Here we collect what we want to do in the upcoming development (Helios) for Xtext.

Clean ups

Build infrastructure

We need to improve our build infrastructure as well as the general workspace setup. Some requirements:

  • use a dedicated target platform
  • be able to do a local build

It would be quite nice to have a sort of deferred commit: The data is send to the integration server, it compiles everything and executes the test and refuses the commit if something fails. Otherwise the patch is finally committed to the version control system.

Development Infrastructure

We should come up with a minimal set of style guidelines and conventions to minimize the necessary effort to merge changes and understand code. Ideally, we should try to stick to Eclipse defaults as much as possible, as this minimized configuration efforts. We should use project specific settings wherever applicable to minimize the risk of the I-forgot-to-import-some-settings-scenario. This includes

  • encoding,
  • code style,
  • save participants,
  • compiler warnings,
  • organize imports and
  • pde settings.


  • Review the Scoping API. We should provide a sort of IScopeExtension with methods like #getElement(String):IScopedElement and #getElement(EObject):IScopedElement. This allows more sophisticated implementations of the API.
  • Review the ValueConverter API: the method which turns Objects into strings should have the corresponding part of Node Model available, if such a part exists. This is needed to keep comments within Datatype-Rule-Values and it is helpful to keep the escaping of special characters (e.g. unicode) as it is.
  • Review the Highlighting API: Merge the *Configuration-Interfaces.
  • Review the implementation of the Classpath URI. Try to find a solution, that allows this kind of URI even in other ResourceSet implementations. Maybe consider deprecating the classpath scheme and replace it with a functionally equal implementation based on the index.
  • Review the LabelProvider API. Instead returning a file name of an icon, it would be better to return an Image/ImageDestriptor, since this would allow the usage of overlay icons.

Xtext Grammar

The linking of the grammar should be reviewed and updated to reflect the latest changes in the API and to be a better example on how to use the different available APIs. Furthermore there are still some possible links, that are currently plain strings (for example: Assignment.feature is still a string and not a cross-reference to the EStructuralFeature).

Project structure

Move as much as possible from org.eclipse.xtext into org.eclipse.xtext.xtext (similar to org.eclipse.xtext.xtext.ui). Merge org.eclipse.xtext.ui.core and org.eclipse.xtext.ui.common into org.eclipse.xtext.ui as the difference between both plugins is somewhat indistinct.

General Cleanups

  • delete no longer used projects from repository
  • delete dead resources from workspace (e.g. check files and the like)
  • remove Xtend-based scoping and code completion API

Cleanup generator.tests

Observation: It takes an awful amount of time to regenerate all test languages, refresh resources, build java code, synchronize with repository, etc. This large turnaround-time slows down the development process.

  • Create workflow-files for each java-package in /src. We already have that for some packages. This allows to regenerate only the needed languages when working on the tests.
  • GenerateALlTestLanguages.oaw should delegate to the workflows residing in the java-packages. This allows custom generator fragments for each language and avoids having two different generator-configurations at the same time for one language.
  • Each language should only have the minimal amount GeneratorFragments activated. The keeps the amount of Java-Files to a minimum and should thereby improve "refresh resources", "rebuild", "synchronize with rep." etc. Do we still support dynamic EMF?

Third party libs

Update the used libraries:

  • Guice2: Multibinding support
  • Google Collections 1.0: We currently use v 0.8 alpha. The API has changed significantly and we should update to 1.0.
  • ANTLR 3.1.x: A bunch of fixes and new features has been incorporated into ANTLR 3.1. Especially the support for grammar imports seems to be interesting.

New Features

Introduce DSLs for some of the stable viewpoints of Xtext.

  • should extend one basic expression language (something like [1])
  • Validation/Quickfix
  • EMFatic (incl. behavior)
  • Scoping
  • Indexing

Migrate to upcoming EMF Index changes

  • Query Builder
  • API / SPI separation
  • automatic revalidation if referenced resources have changed
  • refactoring of validation so that
    • in the reconciler no markers (just annotations are set)
    • on save markers are set using a builder
    • on save of referenced resources markers are set using a corresponding index hook

Runtime Features

  • Formatting API that allows to define rules depending on the current state of your model.
  • Formatting API that allows to define rules for grammar elements that are not explicitly represent in the Node Model. There are two ways to implement this: Extend the Node Model to contain all Grammar Elements or to derive the Grammar Elements position from the grammar's state machine.
  • find-Methods for GrammarAccess. Example: ga.findKeywords(";"), ga.getMyRuleAccess().findKeywords(","), ga.getMyRuleAccess().findRuleCallsToThisRule(), ga.findAllKeywordPairs("{", "}")
  • Provide a best practice on how to implement a property sheet that (partially) configures formatting. I don't think its a good idea to expose the formating rules on the property sheet. They are to fine grained/technical.
  • Allow tracing / impact analysis to implement partial linking, validation etc.
  • Introduce an official API that allows to define default values of the semantic model. (maybe through above's EMFatic DSL) (moritz: why is the common post processing hook for the semantic model not sufficient?)
  • Derive an EMF Validator from the grammar which checks if a model complies with all the implicit constrains of the grammar. Currently I (moritz) am aware of three kind of constraints:
    • Quantities: QRule: (a+=ID b+=INT); implies that a.size() == b.size()
    • Orders: X: Y | Z; ORule: a+=Y a+=Z; a is of type X, but the grammar implies that it may only contain instances of Y and Z and(!) that all instance of Y must be positioned before Z's instance in list a
    • Values: all values must comply with their TerminalRules/DatatypeRules/ValueConverters

UI Features

  • (delete me if this is already solved) Update the editor if the model is changed. If single EAttributes are modified, only their values need to be updated in the editor. If EReference are modified, the ParseTreeConstructor for the EObject and all of its children need to be executed. If an EAttributes contributes to an EObjects referenceable name, all cross references pointing to this EObject need to be found and updated. Since each grammar implies constraints, not all models are serializeable. Therefore, the model can not be serialized after each modification, but only after "some kind of transaction" has finished.
  • AutoEdit: This would greatly improve the user experience and help to minimize the on-type-changes in the semantic model. Without autoedit, the model becomes totally corrupt when the user tries to write a STRING literal, because everything behind the opening quotes will be consumend as a STRING. In almost any case this is not the intention of the user. Autoedit would insert a matching closing quote after to opening quote and the model will remain much more stable. Btw: Autoedit is not the right term here. JDT implemented this with a VerifyKeyListener (see org.eclipse.jdt.internal.ui.javaeditor.CompilationUnitEditor.BracketInserter).
  • Quick fixes, QuickAssist-API
  • Advanced Navigation / Hyperlinking API: Find References, allow to define references by convention, etc.
  • Highlighting: Allow to merge different highlighting styles that have been applied to the very same region of the model file.
  • a general API that allows to define arbitrary views for model elements (comparable to viewpoints in the Outline)
  • View points in the Outline
  • Hook for tooltips: users might want to display additional information when the mouse hovers over a reference, e.g. they might want to see the rafeerenced element's documentation. We should provide a hook for feeding this information.
  • Wizards
    • Wizard-Framework
    • Enhance "New Xtext Project" wizard: let user choose among various examples (see PDE New Project wizard to see this in action)
    • Add "New DSL Project" wizard to generated DSL, supply hooks for customization
  • Refactoring
    • rename refactoring
    • delete (model element / model file) refactoring - notifies the user if there are any references to the element / file
  • Provide an Eclipse-Builder that runs the user's code generation when the models are modified. The Default-Builder could always run a full build and be customizable to implement an incremental build

Xtext Grammar

  • Terminal fragments: Reusable elements in terminal rules, that are not globally visible (see ANTLR Fragments)
  • Aspects in production rules: Reuse different aspects of the grammar across various parser rules. One may think of infered EObjects that implement different interfaces such as Named, Documentable or FooContainer or BarReference. (You may call it a fragment).
  • Allow a notation to call the super-definition of a overridden parser rule.
  • Allow to define the default of an EEnum (even with an empty literal representation).
  • Try to generate named opposite references for eContainer.
  • add support for semantic predicates
  • add support for syntactic predicates
  • allow mixin of multiple grammars

Xtext Grammar Editor

Debug-support for to Java compiled languages

If one had the trace from the Java Code back to the model, it might be possible to hook JDT's debugger in order to shoe the model instead of the java code during debugging. Also it would be nice if we could come up with suitable abstractions over what is shown in the variables view then.

Back to the top