Jump to: navigation, search

GMF Xpand Migration howto

Revision as of 13:44, 19 March 2009 by Alexander.shatalin.borland.com (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

One of the most noticeable changes developed during GMF 2.2 M4 cycle was integration of QVTO-based model query language with the Xpand template language used throughout GMF. This document is important for those who used custom templates during GMF-based diagram(s) code generation and was designed to help in a migration of existing legacy Xtend-based templates to new QVTO-based queries.

Migration Tool

Migration tool consists of following plugins:

  • org.eclipse.gmf.xpand.migration
  • org.eclipse.gmf.xpand.qvtlibrary
  • org.eclipse.gmf.templates.legacy

    and can be downloaded as a part of GMF 2.2M4 or later. Source code is available from GMF CVS.

    org.eclipse.gmf.xpand.migration

    This plug-in contains slightly modified source code of the legacy GMF Xpand engine, together with migration code including UI contribution (probably this package should be reworked later to separate all three logical parts). From now legacy Xpand builder (org.eclipse.gmf.xpand.oawBuilder) is registered here, so without this plug-in you will not be able to see compilation problems in .xpt and .ext files of the legacy Xpand projects. This plug-in should be installed into the platform to expose the migration action. This plug-in is responsible for the GMF Xpand dialect migration, but with minor modification can also be used to migrate any Xtend language-based queries to the QVTO-based version.

    org.eclipse.gmf.xpand.qvtlibrary

    QVTO and Xtend languages are very similar, but nevertheless some particular Xtend constructions cannot be simply copied using QVTO – this language always has alternative constructions, but the semantic of QVTO construction can be different from the original Xtend query. In such cases migration code will produce .xpt/.qvto files referencing native libraries implementing particular Xtend language functions. All these native libraries are collected in org.eclipse.gmf.xpand.qvtlibrary plug-in. This plug-in should be installed into the platform to make .xpt/.qvto files produced by the migration plug-in compilable/executable.

    org.eclipse.gmf.templates.legacy

    This plug-in contains all templates released with last GMF 2.1.3. Legacy org.eclipse.gmf.xpand.oawBuilder Xpand builder was modified to load GMF templates from this plug-in instead of original template locations:
    /org.eclipse.gmf.codegen/templates
    /org.eclipse.gmf.codegen/templates-dynmodel
    /org.eclipse.gmf.codegen.lite/templates
    /org.eclipse.gmf.graphdef.codegen/templates

    This plug-in is necessary for the successful migration of custom GMF templates.

    How to execute

    Download and install migration tool

    Detailed description of Migration tool distribution you can find above: Migration Tool.

    Ensure existing templates present in the workspace and can be compiled

    Ensure Eclipse project with existing (legacy) Xtend-based templates is available in current workspace and can be properly compiled by legacy Xpand builder (org.eclipse.gmf.xpand.oawBuilder). To make existing templates compilable you have to create valid .xpand-root file in the root of the project with templates. This file was used in past by Xpand project builder to recognize local folders with templates (template roots) and point to the external template roots located in other projects/plugins. This dependency was necessary to resolve original templates called/aspected by template files from this template root. Here you can find examples of valid .xpand-root files from org.eclipse.gmf.codegen plug-in and from org.eclipse.gmf.codegen.lite plug-in. The format of this file was not changed during migration, so new QVTO-based Xpand uses the same style of .xpand-root file.

    As it was mentioned in org.eclipse.gmf.templates.legacy plug-in description, all legacy templates were relocated to org.eclipse.gmf.templates.legacy plug-in. Nevertheless, original template locations should be used inside .xpand-root file like:
    /org.eclipse.gmf.codegen/templates
    /org.eclipse.gmf.codegen.lite/templates

    Execute migration action

    "Project/Migrate to new QVTO-based xpand" main menu action is available then project with legacy Xpand builder (org.eclipse.gmf.xpand.oawBuilder) is selected. You can expect following results of this action execution:

  • New folder with QVTO-based templates will be created as a sibling of each template root specified in .xpand-root file. By default this folder will be named *.migrated.
  • New java source root will be created for each template root containing .ext files referencing java classes (JAVA Xtend construction). Newly created source root will be called *.qvtlib by default. Java classes representing necessary native extensions for QVTO will be generated into this source root.
  • plugin.xml will be modified if necessary to register generated QVTO native extensions
  • .xpand-root file will be updated with references to newly created *.migrated template folders.
  • New org.eclipse.gmf.xpand.xpandBuilder template builder will be registered on this project instead of legacy org.eclipse.gmf.xpand.oawBuilder.
  • QVTO org.eclipse.m2m.qvt.oml.project.TransformationNature and org.eclipse.m2m.qvt.oml.QvtBuilder will be installed on this project.

    Verify .xpand-root file

    This file will be properly updated with new template root folder name(s) created during migration, but all the other referenced template root names (external roots) will be just suffixed with *.migrated extension. During this step you have to open .xpand-root file in a text editor and verify if all (external) QVTO-based template roots specified there are available in your configuration and was properly updated during migration.

    Check QVT builder

    Transformation nature together with QVT builder will be installed upon selected project during migration process. This nature and builder are necessary to compile *.qvto files generated by the migration or created later manually in this project and display detailed error messages if there any. Due to the current QVT builder limitation you can set up only one QVT source container folder in project properties/QVT Settings. In other words, even if you have several Xpand template roots in your project holding *.qvto files only one of these roots can be specified in project properties as source container and will be respected by QVT builder during a build process. By default first migrated template root will be specified as QVT source container. If you want to see QVT builder errors for other template roots then you have to modify corresponding project properties to use proper folder as QVT source container.

    Add org.eclipse.m2m.qvt.oml into required plug-ins list

    Sometimes .java files implementing native extensions will be generated during migration. In these cases generated java code will import org.eclipse.m2m.qvt.oml.blackbox.java.Operation to use it in java annotations. To resolve referenced org.eclipse.m2m.qvt.oml.blackbox.java.Operation you have to add org.eclipse.m2m.qvt.oml plug-in into the list of required plug-ins.

    Repeat these steps for all template projects

    You can simply repeat these steps for all template projects. If you have cross-project dependencies then it is recommended that you migrate the most common project(s) first.
    Example: If you have a project p1 with the templates using templates from another project p2 then you have to:

  • Migrate project p2 first. As a result you’ll have both legacy templates and QVTO-based templates available in p2, p2/.xpand-root file pointing to the QVTO-based templates
  • Migrate project p1.
  • Then you’ll have all the templates successfully migrated and compilable (see: Ensure all projects ... compilable) you can finally delete legacy templates from both p1 and p2 projects.

    Content of p2/.xpand-root before the migration:

    templates, /p2/templates

    Ensure all projects with migrated templates are compilable

    At the end of migration process all the migrated templates should be successfully compiled. If you still have some compilation errors then check GMF Xpand Migration howto#Known problems section of this document describing how to resolve specific situations which are not supported by automated migration now.

    Other resources located below template root

    Migration action recognizes *.xpt and *.ext files only. Content of these files will be updated and stored in a new template root. All the other files located in legacy template root will be just copied into the same relative directory below newly created template root.

    Exposing legacy templates for the migration process

    If GMF Xpand was used to create GMF-independent template set and there can be other customers using these templates then it is possible to expose currently existing legacy templates to allow future migration. There are two possibilities - you can:

  • leave legacy templates in original folder and modify code generator to use newly created “<templates>.migrated” folder as a template root
  • move legacy templates into another folder/plug-in and expose this legacy templates location for org.eclipse.gmf.xpand.oawBuilder by using org.eclipse.gmf.xpand.migration.templates extension-point.

    Known limitations

    Here you can find a list of known limitations of Xtend->QVTO model query language migration tool with proposed solutions. If you find any other limitations please file a bugzilla request to correct corresponding migration tool problem(s) or update this wiki page.

    OclAny is not a super-type of the Collection type in QVTO

    Xtend type system is slightly different from the QVTO one. One of the most important differences is: in QVTO Collection type is not a subtype of OclAny, while an Xtend Collection is always a subtype of Object. This leads us to the following problem: extension/template definition defined using Object as a parameter/contextual type was successfully called for any instance of the Collection in Xtend-based templates, but will produce a compilation error after migration.

    For example following definition:

    «DEFINE generatedMemberComment FOR OclAny»
       «EXPAND generatedMemberComment('')»
    «ENDDEFINE»
    

    was successfully called for a Sequence(gmfgen::GenNode):

    «DEFINE childContainerCreateCommand FOR Sequence(gmfgen::GenNode)-»
    	«EXPAND xpt::Common::generatedMemberComment FOR this»
    «ENDDEFINE»
    

    Now such an EXPAND expression will produce compilation error with following message: “Couldn't find definition xpt::Common::generatedMemberComment for type Sequence(GenNode)”. To fix this error you simply have to define extension/template using corresponding Collection type as a parameter and handle this situation there:

    «DEFINE generatedMemberComment FOR Collection(OclAny)»
       «EXPAND generatedMemberComment('') FOR ''»
    «ENDDEFINE»
    

    Inner collection is not an instance of OclAny too

    As a side-effect of OclAny is not a super-type of the Collection type in QVTO you can face the following problem: extension/definition having Collection[Object] as a parameter/context type was successfully called for an instance of Collection[Collection[Object]] in Xtend-based queries, but will produce compilation problems with QVTO. Suggestion is similar – you have to define query/template with an appropriate collection type and handle this situation there.

    QVTO Collections are always strongly types

    As an another side-effect of OclAny is not a super-type of the Collection type in QVTO you can face following restriction wuring collection instance definition - in Xtend-based query language it was possible to define collection instance like:

    {
      link.getDiagram(), 
      link.getDiagram().getAllNodes(), 
      link.getDiagram().links
    }.flatten()
    

    as a result you’ll get a collection of common super-class of GenDiagram, GenNode and GenLink (GenCommonBase). In terms of QVTO this query is incorrect because there is not commons susper-class for the GenDiagram, Collection(GenNode), Collection(GenLink) combined together inside original collection. In QVTO to be valid this query have to be reformulated as:

    {link.getDiagram()}
       .addAll(link.getDiagram().getAllNodes())
       .addAll(link.getDiagram().links)
    

    Only contextual queries are polymorphic in QVTO

    Polymorphism was used intensively in Xtend – it was possible to define several extensions having different parameter types and let engine select appropriate extension based on the actual variable type at runtime. In addition, all Xtend extension signatures appear as static extensions, but it was possible to call these extensions in both static and contextual manner.

    For example it was possible to call following extension:

    String getExternalizerPackageName(gmfgen::GenEditorGenerator generator) :
    	generator.editor.packageName
    ;
    

    using static:

    String caller(gmfgen::GenEditorGenerator generator) :
    	getExternalizerPackageName(generator)
    ;
    

    or non-static way:

    String caller(gmfgen::GenEditorGenerator generator) :
    	generator.getExternalizerPackageName()
    ;
    

    This was a rather powerful and in the same time error-prone technique. In contrary to Xtend, QVTO supports only limited polymorphism functionality (very similar to Java language). First of all, it is possible to define either contextual:

    helper genModel::GenEditorGenerator::getExternalizerPackageName() : String {
    	return self.editor.packageName
    }
    

    or static queries

    helper getExternalizerPackageName(generator : genModel::GenEditorGenerator) : String {
    	return generator.editor.packageName
    }
    

    Contextual queries can be called using contextual notation only:

    helper caller(generator : genModel::GenEditorGenerator) : String {
    	return generator.getExternalizerPackageName()
    }
    

    and it is possible to define several contextual queries for different concrete sub-types (of genModel::GenEditorGenerator in this example) with different logic – appropriate query will be selected based on the actual type of the contextual variable on query execution. The limitation is: polymorphism supported only for context variable of the query – if you define several queries with different parameter types then the proper query to execute will be selected based on the declared type of parameter variable, but not on the actual parameter type during the execution.

    Migration tool transfer all the Xtend extension into the static QVTO queries if there is only one query with the same signature defined in this .ext file. In case there are several queries (with possible polymorphic calls from the outside) then each of these queries will be transformed into contextual query with first parameter used as a context. This will cover most of the usages of Xtend polymorphism feature – luckily in GMF templates only first extension parameter was used as a polymorphic one. In you have any templates using Xtend polymorphism feature with other extension parameters then you have to rework such a queries manually after migration.

    We saved parameters polymorphism for Xpand definitions for now. This means, applicable definition will be selected on template execution time based on the actual type of all DEFINE parameters.

    There is no cached keyword in QVTO

    Xtend supports cached modifier in an extensions. Except performance optimization this modifier in composition with possibility to modify contextual collection was used to save and access some temporary values during templates execution session from different templates.

    For example following extension:

    cached List[int] getNonNLSCounter() : 
    	{}
    ;
    

    can be called from different templates in a way modifying cached value:

    String incrementNonNLSCounter() :
    	getNonNLSCounter().add(getNonNLSCounter().size() + 1)->""
    ;
    

    or from another template during the same session to access accumulated state.

    QVTO supported module-vide properties for these purposes. You can define property:

    property counter : Sequence(Integer) = Sequence{};
    

    modify property value

    helper incrementNonNLSCounter() : OclVoid {
    	counter := counter->including(counter->size() + 1);
    	return null
    }
    

    and access it using QVTO language constructions. The limitation is: migration tool can not create module property based on cached Xtend modifier now, so one have to review all the cached keyword usages and perform migration manually.

    QVTO compiler performs return type checking

    There was no return type check for Xtend extensions during the compilation. As a result it was possible to successfully declare query like:

    gmfgen::GenNode getEClass(gmfgen::ElementType elementType) :
    	elementType.diagramElement
    ;
    

    and the actual reference type (gmfgen::ElementType::diagramElement is of type gmfgen ::GenCommonBase but not gmfgen::GenNode as declared) will be discovered only during template execution process forcing us to possible runtime errors. In QVTO such declarations will result in compilation errors forcing user to correct queries. It looks like most frequently such “incorrect” queries was used to implicitly cast abstract Collection[Object] received as a result of non-typed collection operations (e.g. flatten) to an actual type of collection (e.g. List[String]). All these situations will not be handled automatically and should be corrected manually after migration action execution.

    Extension files cannot be composed from several template roots anymore

    It was possible to extend a set of the queries defined inside particular *.ext file by adding another *.ext with the same fully qualified name into the different Xpand root folder – legacy Xpand engine will logically merge there two .ext files into the single composite extension exposing joint content of both .ext files. From now it is not possible – in QVTO you can not define *.qvto file with the same fully-qualified name as an existing one. As a solution you can rename such *.qvto files to make their names unique and inject appropriate imports into *.qvto/*.xpt files referencing them.

    Note: it is still possible to create several *.xpt files with the same fully-qualified name to get composite Xpand template as a result.

    .value and .literal properties are invisible for EEnumLiteral in QVTO

    In legacy Xpand it was possible to directly access .value and .literal properties of EEnumLiterals in QVTO:

    «DEFINE rgb FOR gmfgen::GenConstantColor-»
       «IF this.name.value > gmfgen::DiagramColors::darkBlue.value»
          «ERROR "Color constant not supported: " + this»
       «ENDIF-»
       org.eclipse.draw2d.ColorConstants.«name».getRGB()
    «ENDDEFINE»
    

    In QVTO corresponding properties are invisible. You can use only .name property or direct comparison for EEnumLiterals:

    «DEFINE rgb FOR gmfgen::GenConstantColor-»
       «IF this.name.name.startsWith("diagram")»
          «ERROR "Color constant not supported: " + this»
       «ENDIF-»
       org.eclipse.draw2d.ColorConstants.«name».getRGB()
    «ENDDEFINE»
    

    or:

    «DEFINE rgb FOR gmfgen::GenConstantColor-»
       «IF this.name.name = gmfgen::DiagramColors::darkBlue»
          …
       «ENDIF-»
       …
    «ENDDEFINE»
    

    Use underscore to prefix QVT keywords

    QVT has a number or reserved keywords which cannot be used as a part of identifiers.

  • If model contains such keywords then corresponding references should be updated to prefix this keyword with “_” symbol. For example, if model element has “library” property the corresponding property should be called from QVT using following notation:
    self._library
    

    Migration action will automatically prefix all the invalid definition/query/parameter/variable names with underscore symbol.

  • If one of keywords was used as definition or parameter/variable name then corresponding definition/parameter/variable name will be prefixed with "_" symbol automatically by a migration tool. For example, xpand definition called "reject" will be migrated as:
    «DEFINE _reject FOR OclAny»
      ...
    «ENDDEFINE»
    

    As a consequence all external references keeping such definition names should be updated manually (for example java code calling Xpand templates and having template names hardcoded).

    TODO

  • automatically add required org.eclipse.m2m.qvt.oml plug-ins

    Additional References

  • OCL 2.0 specification on OMG web site: http://www.omg.org/spec/OCL/2.0/
  • QVT 1.0 specification on OMG web site: http://www.omg.org/spec/QVT/1.0/
  • M2M/Operational_QVT_Language_(QVTO)