Jump to: navigation, search

Difference between revisions of "ATL VM Testing"

(emfvm missing features)
(updated links to repository using Template:ATLRepositoryLink)
 
(5 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
==Intent==
 
==Intent==
  
This page consists in a report of [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/plugins/org.eclipse.m2m.atl.engine.emfvm/?root=Modeling_Project emfvm] non-regression tests. Those tests are available on [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/?root=Modeling_Project CVS] and reports all performances and regression issues between the two versions of the ATL VM.
+
This page consists in a report of {{ATLRepositoryLink|plugins/org.eclipse.m2m.atl.engine.emfvm/|emfvm}} non-regression tests. Those tests are available in {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/|the ATL repository}} and reports all performances and regression issues between the two versions of the ATL VM.
 
Here are also centralized all development informations about the emfvm, like missing features.
 
Here are also centralized all development informations about the emfvm, like missing features.
  
Line 19: Line 19:
 
! rowspan=2 | Comments
 
! rowspan=2 | Comments
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/plugins/org.eclipse.m2m.atl.engine.emfvm/?root=Modeling_Project emfvm]
+
! colspan=1 | {{ATLRepositoryLink|plugins/org.eclipse.m2m.atl.engine.vm/|Regular VM}}
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/plugins/org.eclipse.m2m.atl.engine.vm/?root=Modeling_Project Regular VM]
+
! colspan=1 | {{ATLRepositoryLink|plugins/org.eclipse.m2m.atl.engine.emfvm/|emfvm}}
<!--
+
TODO
+
======
+
To feed the non-regression table, just copy and paste here the trace.txt file here
+
To keep the sense of this table, be aware of overwriting the old data, EXCEPT old comments -->
+
<!-- BEGIN TESTS -->
+
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/SpreadSheetML2PerformanceUMLStateChart/Trace2PerformanceMetrics/?root=Modeling_Project Trace2PerformanceMetrics]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/SpreadSheetML2PerformanceUMLStateChart/SpreadsheetMLSimplified2Trace|SpreadsheetMLSimplified2Trace}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.015s.
+
! colspan=1 | 0.031s.
 
! colspan=1 | 0.0s.
 
! colspan=1 | 0.0s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/BibTeX2DocBook/?root=Modeling_Project BibTeX2DocBook]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Class2Relational/Ecore2Class|Ecore2Class}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.031s.
+
! colspan=1 | 0.016s.
! colspan=1 | 0.047s.
+
! colspan=1 | 0.016s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/FlattenTest/?root=Modeling_Project FlattenTest]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Families2Persons|Families2Persons}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.047s.
+
! colspan=1 | 0.0s.
! colspan=1 | 0.032s.
+
! colspan=1 | 0.0s.
! colspan=1 | Collections implementation completed on 12/12/2007.
+
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Book2Publication/XML2Book/?root=Modeling_Project XML2Book]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/EMF2DSL/KM32DSL|KM32DSL}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.0s.
+
! colspan=1 | 0.062s.
! colspan=1 | 0.016s.
+
! colspan=1 | 0.032s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Families2Persons/?root=Modeling_Project Families2Persons]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/XML2DSLModel|XML2DSLModel}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.0s.
+
! colspan=1 | 10.078s.
! colspan=1 | 0.0s.
+
! colspan=1 | 0.922s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Class2Relational/Ecore2Class/?root=Modeling_Project Ecore2Class]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/SpreadSheetML2PerformanceUMLStateChart/Trace2PerformanceMetrics|Trace2PerformanceMetrics}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 0.016s.
 
! colspan=1 | 0.0s.
 
! colspan=1 | 0.0s.
! colspan=1 | 0.016s.
 
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/DSL2EMF/DSL2KM3/?root=Modeling_Project DSL2KM3]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/DSL2EMF/XML2DSL|XML2DSL}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.032s.
+
! colspan=1 | 0.468s.
! colspan=1 | 0.031s.
+
! colspan=1 | 0.062s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Book2Publication/Book2Publication/?root=Modeling_Project Book2Publication]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/DSLModel2KM2|DSLModel2KM2}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.016s.
+
! colspan=1 | 0.047s.
! colspan=1 | 0.015s.
+
! colspan=1 | 0.0s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Class2Relational/Class2Relational/?root=Modeling_Project Class2Relational]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/UniqueLazyRule|UniqueLazyRule}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | 0.0s.
 
! colspan=1 | 0.0s.
Line 82: Line 76:
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/UML2Relational/?root=Modeling_Project UML2Relational]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Book2Publication/XML2Book|XML2Book}}
! colspan=1 | <b style="color:red">FAIL</b>
+
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 0.016s.
 +
! colspan=1 | 0.0s.
 
! colspan=1 |  
 
! colspan=1 |  
! colspan=1 | 0.046s.
 
! colspan=1 | Unsupported model elements containing "::".
 
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/EMF2DSL/DSL2XML/?root=Modeling_Project DSL2XML]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/EMF2DSL/DSL2XML|DSL2XML}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.094s.
+
! colspan=1 | 0.172s.
! colspan=1 | 0.156s.
+
! colspan=1 | 0.032s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/AssertionModification/?root=Modeling_Project AssertionModification]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/Maven2XML|Maven2XML}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.078s.
+
! colspan=1 | 0.062s.
! colspan=1 | 0.063s.
+
! colspan=1 | 0.0s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/XML2DSLModel/?root=Modeling_Project XML2DSLModel]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/Ant2Maven|Ant2Maven}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.938s.
+
! colspan=1 | 0.109s.
! colspan=1 | 10.203s.
+
! colspan=1 | 0.063s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/XML2Ant/?root=Modeling_Project XML2Ant]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/KM32ATL_KM22MM|KM32ATL_KM22MM}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.25s.
+
! colspan=1 | 0.047s.
! colspan=1 | 2.063s.
+
! colspan=1 | 0.031s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/DSLModel2KM2/?root=Modeling_Project DSLModel2KM2]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/UML2Relational|UML2Relational}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 0.047s.
 
! colspan=1 | 0.016s.
 
! colspan=1 | 0.016s.
! colspan=1 | 0.031s.
 
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/Ant2Maven/?root=Modeling_Project Ant2Maven]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Book2Publication/Book2Publication|Book2Publication}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.015s.
+
! colspan=1 | 0.0s.
! colspan=1 | 0.031s.
+
! colspan=1 | 0.0s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/EMF2DSL/KM32DSL/?root=Modeling_Project KM32DSL]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/DSL2EMF/DSL2KM3|DSL2KM3}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.032s.
+
! colspan=1 | 0.047s.
! colspan=1 | 0.031s.
+
! colspan=1 | 0.016s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/Maven2XML/?root=Modeling_Project Maven2XML]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Relational2UML|Relational2UML}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | 0.031s.
 
! colspan=1 | 0.031s.
! colspan=1 | 0.047s.
+
! colspan=1 | 0.016s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/ATL2Problem/?root=Modeling_Project ATL2Problem]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/BibTeX2DocBook|BibTeX2DocBook}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.046s.
+
! colspan=1 | 0.062s.
! colspan=1 | 0.078s.
+
! colspan=1 | 0.016s.
! colspan=1 | EnumLiteral support added on 25/01/2008.
+
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/MetamodelBridge/DSL2EMF/XML2DSL/?root=Modeling_Project XML2DSL]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/FlattenTest|FlattenTest}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.078s.
+
! colspan=1 | 0.031s.
! colspan=1 | 0.453s.
+
! colspan=1 | 0.015s.
 +
! colspan=1 | Collections implementation completed on 12/12/2007.
 +
|-
 +
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/AssertionModification|AssertionModification}}
 +
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 0.156s.
 +
! colspan=1 | 0.063s.
 
! colspan=1 |  
 
! colspan=1 |  
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/SpreadSheetML2PerformanceUMLStateChart/SpreadsheetMLSimplified2Trace/?root=Modeling_Project SpreadsheetMLSimplified2Trace]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Ant2Maven/XML2Ant|XML2Ant}}
! colspan=1 | <b style="color:red">FAIL</b>
+
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 2.078s.
 +
! colspan=1 | 0.235s.
 
! colspan=1 |  
 
! colspan=1 |  
! colspan=1 | 0.015s.
 
! colspan=1 | Transformation fails without errors.
 
 
|-
 
|-
! colspan=1 | [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.m2m/org.eclipse.m2m.atl/tests/org.eclipse.m2m.atl.tests/data/inputs/DSLBridge/ModelBridge/KM32ATL_KM22MM/?root=Modeling_Project KM32ATL_KM22MM]
+
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/Class2Relational/Class2Relational|Class2Relational}}
 
! colspan=1 | <b style="color:green">PASS</b>
 
! colspan=1 | <b style="color:green">PASS</b>
! colspan=1 | 0.031s.
+
! colspan=1 | 0.016s.
! colspan=1 | 0.031s.
+
! colspan=1 | 0.016s.
 
! colspan=1 |  
 
! colspan=1 |  
 +
|-
 +
! colspan=1 | {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/inputs/ATL2Problem|ATL2Problem}}
 +
! colspan=1 | <b style="color:green">PASS</b>
 +
! colspan=1 | 0.094s.
 +
! colspan=1 | 0.016s.
 +
! colspan=1 | EnumLiteral support added on 25/01/2008.
 
|-
 
|-
 
! colspan=2 | Total time :  
 
! colspan=2 | Total time :  
! colspan=1 | 1.75s.
+
! colspan=1 | 13.686s.
! colspan=1 | 13.34s.
+
! colspan=1 | 1.567s.
<!-- END TESTS -->
+
 
|}
 
|}
  
Tests realized on 29/01/2008 with :
+
Tests realized on Wed Mar 05 18:39:38 CET 2008 with :
 
+
 
Hardware
 
Hardware
 
*Processor : Pentium 2 GHz
 
*Processor : Pentium 2 GHz
Line 175: Line 179:
 
*Operating system : Windows XP
 
*Operating system : Windows XP
 
*JVM : JRE 1.6
 
*JVM : JRE 1.6
*Eclipse 3.3 Europa
+
*Eclipse 3.4 Ganymede
 
*ATL 2.0
 
*ATL 2.0
  
Line 192: Line 196:
  
 
The "delete" and "unset" new bytecodes will for instance be usefull to implement the refining mode of ATL 2006 (with in-place model transformation support).
 
The "delete" and "unset" new bytecodes will for instance be usefull to implement the refining mode of ATL 2006 (with in-place model transformation support).
 +
 +
==How to use==
 +
 +
org.eclipse.m2m.atl.test is an Eclipse plugin embedding a lot of ATL use cases. It also provides a way to launch several non-regression tests :
 +
* Transformation : a result of each transformation has been stored. Each transformation is launched using one of the atl vm's and the resulting models are compared with the old ones using [http://www.eclipse.org/modeling/emft/?project=compare EMF Compare].
 +
* Parsing : each atl file can be parsed into its .atl.ecore equivalent, in order to compare them.
 +
 +
===Installation===
 +
 +
To be launched, the test needs the org.eclipse.test plugin to be present. The best way to install it is to use an [[ATL/Developer Guide#Install ATL from CVS|atl psf]]. You can also checkout it from the /cvsroot/eclipse repository.
 +
 +
===Launching===
 +
 +
To launch the test, just right-click on the project, select Run As -> Junit Plug-in Test. Then in the launch configuration you will be able to choose the tests classes you want to launch.
 +
If you just want to test the EMFVM, just right click on the {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/src/org/eclipse/m2m/atl/tests/unit/atlvm/TestNonRegressionEMFVM.java|TestNonRegressionEMFVM.java}} file and launch a Junit Plug-in Test.
 +
 +
The {{ATLRepositoryLink|tests/org.eclipse.m2m.atl.tests/data/|data}} directory contains all the tests cases. If you want to avoid one are many tests, simply add their names in the emfvm.properties or vm.properties files (one per line).
 +
 +
====Results====
 +
 +
When the transformation tests are launched, the execution time of each transformation is printed on the console. You can retrieve the results in a simple XML file, in the "results" folder. Basically, those files are used to generate the non-regression table on this page.
  
 
==How to contribute==
 
==How to contribute==

Latest revision as of 07:55, 4 June 2013

Intent

This page consists in a report of emfvm non-regression tests. Those tests are available in the ATL repository and reports all performances and regression issues between the two versions of the ATL VM. Here are also centralized all development informations about the emfvm, like missing features.

Processus

Some relevant EMF-based transformations are integrated to the test set and are launched with the Regular VM and the emfvm. The results of the transformations are comparated using EMF Compare. When a test pass successfully, execution times are reported for both VMs in order to detect performance issues. If a test fails, a short description of the problem (or eventually a bug) is reported in the "Comments" field. The test is excluded from the emfvm test set until it is corrected.

Non-regression tests

Non-regression tests on emfvm
Test case Status Time Comments
Regular VM emfvm
SpreadsheetMLSimplified2Trace PASS 0.031s. 0.0s.
Ecore2Class PASS 0.016s. 0.016s.
Families2Persons PASS 0.0s. 0.0s.
KM32DSL PASS 0.062s. 0.032s.
XML2DSLModel PASS 10.078s. 0.922s.
Trace2PerformanceMetrics PASS 0.016s. 0.0s.
XML2DSL PASS 0.468s. 0.062s.
DSLModel2KM2 PASS 0.047s. 0.0s.
UniqueLazyRule PASS 0.0s. 0.0s.
XML2Book PASS 0.016s. 0.0s.
DSL2XML PASS 0.172s. 0.032s.
Maven2XML PASS 0.062s. 0.0s.
Ant2Maven PASS 0.109s. 0.063s.
KM32ATL_KM22MM PASS 0.047s. 0.031s.
UML2Relational PASS 0.047s. 0.016s.
Book2Publication PASS 0.0s. 0.0s.
DSL2KM3 PASS 0.047s. 0.016s.
Relational2UML PASS 0.031s. 0.016s.
BibTeX2DocBook PASS 0.062s. 0.016s.
FlattenTest PASS 0.031s. 0.015s. Collections implementation completed on 12/12/2007.
AssertionModification PASS 0.156s. 0.063s.
XML2Ant PASS 2.078s. 0.235s.
Class2Relational PASS 0.016s. 0.016s.
ATL2Problem PASS 0.094s. 0.016s. EnumLiteral support added on 25/01/2008.
Total time : 13.686s. 1.567s.

Tests realized on Wed Mar 05 18:39:38 CET 2008 with : Hardware

  • Processor : Pentium 2 GHz
  • RAM : 2 Go

Software

  • Operating system : Windows XP
  • JVM : JRE 1.6
  • Eclipse 3.4 Ganymede
  • ATL 2.0

emfvm missing features

Here is the list of the missing features of the emfvm. The purpose of this list is to attempt to prevent duplicate works, by signalizing current developments. So, if you are committer, or contributor, do not hesitate to update this list by shortly describing your current work (possibly with your contact):

  • Compilation support using one of the following alternatives:
    • Update ACG to directly generate an XML model instead of relying on specific "native types" like ASMEmitter.
    • Add the ASMEmitter type to emfvm, so that compilers generated with the current version of ACG can work.
  • ADWP debugging protocol.
  • Exceptions logger.
  • Superimposition.
  • New bytecodes:
    • "delete": to remove an element, and all references that point to it.
    • "unset": to unset a feature (e.g., to support changing the value of a multi-valued feature).
    • "newin": to create a given element in a specific model.

The "delete" and "unset" new bytecodes will for instance be usefull to implement the refining mode of ATL 2006 (with in-place model transformation support).

How to use

org.eclipse.m2m.atl.test is an Eclipse plugin embedding a lot of ATL use cases. It also provides a way to launch several non-regression tests :

  • Transformation : a result of each transformation has been stored. Each transformation is launched using one of the atl vm's and the resulting models are compared with the old ones using EMF Compare.
  • Parsing : each atl file can be parsed into its .atl.ecore equivalent, in order to compare them.

Installation

To be launched, the test needs the org.eclipse.test plugin to be present. The best way to install it is to use an atl psf. You can also checkout it from the /cvsroot/eclipse repository.

Launching

To launch the test, just right-click on the project, select Run As -> Junit Plug-in Test. Then in the launch configuration you will be able to choose the tests classes you want to launch. If you just want to test the EMFVM, just right click on the TestNonRegressionEMFVM.java file and launch a Junit Plug-in Test.

The data directory contains all the tests cases. If you want to avoid one are many tests, simply add their names in the emfvm.properties or vm.properties files (one per line).

Results

When the transformation tests are launched, the execution time of each transformation is printed on the console. You can retrieve the results in a simple XML file, in the "results" folder. Basically, those files are used to generate the non-regression table on this page.

How to contribute

To contribute, you can add your own test to the non-regression test. Please make sure that:

  • You are allowed to release the work under EPL.
  • The transformation works with EMF (i.e., the only model handler currently supported by emfvm).
  • The transformation correctly runs on the Regular VM.

NOTE : At this time, profiled UML2 target models are not supported. However, you can already submit any related test, in prevision of a future implementation.

To contribute a test, you can either put it into CVS (if you are committer), or contribute it on bugzilla. You may ask questions about how to do this on the M2M newsgroup, using the [ATL] prefix in your subject. You may also directly contact William Piers if you cannot follow the procedures described above.

A non-regression test is never complete, so feel free to contribute your tests!