Revision as of 03:20, 11 August 2015

Frequently Asked Questions

Performance

This content is migrated from http://incquery.net/performance

On this page, we aim to summarize our experiences regarding peformance benchmarking with model transformation and model query tools. In particular, we attempt to provide advice on how to do accurate performance benchmarking and avoid typical pitfalls. We also aim to answer frequently asked questions regarding our technologies and performance/scalability/usability/functionality issues. Finally, we provide a detailed list of references to all academic papers, reports and supplementary material that are related to performance/scalability experiments.

Our most important goals with this page are transparency and reproducibility, that is, to provide precise descriptions, code and model examples, and evaluation guidelines that anyone can use to reproduce and check VIATRA and EMF-IncQuery for performance and scalability.

Benchmarking guidelines

Basics

The most important configuration step is to ensure that the Java Virtual Machine (JVM) running the Eclipse environment (and VIATRA/EMF-IncQuery inside) has access to as much memory (RAM) as possible. The reason for this is that by default, the JVM is not configured (by the settings in eclipse.ini) to be able to use all the available RAM in your computer. If the Eclipse application uses up all memory within the - rather low - default limit, thrashing and other kinds of performance degradation might occur, potentially corrupting performance measurement results.

For information on how to specify JVM boot parameters in Eclipse, we refer the reader to:

• Eclipse boot options http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Freference%2Fmisc%2Fruntime-options.html
• Running Eclipse (wiki page) http://wiki.eclipse.org/FAQ_How_do_I_run_Eclipse%3F
• Eclipse.ini http://wiki.eclipse.org/Eclipse.ini

For Eclipse applications, a performance benchmark setup typically requires the appropriate setting of two boot JVM parameters:

• maximum heap size: -XmxHEAPSIZEm (larger is better)
  • e.g. -Xmx2048m (for a 2GB heap limit)
  • if you wish to use EMF-IncQuery or VIATRA with large instance models (>100MB in serialized size), specify a limit which is as close as the physical RAM in your computer as possible
• maximum permgen space: -XX:MaxPermSize=PERMSIZEm
  • e.g. -XX:MaxPermSize=256m (for a 256M permgen space limit)

There are a number of other JVM boot parameters as well, which might have a beneficial effect on overall performance. On 64 bit systems, we recommend to use the followings:

• -XX:+UseCompressedOops
• -XX:-UseParallelGC