Skip to main content
Jump to: navigation, search

Papyrus/Oxygen Work Description/Refactoring/PerformancesImprovements

< Papyrus‎ | Oxygen Work Description
Revision as of 02:29, 24 January 2017 by Benoit.maggi.gmail.com (Talk | contribs) (Links)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Thanks to their experience in modeling performance and their expertise on the VIATRA technology, IncQuery Labs contributes to the Oxygen release of Papyrus by:

  • Providing means to assess and monitor some performance critical use cases in Papyrus.
  • Improving the performances on some of these cases.

Use cases:

Basic use cases

  1. Open a model (with many diagram already opened)
  2. Navigation in model explorer (Especially with stereotype with custom icons)
  3. Drop an element from the palette (Especially with a custom css)
  4. Drop an element from the model explorer (Especially with a custom css)
  5. Change the selection with the property view open
  6. Flat View in Select type widget
    • Create a model with many elements
    • Create a class with a property
    • Select the property and set the type in the property view
    • In the popup widget there is a tab named Flat (this tab presents all elements from the model and can be slow in big models)
  7. Create View dialog (Flat tab is no longer displayed in this case!)
    • Open a model with many model elements
    • In the Welcome page of the Papyrus editor, click on the Create View button
    • In case of large models, simply opening the dialog takes minutes or even more
  8. Filter in Model explorer
    • Open a model with many model elements
    • In the model explorer use the search/filter box
    • In case of large models, the model tree takes some times to refresh

Modeling workflow combining the above

  1. Warmup
    1. Open model
    2. Close all diagrams
    3. Open multiple diagrams
    4. Close model (diagrams will open next time)
  2. Open a model
  3. (loop multiple times)
    1. Open diagram
    2. Navigate in model explorer (select multiple elements, with properties view visible)
    3. Modify model (select one)
      • Add element to diagram
        • Drag from Model Explorer (custom CSS?)
        • Drag from palette (custom CSS?)
        • New child menu (custom CSS?)
      • Set type of property (opens dialog with flat tab)

Tasks:

Task 1.

Setting up a continuous scalability evaluation testbed for Papyrus. This testbed will be based on RcpTT tests and Hudson jobs

Its goal is to:

  1. highlight future regressions
  2. validate future enhancements

Further details

Task 2.

Identifying and sharing existing models (UML, notation, css, profile) related to performance bottlenecks in Papyrus. The Papyrus community is expected to share some models, but many performance issues are only present in big proprietary models (Obfuscation will probably be required)

Its goal is to:

  1. share a library of models that can be used for any kind of testing in Papyrus

Further details

Task 3.

Integrate a Papyrus specific indexing module based on the VIATRA base-indexer and query engine to improve model navigation and derived feature usage. Based on the 2 previous tasks, there will be an evaluation before and after the model integration to check performance improvements.

Further details

Links

To be completed:

User interface performance evaluation

The use cases listed above (1-6 at the moment) are each measured by RCPTT testcases that are added to the SysML 1.4 Git repository as the input model used is SysML 1.4. The testcases reuse the already existing common RCPTT contexts from the Papyrus Tools and SysML 1.4 RCPTT projects.

The performance measurement is using the start-measure and stop-measure ECL commands, with the output written to log files in the workspace.

let [val uri "workspace:/logs/case1-open-diagrams.log"] {
  // Empty file (if exists)
  write-lines -uri $uri
 
  start-time-measure "Open Model"
  open-file -project $projectName -file $modelName
  stop-time-measure "Open Model" | write-lines -uri $uri -append
}

In the most basic case, the above example measures the time it takes Papyrus to open an existing model.

proc "measure-property-view-selection" [val tree -input] [val element] [val message] [val uri] {
  try -command {
    with [$tree | get-item $element] {
      start-time-measure $message
      select-item
    }
  } -finally {stop-time-measure $message | write-lines -uri $uri -append}
}

Note that since RCPTT commands are rather slow, we try to minimize the amount of RCPTT commands inside the measurement window. In the example above, we make sure RCPTT finds the item in the tree before we make the selection itself.

The results of the testcases are written to log files that look like the following in the end (duration in milliseconds):

org.eclipse.rcptt.ecl.perf.impl.PerfCounterImpl@22eee37e (name: Open Model, duration: 13588)
org.eclipse.rcptt.ecl.perf.impl.PerfCounterImpl@57cf0bbb (name: Basic navigation, duration: 19521)
org.eclipse.rcptt.ecl.perf.impl.PerfCounterImpl@21531395 (name: Open Diagram, duration: 8679)
org.eclipse.rcptt.ecl.perf.impl.PerfCounterImpl@b9da5a6 (name: Navigate to, duration: 1746)

Shared model library

In order to provide a truly reusable shared model library, there should be a single entry point for all information on the models, this is probably a Wiki page with a link to the repository location of the model library.

The following is an incomplete, draft list of information that could be useful in a model library:

  • Model library level information:
    • List of models
    • General description on library structure (how to use the models)
    • Guide on adding new models
    • Guide for using models without copy-pasting
      • update site / feature, Maven repository and group:artifact identifiers
      • Java code for accessing models from tests
      • RCPTT context with procedures for adding the project to the workspace
  • Model level information:
    • Unique identifier to be used for referencing the model from list of models, issues or documentation
    • Model files
    • Short description
    • Usage guide (profiles/features needed)
    • Model metrics (number of elements, references, etc.)
    • Copyright information (who contributed it)

Papyrus specific indexing module

Following a detailed performance analysis of the cases 1-7, a few code segments were identified where performance issues were caused by repeated access of the model, more specifically repeated calculation of applied profiles and stereotypes. Even through this was only one of the causes of the slowdowns, a generic indexer feature is introduced to address such issues relying on the Base Indexer component of VIATRA and future details on indexer documentation page.

Profile application indexer with VIATRA

The Gerrit changes https://git.eclipse.org/r/#/c/85788/ and https://git.eclipse.org/r/#/c/86487/ provide a prototype implementation for this feature: the first one introduces a Papyrus service to initialize the indexer and make it available for use, while the second updates UMLProfileHelper implementation to use this service.

The proposed functionality is as follows:

  • The freshly introduced service UMLModelIndexingService starts as a Papyrus service, and it initializes the base indexer, and provides UML-specific APIs over the top of it.
  • The single specific ProfileApplicationIndexer provides a lazy cache implementation: when asked, it calculates the profile applications, and stores it in a map; and listens to base index changes to invalidate cache entries on model changes.
  • The service is added as an EMF Adapter to the model to support accessing it from model processing code.

For more details, see the dedicated documentation page.

Performance evaluation with RCPTT

The RCPTT based tests were used to measure on the macro-level how does the VIATRA based indexer affects the performance of Papyrus using the basic use cases from before. For the comparison, Papyrus and SysML 1.4 support was loaded from source, and the scenarios were executed three times. The first round was used as warm-up: as it is possible that some classes/extensions are not yet loaded, it can take longer to execute; the second and third rounds reused the same Papyrus instance. The corresponding commits are described in the table below:

Repository Commit without indexer Gerrit change for indexer
Papyrus core repository http://git.eclipse.org/c/papyrus/org.eclipse.papyrus.git/commit/?id=5b79cfb1b41f2089508ebda7c82675513fd3620c https://git.eclipse.org/r/#/c/86487/
SysML repository http://git.eclipse.org/c/papyrus/org.eclipse.papyrus-sysml.git/commit/?h=committers/bmaggi/oxygen&id=920f7e0d0f97ebfd54f551657e91a36b0cadf19c https://git.eclipse.org/r/#/c/86779/1

The measurements were collected in an Excel sheet were it can be seen that the effects of the indexer were negligible: loading the model initially got a bit slower (setup of the eager indexing), but some steps become somewhat faster later. However, it is important to note that the average difference between the runs using indexer or original one are smaller than the differences between single runs of the same type.

Detailed profiling

A more detailed profiling was executed to identify possible components that are responsible for a lot of execution time. This was done by aggregating the time spent in various Java classes per package (and subpackage) using an exported CSV file from the Yourkit profiler and an R script to do the aggregation. The measurements were executed both with the sampling and tracing mode of Yourkit for use cases 1 (open model), 2 (navigate in model explorer) and 6 (use set type dialog with flat view).

We were not able to provide comparable snapshots with open source profilers, because of limitations in the tools:

  • In case of Java Mission Control (and Flight Simulator) the tool supports only sampling mode, but its sampling frequency was too low to provide meaningful snapshots for small cases (a few seconds long).
  • In case of JVisualVM the sampling frequency was also too low; while the instrumentation required for profiling mode did not work when class loading was necessary.

Yourkit snapshots

The raw Yourkit snapshots are also available in https://incquerylabs-my.sharepoint.com/personal/zoltan_ujhelyi_incquerylabs_com/_layouts/15/guestaccess.aspx?folderid=01381c1a8db004829a4fa7f1841770700&authkey=AY2DHfSugoXxdpy_Ub0rEQE

Processed Files

The R script, the exported and processed CSV files and all the created Excel files are available from https://incquerylabs-my.sharepoint.com/personal/zoltan_ujhelyi_incquerylabs_com/_layouts/15/guestaccess.aspx?folderid=0a777bf0709cb4b5d893712665e2e93c0&authkey=AcHi4rOuqeiGj11oZy7Ku0A

Conclusion

The results were similar in all cases: there is no single location that is responsible for a large amount of time, the time is spread out between various components, such as SWT, MDT-UML, Papyrus services etc.

=> This usually means it is not possible to enhance performance significantly with localized changes, but some overarching changes are necessary.

Back to the top