Jump to: navigation, search

WTP Performance Tests/2008-07-28

Attendees

Gary Karasiuk Y
Angel Vera
Mark Hutchinson Y
Nick Sandonato Y
Carl Anderson Y
Kaloyan Raev Y
Raghu Srinivasan
Neil Hauge
David Williams Y

Agenda

Review action items from previous week and define our next step

Minutes

Running a single performance test
Gary tried to run a single performance tests by following the instructions of the Platform team. Everything is successful until the point of writing the results in the Derby database, inclusively. The problem is the next step of generating graphical comparison of the results.
As promised I spent some time looking into how the platform runs their performance tests. I think I got about as far as Kaloyan did. I was able to write and then run a performance test. This test wrote results into a Derby database. I was able to examine the database and verify that the results were written correctly.
I then spent a significant amount of time debugging the org.eclipse.test.performance.ui.Main application, to try to get it to generate some graphs. This program is very specific to the platform (e.g. they hard coded in a number of standard eclipse components) and would take some work to make it usable by others. I was hitting various NPEs while trying to run it against my database. In the time that I had set aside I couldn’t get it to generate results.
The graph generation tool has a lot of hard coded parameters. We should generalize the tool, so it can be configured and used for WTP.
Nick volunteered to proceed with Gary's efforts and try to run the graph generation tool.
David asked if we need generating such graphics. Gary replied that such graphics are vital part of analyzing performance tests result and we should achieve this step before moving forward.
WTP performance wiki page
The current WTP Performance Tests wiki page has some outdated contents. Kaloyan has reused the page for the needs of the Performance team.
We should remove the Performance Tests Owners section, because it makes no sense if we do not execute performance tests. We have the Team members table that represents the people involved with the performance tests at the moment. The owners list could be built again in the future.
Other
There is an interesting article about automating PDE JUnit tests.
It seems that the described techniques are different than those used in the WTP performance test procedure.
Kaloyan has asked the authors if they can extend it with describing how to automate PDE performance tests.

Action items

Nick - try running the visualization tool
Kaloyan - refactor the WTP Performance Tests wiki page

References

How to Write an Eclipse Performance Test
Sample Eclipse Performance Results
Article Automating Eclipse PDE Unit Tests using Ant