Difference between revisions of "WTP Performance Tests/2008-07-28"

From Eclipsepedia

Jump to: navigation, search
(Attendees)
(Minutes)
 
(5 intermediate revisions by one user not shown)
Line 33: Line 33:
 
== Agenda ==
 
== Agenda ==
  
:
+
:Review action items from previous week and define our next step
  
 
== Minutes ==
 
== Minutes ==
  
:
+
;Running a single performance test
 +
:Gary tried to run a single performance tests by following the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html?view=co instructions] of the Platform team. Everything is successful until the point of writing the results in the Derby database, inclusively. The problem is the next step of generating graphical comparison of the results.
 +
::''As promised I spent some time looking into how the platform runs their performance tests. I think I got about as far as Kaloyan did. I was able to write and then run a performance test. This test wrote results into a Derby database. I was able to examine the database and verify that the results were written correctly.''
 +
::''I then spent a significant amount of time debugging the org.eclipse.test.performance.ui.Main application, to try to get it to generate some graphs. This program is very specific to the platform (e.g. they hard coded in a number of standard eclipse components) and would take some work to make it usable by others. I was hitting various NPEs while trying to run it against my database. In the time that I had set aside I couldn’t get it to generate results. ''
 +
:The graph generation tool has a lot of hard coded parameters. We should generalize the tool, so it can be configured and used for WTP.
 +
:Nick volunteered to proceed with Gary's efforts and try to run the graph generation tool.
 +
:David asked if we need generating such graphics. Gary replied that such graphics are vital part of analyzing performance tests result and we should achieve this step before moving forward.
 +
 
 +
;WTP performance wiki page
 +
:The current [[WTP Performance Tests]] wiki page has some outdated contents. Kaloyan has reused the page for the needs of the Performance team.
 +
:We should remove the ''Performance Tests Owners'' section, because it makes no sense if we do not execute performance tests. We have the ''Team members'' table that represents the people involved with the performance tests at the moment. The ''owners'' list could be built again in the future.
 +
 
 +
;Other
 +
:There is an interesting [http://www.eclipse.org/articles/article.php?file=Article-PDEJUnitAntAutomation/index.html article about automating PDE JUnit tests].
 +
:It seems that the described techniques are different than those used in the WTP performance test procedure.
 +
:Kaloyan has [https://bugs.eclipse.org/bugs/show_bug.cgi?id=140503#c33 asked] the authors if they can extend it with describing how to automate PDE performance tests.
  
 
== Action items ==
 
== Action items ==
  
:
+
:Nick - try running the visualization tool
 +
:Kaloyan - refactor the [[WTP Performance Tests]] wiki page
  
 
== References ==
 
== References ==
  
:
+
:[http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html?view=co How to Write an Eclipse Performance Test]
 +
:[http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/performance/performance.php Sample Eclipse Performance Results]
 +
:Article [http://www.eclipse.org/articles/article.php?file=Article-PDEJUnitAntAutomation/index.html Automating Eclipse PDE Unit Tests using Ant]

Latest revision as of 13:02, 28 July 2008

Contents

[edit] Attendees

Gary Karasiuk Y
Angel Vera
Mark Hutchinson Y
Nick Sandonato Y
Carl Anderson Y
Kaloyan Raev Y
Raghu Srinivasan
Neil Hauge
David Williams Y

[edit] Agenda

Review action items from previous week and define our next step

[edit] Minutes

Running a single performance test
Gary tried to run a single performance tests by following the instructions of the Platform team. Everything is successful until the point of writing the results in the Derby database, inclusively. The problem is the next step of generating graphical comparison of the results.
As promised I spent some time looking into how the platform runs their performance tests. I think I got about as far as Kaloyan did. I was able to write and then run a performance test. This test wrote results into a Derby database. I was able to examine the database and verify that the results were written correctly.
I then spent a significant amount of time debugging the org.eclipse.test.performance.ui.Main application, to try to get it to generate some graphs. This program is very specific to the platform (e.g. they hard coded in a number of standard eclipse components) and would take some work to make it usable by others. I was hitting various NPEs while trying to run it against my database. In the time that I had set aside I couldn’t get it to generate results.
The graph generation tool has a lot of hard coded parameters. We should generalize the tool, so it can be configured and used for WTP.
Nick volunteered to proceed with Gary's efforts and try to run the graph generation tool.
David asked if we need generating such graphics. Gary replied that such graphics are vital part of analyzing performance tests result and we should achieve this step before moving forward.
WTP performance wiki page
The current WTP Performance Tests wiki page has some outdated contents. Kaloyan has reused the page for the needs of the Performance team.
We should remove the Performance Tests Owners section, because it makes no sense if we do not execute performance tests. We have the Team members table that represents the people involved with the performance tests at the moment. The owners list could be built again in the future.
Other
There is an interesting article about automating PDE JUnit tests.
It seems that the described techniques are different than those used in the WTP performance test procedure.
Kaloyan has asked the authors if they can extend it with describing how to automate PDE performance tests.

[edit] Action items

Nick - try running the visualization tool
Kaloyan - refactor the WTP Performance Tests wiki page

[edit] References

How to Write an Eclipse Performance Test
Sample Eclipse Performance Results
Article Automating Eclipse PDE Unit Tests using Ant