TPTP User Experiences Profiling

From Eclipsepedia

Jump to: navigation, search

Note that when you edit a page on the wiki, there is a checkbox you can click to be notified of updates to the page. E-mail notification should be turned on at your Wiki Preferences, "my preferences" at top of page. I'm not 100% sure if it works but if it does, it will be a good way to avoid having to cc a mailing list whenever you update the wiki...


Please use Discussion page for suggestions, questions and thoughts


This page discusses user types that we are hoping to solicit feedback regarding user experiences. In particular, we hope to get pointed feedback and make TPTP better for these classes of users.

Contents

Come to TPTP as Lead user

The tptp profiler

J2SE 5 and J2SE 6 are supported in TPTP by the new JVMTI interface while J2SE 1.4 and below are supported using the old JVMPI interface. The components of TPTP profiler that are evolving the most relate to the JVMTI profiling experience. If you don't know what version of Java you are using, please use java -version.

The tptp team is undertaking an effort to improve the usability and out-of-the-box experience for the profiler. We are going to do that by working closely with some lead users. The term lead user comes from the work of Eric von Hippel at MIT. The lead users will use the tptp profiler and will be closely supported by the tptp team. The lead users will get usage (and bug) feedback to the tptp team, who will expedite the resolution of issues. The lead user would expect to get a number of deliveries of the profiler during the course of their usage, each improving on the usability and out-of-the-box experience. The lead user may be given prototypes to try. It is anticipated that tptp will work with a number of lead users. Our desire is to get an improved user experience into Gannymede.

There will be two related efforts for this:

  • J2SE, and
  • J2EE

The primary difference in the use cases is that a J2SE application reinvokes the JVM upon each execution whereas a J2EE application runs on an already running JVM, so must attach.

Ideal J2SE Candidate Criteria For This Survey

Here are the characteristics of the ideal J2SE lead user.

  • J2SE
  • The lead user's application runs on the tptp reference platform: Windows XP and the IBM JVM 1.5. We will also consider other 1.5 JVMs.
  • To make debugging and support simpler, the lead user's application is easily executed by the tptp test team on their system (the reference platform). This implies that there should not be a lot of external "strange" dependencies in the lead user's application, as the tptp team will not be able to easily recreate the lead user's environment.

If you would like to be a lead J2SE user, please contact Eugene Chan directly by e-mail.

Below is a more formal definition of "strange" for a lead user's project. These are not mandatory but help understand our position. Anyway, please contact, and we will try to find convenient cooperation process.

(in priority order):

  • Doesn't require third party's software
  • Can work on single computer, doesn't require distributed environment. Can access the Internet.
  • Easy to deploy
  • Isn't a very specific application field. The purpose and use cases are understandable without special knowledge. This will result in a short time for the tptp team to familiarize themselves with the application.
  • Absence or minimum of native libraries
  • Can be downloaded from CVS and built from scratch in Eclipse

Ideal J2EE Candidate Criteria For This Survey

  • J2EE
  • Optionally a WTP user
  • Tomcat (version 5.5 or newer and therefore Java 5) as the application server. Do we have other requirements?
  • Otherwise the same as the J2SE case above.

If you would like to be a lead J2EE user, please contact Eugene Chan directly by e-mail.

Our expectation from this effort

In brief, we want to improve the usability and out-of-the-box experience for the profiler for the next major iteration of Eclipse, called Gannymede, currently scheduled for general availability June 25th, 2008. We anticipate that the increased usability and out-of-the-box experience will increase the number of users of the profiler.

The TPTP team is using an iterative development process, which has a number of interim releases leading up the the June 25, 2007 delivery. The detailed TPTP schedule can be found at http://www.eclipse.org/tptp/home/project_info/releaseinfo/4.5/schedule.html.

The dates of interest for this effort are the M releases, specifically, M4, M5, M6 and M7. The lead users are expected to try each new M release and write down their experiences and suggestions (and, yes, problems) for each M release. For example, the next M release is January 4th (M4). It is expected that each lead user download and try the tptp M4 release and get their feedback to us. The lead user should be able to be done pretty quickly with this effort. Of course, the lead user is encouraged to continue to use the profiler and get feedback to us, but that is up to the lead user. We will notify you soon before each M release to schedule some time to try it out.

The TPTP Profiler is a general purpose tool which provides a broad range of functionality for applications tuning. It can do things such as execution times, heap usage, threading, filtering a narrow set of events, specific requirements for data depiction (e.g. reverse call tree) or other specific analytical features.

In this effort, our goal is to define a small set of clearly defined high-value use-cases in collaboration with Lead Users and focus on the usability and out-of-the-box experience for these few high value use cases. The use cases will be specified and documented in Wiki so the lead users can collaborate to get the best input.

Brief list of expectations for the Lead Users.

  1. Help define what are the high value use cases. This can be quite simple for the lead users: just tell us the task that you need to do!
  2. Work with the tptp team to define how tptp should implement the use cases. One of the efforts coming out of this will undoubtedly be targeted documentation, but is quite likely that the tptp software itself should be enhanced.
  3. Close collaboration for product evaluation; we may want to get you quick deliveries to try out.
  4. Use Bugzilla for your problem reporting. Place the string "[POG]" as the beginning of the bugzilla description, so we can properly track them. If you are unfamiliar with bugzilla, we will help you use it (it is not hard). Yes, POG does stand for "Profiler of the Gods".
  5. At least a few hours to try each M delivery and then report of it's usability and out-of-the-box experience.

Getting started with TPTP

Information Sources

Stable TPTP usage scenarios

Profiling types

Performance issues which can be identified with TPTP

Known TPTP profiler weaknesses

Some User Initial Experiences

As users come forward, we want to capture their initial experiences. We hope that some of these users will choose to become "lead user's" and migrate to the next section. :).

Profile on Server and Enabled mode Collection

A number of folks are trying to use TPTP to profile Tomcat and/or RCP applications and are launching/starting their app either from within the workbench or from a command line script. The summary of one user's initial experiences with the TPTP 4.4 profiler for a Tomcat based application are here

Using TPTP to profile JRE class libraries

JRE as all other application can have performance issues and hence area for its improvements. It consists from native and managed code, and portion of the last one is much bigger then often expected. So, for that portion we can use TPTP as profiler and localize with its help potential performance or resources usage issues. Also one of the goal is to estimate possibility of Java profiler usage for JRE profiling. Harmonyis used as target JRE. For details watch page Harmony profiling with TPTP

Using TPTP to profile Eclipse Platform UI

There isn't many details yet. All initial input regarding user experience Eclipse Platform UI with TPTP

Using TPTP to profile Eclipse Plugin

The initial case of profiling Eclipse plugin for eclipse plugin developer.

We take TPTP itself as a example and profile Eclipse plugins to get big statistics filesProfile Eclipse plugins,

Then, use TPTP to profile itself in loading and viewing large statistics result file to identify the performance bottleneck. Profile TPTP itself

Accepted Lead User's Use Cases

These use cases have been accepted by the tptp team as part of the profiler initiative. If you feel that you too would like to try this use case, please let us know. We would love more than one user for each use case, as that will better and more efficiently get results.

J2EE Scenario: Using TPTP in a hybrid situation

The basic premise is that a developer wants to use SVG and Batik in their application. In order to get a sense of the responsiveness they try the SVG jspx example that comes with Tomcat. They will notice a delay when using the Batik squiggle browser which is a Java application itself. This will require profiling of the jsp as well as the squiggle browser. Hybrid TPTP scenario

User guide

As a part of initiative we are going to develop brand new user guide for TPTP profiler.

Meanwhile this is under development (actually on very beginning).

Here you can find draft or something that just under development Profiler User Guide

Also please don't forget about Eclipse help section "Profiling an application"

Contact points

Places where you can ask questions, discuss and of course proved feedback for the tool:

We get all information from these sources, analyse and put structured in Wiki. This is important for us and even small observation which published and discussed can affect overall quality and create product more attractive. So, please participate in community process and make TPTP better.

We have had some meetings via telecon to pursue this.