TPTP-PMC-F2F-200803

Starting Logistics

Attending: Harm, Oliver, AlexA, MikhailV, Chris, Joanna(phone)

Regrets: Paul, AlexN

• Joanna is on the road but has no time constraints
• Oliver asks if there are any additional agenda items?
  • Chris wanted to add an item about a specific enhancement

Action Items summarized

See full context below for details. I've tried to catch all the action items in one place for easier tracking:

• Joanna to confirm that the issues (for IBM related to 148629) are long term so that if there does end up being a need to defer due to legal or technical difficulties that it will not cause a panic.
• Eugene to try out the (SAP) heap analyzer on trace leak issue
• Harm to fwd ((SAP) heap analyzer) details to leakbot team at IBM
• Chris to fwd ((SAP heap analyzer) details to Intel memory profiler guys (Sri)
• AlexA and AlexN to cooperate to define responsibilities and metrics for Richard (ensure completion of Trace and Monitoring requirements)
• All leads (AlexA, Paul, AlexN, Joanna) to create a strawman proposal for items that should be under your project. Probably use AG for followup discussions before we take official action.
  • Sample question: where for example does probekit fall?
• AlexA/Joanna/Mikhail to discuss near term efficiency ideas related to profiling management of resources between IBM/Intel
  • Bring joint proposal to pmc and just make it so.
• Harm to create page on wiki and send pointer to team (erikson and co) to let them start brainstorming in wiki
• AlexA to continue driving the POG engineering effort
• Joel to come to to PMC and discuss results (of trying to get J2EE bundle packaging/builds to work together)

Discussion of 4.5 Release

Chris raised question about stack map attribute enhancement request

• A few weeks ago IBM mentioned that they really wanted this to be in the release.
• We wanted to clarify that this defect is actually about forward looking support for when VMs do not support backoff verification ncode for missing stackmap attributes.
• Wanted to find if there is some immediate consumption from IBM for this enhancement
  • Is there a Java6 JRE that does not implement the fallback verifier for missing Stackmaps?
  • Or is it just risk of it going away at some future Java
• Feature is currently on track but engineering is not yet 100% and IPzilla still needs approval so there is some risk.
• Harm mentioned that this was one of the few bugzilla entries that spoke directly to Java6 support and that IBM wants to be sure of Java6 support if 4.5
  • Does not think that this particular feature is critical but will check to verify
  • Joanna raised question regarding line level coverage
    • There was a problem with Emma at one point that might have required this feature.
    • Harm noted that there are folks that use Emma today.
    • Joanna noted that code coverage seems to work for 1.6 now.
    • However, it is not really supported by developers
    • There is no commitment to get support for Java6+ (probekit) for line level coverage.
    • IBM builds line level coverage on top of TPTP
• AI to Joanna to confirm that the issues are long term so that if there does end up being a need to defer due to legal or technical difficulties that it will not cause a panic.

Question whether this is the same of different from the issue around probekit using deprecated byte codes.

• Different Item

Oliver asks if we know what big items need to fall off the list?

• Leads should be working thru the process of identifying items
• Joanna and Chris should be involved when removal happens

Staffing is open/questionable on this feature.

• Staffing issue (committers resigned)
• Tivoli trying to find resource but if they cannot then feature will not be in.
  • Oliver asks if IBM understands and agrees to risk
  • Harm says that they have to be okay with it because there is no choice
  • Intel has no dependence on this bugzilla.

Enhancements slipping out of I6

• Filtering (updates to predefined filters) and make them more generic and "obvious"
  • Enriching command line will probably involve adding more parameters
  • This will make command lines more complex
    • Will need some way to simplify the command line
• JVMTI profiler documentation and command line help/wizard

Team discussed possibility of deferring some of the documentation and command line wizards to 4.5.1

• Too early to tell what translation support will be in 4.5.1
• Could in theory do documentation part on the web site and roll in when translation support is provided
• Alternately, can point to wiki documentation and bypass translation processes altogether
  • Some other projects are doing this.
  • Would enable us to be most agile.

For documentation that team will provide, we need to figure out how to review for English as second language.

• In discussion there was a general belief that if it is a reasonable amount of content and we have a few weeks notice we can probably get reviews/edits from folks on team with English as a first language.
  • Easiest way would be to do it on the wiki and let us just edit it in place.

Andreas arrived so we tabled 4.5 discussion.

Memory Analyzer Discussion

Andreas (SAP) came in to discuss their memory analyzer that extracts interesting information from heap dumps and visualizes it.

If one has a 4g heap dump and uses brute force techniques will need 4-5x that much space for analysis. They reduce this overhead with intelligence in parser/indexer. Reduced amount of information available but makes it quite efficient.

• classloader details, lifetimes
• dominator tree to identify heap details (probable heap reduction from freeing objects)
  • Identify what objects maintain references, sizes, classloaders, etc, possible bad guy.

Andreas provided a demo of a 2.4g heap system.

• currently tied to sun format
• talking also (for example) to IBM J9 guys for other heap dump formats
  • IBM vm missing some information about GC roots
  • J9 open to changing.
  • AI to Eugene to try out the heap analyzer on trace leak issue
  • AI Harm to fwd to leakbot team at IBM
  • AI Chris to fwd to Intel memory profiler guys (Sri)

Everyone was quite excited. Project has a finely tuned data model for indexing heap data. Unclear how much the models for this could be fully integrated with models for Trace. There was discussion around when and how to incubate the project. TPTP is definately a possible host for the effort. The suggestion of talking to the JDT guys too also came up.

Netbeans Comparison

OC systems engineer dialed in and discussed their initial comparison of Netbeans to Eclipse profiler. He did several demos and then we discussed and gave several ARs for followup.

Questions that arose:

• can one run with command line mode and then do post-hoc analysis w/ netbeans?
  • unsure... looked at web documentation... did not find such a mechanism
• Is the data stream in a format where it can be externally consumed?
• Does the data stream consist of aggregated stats or full data streamed
  • Suspect aggregation since there are no views that show overall execution sequences that would require full sequence.
• Do we have any idea of the overhead of their data transport model (e.g., sockets, sharedmem/etc). We wonder if there is some magic there.

Ability to attach to a running JVM w/ no command line options in netbeans

• Chris noted that before Java6 Netbeans would ship with a custom Sun JVM to allow this.
  • In Java6 Sun partially documented in the JVMTI spec the feature that the JVM would need to implement to get this attach to work. Unfortunately the specification is incomplete so it is unclear how other VMs could implement the functionality.

Working Globally

Oliver had asked each lead to come prepared to discuss methods they use for working with globally distributed teams. Oliver asks: You have people that you work with all over the world. How do you deal with the folks that you only see virtually?

Platform does not have a huge problem. The majority of engineers involved report directly to Joanna

• One exception is Eugene and there are no communication issues there
• Also the Intel folks do not report to Joanna but she is not having issues there right now either
  • A few times has asked Chris to get involved but those times are rare.
• The largest examples of concern related to code analysis team in ottowa
• Paul had seen some difficulaty because although he is the TPTP lead for his project he is not a manager at IBM
  • Caused some confustion nin relations
  • Joanna has had to step in and explain role to IBM mgt chain so that Paul is paid attention to as he should be

Oliver asks: How do you know if someone is fulfulling their potential? (i.e., is 1 or 2 bugs a week good or not?)

• Joanna runs reports about bug fix rates and passes to mgt chain
• This process calls attention to outliers
• Typically when the numbers are off it means that someone is working on a single large big defect
  • Otherwise (rare) she will followup.
  • In severe cases @IBM would cc Harm to get additional questions asked
  • W/ Intel regular meetings used to raise most problems. In a few cases have involved Chris (quite rare)

Paul apparantly has the team do some tracking of hours worked on defects within their bugzillas to call out defects that are taking "too long" to complete and give collateral for discussion.

Joanna looks at imbalances in upcoming defect backlog and reassigns defects accordingly

Oliver asks what metrics Joanna uses to track Intel team.

• AlexA owns has majority of platform defects
• Some rerouting of defects between Igor and Stanislav (Agent controller support)
  • This is becoming more clear nowadays because Stanislav is on the AC now and Igor on profiler

Oliver asks what Joanna knows about AlexN's processes

• As far as she knows, they are similar to what Joanna does for platform

Oliver doesn't want to break stuff but asks if it would be valuable for leads to gather the same metrics.

• Not all folks are diligent about sizing defects (some over some underestimate)
• deferral rates might be interesting
• Chris suggests that at the very least leads should exchange their methodologies for tracking

Resourcing Discussion

The question of what constitutes 10% of Joe toomey's time came up.

• In the end rational and tivoli talked and came up w/ Richard to support Joe's feature
• There was argument that got the 10% of value from Toomey was his discussion on test project defects
  • Team discussed fact that in the plan Joe's 10% was to pursue a specific action item.
  • Spending time on another project, while a real portion of his time doesn't really accomplish the original assignment.
    • Chris commentary as he transcribes this... -- Perhaps need to be a bit better about annotating specific line items/expectations for 10%

Richard is 70% TPTP

• Time is split between monitoring and trace projects
• This split makes it difficult to track whether (a) aggregate 70% is done and (b) sufficient time is saved for accomplishing trace deliverable
  • AI: AlexA and AlexN to cooperate to define responsibilities and metrics for Richard

Component Assignments to Projects

The question came up asking if we are being sufficiently efficient in our project management. Is there extra unnecessary cost associated with managing projects from a different site where the engineering is done?

• It is time to review association of plugin components to projects
• Platform used to be where things used by more than 1 project
  • We may be taking this rule of thumb too seriously
  • For example, there are other projects that use trace but the vast majority is currently in profiling project
    • There had been an intent to use more of the platform trace views across test, monitoring, etc
    • intent was not completely followed thru and does not really have resourcing today.
    • Monitoring does use some of it(trace) but vast majority is profiling related
• The platform has gotton huge and it is time to discuss best location for stuff like AC
• It is somewhat strange to have AlexA leading Trace where pretty much all the resourcing is from IBM while most of the profiling work is done under Joanna's project with Intel resources.

Mikhail believes that with an active AG we can handle integration problems between the projects

• used to have these weekly forums but went away for a while

Are there specific points to fix now versus things to plan to do?

• Main buckets: profiler and test
  • Monitoring is also pretty separable
• Harm suggests that we don't rename plugins but start managing more of profiling stuff under trace

Harm okay w/ 3 basic components/use cases

• Test -- testing stuff
• Trace -- profiler stuff (gets bigger)
• Platform -- truely common stuff (gets smaller)
• Chris' commentary transcoding -- I think we also agreed that Monitoring should be a 4th bucket here.

AI to each lead -- AlexA, Paul, AlexN, Joanna: Create a strawman proposal for items that should be under your project. Probably use AG for followup discussions before we take official action.

• Sample question: where for example does probekit fall?

Mikhail notes that we could do this in two phases. Longer term bucketing design above is good but there are some specific inefficiencies now. Can we do a few critical tweaks now and strategic updates later?

• General consensus this is okay.
  • AI AlexA/Joanna/Mikhail to discuss near term efficiency ideas.
    • Bring joint proposal to pmc and just make it so.

Trace Consolodation Discussion

Erikson came in to discuss new project coming into Eclipse. Merging/correlating traces from different sources. Before F2F they sent out link to trace conference material. A group of primarily mobile companies note that they have a lot of common functionality... primarily for collecting/analyzing data streams (traces) of behavior of native apps.

Intercompany tracing summit used to start discussing non-java tracing requirements

• Engineers currently finishing up some common debugging components (will be done this summer)
• Team will then will pursue tracing
• There is not existing code (many Eclipse projects start from initial bulk contribution). Team is starting from blank slate to design approach and will then develop it in open source.
• Research project w/ universities is starting as well.
  • hope for some synergy
  • Team also doing linux next gen tracing LTT NG (a data source for one class of trace/data stream)
  • This has involved mods to the linux kernel

Harm talked a bit from looking at wiki

• does both stack trace logging as well as logging of system behavior
• main component is tracing itself including logging from static markers in kernel.

Question: Does team envision a trace as a full sequential sequence or a statistical aggregate?

• eventually a framework should handle both
  • there are many layers : os, hypervisor, etc.

Team is interested in more than just low level devices.

• examples, combine with data from additional data sources (e.g., memory analyzer) and correlate.

When and what do we do?

• They intend to go talk to other vendors to collect details
• AI Harm to create page on wiki and send pointer to team to let them start brainstorming in wiki

NOTE -- I DID NOT CAPTURE THE OTHER GUYS WHO CAME IN LATER... PLEASE FILL IN...

Planning Oliver Report to Board

Oliver will report to board regarding TPTP on June 18th

• Everyone agrees community is important but are hesitant to pay for the overhead of coordinating w/ different company teams
• Some questions have come up regarding review by companies before/after plan/design. To some extent companies need to give up control.
• Oliver is in control for the presentation :-)

Fundamental question for TPTP: Are we a target for the community at large to use directly or are we simply a vehicle for consumption in products?

• TPTP believes that TPTP as a component directly usable by end users is a key component
  • Examples, POG/usability, etc.
• Ideally will get board to say that TPTP seems to be doing the right things in this space

It is possible IBM perspective would be that it is not desirable to resource anything unless it is for a consuming product. Further discussion around this may be needed.

Question: Why ask board this question given we know that they are of two minds on this general topic already?

• We are seeing companies who want all these community things built up and working but they dont want to pay for them w/ resources.
  • Question nto the board: What are you doing to increase desirability of participating in community.

POG Effort

Oliver asked Harm to summarize his perspective as a lead user

• Big blocking defects detected from first few passes pretty much okay now
• Team is now touching on bits and pieces of data correctness questions
  • e.g., reports that certain conditions don't make sense.
• By end of I6 will have done things targeted for 4.5 that impact UI
  • Harm likes the idea of using wiki for documentation to avoid formalized translation deadlines
• Harm has been spending a bit of time looking at model/loader for opportunities

AI AlexA to continue driving the POG engineering effort

• Triage all existing defects
• Keep fixing important defects

Intel performance team planning April-June to stress with their use cases. Hope to uncover

• Some more low hanging fruit
• Some long term requirements

Because of Ganymede rampdown and translation deadlines, we will be limited in how much we can target in I7/I8. How do we deal with lead user issues in this 4.5 timeframe?

• We will need to be careful
• I7/I8 development where possible
• Offload some documentation to wikis (bypasses translation)

We had a discussion about tuning default settings for profiler to be the "best recommended" settings out of the box.

• There was some confusion about IBMs position here. Some thought that IBM wanted default to be legacy (e.g., default to XML instead of binary).
• Harm clarified IBM position
  • Should be possible to configure legacy mode (e.g., XML)
  • When old workbench or old AC is used that does not support "new better mode", should automatically detect and fallback to legacy
    • Harm believes that the design of current features accomodates this. However, if this autodiscovery of legacy cannot be accomodated we should talk...
  • Subject to these constraints, the defaults for Ganymede workbench w/ Ganymede agent can be tuned to be optimal for our use cases.

EPP Bundling Discussion

J2EE developer EPP bundle owner came in to discuss logistics for including TPTP in J2EE developer bundle. Part of discussion was a review of items that we already knew.

Goal: TPTP as part of EPP for J2EE developer bundle

• This bundle currently includes Mylin
• some subset of TPTP good for profiling and/or junit testing would be a good value add

requirements

• Decide which features to include
• Configure so that they can be pulled automatically from ganymede update site
• In addition
  • Document (preferably with a tutorial) the key use case scenarios
    • Use test cases are things that we say we do well today.
    • Some of these tutorials already exist.
    • Where should tutorial go? our website or elsewhere?
    • Tutorial on our website probably sufficient.
    • Question came up regarding tutorial content that should show up on welcome page.
  • someone who tests on weekly basis (each integration build)
    • Currently there is wiki page in WTP where people document test updates
    • Cost guestimate: ~30min/wk ... May actually be the same as existing testing especially if we start pulling from J2EE bundle for our all in one builds.
    • Before each milestone, someone goes and stresses it.

EPP is home site for the builds

How to deal with native code within the targets?

• Problem: Want to only download native bits that correspond to desired platform
• Update manager tends to be poor about accomodating platform specific code in subprojects
  • You will get features thru update mgr
  • There is work underway to get targeted platform sensitivity
    • EPP is all in one package
    • EPP makes zips for each platform (windows, linux, linux-gtk, possibly mac)
    • zips are made via update mgr so it might "just work"

Joanna will try to get Joel to pull from JEE EPP and see how close it is to working

• AI for Joel to come to to PMC and discuss results

From build point of view because of the way the EPP bundles are built from day to day perspective from the update site, we may need to tweak how (frequently) we upload our builds

• May need to start dropping more builds up to site.
• Should not be a bit problem because can update the builds that are uploaded any number of times.