TPTP Sep 07 face to face

TPTP Leadership

The TPTP PMC Face to Face meeting has ratified the following roles and responsibilities. (unanimous approval):

PMC:

• Oliver Cole -- lead
• Harm Sluiman
• Chris Elford

Project Leads for TPTP are:

• Test Project – Paul Slauenwhite
• Platform Project – Joanna Kubasta
• Monitoring Project – Alex Nan
• Trace/Profile Project – Guru Nagarajan

AG lead – Paul Slauenwhite Planning and Requirements – Joanna Kubasta

Nomination of Paul as AG lead seconded and passed unanimously. Paul to represent TPTP to Eclipsecon Program Committee – Congratulations Paul!

AI: Everyone provide BIO update to Paul for web site update

Upcoming Release Schedule

Old “4.4.1” schedule for shutdown testing becomes 4.4.0.3 shutdown test schedule with new Europa driver.

• Regression Test with latest Europa milestone driver
  • 1 week
  • Mon 09/17/2007-Fri 09/21/2007
  • Smoke Test
• Regression Test with Europa Maintenance Release candidate
  • 1 week
  • Mon 09/24/2007-Fri 09/28/2007
  • Smoke Test
• 4.4.0.2 release driver becomes candidate driver for 4.4.0.3
• 4.4.0.3 is 4.4.0.2 tested on new Europa base (3.3.1)
• No patches (unless some Europa update failure triggers it…)
  • PMC approval definitely required for any exceptions
• Each individual project decides which smoke tests to run that are appropriate.

AI: Alex to get list of all emergency defects for Monitoring by Monday noon (Eastern)

Version to coincide w/ Europa winter update (Feb29)

• The name for this release is 4.4.1.
• PMC approval required for any addition
• Intent is to include security update and issues related to IBM JVM update coming

AI: Joel to split CVS now calling it 4.5 for now. May decide to “split”/rename again later calling it 5.0.

TPTP 4.4 Post Mortem

From outside

• point deliveries
• lots of churn

to address Paul noted had to lean on committers more than previously

• Paul came on in 4.3 and 4.4. Doesn’t have long history to see if it is getting worse or if it is better.
• Oliver makes a point that lead is not necessarily easy and is not necessarily being the friend.
• Paul notes with most people, pushing works and makes things better. Oliver offers to talk to specific places where there is a gap.
• You can’t micromanage. It is not an efficient use of time.
• Haven’t aggressively kept lines of communication open with managers in the past.
• Harm notes that it is fairly recent that project leads are not part of management chain of the engineers.
• How applies to mgt. chain.
• In places where there are issues discuss time versus service agreement.

Stopping iterations – how do we stop it.

• We took a risk in Jan by cutting the ties on the old/new AC and old/new profiler
• We made the right call but didn’t have full knowledge of impact.
• Revisited in March/April –
• Assumptions made regarding use case assumptions between different subteams. – We didn’t have PI guys testing TI and vice versa and old AC guys testing new AC and vice versa.
• Distinguish between developers and testers for given projects.

Does it make any sense to merge testing strategy for whole TPTP into one team.

• Joanna had tossed idea around of asking for a student to facilitate some of this.
• Testing can cause burnout; automate as much as possible to help.
• Goal: get to point where we smoke testing every third week potentially.

One reason test passes take so long is because they fail.

• Potentially change models to help discover bugs sooner in proess.
• Oliver asks if using “Test” project s harness for automated/nightly could help here.

Consumers of TPTP test using their own scenarios/use models

• Oliver asks if this is replication of effort and if test project is being used.

Could try to use test for : current automated tests, junit tests, properly wrapped C++ automated agent controller tests

• AI: Paul & Kiryl to talk about using Test to wrap automated AC tests.

Who owns the role of isolating issues once a failure occurs.

• Developers will need to own resolution on failures

Propose build team as owning automated test initialization (smart way to automate tests)

• Oliver to meet with some folks during PMC meeting

AI: Alan, Guru, possibly Alexander -- Need to have test for PI & TI sit down and discuss coverage and overlap of the different test suites.

Bring back technical meetings: AG calls. – (9a pacific on Fri – potential other times) AI: Paul to post on mailing list a plea for times AI: come back with potential time for AG call to pull in more attendees.

Potential to merge TI,AC,Platform meetings

• Project now smaller than when all the meetings made sense.
• AI: Joanna and Guru to sync on doing full merge.

Ad hoc meetings are a useful thing but…

• Outcome needs to be documented/ratified so that key points/decisions are not “lost” in the shuffle.
• Spin off work that comes back and reports back to meetings.
• AI: Leads to make sure results of the ad-hoc discussions are made public

AI: Oliver to have discussions w/ mgt teams about stabilizing contributions. Potential value from Oliver road trip.

Eclipse users – Want to profile the workbench External users – Want to profile other apps (e.g., tomcat, etc)

• For external users need more emphasis on capturing alpha/beta users to provide true feedback.

May have to make a hard call to decide between needs of External users versus Eclipse users

• Be good for a smaller user base.

Overall objective is to be able to characterize “large” Java apps.

Question: Are there any IAC test cases anymore? -- yes

Question, can Abbot/junit be tweaked to run with TPTP.

• If Junit was going to be around then this would be a potential target audience since Abbot has a big community. Since not, no pursuit warranted at this time.

Question: Are we handling rampdown effectively. Definitely some issues “just happen” and they cannot be handled with more effective rampdown.

Question: Do we need to step back on a planning side to ensure that there are consistent understandings about level of focus for our release. Buy-in versus agree-to-disagree.

• Clear on what is targeting a release. Be more careful about what else is allowed to slip in.

Ohloh.net tracks asst. open source projects and committer contribution rates. It would be good to use this as a resource to help track which of our committers have gone defunct.

TPTP 4.4 Maintenance Release Discussion Items

Editors note: Much of the discussion until the horizontal rule below is defunct. We discussed 4.4.1 extensively and eventually decided on the release details above:

---

Meeting started with Discussion of upcoming releases 4.4.1 5 blocking or critical Entry points blocked on profiling

Statistical views results should be checked in now

• JMX, etc.

There is an issue running tests on “additional” platforms. There are some BIRT issues. Trying to determine whether it is the version of BIRT that we have committed to support or not.

• Fix already checked into BIRT. Will be shipped in europa 3.3.1 and available in next milestone drop for next week testing

7 P1s targeted to 4.4.1

We are obviously running into issues satisfying our support obligations.

• People not doing what they are committed to do.

4.4.1 discussion moving to afternoon today.

4.4.1 – Test – nothing 4.4.1 – Platform – 4.4.1 – Monitoring – pretty much done

• Remote logging tests (Z/OS, OS/400, AIX, PPC-64 linux) – will be done soon.
• Finished as of Thursday
• JBOSS
• Finished as of Thursday

Platform –

• 3 days to fix items
• Question whether they are important. Question about whether they are new regressions.

4.4.1 update – all issues for Trace/Profiling except 202550 have been resolved (IBM JVM specific issue to call out in release notes). Remaining issues are AC related.

Statistical agents --

Birt 2.2 versus 2.2.1 TPTP 4.4 GA w/ Birt 2.2. We now fail w/ 2.2.1.

1 critical against JVMTI 203018

Note to steve fransicso – to take 4.4.1 requiring an IBM VM upgrade. Or 4.4.0.2. If 4.4.1, make sure we clearly document the issue

OpenSSL was not used previously JSSE was used via JNI. If review of OpenSSL is necessary, this tells us we HAVE to ship 4.4.0.2.

Decision made – 4.4.0.2 ships w/ Europa SP1; look into reworking security patch to avoid openSSL dependence; 4.4.1 moves to Europa SP2 (Feb 29)

---

Editors note: Much of the discussion above the horizontal rule below is defunct. We discussed 4.4.1 extensively and eventually decided on the release details at start of document. The below details the NEW 4.4.1 [set to coincide with the winter Europa maintenance release]

4.4.1 iteration 1 – test when (a) J9 available and (b) security fix reworked (Oct)

• http://wiki.eclipse.org/Ganymede_Simultaneous_Release
• http://wiki.eclipse.org/Europa

4.4.1 iteration 2 – test when Europa SP2 driver available ~6wks before Feb 29

4.4.1 bugs case by case cut over based on PMC approval

What needs to happen to avoid 4.4.1 pullback issue in the future.

• A combination of decisions triggered a sequence of events that led to the OpenSSL popping up in a point release as a new API dependences. We will watch more carefully in future.

TPTP 4.5 Resources

IBM and Intel discussed the org structures that contribute to TPTP.

TPTP 4.5 Feature Adds

The team used a spreadsheet Media:TPTP-F2F-0709-xls.zip‎ to discuss features that have resourcing issues.

Intel discussed some suggested profiler enhancements for 4.5. See Media:TPTP-F2F-0709-ppt.zip‎.

Comments:

Discussion of Bug backlog in trace/profiling project

• AI: Guru to talk to TI profiling bug backlog.

Note: ensure that old agent doesn’t barf on receiving binary request

Question to other VM vendors about their methodology for StackMap.

Filtering gauge – how many classes are you going to be covering Nonexistant F1 help Nobody on board writing help/documentation. Cheat cheats instead of wizards.

• Potential to have them as “useful” tech previews.

Steps… Harvest mailing list and newsgroup for scenarios that tend to cause confusion

• Build cheatsheets for these items

4.5 discussion (enhancement ideas)

• Some profiling features
• Some monitoring apis
• IP v6
• Candidates w/ general agreement to make defunct (not GA anymore)
• See spreadsheet
• Demotion from GA to tech preview. Disclose on lists. Make items separately downloadable.

Potential TPTP 4.5 Feature Removals

The team used a spreadsheet Media:TPTP-F2F-0709-xls.zip‎ to discuss features that have resourcing issues. Areas of staffing concern

• Profiler UI, Probekit, perfmon, trace model

Test potential items to help w/ resourcing limits

• API recorder
• AGR
• Manual test
• Deprecate or not move forward.
• Essentially, only remaining test modality would be Junit
• Alternately, could cut down on supported (tested) platforms
• Linux X, WindowsXP; Java5
• Less work on other oses (e.g., Vista
• Questions about whether Vista should be default… nasty question about Java 7

Basis level of support

• How much to:
• Answer questions on mailing list (how to make something work, state something doesn’t work).
• Test cycles with new drivers
• Account it and apply remaining to bug fix.

Except for items on spreadsheet, Oliver asks if all leads have resources to cover support for other items

• There are some “promises” of potential resources to help cover on Monitoring
• There are some open questions about dev items for Test
• There is one ultra high priority enhancement item for Platform
• There are dependences from Profile on some top lines in spreadsheet

AI: all leads to come to PMC meeting Sep 26 w/ feedback regarding spreadsheet and line item coverage.

Cleaning up of 1.4 dependences so runtime can be 1.5 specific.

• Separate issue from targeting Java 1.4 vms (e.g., for profiler)

Today, because some components need 1.4, everything targets 1.4.

AI: Joanna -- triage rows 22-26 in spreadsheet to find out if there are potential consumers.

CBE (common base event) – If ship Java5 class files could impact adoption (e.g., folks shipping the CBE binary jar).

• Question to consumers – is a binary release capable of running on Java1.4 necessary in 4.5 timeframe.
• Motivation: Allow us to have our builds target 1.5 classes. Why? It reduces test load.
• Is it acceptable for them to build it themselves. If no, how much willing to pay for this

Testing Framework Update

Ratify developers work on Java6/XP/x86. Test cycles starting w/ Oct 4.4.1… Need grand plan from Alan regarding how to cross platform test in a non-redundant fashion.

• Documentation of reduction in overhead to help assuage fears that TPTP wastes time on excessive test cycles.

AI: Paul to indicate if a single reference platform that is checked before all checkins would simplify test & test cycles. If yes, AI: to Chris to double check if this platform is viable choice for profiling work. AI: Joanna -- Define assumptions when entering a test pass: Is it expectation of success on reference platform or expectation of success on all platforms.

AI: Joanna & Guru -- Discussion between Joanna and Guru teams regarding what types and quantities of bugs being detected by JVMTI standalone tests being done by Guru and co.

Ubertesting framework: October goals:

• Each team automates component tests that intend to be automated
• Tests that are already automated use existing frameworks (over time converge)
• New tests automated use common framework (“This” team owns defining the constraints on this common framework)

Eventual goal

• Hook into build framework to have them automatically run as part of build

AI: Paul to provide a date where framework is ready so that when tests are automated “correctly”, they will work within the harness so that they run within build framework.

AI: Paul/Jonathan to talk about plugging in test automation bits with TPTP Test harness.

Uncategorized Content

New committer processes… Joanna did not have access to create right for folks. Christopf was working on it.

• Agenda add: committer lists being straightened out
• AI: Each lead to poll potentially defunct committers asking if they want to be removed. Give them time to respond before blanket removal for lack of responsiveness.

AI: Oliver to nag Toni about removal of WSDM tooling over to COSMoS

Question for Oliver to ask at Ganymede F2F (EclipseWorld) – for Joel

• Are they going to have a new update manager? There were issues with previous one.

AI: Joel – follow up on update mgr item and follow up with Oliver if there is still a question for Eclipseworld Ganymede F2F.

AI: Each lead should send name, email of mgr of each team member to Oliver.

• One line item per contributor along with their mgr, contrib %, and their mgr’s email.