TPTP-Test-Tools-20080512

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Tony Wang
  • Jerome Bozier
  • Joseph Toomey

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• This week is the first week of I8 TP1 (target: May 23).
  • All defects require patch reviews, Stop Ship Template, and Project/PMC approval.

Testing

• We want to complete the I8 TP1 by Friday, May 16.

• The candidate driver will be announced by the Release Engineering Team later today.

• A note has been sent to the originators of all resolved defects reminding them to verify/close these defects before Ganymede releases.

• Please verify/close your resolved defects during the TP1:
  • Paul:
    • (Trace Project) Create Bugzilla component for trace reports.
    • TPTP test results are not reported correctly.
    • Incorrect project names in .project files.
    • Fix broken externalized strings for the Monitoring Project.
    • CTRL-S does not work in the XML Source tab of the *.rptdesgin editor.
    • Support capturing additional draw2d related events when recording user's interactions with a GEF editor.
    • No data is recorded when recording after a hung/cancelled test execution.
    • URL Recorder operational messages require cleanup.
    • BIRT Trace Report does not appear in the Trace Report wizard.
    • Instrument the Data Collection Engine to log native Common Base Events.
    • Incorrect dependencies in TPTP features when installing from the Callisto Discovery Site.
    • Warnings in .log file.
    • 4.4.0.3 copyright build report is incorrect.
    • Provide reference for CDT in Requirements section of the download page.
    • Update project sets for TPTP 4.5.
    • Refactor TPTP project sets for component reorganization.
    • (Vista) File paths are too long when unzipping all-in-one.
    • Add a 'Bundle-ActivationPolicy' header to all manifest.mf files.
    • Duplicate record error when import a CSV into an existing datapool.
    • sun.io.MalformedInputException when deleting a test asset in the Test Navigator.
    • (Functionality, Usability) Testing resources cannot be renamed, deleted and/or moved without breaking associations.
    • Update Platform section of TPTP Project Matrix.
    • Missing link.
    • Exceptions when opening the Platform documentation.
    • (Platform Project) Create Bugzilla component for statistical reports.
    • TPTP 4.5 release notes are empty.
    • TPTP test results are not reported correctly.
    • Leverage/integrate PIN for line level code coverage of TPTP's native code.
    • Update copyright dates.
    • 4.4 API change report is not accurate.
    • Reinstate features and 'empty' plug-ins for as-is components.
    • (Web) Test execution periodically hangs.
    • (Web) (Vista) Test execution terminates abnormally.
    • Workbench location wizard switches values for -arch and -ws.
    • Update copyright dates and translatability errors.
    • (Web) (Linux) Cannot not delete test assets in the Test Navigator.
    • Exception in the deployment editor when a referenced artifact is deleted.
    • (ASF) FileNotFoundException when interrogating test execution results.
    • Resolving a boolean value from a datapool always returns false.
    • Generating a URL test class in the root Java package causes a compilation error.
    • Datapool cells are not created for CSV ending with a comma.
    • Recorder Control view is not opened or has focus when a recording is started
    • Test Tools about.properties files contain incorrect descriptions.
    • Importing large datapool freezes wizard.
    • Move JUnit Plug-in test tools documentation to a separate plug-in.
    • Clicking duplicate topic headings jumps to the first instance.
    • Create TVT/enablement test cases for new Test Project UI.
    • Vista-specific error dialog when recording using IAC/IE on XP.
    • NullPointerException when deleting artifact location pairs.
    • Time out error message during test execution is not resolving.
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.
    • Deprecate JScrib and JScrib report generators.
    • Test Tools documentation I5/I6 smoke test failures.
    • System default browser is not used as the default browser for URL recording.
    • Extra .execution appended to the results property.
    • Remove references to internal APIs.
    • Exported datapool is incorrect.
    • Cannot import HTTP recording file.
    • (Usability) Logical and physical names for test assets is confusing.
    • Failures in /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.EncryptedDatapoolEditor_running.testsuite.
    • ClassNotFoundException when running ASF automation services without OSGi/ICU.
    • Duplicate wizards for creating a URL recording/test.
    • UI allows to launch non-profilable tests in profile mode
    • Leverage/integrate EMMA for line level code coverage of TPTP's Java code.
    • Fix broken externalized strings for the Trace Project.
    • (Platform Project) Remove duplicate/unused messages
    • (Trace Project) Remove duplicate/unused messages
    • Eventformatter does not correctly serialize the any properties.
    • Exception when de-serializing nested Common Base Event XML fragments/documents.
    • Remove legacy IBM/Rational comment/string references.
    • Remove Hyades references from logging examples.
    • API Recorder's Generate Test Suite dialog does not select a default selection.
    • (Usability) Switch to Test Perspective after creating a test asset.
    • Test.Execution.JUnitRunner.testsuite and Test.Execution.JUnitRunner.Bug186196.testsuite are failing.
    • Cannot install BIRT without WTP with Eclipse M6.
    • Connection refused when lauching tests on Linux.
    • IAC does not configure plug-in's pluginconfig.xml at startup.
  • Tony:
    • Connection refused on local host during test execution.
    • The 'Encrypt' option doesn't work with keyboard selection
    • "Paste" doesn't not complete normally
    • The challenge is not removed from the datapool when the last encrypted variable is removed
    • Datapool can not be created with more than one variable
    • The invocation for some deleted testcases can't be removed in behaviour tab
    • IAC can't record a Java application
  • Duwayne:
    • Test Navigator doesn't always show new assets
    • (regression) tptp:test service fails when using the suite parameter
    • Broken strings in ASF command shell adapter
    • Consumming Product: ASF Throws Modal Dialog Box.
    • SSL Recording not working, regression
    • Test Execution Fails
    • (Vista) HTTP Recorder requires work-around on Windows Vista for Internet Explorer
    • All In One TPTP Package Does not have Test Perspective
    • All-in-One build for Linux File Permissions Issue
    • Regression, test execution on Vista crashes the AC
    • IAC will not start on Linux IA32
    • Extra java.exe left running after each HTTP Recording
    • Documentation for HTTP Recording with IE 7
    • (Web Doc) IPV6 not supported for Internet Explorer
    • (Web Doc) IPv6 Limitation on Firefox
    • Regression, URL Test Does Not Get Generated
    • Import fails if project name contains special characters
    • Foreign Language RAC Install Error
    • Regression in RAC, Hyades URL Test fails to execute properly
    • HTTP Recorder does not work with IAC in TPTP 4.3.1 and 4.4
    • non-externalized strings in org.eclipse.hyades.execution
    • IAC broken for URL Tests on 4.3
  • Jerome:
    • persisted proxy fail to load
    • log viewer : change adress preference for defect is not applied to already entered defect
    • lack of error message when remote execution fail

• Please use the TPTP all-in-one package and install dependent/As-Is/Technology Preview components in the <Eclipse installation directory>/eclipse/dropins directory.
  • Note: When testing in a Java 1.4.x environment, do NOT use the TPTP all-in-one package since it requires Java 1.5.x+.

4.5 I8 TP1

• TP1 testing assignments:
  • See Test Tools Project Tests.

News Group

• The following news group posts require a reply by Friday, May 16:
  • Duwayne:
    • Re: Using wait command in AGR script (March 11, 2008 2:07 PM)
    • Re: HTTP Recording stops immediately after start-up (FireFox) or never records data (IE) (March 25, 2008 1:15 PM)
    • tptp execuation task issue (April 2, 2008 3:12 PM)
    • Multi-Language tests with AGR (April 7, 2008 3:11 PM)
    • Testing an application with Custom component (April 8, 2008 11:03 AM)
    • Re: NoClassDefFoundError: org.eclipse.hyades.test.common...? (April 10, 2008 5:33 PM)
    • Re: Multiple instances for Verification Class (April 28, 2008 3:11 PM)
    • Re: Reusable Thread Error in standard mode (April 29, 2008 6:12 PM)

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • For more information, see the WIKI page.
  • To see the BVT results for a specific build, open the Build Reports page for a specific build and select BVT Results.
  • Outstanding defect against the Platform Project include:
    • Execution of any testsuite fails using the IAC - InactiveProcessException
  • Outstanding defect against the Test Project include:
    • Paul:
      • JUnit Plug-in runner incorrectly modifies deployed plugin.xml files.
      • AGR Tests require absolute path to Equinox JAR
      • Clear remote workspace launch does not work for remote test execution
      • AGR Tests cannot be run with start ACServer
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests
      • Content missing in Tabular Report
      • Connection property does not deploy all dependencies
      • Connection property does not allow JUnit Plug-in Tests to be executed
    • Duwayne:
      • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
      • Confusing exceptions when running automated services.
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results
      • ANT test results property requires trailing slash to specify location
      • Exceptions when running ASF tasks.
      • Only the name of the execution result is used from the results property.

Enhancements

• Queries.
• Schedule.

• Alexander Nyßen:
  • Providing the ability to record user's interactions with GEF objects in a GEF editor: Alexander (TPTP user and contributor) has contributed a patch and documentation for basic GEF/Draw2D support in the AGR. AG/PMC approved and Contribution Questionnaire (CQ) approved. Paul did not complete the integration, testing, and check-in during I8. Since the AGR is an As-Is component, Paul will complete this integration after we complete I8 TP1.

Defects

• Queries.

• Schedule.

• Defects require a reply:
  • Duwayne:
    • jdt.ui bundle activator assumes PlatformUI.getWorkbench(), causing autobuild failure when running headless

• Defects require a sizing:
  • Jerome:
    • Test Navigator link to editor doesn't work when multiple editors open on file mapping to same proxy node.

• Platform Project impacting the Test Project:
  • Execution of any testsuite fails using the IAC - InactiveProcessException: Bing cannot reproduce. Working with originator to determine if a configuration issue.
  • Random intermitten errors while launching a test against remote agents: Will be resolved during I8 TP1. This is stop-ship issue for the consuming product.
  • RPT 8.0 command line and runtime workbench unsatisfied link error
  • NullPointerException when opening Eclipse with an opened editor.
  • The edit option is not working properly in probe targets
  • Cannot add method entry fragment data items [regression]

• Defects required in I8:
  • Paul:
    • Provide ISV documentation for enhancement 166025.: Deferred to Future
    • TPTP EMF models needs to update: Deferred to Future
    • Context sensitive help requires refactoring and new content.: Deferred to Future
    • (Usability) HTTP tests do not automatically generate JUnit Java code like JUnit tests.: Deferred to Future
    • User can create test suite in non-Java project when generating test from existing recmodel (right-clicking on it): Will be resolved during I8 TP1
    • New recording wizard does not enable the Finish button.: Will be resolved during I8 TP1
    • Test editor is too wide for users to read: Will be resolved during I8 TP1
    • Technical review of Test Project documentation before 4.5 release.: Will be resolved during I8 TP1
    • Fix spelling/broken links/accessibility errors in JUnit Plugin/Manual Test/BIRT Reports plug-ins.: Will be resolved during I8 TP1
    • Documentation smoke test failures.: Will be resolved during I8 TP1
    • Providing the ability to record user's interactions with GEF objects in a GEF editor: Will be resolved during I8 TP1
    • Clean-up the TPTP committer lists and project matrix.: Will be resolved during I8 TP1
    • Regression, URL Test Does Not Get Generated: Resolved
    • Starting Recording forces change to Test Perspective: Resolved
    • Need new icon for testlog files: Resolved
    • Creating resources with # in name is not prevented but causes problems loading the model: Resolved
    • URL test type does not support encrypted datapools.: Resolved
    • Junit plug-in test type does not support encrypted datapools.: Resolved
    • Resolving a boolean value from a datapool always returns false.: Resolved
    • Generating a URL test class in the root Java package causes a compilation error.: Resolved
    • 'asked for password' dialog still pop up when running the test after encrypt--then decrypt all variables in datapool (no encrypted data when running): Resolved
    • The challenge is not removed from the datapool when the last encrypted variable is removed: Resolved
    • Test Tools about.properties files contain incorrect descriptions.: Resolved
    • Deprecate JScrib and JScrib report generators.: Resolved
    • testgen wizard does not preselect options: Resolved
    • profiling trace generated from running a java app instrumented with api recording probe does not contain information: Resolved
    • ClassCastException with DPLVariable in DatapoolTable: Resolved
    • Valid IPv6 Addresses produce 'Invalid IP Address' error in a Test Location: Resolved
    • lack of error message when remote execution fail: Resolved
    • (Usability) Switch to Test Perspective after test execution.: Resolved
    • Extra .execution appended to the results property.: Resolved
    • Datapool cells are not created for CSV ending with a comma.: Resolved
    • NoClassDefFound junit TestCase: Resolved
  • Duwayne:
    • Exception when canceling a recording (before "recording "actually starts): Deferred to Future
    • Cannot remove a test invocation to a removed HTTP request.: Will be resolved during I8 TP1
    • Parallel agent startup errors, InactiveProcessException: Will be resolved during I8 TP1. Blocked by defect Random intermitten errors while launching a test against remote agents.
    • SSL Recording not working, regression: Resolved
  • Jerome:
    • Test Navigator link to editor doesn't work when multiple editors open on file mapping to same proxy node.: Will be resolved during I8 TP1. Patch does not resolve the issue in the consuming prodcut. Attempt to resolve using the consuming product.
    • Unable to create proxy for testsuite: Will be resolved during I8 TP1. Patch does not resolve the issue in the consuming product. Attempt to resolve using the consuming product.
    • rename does not call create update change on client proxies?: Will be resolved during I8 TP1.
    • exception raised when moving an element without selecting a destination: Will be resolved during I8 TP1.
  • Tony:
    • Accessibility-Need shortcuts for all Test Navigator pulldowns: Deferred to Future
  • Kit:
    • Incorrect german translation of the word "Pass": Resolved

• Test case/automation defects to be resolved during I8 TP1:
  • Tony:
    • (Linux) AGR test suite does not run.
  • Duwayne:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
  • Jerome:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
  • Paul:
    • All external APIs should have JUnit tests
    • Automated Test Execution Harness Tests

Defects to be resolved during I8 TP1/TP2

• Defects to be resolved during I8 TP1/TP2

Defects to be deferred to Future

• Defects to be deferred to Future

Round-table

Paul

• Last Week:
  • Resolved defects:
    • Deprecate JScrib and JScrib report generators.
    • Regression, URL Test Does Not Get Generated
    • profiling trace generated from running a java app instrumented with api recording probe does not contain information
    • ClassCastException with DPLVariable in DatapoolTable
    • Valid IPv6 Addresses produce 'Invalid IP Address' error in a Test Location
    • 'asked for password' dialog still pop up when running the test after encrypt--then decrypt all variables in datapool (no encrypted data when running)
    • Resolving a boolean value from a datapool always returns false.
    • Generating a URL test class in the root Java package causes a compilation error.
    • Starting Recording forces change to Test Perspective
    • Creating resources with # in name is not prevented but causes problems loading the model
    • Need new icon for testlog files
    • lack of error message when remote execution fail
    • Junit plug-in test type does not support encrypted datapools.
    • URL test type does not support encrypted datapools.
    • Test Tools about.properties files contain incorrect descriptions.
    • The challenge is not removed from the datapool when the last encrypted variable is removed
    • testgen wizard does not preselect options
    • (Usability) Switch to Test Perspective after test execution.
    • Extra .execution appended to the results property.
    • Datapool cells are not created for CSV ending with a comma.
    • NoClassDefFound junit TestCase
    • Duplicate record error when import a CSV into an existing datapool.
    • Problem with testing JSF application.

• This Week:
  • 4.5 I8 TP1.

Duwayne

• Last Week:
  • Resolved defect:
    • SSL Recording not working, regression
  • Reviewing I8 patches.
  • Triaging:
    • Cannot remove a test invocation to a removed HTTP request
    • Parallel agent startup errors, InactiveProcessException
    • Random intermitten errors while launching a test against remote agents

• This Week:
  • Out of office: 1 day
  • 4.5 I8 TP1.

Jerome

• Last Week:
  • Out of office: 2 days
  • Resolved defect:
    • Refactoring error on attempt to Delete multiple Results from TN view.
  • Working on:
    • Test Navigator link to editor doesn't work when multiple editors open on file mapping to same proxy node.
    • Unable to create proxy for testsuite

• This Week:
  • Non-TPTP responsibilities.

Tony

• Last Week:
  • Working on defect Accessibility-Need shortcuts for all Test Navigator pulldowns.
  • Assisted Platform Project with I7 testing.

• This Week:
  • Finish assisting the Platform Project with I7 testing (target: EOD Monday).
  • 4.5 I8 TP1.

Action items

• None.

Notes

• Next week's call (May 19) will be canceled due to a Canadian holiday (Victoria Day).

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.