Difference between revisions of "EPP/Obsolete/Logging/Testplan"

Revision as of 10:37, 19 August 2016

Manual Release Tests

Before starting, the value org.eclipse.epp.logging.aeri.core/debug in org.eclipse.epp.logging.aeri.core/.options should be set to true. The application should be started with a debug configuration and tracing enabled with the plugin checked (Debug Configuration - Tracing). Please ensure that the org.eclipse.epp.logging.aeri.ide.tests-plugin is enabled. The workspace should be cleared before each test. The product configuration org.eclipse.epp.logging.aeri.ide.tests/aeri.ide.tests.product is recommended. The tests-plugin will contribute a Test Errors dropdown to the toolbar. This menu is used to trigger various test-errors which are referred as underlined in the next steps. If a testcase contains no instructions for the configuration dialog, the dialog can be skipped with 'Enable' and default settings.

Remote Data Download

• Start with default settings
• Wait for the Updating Error Reports Database-job to finish
• Verify: in the runtime-workspace: .metadata/.plugins/org.eclipse.epp.logging.aeri.ide/org.eclipse.epp.logging.aeri.ide.server/remote-index/ exists
• Verify: in the runtime-workspace: .metadata/.plugins/org.eclipse.epp.logging.aeri.ide/org.eclipse.epp.logging.aeri.ide.server/server-config.json exists
• Restart the instance
• Verify: remote-index folder was not changed

Configuration

• Start with default settings
• Log Error
• Open configuration-dialog from popup link ('Enable')
• In the configuration-dialog, enter a name, an email and set both anonymize-settings to checked
• Open the details-dialog for the error in the upcoming popup ('View Details')
• Verify: name end email of REPORT are the same as configured
• Verify: Status and Exception messages are replaced by HIDDEN
• Verify: second stackframe anonymized to

  at HIDDEN.HIDDEN(HIDDEN:-1)

• Verify: Anonymize Messages and Anonymize Stack Trace icons checked.
• Cancel sending ('Don't send')
• Open Preferences - General - Error Reporting
• Change name and email, deselect both anonymize fields and select enable debug mode
• Log error again and open details view
• Verify: name end email of REPORT are the new configured values
• Verify: No messages replaced by HIDDEN
• Verify: second stackframe

  at any.third.party.Clazz.thirdPartyMethod(Clazz.java:42)

  is not anonymized
• Verify: Anonymize Messages and Anonymize Stack Trace icons not checked.

Disable

• Start with default settings
• Log Error
• Click disable in upcoming popup
• Log Error again
• Verify: No error popup
• Verify: Both errors in the error log

Temporary Disable

• Start with default settings
• Log Error
• Wait for popup fade-out
• Log Error again
• Log MS Error
• Verify: No further popups
• Restart without cleaning workspace
• Log ClassNotFound
• Verify: Configure popup shows

Temporary Disable 2

• Start with default settings
• Log Error
• Use 'x' to cancel the popup
• Log Error again
• Log MS Error
• Verify: No further popups
• Restart without cleaning workspace
• Log ClassNotFound
• Verify: Configure popup shows

Report skipped

• Set the debug option to false for this test
• Start with default settings
• Log Error
• Confirm all settings and use 'x' to cancel the report preview popup
• Log Error again
• Verify: No further popups
• Log MS Error
• Verify: Popup opens and both errors visible in the preview dialog

Report skipped 2

• Set the debug option to false for this test
• Start with default settings
• Log Error
• Confirm all settings and wait until the report preview popup fades out
• Log Error again
• Verify: No further popups
• Log MS Error
• Verify: Popup opens and both errors visible in the preview dialog

Disable by system parameter

• Start with the VM argument

  -Dorg.eclipse.epp.logging.aeri.skipReports=true

• Verify: No startup-progress and download job visible in progress view
• Log Error
• Verify: No upcoming error popups
• Verify: Error log contains only the error but no predicate debug output
• Verify: .metadata/.plugins/org.eclipse.epp.logging.aeri.ide/* not created

Multistatus Handling

• Start with default settings
• Log MS Error
• Open the details-dialog for the error
• Verify: 4 status objects logged, each with a different message

UI Freeze

• Start with default settings
• In Preferences-General-UI Responsiveness check the Detect periods of unresponsive UI checkbox and save the settings
• UI Freeze
• Open the details view
• Verify: Multiple status objects, message of the first one has suffix [n child-status duplicates removed by Error Reporting]

CoreException Handling

• Start with default settings
• Log CE Error
• Open the details view
• Verify: First status with message: Core-Exception outer status
• Verify: Second status with message: Core-Exception inner status [detached from CoreException of Status 'Core-Exception outer status' by Error Reporting]

No UI Deadlock

• Start with default settings
• Log Error and configure system, close error popup
• Test UI Deadlock
• Confirm first dialog ('Test lock')
• Verify: Second dialog opens ('No lock')
• Verify: Error popup shows up
• Verify: Eclipse interface still responsive

Error Severity

• Start with default settings
• Log Error
• Open the details view
• Verify: severity is UNKNOWN
• Change the This error's severity is... combobox value
• Verify: The report severity reflects the setting of the combobox (true/false)

Report Processors

• Start with default settings
• Log Error
• Open the details view
• Verify: Extended Classpath Analysis not selectable and all other processors disabled.
• Enable all possible processors
• Verify: Section in the preview are created for each processor
• Edit one section and change the text to another value
• Verify: Reset option visible for the edited section
• Reset the edited section
• Verify: Section reset to previous state and reset option no longer visible

Wiring Analysis

• Start with default settings
• Log ClassNotFound
• Verify: In the report details dialog, the preview field should contain a section like:

Extended Classpath Analysis
The problematic package ‘org.eclipse.epp.logging.aeri.core.util’ may originate in the following bundles:
org.eclipse.epp.logging.aeri.core 2.0.3.qualifier, imported by:
    org.eclipse.epp.logging.aeri.ide 2.0.3.qualifier

• Disable and enable the Extended Classpath Analysis icon.
• Verify: The section is removed and added accordingly to the icon state

Problem State Dialog

• Start with default settings
• Log ClassNotFound
• In the report details dialog, open the problem state dialog (info icon)
• Verify: Server message shown and additional information request for the wiring analysis visible