Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "COSMOS QA i10 Activities"

('''In scope i10 ERs''')
('''Scope Definition''')
 
(12 intermediate revisions by 2 users not shown)
Line 2: Line 2:
  
 
This has been put together to address [https://bugs.eclipse.org/bugs/show_bug.cgi?id=222932 Bugzilla ER 222932].
 
This has been put together to address [https://bugs.eclipse.org/bugs/show_bug.cgi?id=222932 Bugzilla ER 222932].
 +
 +
All these activities, test execution has been tracked over http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution
  
 
== '''Terminologies/Acronyms''' ==
 
== '''Terminologies/Acronyms''' ==
Line 26: Line 28:
 
''Primary Tasks:''  
 
''Primary Tasks:''  
  
#E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0  
+
#E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10)
(http://wiki.eclipse.org/COSMOS_UG_Running_the_Example)
+
#i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)  
#i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_9_Enhancements)  
+
 
#Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)  
 
#Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)  
  
Line 48: Line 49:
  
 
For i10, QA will use following softwares:
 
For i10, QA will use following softwares:
# Tomcat 5.5.25
+
# Tomcat 5.5.25 & Axis2 1.3
 
# SUN JDK 1.5.0_15  
 
# SUN JDK 1.5.0_15  
 
# Firefox version (2.0.0.11)
 
# Firefox version (2.0.0.11)
 
# Internet Explorer (6.0.2900.2180.XP SP2)
 
# Internet Explorer (6.0.2900.2180.XP SP2)
# Eclipse 3.3.4
+
# Eclipse 3.4
 +
# DOJO 1.0.2
 +
# BIRT 2.3
 +
 
 +
http://www.eclipse.org/downloads/moreinfo/jre.php
 +
 
 +
Align with Ganymede: http://www.eclipse.org/projects/ganymede.php
  
 
== '''Iteration QA Entrance and Exit Criteria''' ==
 
== '''Iteration QA Entrance and Exit Criteria''' ==
Line 59: Line 66:
  
 
# A stable i10 build that has undergone multiple integration build during the course of the Development phase which ends on April 18, 2008
 
# A stable i10 build that has undergone multiple integration build during the course of the Development phase which ends on April 18, 2008
# The wiki page http://wiki.eclipse.org/COSMOS_i10_e2e_testing is completed (by Development)
+
# The wiki page http://wiki.eclipse.org/COSMOS_DEMO_i10 is completed (by Development)
  
 
The following items need to be in place before the 2 week i10 QA phase can be declared complete:
 
The following items need to be in place before the 2 week i10 QA phase can be declared complete:
  
 
# All ERs should have JUnits or a manual TPTP test in place.  If JUnits are not applicable to a given ER, alternative verification means need to be specified.  To reiterate, QA is not expected to run the JUnits. In i10, this activity will be completed as part of the weekly iteration builds. In future, this will be fully automated once ER 215135 is completed.
 
# All ERs should have JUnits or a manual TPTP test in place.  If JUnits are not applicable to a given ER, alternative verification means need to be specified.  To reiterate, QA is not expected to run the JUnits. In i10, this activity will be completed as part of the weekly iteration builds. In future, this will be fully automated once ER 215135 is completed.
# The e2e tests for i10 (http://wiki.eclipse.org/COSMOS_i10_e2e_testing) are run against the final i10 build
+
# The e2e tests for i10 (http://wiki.eclipse.org/COSMOS_DEMO_i10) are run against the final i10 build
  
 
== '''i10 Test Cases''' ==
 
== '''i10 Test Cases''' ==
Line 76: Line 83:
 
This section includes the tasks required to complete this enhancement.
 
This section includes the tasks required to complete this enhancement.
  
# Srinivas Reddy has generated this page to address [https://bugs.eclipse.org/bugs/show_bug.cgi?id=222932 bugzilla 216529]
+
# Srinivas Reddy has generated this page to address [https://bugs.eclipse.org/bugs/show_bug.cgi?id=222932 bugzilla 222932]
 
# The COSMOS team needs to review the relevant sections for this page.
 
# The COSMOS team needs to review the relevant sections for this page.
 
# Kishore Adivishnu/Srinivas, representing the QA team, is supposed to complete this activity by April 18, 2008.  This is prior to the commencement of the QA phase for i10.
 
# Kishore Adivishnu/Srinivas, representing the QA team, is supposed to complete this activity by April 18, 2008.  This is prior to the commencement of the QA phase for i10.

Latest revision as of 15:48, 28 April 2008

COSMOS QA Activities for i10

This has been put together to address Bugzilla ER 222932.

All these activities, test execution has been tracked over http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution

Terminologies/Acronyms

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.

Term Definition
Quality Expectations Is a statement of some behaviour, characteristic or operational facility that a product must exhibit for it to be deemed ‘fit for purpose’. Quality expectations are normally grouped into four main categories: functional/behavioural, operational efficiency, inter operability factors; and admin/management factors (to control TCO).
Acceptance Criteria This is a quantification of how a quality expectation is to be validated. For functional/behavioural quality expectations this is a simple Boolean test – it either works or it doesn’t. Hence, for most scope docs there is no need to specifically define functional acceptance criteria. However, other types of quality expectations – especially performance related areas – do require specific acceptance criteria because the quantification is normally some form of numeric threshold (with optional margin/tolerance) that states minimum levels of acceptable operational efficiency.

Scope Definition

The COSMOS quality expectations and the matching acceptance criteria, that would serve as a preamble to the COSMOS QA team while executing their work, were completed via ER 214576.

QA Scope for i10 testing:

Primary Tasks:

  1. E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10)
  2. i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)
  3. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)

Secondary Tasks:

  1. Execution of i10 supplement e2e tests(negative) from QA.
  2. Basic operational efficiency tests (limited performance tests)

In scope i10 ERs

Please refer to http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements for the list of ERs that QA will need to test as part of the i10 testing phase that runs from April 21 thru 2nd May 2008.

In scope platforms, OS's & configurations

For i10, QA will certify the following platforms:

  1. Microsoft Windows XP SP2
  2. Micorsoft Windows 2003 Server SP1
  3. RedHat Linux Enterprise Server 5.0 (Smoke test/Limited End2End testing only)

For i10, QA will use following softwares:

  1. Tomcat 5.5.25 & Axis2 1.3
  2. SUN JDK 1.5.0_15
  3. Firefox version (2.0.0.11)
  4. Internet Explorer (6.0.2900.2180.XP SP2)
  5. Eclipse 3.4
  6. DOJO 1.0.2
  7. BIRT 2.3

http://www.eclipse.org/downloads/moreinfo/jre.php

Align with Ganymede: http://www.eclipse.org/projects/ganymede.php

Iteration QA Entrance and Exit Criteria

The following items need to be in place before the 2 week i10 QA phase can commence:

  1. A stable i10 build that has undergone multiple integration build during the course of the Development phase which ends on April 18, 2008
  2. The wiki page http://wiki.eclipse.org/COSMOS_DEMO_i10 is completed (by Development)

The following items need to be in place before the 2 week i10 QA phase can be declared complete:

  1. All ERs should have JUnits or a manual TPTP test in place. If JUnits are not applicable to a given ER, alternative verification means need to be specified. To reiterate, QA is not expected to run the JUnits. In i10, this activity will be completed as part of the weekly iteration builds. In future, this will be fully automated once ER 215135 is completed.
  2. The e2e tests for i10 (http://wiki.eclipse.org/COSMOS_DEMO_i10) are run against the final i10 build

i10 Test Cases

QA will add End2End i10 test cases and will provide the link over here.

Resources & timeframe

i10 QA will be completed by two dedicated CA resources. The i10 QA phase will run from April 21 thru May 2.

Task Breakdown

This section includes the tasks required to complete this enhancement.

  1. Srinivas Reddy has generated this page to address bugzilla 222932
  2. The COSMOS team needs to review the relevant sections for this page.
  3. Kishore Adivishnu/Srinivas, representing the QA team, is supposed to complete this activity by April 18, 2008. This is prior to the commencement of the QA phase for i10.

Open Issues/Questions

All reviewer feedback should go in the Talk page for 222932 "Define scope and detail the i10 QA activities".


Copyright © Eclipse Foundation, Inc. All Rights Reserved.