Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS i10 QA Phase Execution"
(→i10 End2End Supplement Tests (Negative)) |
(→i10 End2End Supplement Tests (Negative)) |
||
Line 668: | Line 668: | ||
|- | |- | ||
− | |Test LoggingDataManager, Top 10 Log Report generation: | + | |Test "LoggingDataManager", Top 10 Log Report generation: |
#LoggingDataManager : access_log.cbe.xml | #LoggingDataManager : access_log.cbe.xml | ||
Expected Result: | Expected Result: | ||
Line 737: | Line 737: | ||
|Passed | |Passed | ||
|Using Query builder able to execute and validate queries properly | |Using Query builder able to execute and validate queries properly | ||
+ | |||
+ | |- | ||
+ | |Test AperiMDR with proper db configuration and with invalid query | ||
+ | # select "AperiMdr" and create invalid query using Query Builder | ||
+ | Expected Result: | ||
+ | # Query execution should fail and display appropriate error message | ||
+ | |Passed | ||
+ | |Passed | ||
+ | |Query execution failed and displayed error message | ||
|- | |- | ||
Line 756: | Line 765: | ||
|Passed | |Passed | ||
|Query executed successfully | |Query executed successfully | ||
+ | |||
+ | |- | ||
+ | |Test "Sample Repository with Query", with invalid query | ||
+ | # select "Sample Repository with Query" | ||
+ | # Create an invalid query from query builder and submit it | ||
+ | Expected Result: | ||
+ | #Should display Empty Query response under GraphResponseViewer | ||
+ | |Passed | ||
+ | |Passed | ||
+ | |Empty Query response displayed under GraphResponseViewer | ||
|} | |} | ||
Revision as of 06:21, 30 April 2008
This page is created by QA to track all i10 test execution activities from 21 Apr'08 to 02 May'08. This page will be filled in by the QA team as i10 testing phase proceeds.
Contents
- 1 QA Test Execution Activities for i10
- 2 QA Status Summary
- 3 E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0)
- 4 Data Visualization TPTP Manual Tests
- 5 i10 ERs - JUnits/Manual Tests Inspection
- 6 i10 End2End Supplement Tests (Negative)
- 7 Basic Operational efficiency tests
- 8 i10 Test Execution Results
QA Test Execution Activities for i10
Primary Tasks:
- E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10 )
- i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)
- Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)
Secondary Tasks:
- Execution of i10 supplement e2e tests (negative) from QA.
- Basic operational efficiency tests (limited performance tests)
For complete i10 QA Activities, refer http://wiki.eclipse.org/COSMOS_QA_i10_Activities
QA Status Summary
E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0)
Build Installation and Configuration
Test Description | Test Status
Windows XP SP2 |
Test Status
Windows 2003 SP1 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|---|
<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path> | Passed | Passed | Passed | None |
Starting COSMOS components : Management Console
Test Description | Test Status
Windows XP SP2 |
Test Status
Windows 2003 SP1 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|---|
<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker | Passed | Passed | Passed | None |
COSMOS> broker registerAllDataManagers | Passed | Passed | Passed | Found warnings - "log4j:WARN No appenders could be found for logger (org.apache.axis2.util.Loader).
log4j:WARN Please initialize the log4j system properly." |
COSMOS> broker getDataManagers | Passed | Passed | Passed | None |
COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp StatDataManager | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp LoggingDataManager | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp TestFederatingCMDB | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp ExampleMdr | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp SMLMDR | Passed | Passed | Passed | None |
COSMOS> broker deregister domsr01-xp AperiMdr | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt | Passed | Passed | Passed | None |
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt | Passed | Passed | Passed | None |
COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt | Passed | Passed | Passed | Empty Response, no data found. |
Using the COSMOS Web User Interface
Test Description | Test Status
Windows XP SP2 |
Test Status
Windows 2003 SP1 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|---|
http://localhost:8080/COSMOSUI/?page=cosmos | Passed | Passed | Passed | Found All 6 DataManagers under COSMOS Web UI |
StatDataManager
server 33 tomcat1181829221515 CBE |
Passed | Passed | Passed | Statistical Report generated successfully |
StatDataManager
server 33 tomcat1181829221515 TomcatSet |
Passed | Passed | Passed | Statistical Report generated successfully |
LoggingDataManager
access_log.cbe.xml |
Passed | Passed | Passed | Top 10 Log Report generated successfully |
LoggingDataManager
windows_app_log.cbe.xml |
Passed | Passed | Passed | Top 10 Log Report generated successfully |
Sample Repository with Registration
Submit Query |
Passed | Passed | Passed | Query for all-items resulted Empty Reponse, no data found. |
Sample Repository with Registration
Register Configuration Items... |
Passed | Passed | Passed | Registration with Example MDR is successful |
Sample Repository with Registration
Deregister Configuration Items... |
Passed | Passed | Passed | Deregistration with Example MDR is successful |
Sample Repository with Registration
Submit Query |
Passed | Passed | Passed | Query for all-items resulted Empty Reponse, no data found. |
Sample Repository with Registration
View MetaData |
Passed | Passed | Passed | No Metadata available |
Resource Repository
Submit Query |
Passed | Passed | Passed | Queries executed successfully with proper results |
Resource Repository
View MetaData |
Passed | Passed | Passed | No Metadata available |
AperiMDR
Submit Query |
Passed | Passed | Passed | Using Query builder able to execute and validate queries properly |
AperiMDR
View MetaData |
Passed | Passed | Passed | Displayed MetaData properly |
Sample Repository with Query
Get All Students |
Passed | Passed | Passed | Displayed Query Result for Students data correctly |
Sample Repository with Query
Get All Teachers |
Passed | Passed | Passed | Displayed Query Result for Teachers data correctly |
Sample Repository with Query
Get All Classes |
Passed | Passed | Passed | Displayed Query Result for Classes data correctly |
Sample Repository with Query
Staff01 |
Passed | Passed | Passed | Displayed data for 'students taught by teacher-staff01' correctly |
Sample Repository with Query
Staff02 |
Passed | Passed | Passed | Displayed data for 'students taught by teacher-staff02' correctly |
Sample Repository with Query
Submit Query |
Passed | Passed | Passed | Using Query Builder, able to create and execute queries correctly |
Sample Repository with Query
View MetaData |
Passed | Passed | Passed | Displayed MetaData information correctly |
Data Visualization TPTP Manual Tests
/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
Test Description | Test Status
Windows XP SP2 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|
navigate | Passed | Passed | 6 nodes should be shown under "Data Managers"
No "Monitoring Data Repository" node, it should be "StatDataManager" node |
navigate2 | Passed | Passed | No "Create Query" found, it should be "Submit Query" |
properties | Passed | Passed | Given EPR is wrong, it should be http://localhost:8080/axis2/services/ExampleMdrDataManager |
visualize log report | Passed | Passed | No "Monitoring Data Repository" available, it should be "StatDataManager |
visualize stat report | Passed | Passed | None |
properties 2 | Passed | Passed | None |
example cmdbf query | Passed | Passed | XML output contents seems to be wrong, actual result shows only one student detail - "firstName="Mike" id="03 lastname="Lee"" |
SML CMDBF Graph Response Viewer | Passed | Passed | None |
Example MDR CMDBF Graph Response Viewer | Passed | Passed | No cancel button available under details view window, it should be OK button. |
Multiple Graph Response Viewers | Passed | Passed | No "Cancel" button available under detail area, it should be Ok button |
Registration Example | Passed | Passed | None |
DeRegistration Example | Passed | Passed | None |
Partial Registration of MDR with CMDBf | Passed | Passed | None |
Partial DeRegistration of MDR with CMDBf | Passed | Passed | None |
/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Reports.testsuite
Test Description | Test Status
Windows XP SP2 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|
LogReport | Passed | Passed | None |
StatReport | Passed | Passed | None |
i10 ERs - JUnits/Manual Tests Inspection
Bugzilla ID | ID Summary | JUnits Status | Manual Tests | Remarks |
---|
i10 End2End Supplement Tests (Negative)
COSMOS Command Line Client : Management Console
Test Description | Test Status
Windows XP SP2 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|
Configure tomcat port to 9080 (other than default port : 8080) and test the following:
Expected Result:
|
Passed | Passed | None |
Test Broker registration other than defaul port 8080 (say 9080):
Expected Result:
|
Failed | Failed | DataManagers are getting registered on default port 8080 only, ignored actual port 9080 |
Test Broker registration multiple times:
Expected Result:
ed." |
Passed | Passed | Found expected message under tomcat console window |
Without broker registration, test the following:
Expected Result:
|
Passed | Passed | None |
After succesful broker registration, test the following:
Expected Result:
|
Passed | Passed | None |
Without any broker registration, test the following
Expected Result:
|
Passed | Passed | None |
Test deregistration of StatDataManager with valid hostname and valid datamanager name :
Expected Result:
|
Passed | Passed | None |
Test deregistration of StatDataManager with invalid hostname and valid datamanager name :
Expected Result:
|
Failed | Failed | No Failure message, it still shows "DataManager deregistered" |
Test deregistration of StatDataManager with valid hostname and invalid datamanager name :
Expected Result:
|
Failed | Failed | No Failure message, it still shows "DataManager deregistered" |
Test graphQuery with valid hostname,valid DataManager,valid query path:
Expected Result:
|
Passed | Passed | This is depended on Broker registration issue, fixed manually to test this. |
Test graphQuery with invalid hostname,valid DataManager,valid query path:
Expected Result:
|
Passed | Passed | Displaying error message as "SMLMDR is not registered at the broker." |
Test graphQuery with valid hostname,invalid DataManager,valid query path:
Expected Result:
|
Passed | Passed | Displayed error message as "StatDataManager is not registered at the broker." |
Test graphQuery with valid hostname,valid DataManager,invalid query path:
Expected Result:
|
Passed | Passed | Displayed "java.io.FileNotFoundException:" error message tree |
Test graphQuery with valid hostname,valid DataManager,valid query path with spaces:
Expected Result:
|
Failed | Failed | Displayed "java.io.FileNotFoundException:" error message tree |
Test graphQuery for AperMdr with valid data, with invalid Aperi database settings:
Expected Result:
|
Passed | Passed | Displayed "aperiDB.getStorageSystems error" |
Using the COSMOS Web User Interface
Test Description | Test Status
Windows XP SP2 |
Test Status
RHES 5.0 |
Remarks |
---|---|---|---|
Open COSMOSUI with 8090 port (Make sure all COSMOS Components are up and running at this port)
Expected Result:
|
Passed | Passed | Found All 6 DataManagers under COSMOS Web UI |
Open COSMOSUI, Verify Properties of all DataManagers
Expected Result:
|
Passed | Passed | All EPR's under properties window displayed correctly |
Test StatDataManager, "Statistical Report" generation:
Expected Result:
|
Passed | Passed | Statistical Report generated successfully |
Test "LoggingDataManager", Top 10 Log Report generation:
Expected Result:
|
Passed | Passed | Top 10 Log Report generated successfully |
Test "Sample Repository with Registration",self registration
Expected Result
|
Failed | Failed | Registration successful, but no result data |
Test "Sample Repository with Registration",Resource Repository registration
|
Failed | Failed | Registration successful, but no result data |
Test "Sample Repository with Registration",AperiMdr registration
Expected Result
|
Failed | Failed | Error "org.apache.axis2.AxisFault: AperiMDR error: relationshipTemplate 'all-relationships' is missing a recordConstraint" |
Test Resource Repository, with valid query
Expected Result:
|
Passed | Passed | Queries executed successfully with proper results |
Test Resource Repository, with invalid query
Expected Result:
|
Passed | Passed | Query execution failed with appropriate error message |
Test AperiMDR with proper db configuration and with valid query
Expected Result:
|
Passed | Passed | Using Query builder able to execute and validate queries properly |
Test AperiMDR with proper db configuration and with invalid query
Expected Result:
|
Passed | Passed | Query execution failed and displayed error message |
Test AperiMDR with invalid db configuration and with valid query
Expected Result:
|
Passed | Passed | Query failed with db connection error |
Test "Sample Repository with Query", with valid query
Expected Result:
|
Passed | Passed | Query executed successfully |
Test "Sample Repository with Query", with invalid query
Expected Result:
|
Passed | Passed | Empty Query response displayed under GraphResponseViewer |
Basic Operational efficiency tests
Test Description | Output | Status | Remarks |
---|