Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "COSMOS QA End2End Tests"
(→Concurrency Tests) |
|||
Line 474: | Line 474: | ||
= Basic Operational efficiency tests = | = Basic Operational efficiency tests = | ||
== Concurrency Tests == | == Concurrency Tests == | ||
+ | |||
+ | Test Description: | ||
+ | *Testing the scenario where COSMOSUI running on windows and all COSMOS services running on Linux | ||
+ | #Install COSMOS service components under linux box | ||
+ | #Install COSMOSUI,COSMOSReportViewer webapps application under windows | ||
+ | #Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to linux hostname | ||
+ | #From windows, open http://localhost:8080/COSMOSUI | ||
+ | Expected Result: | ||
+ | COSMOS UI should display all 6 Data Managers and all it's operations should work normally | ||
+ | ---- | ||
+ | |||
== Volume/Performance Tests == | == Volume/Performance Tests == | ||
== Scalability/Stability Tests == | == Scalability/Stability Tests == |
Revision as of 04:56, 23 May 2008
Contents
Overview
The purpose of this page is to articulate all COSMOS End-to-End test scenarios for QA testing. This page will be filled in by the QA team and will be updated with every new iteration.
COSMOS Installation
As a pre-requisite for below test scenarios, install the COSMOS application as per the instructions available at http://wiki.eclipse.org/COSMOS_DEMO_i11#Installation
COSMOS End2End Tests
Build Installation and Configuration
Test Description:
- <cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path>
Expected Result:
Starting COSMOS components : Management Console
Test Description:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker
Expected Result:
Test Description:
- COSMOS> broker registerAllDataManagers
Expected Result:
Test Description:
- COSMOS> broker getDataManagers
Expected Result:
Test Description:
- COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp StatDataManager
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp LoggingDataManager
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp TestFederatingCMDB
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp ExampleMdr
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp SMLMDR
Expected Result:
Test Description:
- COSMOS> broker deregister domsr01-xp AperiMdr
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt
Expected Result:
Test Description:
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt
Expected Result:
Test Description:
- COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt
Expected Result:
Using the COSMOS Web User Interface
Test Description:
Expected Result:
Test Description:
- StatDataManager
- server 33 tomcat1181829221515 CBE
Expected Result:
Test Description:
- StatDataManager
- server 33 tomcat1181829221515 TomcatSet
Expected Result:
Test Description:
- LoggingDataManager
- access_log.cbe.xml
Expected Result:
Test Description:
- LoggingDataManager
- windows_app_log.cbe.xml
Expected Result:
Test Description:
- Sample Repository with Registration
- Submit Query
Expected Result:
Test Description:
- Sample Repository with Registration
- Register Configuration Items...
Expected Result:
Test Description:
- Sample Repository with Registration
- Deregister Configuration Items...
Expected Result:
Test Description:
- Sample Repository with Registration
- Submit Query
Expected Result:
Test Description:
- Sample Repository with Registration
- View MetaData
Expected Result:
Test Description:
- Resource Repository
- Submit Query
Expected Result:
Test Description:
- Resource Repository
- View MetaData
Expected Result:
Test Description:
- AperiMDR
- Submit Query
Expected Result:
Test Description:
- AperiMDR
- View MetaData
Expected Result:
Test Description:
- Sample Repository with Query
- Get All Students
Expected Result:
Test Description:
- Sample Repository with Query
- Get All Teachers
Expected Result:
Test Description:
- Sample Repository with Query
- Get All Classes
Expected Result:
Test Description:
- Sample Repository with Query
3Staff01 Expected Result:
Test Description:
- Sample Repository with Query
- Staff02
Expected Result:
Test Description:
- Sample Repository with Query
- Submit Query
Expected Result:
Test Description:
- Sample Repository with Query
- View MetaData
Expected Result:
End2End Supplement Tests (Negative)
COSMOS Command Line Client : Management Console
Test Description:
- Configure tomcat port to 9080 (other than default port : 8080) and test the following:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
Expected Result:
- Should display COSMOS prompt without any errors
Test Description:
- Test Broker registration other than defaul port 8080 (say 9080):
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
Expected Result:
- All DataManagers should successfuly get registered to broker on port 9080
Test Description:
- Test Broker registration multiple times:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
- COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
Expected Result:
- Under tomcat console window, it should display "Service has already been registered at this broker. No new brokers are registered."
Test Description:
- Without broker registration, test the following:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker getDataManagers
Expected Result:
- Should display empty list of DataManagers
Test Description:
- After succesful broker registration, test the following:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker getDataManagers
Expected Result:
- Should display all 6 DataManagers registered with broker
Test Description:
- Without any broker registration, test the following
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> finddatamanagers http://localhost:9080/axis2/services/CosmosServiceFinder
Expected Result:
- Should display all 6 DataManagers deployed under tomcat with port 9080
Test Description:
- Test deregistration of StatDataManager with valid hostname and valid datamanager name :
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker deregister domsr01-xp StatDataManager
Expected Result:
- Deregistration should be successful
Test Description:
- Test deregistration of StatDataManager with invalid hostname and valid datamanager name :
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker deregister unknown StatDataManager
Expected Result:
- Deregistration should fail and should display appropriate error message
Test Description:
- Test deregistration of StatDataManager with valid hostname and invalid datamanager name :
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> broker deregister domsr01-xp StatDataMgr
Expected Result:
- Deregistration should fail and should display appropriate error message
Test Description:
- Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
Expected Result:
- Query execution should be succesful.
Test Description:
- Test graphQuery for SMLMDR, with invalid hostname,valid DataManager,valid query path:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery unknown SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
Expected Result:
- Query execution should fail and throw appropriate error message.
Test Description:
- Test graphQuery for SMLMDR, with valid hostname,invalid DataManager,valid query path:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp StatDataManager D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
Expected Result:
- Query execution should fail and throw appropriate error message.
Test Description:
- Test graphQuery for SMLMDR, with valid hostname,valid DataManager,invalid query path:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDRTest\testInstanceIdQuery.txt
Expected Result:
- Query execution should fail and throw appropriate error message.
Test Description:
- Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path with spaces:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp SMLMDR "D:\cosmos i10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt"
Expected Result:
- Query execution should be successful
Test Description:
- Test graphQuery for AperMdr with valid data, with valid Aperi database settings:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
Expected Result:
- Query should execute successfully
Test Description:
- Test graphQuery for AperMdr with invalid query data, with valid Aperi database settings:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
Expected Result:
- Query execution should fail and display appropriate error message
Test Description:
- Test graphQuery for AperMdr with valid data, with invalid Aperi database settings:
- <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
- COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
Expected Result:
- Query execution should fail, with DB Connection error
Using the COSMOS Web User Interface
Test Description
- Open COSMOSUI with 8090 port (Make sure all COSMOS Components are up and running at this port)
Expected Result:
- Should display all 6 DataManagers properly
Test Description:
- Open COSMOSUI, Verify Properties of all DataManagers
- StatDataManager
- LoggingDataManager
- Sample Repository with Registration
- Sample Repository with Query
- Resource Repository
- AperiMdr
Expected Result:
- All properties should display the correct EPR with port 9080
Test Description:
- Test StatDataManager, "Statistical Report" generation:
- StatDataManager : server 33 tomcat1181829221515 CBE
Expected Result:
- Statistical Report should get generated properly
Test Description:
- Test "LoggingDataManager", Top 10 Log Report generation:
- LoggingDataManager : access_log.cbe.xml
Expected Result:
- Top 10 Log report should get generated properly
Test Description:
- Test "Sample Repository with Registration",self registration
- Select "Sample Repository with Registration"
- Click on "Register Configuration Items..."
- Select "Sample Repository with Registration" under registration window
Expected Result
- Either it should not be allowed or it should display appropriate error message
Test Description:
- Test "Sample Repository with Registration",Resource Repository registration
- Select "Sample Repository with Registration"
- Click on "Register Configuration Items..."
- Select "Resource Repository" under registration window
Expected Result
- Either registration should not be allowed or it should display appropriate error message
Test Description:
- Test "Sample Repository with Registration",AperiMdr registration
- Select "Sample Repository with Registration"
- Click on "Register Configuration Items..."
- Select "AperiMdr" under registration window
Expected Result
- Either registration should not be allowed or it should display appropriate error message
Test Description:
- Test Resource Repository, with valid query
- select "Resource Repository" and click on "Submit Query"
- Provide valid query and submit
Expected Result:
- Query should execute successfully
Test Description:
- Test Resource Repository, with invalid query
- select "Resource Repository" and click on "Submit Query"
- Provide invalid query and submit
Expected Result:
- Query should fail with appropriate error message
Test Description:
- Test AperiMDR with proper db configuration and with valid query
- select "AperiMdr" and create valid query using Query Builder
Expected Result:
- Query should execute successfully
Test Description:
- Test AperiMDR with proper db configuration and with invalid query
- select "AperiMdr" and create invalid query using Query Builder
Expected Result:
- Query execution should fail and display appropriate error message
Test Description:
- Test AperiMDR with invalid db configuration and with valid query
- select "AperiMdr" and create valid query using Query Builder
Expected Result:
- Query execution should fail with db connection error
Test Description:
- Test "Sample Repository with Query", with valid query
- select "Sample Repository with Query"
- Create a valid query from query builder and submit it
Expected Result:
- Query execution should be successful
Test Description:
- Test "Sample Repository with Query", with invalid query
- select "Sample Repository with Query"
- Create an invalid query from query builder and submit it
Expected Result:
- Should display Empty Query response under GraphResponseViewer
Basic Operational efficiency tests
Concurrency Tests
Test Description:
- Testing the scenario where COSMOSUI running on windows and all COSMOS services running on Linux
- Install COSMOS service components under linux box
- Install COSMOSUI,COSMOSReportViewer webapps application under windows
- Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to linux hostname
- From windows, open http://localhost:8080/COSMOSUI
Expected Result: COSMOS UI should display all 6 Data Managers and all it's operations should work normally