Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

COSMOS i10 QA Phase Execution

This page is created by QA to track all i10 test execution activities from 21 Apr'08 to 02 May'08. This page will be filled in by the QA team as i10 testing phase proceeds.

QA Test Execution Activities for i10

Primary Tasks:

  1. E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10 )
  2. i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)
  3. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)

Secondary Tasks:

  1. Execution of i10 supplement e2e tests (negative) from QA.
  2. Basic operational efficiency tests (limited performance tests)

For complete i10 QA Activities, refer http://wiki.eclipse.org/COSMOS_QA_i10_Activities

QA Status Summary

Iteration 10 QA testing effort and completion status (21 Apr’08 – 02 May’08)

  • QA announces the finish of our testing effort for COSMOS iteration 10 with 1 day ahead of scheduled completion date (02 May, 2008)
  • The following activities were completed by QA, as per defined scope of i10 QA activities (http://wiki.eclipse.org/COSMOS_QA_i10_Activities)

i10 QA Execution Summary:

  1. End2End testing on 3 platforms (Windows XP,2003 and Red Hat Enterprise Server 5.0) based on i10 demo page instructions (http://wiki.eclipse.org/COSMOS_DEMO_i10)
  2. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite and DataVisualization.UI.Reports.testsuite)
  3. Review of i10 ER’s for JUnits and Manual Tests
  4. Additional E2E tests execution (Negative tests) (http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution#i10_End2End_Supplement_Tests_.28Negative.29)
  5. Limited Performance Tests (Configuration tests & Concurrency tests)

i10 QA Execution Status Page:

i10 bugs list filed by QA:

E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0)

Build Installation and Configuration

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path> Passed Passed Passed None

Starting COSMOS components : Management Console

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker Passed Passed Passed None
COSMOS> broker registerAllDataManagers Passed Passed Passed Found warnings - "log4j:WARN No appenders could be found for logger (org.apache.axis2.util.Loader).

log4j:WARN Please initialize the log4j system properly."

COSMOS> broker getDataManagers Passed Passed Passed None
COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder Passed Passed Passed None
COSMOS> broker deregister domsr01-xp StatDataManager Passed Passed Passed None
COSMOS> broker deregister domsr01-xp LoggingDataManager Passed Passed Passed None
COSMOS> broker deregister domsr01-xp TestFederatingCMDB Passed Passed Passed None
COSMOS> broker deregister domsr01-xp ExampleMdr Passed Passed Passed None
COSMOS> broker deregister domsr01-xp SMLMDR Passed Passed Passed None
COSMOS> broker deregister domsr01-xp AperiMdr Passed Passed Passed None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt Passed Passed Passed None
COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt Passed Passed Passed Empty Response, no data found.

Using the COSMOS Web User Interface

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
http://localhost:8080/COSMOSUI/?page=cosmos Passed Passed Passed Found All 6 DataManagers under COSMOS Web UI
StatDataManager

server 33 tomcat1181829221515 CBE

Passed Passed Passed Statistical Report generated successfully
StatDataManager

server 33 tomcat1181829221515 TomcatSet

Passed Passed Passed Statistical Report generated successfully
LoggingDataManager

access_log.cbe.xml

Passed Passed Passed Top 10 Log Report generated successfully
LoggingDataManager

windows_app_log.cbe.xml

Passed Passed Passed Top 10 Log Report generated successfully
Sample Repository with Registration

Submit Query

Passed Passed Passed Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration

Register Configuration Items...

Passed Passed Passed Registration with Example MDR is successful
Sample Repository with Registration

Deregister Configuration Items...

Passed Passed Passed Deregistration with Example MDR is successful
Sample Repository with Registration

Submit Query

Passed Passed Passed Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration

View MetaData

Passed Passed Passed No Metadata available
Resource Repository

Submit Query

Passed Passed Passed Queries executed successfully with proper results
Resource Repository

View MetaData

Passed Passed Passed No Metadata available
AperiMDR

Submit Query

Passed Passed Passed Using Query builder able to execute and validate queries properly
AperiMDR

View MetaData

Passed Passed Passed Displayed MetaData properly
Sample Repository with Query

Get All Students

Passed Passed Passed Displayed Query Result for Students data correctly
Sample Repository with Query

Get All Teachers

Passed Passed Passed Displayed Query Result for Teachers data correctly
Sample Repository with Query

Get All Classes

Passed Passed Passed Displayed Query Result for Classes data correctly
Sample Repository with Query

Staff01

Passed Passed Passed Displayed data for 'students taught by teacher-staff01' correctly
Sample Repository with Query

Staff02

Passed Passed Passed Displayed data for 'students taught by teacher-staff02' correctly
Sample Repository with Query

Submit Query

Passed Passed Passed Using Query Builder, able to create and execute queries correctly
Sample Repository with Query

View MetaData

Passed Passed Passed Displayed MetaData information correctly

Data Visualization TPTP Manual Tests

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
navigate Passed Passed 6 nodes should be shown under "Data Managers"

No "Monitoring Data Repository" node, it should be "StatDataManager" node

navigate2 Passed Passed No "Create Query" found, it should be "Submit Query"
properties Passed Passed Given EPR is wrong, it should be http://localhost:8080/axis2/services/ExampleMdrDataManager
visualize log report Passed Passed No "Monitoring Data Repository" available, it should be "StatDataManager
visualize stat report Passed Passed None
properties 2 Passed Passed None
example cmdbf query Passed Passed XML output contents seems to be wrong, actual result shows only one student detail - "firstName="Mike" id="03 lastname="Lee""
SML CMDBF Graph Response Viewer Passed Passed None
Example MDR CMDBF Graph Response Viewer Passed Passed No cancel button available under details view window, it should be OK button.
Multiple Graph Response Viewers Passed Passed No "Cancel" button available under detail area, it should be Ok button
Registration Example Passed Passed None
DeRegistration Example Passed Passed None
Partial Registration of MDR with CMDBf Passed Passed None
Partial DeRegistration of MDR with CMDBf Passed Passed None

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Reports.testsuite

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
LogReport Passed Passed None
StatReport Passed Passed None

i10 End2End Supplement Tests (Negative)

COSMOS Command Line Client : Management Console

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
Configure tomcat port to 9080 (other than default port : 8080) and test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker

Expected Result:

  1. Should display COSMOS prompt without any errors
 Passed Passed None
Test Broker registration other than defaul port 8080 (say 9080):
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. All DataManagers should successfuly get registered to broker on port 9080
 Failed Failed DataManagers are getting registered on default port 8080 only, ignored actual port 9080
Test Broker registration multiple times:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
  3. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. Under tomcat console window, it should display "Service has already been registered at this broker. No new brokers are register

ed."

Passed Passed Found expected message under tomcat console window
Without broker registration, test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker getDataManagers

Expected Result:

  1. Should display empty list of DataManagers
 Passed Passed None
After succesful broker registration, test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker getDataManagers

Expected Result:

  1. Should display all 6 DataManagers registered with broker
 Passed Passed None
Without any broker registration, test the following
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> finddatamanagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. Should display all 6 DataManagers deployed under tomcat with port 9080
 Passed Passed None
Test deregistration of StatDataManager with valid hostname and valid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister domsr01-xp StatDataManager

Expected Result:

  1. Deregistration should be successful
 Passed Passed None
Test deregistration of StatDataManager with invalid hostname and valid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister unknown StatDataManager

Expected Result:

  1. Deregistration should fail and should display appropriate error message
 Failed Failed No Failure message, it still shows "DataManager deregistered"
Test deregistration of StatDataManager with valid hostname and invalid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister domsr01-xp StatDataMgr

Expected Result:

  1. Deregistration should fail and should display appropriate error message
 Failed Failed No Failure message, it still shows "DataManager deregistered"
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should be succesful.
 Passed Passed This is depended on Broker registration issue, fixed manually to test this.
Test graphQuery for SMLMDR, with invalid hostname,valid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery unknown SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displaying error message as "SMLMDR is not registered at the broker."
Test graphQuery for SMLMDR, with valid hostname,invalid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp StatDataManager D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displayed error message as "StatDataManager is not registered at the broker."
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,invalid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDRTest\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path with spaces:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR "D:\cosmos i10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt"

Expected Result:

  1. Query execution should be successful
 Failed Failed Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery for AperMdr with valid data, with valid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query should execute successfully
 Passed Passed Query executed successfully
Test graphQuery for AperMdr with invalid query data, with valid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query execution should fail and display appropriate error message
 Passed Passed Query execution failed with appropriate error message
Test graphQuery for AperMdr with valid data, with invalid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query execution should fail, with DB Connection error
 Passed Passed Displayed "aperiDB.getStorageSystems error"

Using the COSMOS Web User Interface

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
Open COSMOSUI with 8090 port (Make sure all COSMOS Components are up and running at this port)
  1. http://localhost:8090/COSMOSUI/?page=cosmos

Expected Result:

  1. Should display all 6 DataManagers properly
 Passed Passed Found All 6 DataManagers under COSMOS Web UI
Open COSMOSUI, Verify Properties of all DataManagers
  1. StatDataManager
  2. LoggingDataManager
  3. Sample Repository with Registration
  4. Sample Repository with Query
  5. Resource Repository
  6. AperiMdr

Expected Result:

  1. All properties should display the correct EPR with port 9080
 Passed Passed All EPR's under properties window displayed correctly
Test StatDataManager, "Statistical Report" generation:
  1. StatDataManager : server 33 tomcat1181829221515 CBE

Expected Result:

  1. Statistical Report should get generated properly
 Passed Passed Statistical Report generated successfully
Test "LoggingDataManager", Top 10 Log Report generation:
  1. LoggingDataManager : access_log.cbe.xml

Expected Result:

  1. Top 10 Log report should get generated properly
 Passed Passed Top 10 Log Report generated successfully
Test "Sample Repository with Registration",self registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "Sample Repository with Registration" under registration window

Expected Result

  1. Either it should not be allowed or it should display appropriate error message
 Failed Failed Registration successful, but no result data
Test "Sample Repository with Registration",Resource Repository registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "Resource Repository" under registration window
  4. Expected Result
  5. Either registration should not be allowed or it should display appropriate error message
 Failed Failed Registration successful, but no result data
Test "Sample Repository with Registration",AperiMdr registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "AperiMdr" under registration window

Expected Result

  1. Either registration should not be allowed or it should display appropriate error message
 Failed Failed Error "org.apache.axis2.AxisFault: AperiMDR error: relationshipTemplate 'all-relationships' is missing a recordConstraint"
Test Resource Repository, with valid query
  1. select "Resource Repository" and click on "Submit Query"
  2. Provide valid query and submit

Expected Result:

  1. Query should execute successfully
 Passed Passed Queries executed successfully with proper results
Test Resource Repository, with invalid query
  1. select "Resource Repository" and click on "Submit Query"
  2. Provide invalid query and submit

Expected Result:

  1. Query should fail with appropriate error message
 Passed Passed Query execution failed with appropriate error message
Test AperiMDR with proper db configuration and with valid query
  1. select "AperiMdr" and create valid query using Query Builder

Expected Result:

  1. Query should execute successfully
 Passed Passed Using Query builder able to execute and validate queries properly
Test AperiMDR with proper db configuration and with invalid query
  1. select "AperiMdr" and create invalid query using Query Builder

Expected Result:

  1. Query execution should fail and display appropriate error message
 Passed Passed Query execution failed and displayed error message
Test AperiMDR with invalid db configuration and with valid query
  1. select "AperiMdr" and create valid query using Query Builder

Expected Result:

  1. Query execution should fail with db connection error
 Passed Passed Query failed with db connection error
Test "Sample Repository with Query", with valid query
  1. select "Sample Repository with Query"
  2. Create a valid query from query builder and submit it

Expected Result:

  1. Query execution should be successful
 Passed Passed Query executed successfully
Test "Sample Repository with Query", with invalid query
  1. select "Sample Repository with Query"
  2. Create an invalid query from query builder and submit it

Expected Result:

  1. Should display Empty Query response under GraphResponseViewer
 Passed Passed Empty Query response displayed under GraphResponseViewer

i10 ERs - JUnits/Manual Tests Inspection

http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements, QA has verified only following ER's as most of the others are documentation related.

Bugzilla ID ID Summary JUnits Tests Manual Tests Remarks
214153 Data Visualization Customization - Customized CMDBf Query Builder NA Available Found Manual tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
214143 Predefined CMDBf Queries NA Available Manual Tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
220334 Additional Record Types for the Graph Reponse Viewer NA Available Found TPTP Manual tests defined under /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
221554 Provide a simple view to show the service meta data from an MDR. NA Available No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
217303 Support partial registering and deregister configuration Items NA Available No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
221901 Service side responses should be cached to reduce network calls Available NA No Manual tests found, seems to have JUnits, but unable to find it's location
205825 Update SML validator implementation based on changes to the SML latest draft NA NA This ER related to documentation
222982 Widget factory should generate unique ids when instantiating widgets NA Available No explicit tests found, I guess some tests under COSMOSUITests suite would cover this as well.
220949 MDRs need to accept vanilla web service requests NA Available No explicit tests provided, however this has been covered with End2End tests available at http://wiki.eclipse.org/COSMOS_DEMO_i10
220952 Broker to manage MDRs that do not support WSA and WSRF NA NA No Manual/Junits tests found for testing
220955 deregister data managers from broker NA Available covered under http://wiki.eclipse.org/COSMOS_DEMO_i10
221324 Additional Record Types for the Graph Reponse Viewer - Widget Available Available Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
221322 Additional Record Types for the Graph Reponse Viewer Available Available Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite

Basic Operational efficiency tests

Configuration Tests

Test Description Test Status Remarks
Testing the scenario where COSMOSUI running on windows and all COSMOS services running on Linux
  1. Install COSMOS service components under linux box
  2. Install COSMOSUI,COSMOSReportViewer webapps application under windows
  3. Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to linux hostname
  4. From windows, open http://localhost:8080/COSMOSUI

Expected Result:

  1. COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 Passed COSMOSUI from windows is working fine.
Testing the scenario where COSMOSUI running on linux and all COSMOS services running on windows
  1. Install COSMOS service components under windows
  2. Install COSMOSUI,COSMOSReportViewer webapps application under linux
  3. Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to windows hostname
  4. From linux, open http://localhost:8080/COSMOSUI

Expected Result:

  1. COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 Passed COSMOSUI from linux is working fine.

Concurrency Tests

Test Description Test Status Remarks
Concurrency tests, where multiple COSMOSUI's running on one machine
  1. Install all COSMOS components on windows PC
  2. open 2 COSMOSUI's under IE and 2 COSMOSUI's under Firebox browsers
  3. Perform various test operations - Generating Stat/Log reports on all COSMOSUI's simultaneously,Executing CMDBf queries on all COSMOSUI's simultaneously

Expected Result:

  1. All COSMOS UI's should work normally and should get appropriate results without any errors
 Passed All COSMOS UI's working normally
Concurrency tests, where multiple COSMOSUI's and multiple COSMOS Command Line Clients running on one machine
  1. Install all COSMOS components on windows PC
  2. open 1 COSMOSUI under IE and 1 COSMOSUI under Firebox browser
  3. open 2 COSMOS Command Line Shells
  4. Perform various test operations - Execute CMDBf queries on all COSMOS Clients simultaneously

Expected Result:

  1. All COSMOS clients should work normally and should get appropriate results without any errors
 Passed All COSMOS clients working properly

Volume/Performance Tests


Apache JMeter Tool - Setup Instructions

  1. Open Jmeter window (jmeter-install-dir\bin\jmeter.bat)
  2. On the left side, right click on the ‘Test Plan’ node and select ‘Add->Thread Group’. This inserts one child node called ‘Thread Group’ under ‘Test Plan’.
  3. Select the ‘Thread Group’ node. On the right side for ‘Number of Threads’ text box, give the number of samples.
  4. Right click on ‘Thread Group’ node, and select ‘Add->Sampler->SOAP/XML- RPC Request’. This inserts one child node called ‘SOAP/XML-RPC Request’ under ‘Thread Group’
  5. Select the ‘SOAP/XML-RPC Request’ node. On the right-side, in text box URL: give the URL to which the request to be send. In the ‘Soap/XML-RPC Data’ text area type the ‘SOAP request’
  6. Right click on ‘Thread Group’ node, and select ‘Add->Listener->Summary Report’. This inserts one child node called ‘Summary Report’ under the ‘Thread Group’. We can select our own format for displaying results (summary, table, tree…etc)
  7. Save the test
  8. Now, to run the test, from menu bar select Run->Start for running a test.
  9. Run->Stop for stopping test
  10. The results will be populated in the Listeners that we have added to the test.


Test Description Test Status Remarks
Using Apache JMeter Tool,testing Broker with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of Broker to Jmeter tool and run it with users simulation of 10,50,100,150,200.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" /> </soapenv:Body></soapenv:Envelope>

Expected Result:

  1. Broker requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing StatDataManager with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of StatDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.

Expected Result:

  1. StatDataManager requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing LoggingDataManager with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of LoggingDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.

Expected Result:

  1. LoggingDataManager requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing FederatingCMDB with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP requests of FederatingCMDB to Jmeter tool and run it with users simulation of 10,50,100,150,200.


<cmdbf:itemTemplate id="all-items"> </cmdbf:itemTemplate> <cmdbf:relationshipTemplate id="all-relationships"> <cmdbf:sourceTemplate ref="all-items" /> <cmdbf:targetTemplate ref="all-items" /> </cmdbf:relationshipTemplate> </cmdbf:query></soapenv:Body></soapenv:Envelope>


<cmdbf:mdrId /> <cmdbf:itemList> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Mike" id="03" lastName="Lee"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>03</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Jane" id="02" lastName="Ryerson"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>02</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Bob" id="01" lastName="Davidson"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<teacher>

<identity firstName="Heather" id="staff02" lastName="Reeba"> </identity>

</teacher> <cmdbf:recordMetadata> <cmdbf:recordId>staff02</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<teacher>

<identity firstName="Dawn" id="staff01" lastName="Johnson"> </identity>

</teacher> <cmdbf:recordMetadata> <cmdbf:recordId>staff01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> </cmdbf:itemList> <cmdbf:relationshipList> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="PHY01" name="Physics">

<students>

<enrolledStudent idRef="02"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>PHY01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>PHY01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="MAT01" name="Mathematics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="02"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>MAT01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>MAT01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="PHY01" name="Physics">

<students>

<enrolledStudent idRef="02"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>PHY01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>PHY01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="MAT01" name="Mathematics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="02"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>MAT01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>MAT01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="ECM01" name="Economics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff01"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>ECM01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>ECM01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="ECM01" name="Economics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff01"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>ECM01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>ECM01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> </cmdbf:relationshipList> </cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>

Expected Result:

  1. FederatingCMDB requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing ExampleMDR with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of ExampleMDR to Jmeter tool and run it with users simulation of 10,50,100,150,200.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing SMLMDR with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of SMLMDR to Jmeter tool and run it with user samples of 10,50,100,150,200

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="cableTemplate"> <st:recordConstraint> <st:propertyValue localName="format" namespace="propertyModel"> <st:equal>RJ45</st:equal> </st:propertyValue> </st:recordConstraint> </st:itemTemplate> </st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. SMLMDR requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing AperiMdr with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of AperiMdr to Jmeter tool and run it with 10,50,100,150,200 users simulation

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="StoragePool" namespace="http://schemas.aperimdr.org/schema/AperiMDR" /> </st:recordConstraint> </st:itemTemplate> </st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. AperiMdr requests with all above user samples should work fine with appropriate turn around time
 Passed None


Scalability/Stability Tests

Test Description Test Status Remarks
Using Apache JMeter Tool,testing ExampleMDR with multiple volumes of test data (data.xml) 1MB,

2MB,2.5MB with user samples of 10,50,100,150,200.

  1. Modify data.xml file to insert test data with 5000,10000,15000 student records
  2. Follow the procedure to create the test, running the test using JMeter.
  3. Provide below SOAP request of ExampleMDR to Jmeter tool and run it with 10,50,100,150,200 users simulation

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 Passed Found "Java Heap Space" and "Post too Large" configuration issues while testing with 2MB data.mxl file

QA Test Results

  • DataVisualization.UI.Component.Widget_1209468870674.execution

http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/common/org.eclipse.cosmos.examples.e2e.tests/manual-results/DataVisualization.UI.Component.Widget_1209468870674.execution?root=Technology_Project&view=log

http://wiki.eclipse.org/images/9/90/Cosmos_i10_performancetestResults.zip

Retrieved from "https://wiki.eclipse.org/index.php?title=COSMOS_i10_QA_Phase_Execution&oldid=96352"

This page was last modified 07:03, 7 May 2008 by Srinivas Reddy.

Back to the top