COSMOS Design 200222

Adding support for the CMDBf query service on top of the SML repository

Change History

Name:	Date:	Revised Sections:
David Whiteman	8/21/2007	Initial version
David Whiteman	8/27/2007	Added use cases

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Design	1	David/Ali
Code	2	David/Ali
Test	1	David/Ali
Documentation	0.4
Build and infrastructure	0.2
Code review, etc.*	0.2
TOTAL	4.8

• - includes other committer work (e.g. check-in, contribution tracking)

Terminologies/Acronyms

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document:

Term	Definition
MDR	management data repository
CMDBf	specification for a CMDB that federates between multiple MDRs[1]
federating CMDB	the CMDB that federates between MDRs, offering a common access point to clients and correlating identifying record data
CMDB	configuration management database
SML Repository	repository of SML documents hosted in COSMOS with an established API (see bugzilla 179828)
Query service	MDRs make data available to Clients via a Query service. Queries may select and return items, relationships, and/or graphs containing items and relationships.

Purpose

According to the CMDBf specification, a repository will need to provide two services for it to be considered as an MDR:

1. Registration
2. Query

The implementation of these services is the minimum requirement for a repository to be integrated with a federating CMDB. During the i6 time-frame a query service will be implemented for the SML repository. The service will allow clients to query the available data with the XML query language described in the CMDBf specification. The output of the query will also be in XML format, which the client is expected to understand.<vp>Re:'The output of the query will also be in XML format, which the client is expected to understand.' I had a discussion with Ali today and went over what a client can and should understand from a CMDBf query. In summary : a CMDBf query is defined using the CMDBf query language ( the query set, containing one or more templates, etc ). This is part of the CMDBf spec and yes, it is straight forward that it is expected to be understood by the client initiating the query. But this is not enough to create a query. Aside of the CMDBf query language, which allows the user to format a query request, the actual 'content' of the query should use some sort of contract or understanding between the client and the Query Service implementor. So, for example, if a client defines a CMDBf query containing a template (I forgot the exact CMDBf naming for this.. ) asking for a 'user' element, the QueryService implementation should be able to map this 'user' with a type in the MDR's data source. It should know that when the query asks for a 'user', the MDR should actually get back all records with type = 'employee' ( user would map to an employee in this specific MDR. I think the way to achieve this is by having a common set of types at the federating CMDB level, and have all MDR's/QueryServices know about these types and know how to map their specific data to/from this generic types. We can start by defining this common types as being a subset of the current COSMOS SML Data Center Repository ( we can choose for example only machine, vm and os ); we should aim to have the common types be the CML library, once one is made available. Ali can probably explain all this in more detail. </vp>

Currently clients are expected to interact with the SML repository via a set of convenience APIs. This enhancement does not imply the removal of such APIs. They may need to slightly be modified to accompany the new changes but they can still be reused by clients who have no intentions of leveraging the new query service.<vp>I don't think keeping the existing API's as an alternative to the QueryService will be a long lasting option; it should stay until COSMOS moves to CML. Once the COSMOS Data Center types will be replaced by a CML library type, these API's will cease to be as useful as they are now; the scope of the data will be extended to more than just machines, os, etc; it may be hard for the API to satisfy this extended spec</vp>

There will likely be common set of code between the COSMOS client and the SML repository. It's expected that this common set will reside under the data collection subproject. One of the primary objectives of this document is to identify this reusable set of common code.

Requirements

This section lists the requirements of this enhancement:<vp>It is implied from the usecases section that the clients will be only able to query the COSMOS Data Center repository. We should make this clear here; I understand this is how much will be delivered in i6 and I am okay with this but we need to clearly state it so we don't loose track of the fact that we address only this part of the puzzle. The implications for this restriction :

• the user will have to understand the Data Center data types, or at least the ones that are made available for query
• if not all the DataCenter types are planned to be made available for query, the MDR/Query Service will need a mechanism for registering the types it wants to make available; the user must know about this registration and must understand the content of the registry to the point where he can create a query using those types.

What needs to be done post i6:

• build a generic, common set of types a client can use to define a CMDBf query. This common type can be registered at the federation level; the common type should ideally be a subset of CML or the Data Center types
• every MDR/Query Service will know how to map its data to/from these common types
• a client will know how to create a query to address to the CMDB federation using these types; will know how to read the result, again containing common type-based information</vp>

1. Querying the SML repository using the XML query format specified in the CMDBf specification
2. Producing query results to clients using the XML response format specified in the CMDBf specification

Use Cases

For the following use cases, the SML repository acting as an MDR will contain the data described in the org.eclipse.cosmos.rm.example.datacenter COSMOS plug-in:

Use Case 1: User submits a query to retrieve all computers that have an installed OS with a brand name of "Red Hat 8"

Use Case 2: User submits a query to retrieve all computers with a RAM value greater than or equal to 2000 MB.

Use Case 3: User submits a query to retrieve all server software.

Use Case 4: User submits a query to retrieve the serial numbers and disk drive capacities of all computers.

Use Case 5: User submits a query to retrieve all software installed on Linux machines that is not a relational database.

Implementation Detail

The following section details out the implementation of the CMDBf query service for the SML repository. As mentioned before, the SML repository has defined a set of convenience APIs that it is currently leveraged by clients. The implementation of this enhancement will need to understand queries based on the CMDBf specification and make use of the convenience APIs to fetch the required data.

Scope

Using the instanceIdSelector under the itemTemplate is equivalent to fetching documents with an org.eclipse.cosmos.rm.repository.provisional.resource.ISMLMetadata that has a specific ID. The property, xpath, and the record type selectors will all be supported under the itemTemplate element. Clients will also be able to use the propertySubsetDirective to only return a set of desired properties.

The only valid relationship between items stored in an SML repository is the 'references' relationship. Item i1 references i2 if and only if there exists an inter-document reference of i2 in the SML document representation of i1. All relationshipTemplate elements should have a record type selector with:

• namespace = "http://cosmos.org"
• localName = "references"

No other selectors should be used in conjunction with this relationship.

Architecture

The flow diagram below depicts how the query service will process an input. The service will apply the selectors in the following order:

1. instanceIdSelector
2. recordTypeSelector
3. propertyValueSelector
4. xpath1Selector

The instance and record selectors are faster and effective mechanisms of narrowing the result set for a query, whereas the property and xpath selectors are more processor intensive operations that are applied in the end.

Class Diagrams (optional)

<This section is only required for features that are introducing APIs that will be leveraged by other components>

Extension Points (optional)

<This section should only be included if new extension points are being introduced>

<Include any additional topics that will help in completing the implementation>

<Content related to the topic above>

Test Coverage (required)

<Include a description of the unit, functional, and system test cases that will be required to test the overall quality of the feature>

Unit tests will need to perform the following tests:

• Complex tests outlined in the Use Cases section of this document
• Tests using an identity selector, one with 0 results and one with exactly 1 result. More than one result will be logged as a failure.
• Tests using a property value selector, one for each of the following operators:
  • equal (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted; string, boolean, time, and integer types used)
  • less (negate set to true, false, or omitted; string, time, and integer types used)
  • lessOrEqual (negate set to true, false, or omitted; string, time, and integer types used)
  • greater (negate set to true, false, or omitted; string, time, and integer types used)
  • greaterOrEqual (negate set to true, false, or omitted; string, time, and integer types used)
  • contains (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted)
  • like (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted)
  • isNull (negate set to true, false, or omitted)
• Tests using a record type selector:
  • type is exact type match
  • type requested is supertype of existing record
• Tests using an XPath selector:
  • using correct syntax for expression, with and without results
  • using incorrect syntax for expression
• Tests combining 2 item templates and a relationship template, that use a drop selector to exclude the relationship template and the 2nd item template
• Tests using a property subset directive
  • with two properties listed
  • with no properties listed
• Tests using relationship cardinality, using minimum, maximum, and both attributes

Open Issues/Questions

All feedback should go in the Talk page for 200222.

---