COSMOS Design 218313

Extend the registration libraries to make it easier to maintain data integrity between MDRs and federating CMDBs

Change History

Name:	Date:	Revised Sections:
Ali Mehregani	03/06/2008	Initial creation

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Design	0.5	Ali
Code	0.5	Ali
Test	1	Ali
Documentation	0.5
Build and infrastructure	0.1
Code review, etc.*	0.1
TOTAL	2.7

• - includes other committer work (e.g. check-in, contribution tracking)

Terminologies/Acronyms

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document:

Term	Definition
MDR	management data repository
CMDBf	specification for a CMDB that federates between multiple MDRs[1]
federating CMDB	the CMDB that federates between MDRs, offering a common access point to clients and correlating identifying record data
CMDB	configuration management database
Query service	MDRs make data available to Clients via a Query service. Queries may select and return items, relationships, and/or graphs containing items and relationships.
Registration service	MDR configuration items and relationships are reconciled by a federating CMDB via the registration service provided by the federating CMDB

Purpose

MDRs are required to periodically update/notify a federating CMDB of any changes to configuration items or relationships that have been registered. This will ensure data integrity between what is stored in MDRs and federating CMDBs. As part of enhancement https://bugs.eclipse.org/bugs/show_bug.cgi?id=215267, COSMOS provides a set of libraries for the registration service and client. The intend of this enhancement is to extend the registration client to make it easier for adopters to maintain data integrity. The code for this enhancement will reside in org.eclipse.cosmos.dc.mdr.registration.client.

Implementation Detail

A new class called RegistrationManager will be added to org.eclipse.cosmos.dc.mdr.registration.client to handle update notifications. The diagram below pictorially depicts the work flow for providing periodic updates.

Scope

Using the instanceIdSelector under the itemTemplate is equivalent to fetching documents with an org.eclipse.cosmos.rm.repository.provisional.resource.ISMLMetadata that has a specific ID. The property, xpath, and the record type selectors will all be supported under the itemTemplate element. Clients will also be able to use the propertySubsetDirective to return a set of desired properties, instead of full records.

The only valid relationship between items stored in an SML repository is the 'references' relationship. Item i1 references i2 if and only if there exists an inter-document reference of i2 in the SML document representation of i1. All relationshipTemplate elements should include a record type selector with:

• namespace = "http://cosmos.org"
• localName = "references"

No other selectors should be used in conjunction with this relationship.

The list below summarizes the scope of this enhancement:

• This enhancement will not provide an implementation of a federating CMDB
• The client will directly access the MDR and get data through the query service
• There will only be one MDR in COSMOS for i6. This will be the COSMOS SML repository
• The Registration service, as described by the CMDBf spec will not be implemented for i6 (Since there is no federating CMDB available)
• The SML Repository MDR can be referenced with the ID: http://cosmos.org/rm/sml/repository

Architecture

The flow diagram below depicts how the query service will process an input. The service will apply the selectors in the following order:

1. instanceIdSelector
2. recordTypeSelector
3. propertyValueSelector
4. xpath1Selector

The instance and record type selectors are faster and effective mechanisms of narrowing the result set for a query, whereas the property and xpath selectors are more processor intensive operations that are applied in the end.

The CMDBf query service will be implemented as an operation of the SML repository. Similar to the validation, import, and export operations, a client will first need to establish a connection to the repository to obtain the CMDBf query operation. The operation will take an input stream as argument and will produce an input stream as output. The argument is expected to be set to the XML query and the output will be the query response.

There will be a reusable component under the data collection subproject that will need to transform an XML query into a graph that is understandable by the query service. There also needs to be a common component to convert a graph representing a query response into an XML document that conforms to the CMDBf standard. The image below depicts the interaction between the query service provider, the SML repository, and the query service. The input and output transformers are reusable components that will reside under the data collection subproject. There needs to be a mirror operation provided for the output transformation to convert an XML query response into a graph representation. This is useful for clients that intend to traverse through a POJO representation of the response.

The diagram above does not include the sequence of steps needed for a client to access a query service end point. It only depicts the layer that will reside behind a WSDM end point that the COSMOS client will remotely access. The end-to-end interaction between the client and the query service should be described in the design of: https://bugs.eclipse.org/bugs/show_bug.cgi?id=200244. There needs to be a systematic method defined for a client to access the query service of MDRs.

Test Coverage

Unit tests will need to perform the following tests:

• Complex tests outlined in the Use Cases section of this document
• Tests using an instance selector, one with 0 results and one with exactly 1 result. More than one result will be logged as a failure.
• Tests using a property value selector, one for each of the following operators:
  • equal (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted; string, boolean, time, and integer types used)
  • less (negate set to true, false, or omitted; string, time, and integer types used)
  • lessOrEqual (negate set to true, false, or omitted; string, time, and integer types used)
  • greater (negate set to true, false, or omitted; string, time, and integer types used)
  • greaterOrEqual (negate set to true, false, or omitted; string, time, and integer types used)
  • contains (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted)
  • like (caseSensitive attribute set to true, set to false, and omitted; negate set to true, false, or omitted)
  • isNull (negate set to true, false, or omitted)
• Tests using a record type selector:
  • type is exact type match
  • type requested is supertype of existing record
• Tests using an XPath selector:
  • using correct syntax for expression, with and without results
  • using incorrect syntax for expression
• Tests combining 2 item templates and a relationship template, that use a drop selector to exclude the relationship template and the 2nd item template
• Tests using a property subset directive
  • with two properties listed
  • with no properties listed
• Tests using relationship cardinality, using minimum, maximum, and both attributes

Task Breakdown

The following section includes the tasks required to complete this enhancement along with a PERT chart that displays the dependencies of each task.

create sql methods to work with existing CBE COSMOS schema File:QueryService-PertChart.png

1. Create graph representation of query input and response
2. Create the input transformer
3. Create the output transformer
4. Add in the query service as an SML repository operation
5. Create a generic framework for implementing selectors
6. Add in the instance id selector
7. Create JUnit tests for the instance id selector
8. Add in the record type selector
9. Create JUnit tests for the record type selector
10. Add in the property value selector
11. Create JUnit tests for the property value selector
12. Add in the xpath selector
13. Create JUnit tests for the xpath selector
14. Implement the property subset directive
15. Ensure that all selectors work with itemTemplate elements
16. Implement the relationship template constraint
17. Add Junit tests for querying items and relationships

Related Links

• 204959 Design for refactoring of this work to be extensible

Open Issues/Questions

All reviewer feedback should go in the Talk page for 200222.

Issues (and potential errors) regarding the CMDBf spec, v0.95. Email responses from a member of the CMDBf expert group are marked in red.

• All of the examples and psuedo-schema in section 4.3.1 for <prefixMapping> have the attributes "prefix" and "value". However, the data model XSD in appendix B (and on the CMDBf site) lists the following as the two prefixMapping attributes as "prefix" and "namespace". When I wrote some <query> testcases based on the samples in the CMDBf spec, the schema validation failed.
• In the query result example in section 4.4, both <relationship> instances have the same recordId as each other. Agree
• In the relationship records in that example, the "foo" namespace maps to http://example.com/computerModel in both cases? Earlier, "comp" was used with that same URL, so it would seem the URL would need to be changed in the "foo" case, or the "foo" example should use "comp" again. Agree
• On line 1205, it says "http://example.com/administers/PeteTheLabLabTechToLabMachineA". Note the "LabLab" typo. Agree
• All of the included examples (such as on line 1086) begin with <query>. I believe they should instead look like this: <query xmlns="http://schemas.cmdbf.org/0-9-5/datamodel"> Making this change will make the query service code a lot easier to write, as the SAX validating parser only works if it has the correct xmlns value. It looks like some revisions of the document did include an xmlns attribute, but for some reason it was not included in the current one available at cmdbf.org. That's definitely the namespace for the <query> so including it is certainly correct & you should feel free to do so.
• Line 757 should be </contains> instead of </stringContains> Agree
• According to the pseudo schema of xpath1Selector, there are 0 or more prefix mappings that can be included under <xpath1Selector>. The CMDBf data model schema does not specify a minOccurs/maxOccurs for the prefixMapping element. <xs:element name="prefixMapping" type="cmdbf:PrefixMappingType" /> should be: <xs:element name="prefixMapping" type="cmdbf:PrefixMappingType" minOccurs="0" maxOccurs="unbounded"/>

---