View source for Talk:COSMOS Design 200222

This page includes all the discussion for [[COSMOS Design 200222]].

<hr>

== Valentina Popescu's Comments ==
<font color="#663300">
# Re: [[COSMOS Design 200222#Terminologies.2FAcronyms|SML Repository term]]: Shouldn't this be called COSMOS SML Repository to differentiate between this specific SML rep and a generic one ?
# Re: [[COSMOS Design 200222#Terminologies.2FAcronyms|Service description term]]: How exactly are supported record types defined? Is there a configuration file where SML record types to be used by the Query Service are logged ? Or every SML type defined by the Data Center is required to be supported?
# Re:[[COSMOS Design 200222#Purpose|'The output of the query will also be in XML format, which the client is expected to understand.']] I had a discussion with Ali today and went over what a client can and should understand from a CMDBf query. In summary : a CMDBf query is defined using the CMDBf query language ( the query set, containing one or more templates, etc ). This is part of the CMDBf spec and yes, it is straight forward that it is expected to be understood by the client initiating the query. But this is not enough to create a query. Aside of the CMDBf query language, which allows the user to format a query request, the actual 'content' of the query should use some sort of contract or understanding between the client and the Query Service implementor. So, for example, if a client defines a CMDBf query containing a template (I forgot the exact CMDBf naming for this.. ) asking for a 'user' element, the QueryService implementation should be able to map this 'user' with a type in the MDR's data source. It should know that when the query asks for a 'user', the MDR should actually get back all records with type = 'employee' ( user would map to an employee in this specific MDR. I think the way to achieve this is by having a common set of types at the federating CMDB level, and have all MDR's/QueryServices know about these types and know how to map their specific data to/from this generic types. We can start by defining this common types as being a subset of the current COSMOS SML Data Center Repository ( we can choose for example only machine, vm and os ); we should aim to have the common types be the CML library, once one is made available. Ali can probably explain all this in more detail.
# Re: [[COSMOS Design 200222#Purpose|SML repository convenience APIs]]: I don't think keeping the existing API's as an alternative to the QueryService will be a long lasting option; it should stay until COSMOS moves to CML. Once the COSMOS Data Center types will be replaced by a CML library type, these API's will cease to be as useful as they are now; the scope of the data will be extended to more than just machines, os, etc; it may be hard for the API to satisfy this extended spec
# Re: [[COSMOS Design 200222#Requirements|Requirements]]: It is implied from the usecases section that the clients will be only able to query the COSMOS Data Center repository. We should make this clear here; I understand this is how much will be delivered in i6 and I am okay with this but we need to clearly state it so we don't loose track of the fact that we address only this part of the puzzle.  The implications for this restriction: 
##the user will have to understand the Data Center data types, or at least the ones that are made available for query
##if not all the DataCenter types are planned to be made available for query, the MDR/Query Service will need a mechanism for registering the types it wants to make available; the user must know about this registration and must understand the content of the registry to the point where he can create a query using those types.

What needs to be done post i6:

* build a generic, common set of types a client can use to define a CMDBf query. This common type can be registered at the federation level; the common type should ideally be a subset of CML or the Data Center types
* every MDR/Query Service will know how to map its data to/from these common types
* a client will know how to create a query to address to the CMDB federation using these types; will know how to read the result, again containing common type-based information
</font>

== Replies to Valentina Popescu's Comments ==
<font color="#336633">
# I changed the text preceding the list of terms to indicate that this is regarding how the terms are used in this document.  I.e. when we refer to SML repository in this document, we mean the COSMOS one.
# I deleted the term from the list, since it really was outside the scope of this design
# This is something that should be considered down the road after i6.  It doesn't fall under the scope of 200222.  COSMOS does not intend to provide an implementation of a federating CMDB within i6.
# The convenience APIs are just an alternative way of querying for data.  I do agree that they will likely be deprecated when CML comes out but I have no intention of removing or deprecating them at this point.
# It is true that the COSMOS client will be tied to the data center examples; however the query service does not make any assumptions about the type of data stored in the SML repository.  I don't agree with adding a description that would imply a tight coupling between the implementation of 200222 and the data center example.
</font>

== Joel Hawkins' Comments ==
<font color="#663300">
# The Architecture diagram doesn't show the Property Subset directive implementation - it might be nice to pencil it in.
</font>

== Reply to Joel Hawkins' Comments ==
<font color="#336633">
# The flow diagram is missing the step of applying the property subset directive.  This was brought up by David during the review and acknowledged.  It will be handled as part of the implementation.  Thanks Joel for bringing that up.
</font>

== Sheldon Lee-Loy Comments ==
<font color="#663300">
# Would it be possible to provides examples how items, facets and properties are modeled in the GraphQuery response.   This will give the client a better understanding how to consume the response.
# I'm also wondering how references are modeled in the graph query.  If you have a SML facet that references an item will the reference contain an itemTemplate id?  This will allow the client to construct a CMDBf query to get the reference items. 
# From the CMDBf spec it seems that I can only query for items from the repository.  In which case the response will always be a list of items.  Therefore I assume that graph response will contain a list of items that contain its nested facets and properties.  If an item references another item the client would have to make another query.  Is this correct?
</font>

== Replies to Sheldon Lee-Loy's Comments ==
<font color="#336633">
# The query results are in a specific format described in section 4.3.2 of the CMDBf spec, and will be wrapped in an InputStream.  An example is found in section 4.4 of the CMDBf spec.  Do you think we should copy that example to the design for 200222?  [Sheldon's reply] No it's not necessary.
# Hopefully the example query response will help answer the question for you.  In short, the relationships are modeled in the <edges> section of the response, and the source and target denote the relationship endpoints.  The localId is what will allow you to correlate those endpoints with the items.  Does that answer your question?  [Sheldon's reply] This answers my question.  Thanks.
# I believe what you want is a <tt><propertySubsetDirective></tt>, which only returns properties instead of full items.  Also you can exclude other items or relationships by using a dropDirective.  I believe relationships are maintained separately so retrieving an entire item shouldn't give you its relationships unless you explicitly request them.
</font>

== Valentina Popescu's Comments ==
<font color="#663300">
# I don't think this comment is accurate 'According to the CMDBf specification, a repository will need to provide two services for it to be considered as an MDR : query and registration'. <br/> I searched the CMDBf spec and couldn't see any normative requirements that a database MUST implement both services in order to be an MDR. Since I can't find this in the spec, using my own logic makes me think that an MDR MUST implement only the QueryInterface since a client seems to be able to connect directly to an MDR and ask for data. The registration service is requited when the MDR is part of a CMDB federation where the federating CMDB uses the push mode to get the data.
# I understand that this design focuses only on the i6 deliverables which as far as I get from this document are as below. We can probably call this out more clearly: 
#* no federating CMDB
#* the client will directly access the MDR and get data through the query service
#* there is going to be only one MDR in COSMOS i6 the user can query from. This will be the COSMOS SML repository; 
#* a Query Service will be implemented for the COSMOS SML repository
#* the Registration service, as described by the CMDBf spec will not be implemented for i6 ( especially since no federating CMDB is around to make use of it )
# The set of goals we have for i6 make sense to me, having the timeframe and resources allocate to do this work. I think we should also mark here the work that needs to be done next in order to declare full CMDBf support. <br/> The items that I would note so that we don't loose track of them:
#* taxonomy for common data query/registration support
#* The usecase for i6 has only one MDR, which is the COSMOS SML repository; the data queried by the client is COSMOS SML type agnostic, which means the client creates a CMDBf query and asks for information using the COSMOS Data Center model structure. Once we move one step further and implement multiple MDR support, the client will need a common language for building a query (I don't think it's viable to assume  a client must know the data structure for every MDR he may communicate with and how this maps to his own understanding of those notions; as an example, if the user intention is to query for all computers with os=linux then he will have to translate this human readable query in a data format that can be understood  by any MDR he is trying to query; in one MDR, a 'computer' may be called 'asset' while in another it may be  called 'node'). 
#* the obvious one is implementing a federated CMDB
</font>

== Replies to Valentina Popescu's Comments ==
<font color="#336633">
# This was also raised during the review.  The CMDBf specification does imply that two services are required to be implemented by a repository.  This is also evident in their overall architectural diagram.  I have reworded the sentence a bit to avoid implying anything that the specification does not state.
# I have included your items under the scope section
# I don't think the items listed fall under the design of this enhancement.  I do agree that these are items that should be in the back of our mind for future considerations.
</font>

== General Comment ==
<font color="#663300">
# Why is it that we store SML documents as part of a CMDBf query response instead of embedding SML-IF content
</font>

== Reply to General Comment ==
<font color="#336633">
The primary purpose of an SML-IF document is to wrap a set of SML documents into one.  The meat of what is represented in SML is contained by the SML documents and not the SML-IF document.  The CMDBf spec already does that for us because of defining a structured response that we can send back to the client (i.e. we can achieve with a CMDBf response what SML-IF is there to do).  
<br/><br/>
Embedding SML-IF documents inside CMDBf response will be similar to having an envelope inside another envelope.  A client consuming the data will need to first parse through the CMDBf response document to extract items and relationships followed by parsing of SML-IF documents to extract SML documents representing resources.
<br/><br/>
The current approach COSMOS takes is to embed SML documents in CMDBf query replies.  Alternative approaches include:
<br/><br/>
# <b>CMDBf(SML-IF(SML)))</b> - i.e. wrapping each SML document in an SML-IF document that gets embedded in a CMDBf query reply.
# <b>CMDBf(One SML-IF (all SML documents)))</b> - i.e. wrapping all SML documents in one SML-IF document that gets embedded in a CMDBf query reply
# <b>One SML-IF (all SML documents)</b> - i.e. returning an SML-IF document that wraps all SML documents
<br/>
Here are the disadvantage of each approach:
<br/>
<b>Aproach 1:</b>
* Envelope within an envelope structure
* Multiple SML-IF documents to parse
* Wrapping one SML document inside an SML-IF document provides no benefit
<br/>
<b>Aproach 2:</b>
* Envelope within an envelope structure
* Inability to determine whether the SML-IF document should be included as an item or a relationship
<br/>
<b>Aproach 3:</b>
* Awkward to return a non-CMDBf structure in response to a CMDBf query

</font>