From Eclipsepedia

This page has been marked for deletion by the administrators. If this request is rescinded, please explain in Discussion.

IMPORTANT: this document is out of date. Please instead read the "Creating a Data Manager" section of the COSMOS Developer's Guide, available on the driver downloads pages.

Providing a CMDBf Query and Registration Service

Introduction

COSMOS provides a set of reusable, extensible classes and interfaces that can be utilized by others who intend to provide a CMDBf query service and/or a CMDBf registration service for a data provider. COSMOS itself uses these interfaces to register its own SML repository as an MDR that provides a query service. This document describes how adopters can accomplish the same for a different data provider.

The support for building a CMDBf query and/or registration service is broken into the following 5 modules:

1. CMDBfCommon.jar
2. CMDBfQueryTransformation.jar
3. CMDBfQueryService.jar
4. CMDBfRegistrationTransformation.jar
5. CMDBfRegistrationService.jar

This structure allows adopters to use the minimum set of modules needed for a service.

The CMDBfCommon.jar contains code that is common to all other modules, and must therefore be included in the runtime path in all cases.

Adopters can choose to use CMDBfQueryTransformation.jar as is or have it tweaked based on their needs. This jar file includes a concrete implementation of the transformations. Similarly, CMDBfRegistrationTransformation.jar includes a concrete implementation of the transformations required from an XML registration request to a POJO representation (and vice-versa), and also deregistration requests.

CMDBfQueryService.jar contains the necessary code for processing a query that conforms to CMDBf. This set will allow the user to provide implementation of handlers that are specific to a data provider. CMDBfQueryService.jar cannot be used as-is and will require a minimum API set to be implemented. Similarly, CMDBfRegistrationService.jar can be used by contributors to help in processing a registration request, and also requires a minimal implementation by consumers.

It is important to note that APIs found in the classes with the @provisional annotation have the long term intent of being public APIs but could change shape before they are finalized. You should reference the provisional classes in your code, instead of the internal classes, which are not public and will not be public in the future.

Overall Architecture

COSMOS' CMDBf library can be downloaded from the project's CVS. Use the following to setup a CVS connection to the project's repository:

Host: dev.eclipse.org
Repository Path: /cvsroot/technology
User: anonymous
Password: <Leave empty>
Connection type: pserver

Once a connection is successfully established, check-out the following project:

1. HEAD/org.eclipse.cosmos/data-collection/org.eclipse.cosmos.dc.cmdbf.services

The plug-in contains the source code for the common CMDBf services. Notice that there are five source folders corresponding to the modules mentioned in the previous section:

1. src-common
2. src-registration-transformation
3. src-query-transformation
4. src-query-service
5. src-registration-service

Each folder corresponds to a Jar file of its own. Adopters can pick and choose the Jar file that they intend to use as part of their CMDBf service implementation. If for example an adopter intends to only provide a CMDBf query service, then they will only need to include CMDBfCommon.jar, CMDBfQueryTransformation.jar, and CMDBfQueryService.jar in their class path. Keep the dependency of the modules in mind when determining which modules to include:

The transformation and the service code is described in more details in the next section.

Query and Registration Service

The query and the registration service code gives adopters the ability to expedite the implementation of a CMDBf query and registration service. The following sections will explore the details of what needs to be provided by adopters to provide or consume a CMDBf service.

CMDBf Query Service

The class diagram below depicts the relationship between the primary classes of the query service library. An ICMDBfQueryOperation is a general interface that represents the CMDBf query operation. CMDBfQueryOperation is a concrete implementation of the operation provided by COSMOS.

The following are the steps that the concrete implementation of the CMDBf query operation undergoes to process an incoming query:

1. Using the query transformation module, transform the XML input stream into a POJO representation
2. Walk through each item template and fetch all items that satisfy the constraints of the item template
3. Walk through each relationship template and eliminate any items collected in step (2) that fail to satisfy the constraints of the relationship template
4. Process the suppressFromResult and content selectors associated with each template
5. Return the result in the form of an IQueryResult.

The image below attempts to pictorially describe the process flow of the operation with an example. When it comes to evaluating the constraints of item or relationship templates, the operation will look up a handler factory of type IQueryHandlerFactory that adopters are expected to pass to the constructor of CMDBfQueryOperation. The factory will allow the operation to build a handler for each type of constraint that it encounters. An adopter can provide a query handler factory by either extending the abstract class: AbstractQueryHandlerFactory or by providing a direct implementation of IQueryHandlerFactory. A factory returns an implementation of the following handler types:

1. IItemTemplateHandler - Invoked for every item template encountered
2. IItemConstraintHandler - Invoked for every constraint encountered in an item template. Constraints include: instanceIdConstraint, recordType, propertyValue, and xpathExpression
3. IRelationshipTemplateHandler - Invoked for every relationship template encountered
4. IRelationshipConstraintHandler - Invoked for every constraint encountered in a relationship template. Constraints include: instanceIdConstraint, recordType, propertyValue, and xpathExpression

For each template, the query engine will first invoke the template handler followed by the constraint handler for each constraint. The constraints associated with an item/relationship template are processed in the following order:

1. Instance id constraints
2. Record type constraints
3. Property value constraints

OR

1. XPath constraints

For example an item template with the structure shown below will cause the query engine to invoke:

<itemTemplate ...>
 <instanceIdConstraint>
   <instanceId>...</instanceId>
   <instanceId>...</instanceId>
 </instanceIdConstraint>
 <recordConstraint>
   <recordType.../>
   <recordType.../>
   <propertyValue...> ...</propertyValue>
 </recordConstraint>
</itemTemplate>

1. The item template handler
2. The item constraint handler to handle the first instance id constraint
3. The item constraint handler to handle the second instance id constraint
4. The item constraint handler to handle the first record type constraint
5. The item constraint handler to handle the second record type constraint
6. The item constraint handler to handle the property value constraint

There is a context argument that is passed to each constraint handler based on the evaluation of the previous handler. If context.isStartingContext() returns true, then the handler should consider all items or relationships stored while evaluating the constraint. Each handler is expected to return a subset of the elements in the context argument passed in.

In addition to the IQueryHandlerFactory argument to the operation, there is also an argument to indicate the factory used for instantiating the POJO artifacts representing the query structure. A null value for the argument will use the default implementation of the factory. See IQueryInputArtifactFactory for more information.

The usefulness of the framework is best described using an example. The next section describes an example that builds a CMDBf query service on top of a simple XML-based repository.

CMDBf Query Example

The CMDBf query example can be retrieved from COSMOS CVS. If you are working with the latest codebase, you can check-out the project from: HEAD/org.eclipse.cosmos/examples/org.eclipse.cosmos.samples.cmdbf.services

The example is a Java project that is dependent on org.eclipse.cosmos.dc.cmdbf.services. Make sure that both projects are checked out before proceeding. The code for the query example that will be covered by this section is stored under the package: org.eclipse.cosmos.samples.cmdbf.services.query.

The sample project includes an XML based repository that implements the CMDBf query service. The class XMLRepository.java represents the XML repository. The file src/data.xml acts as the content of the repository. If you open the XML file, you'll notice that it includes 3 students, 2 teachers, and 3 classes. Each class indicates the students enrolled and the teacher supervising the class. The class XMLRepository.java uses the SAX handler SchoolXMLHandler.java to parse the XML file.

The query service implements the following handlers:

1. The item template handler is implemented by ItemTemplateHandler.java
2. The instance id constraint for item templates is implemented by ItemInstanceIdHandler.java
3. The record type constraint for item templates is implemented by ItemRecordTypeHandler.java
4. The relationship template handler is implemented by RelationshipTemplateHandler.java

The instantiation of the handlers is managed by the factory class: QueryHandlerFactory.java. Finally, the query launcher (i.e. QueryLauncher.java) acts as the main class to invoke CMDBf queries against the simple XML repository.

For convenience, there are a few query files provided under the src directory. By default, the query launcher will use teaches-relationship.txt but this can be modified by changing the value of the field queryFile under QueryLauncher.java. The three files are described below:

• all-students.txt - Used to query all students. The expected result is for all three students to be returned.
• all-teachers.txt - Used to query all teachers. The expected result is for both teachers to be returned.
• relationship-all-classes.txt - Queries all classes.
• teaches-relationship.txt - Used to query all students that are taught by the teacher with the id: "staff01". The expected result is for the following teacher, students, and class to be returned:
  • Teacher: Dawn Johnson
  • Students: Mike Lee and Bob Davidson
  • Class: Economics (ECM01)

To run the query launcher, simply have it launched as a Java application (i.e. Right click > Run As > Java Application). The output of the query is displayed in the console. For example, the output of the teaches-relationship.txt query will look similar to:

<queryResult xmlns="http://cmdbf.org/schema/1-0-0/datamodel">
	<nodes templateId="teacher">
		<item>
			<record>

				<teacher>
				 <identity firstName="Dawn" lastName="Johnson" id="staff01"/>
				</teacher>

				<recordMetadata>
					<recordId>staff01</recordId>
				</recordMetadata>
			</record>
			<instanceId>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>staff01</localId>
			</instanceId>
		</item>
	</nodes>
	<nodes templateId="students">
		<item>
			<record>

				<student>
				 <identity firstName="Mike" lastName="Lee" id="03"/>
				</student>

				<recordMetadata>
					<recordId>03</recordId>
				</recordMetadata>
			</record>
			<instanceId>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>03</localId>
			</instanceId>
		</item>
		<item>
			<record>

				<student>
				 <identity firstName="Bob" lastName="Davidson" id="01"/>
				</student>

				<recordMetadata>
					<recordId>01</recordId>
				</recordMetadata>
			</record>
			<instanceId>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>01</localId>
			</instanceId>
		</item>
	</nodes>
	<edges templateId="reference">
		<relationship>
			<source>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>staff01</localId>
			</source>
			<target>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>03</localId>
			</target>
			<record>

				<class name="Economics" courseCode="ECM01">
				 <students>
				  <enrolledStudent idRef="01"/>
				  <enrolledStudent idRef="03"/>
				 </students>
				 <teacher idRef="staff01"/>
				</class>

				<recordMetadata>
					<recordId>ECM01</recordId>
				</recordMetadata>
			</record>
		</relationship>
		<relationship>
			<source>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>staff01</localId>
			</source>
			<target>
				<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
				<localId>01</localId>
			</target>
			<record>

				<class name="Economics" courseCode="ECM01">
				 <students>
				  <enrolledStudent idRef="01"/>
				  <enrolledStudent idRef="03"/>
				 </students>
				 <teacher idRef="staff01"/>
				</class>

				<recordMetadata>
					<recordId>ECM01</recordId>
				</recordMetadata>
			</record>
		</relationship>
	</edges>
</queryResult>

CMDBf Registration Service

The class diagram below depicts the relationship between the primary classes of the registration service library. ICMDBfRegistrationOperation and ICMDBfDeregistrationOperation are general interfaces that represent the CMDBf registration and deregistration operations, respectively. CMDBfRegistrationOperation and CMDBfDeregistrationOperation are concrete implementations of these operations provided by COSMOS.

The following are the steps that the concrete implementation of the CMDBf registration operation undergoes to process an incoming query:

1. Using the registration transformation module, transform the XML input stream into a POJO representation
2. Walk through each item and invoke the item handler for the registration service
3. Walk through each relationship and invoke the relationship handler for the registration service
4. Return the result in the form of an IRegisterResponse

The steps for the concrete implementation of the CMDBf deregistration operation is exactly the same except that handler classes for the deregistration service are invoked.

The same plug-in that contains the query example also contains a registration and a deregistration example that illustrate the use of the APIs. The next section delves into more detail about the examples.

CMDBf Registration and Deregistration Examples

The plug-in org.eclipse.cosmos.samples.cmdbf.services contains both an example of a registration and a deregistration operation. Please see the CMDBf Query Example about how the plug-in can be retrieved.

The registration and deregistration sample share a set of common classes that are stored under the following package: org.eclipse.cosmos.samples.cmdbf.services.common. The registration launcher class, RegistrationLauncher.java, is stored under org.eclipse.cosmos.samples.cmdbf.services.registration. The class attempts to register 2 items and a relationship. The content of the registration request is stored under src/registration-request.txt. The output of the registration launcher should be similar to what is displayed below:

<registerResponse xmlns="http://cmdbf.org/schema/1-0-0/datamodel">
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>staff01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>ECM01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
</registerResponse>

Similarly the launcher class of the deregistration operation, DeregistrationLauncher.java, is stored in org.eclipse.cosmos.samples.cmdbf.services.deregistration. The class attempts to deregister 2 items and a relationship. the content of the deregistration request is stored under src/deregistration-request.txt. Once executed, the output to the console should look similar to what is displayed below:

<deregisterResponse xmlns="http://cmdbf.org/schema/1-0-0/datamodel">
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>staff01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
	<instanceResponse>
		<instanceId>
			<mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</mdrId>
			<localId>ECM01</localId>
		</instanceId>
		<accepted>
		</accepted>
	</instanceResponse>
</deregisterResponse>

The classes are well documented and the reader is recommended to step through the code to better understand the flow of each operation.

The next few sections will explore the transformation code for converting XML-based requests into POJO representations (and vice-versa).

Query Transformations

CMDBf specifies an XML-based syntax by which queries are submitted to the query service, which reports back a query result from the available MDRs, also using XML syntax. As part of this implementation of the query service, we internally accept POJOs to represent a graph of the query, and return the query result as a set of POJOs. The job of the transformers is to convert POJOs to and from the XML syntax detailed in the CMDBf specification.

The code provided in CMDBfQueryTransformation.jar contains the necessary code for clients who intend to build a CMDBf query and consume a query response. It performs the following transformations:

• CMDBf XML query document > POJO representation of the query
• POJO representation of the query > CMDBf XML query document
• CMDBf XML query response > POJO representation of the query response
• POJO representation of the query response > CMDBf XML query response

Adopters can choose to use CMDBfQueryTransformation.jar as is or have it tweaked based on their needs. The jar file includes a concrete implementation of the transformations.

Here is a simple example that illustrates invoking the incoming query input transformation:

InputStream inputStream = new ByteArrayInputStream(example1().getBytes());
IQuery query = QueryInputTransformer.transform(inputStream);

Example notes:

• Assume that example1() returns a String containing the XML syntax for a <query> as specified in the CMDBf spec.
• IQuery is the interface for the object that represents the <query> structure. There are similar interfaces provided for all elements in a query, and a factory class for creating these elements.

Here is a class diagram for the query input:

Here is a class diagram for the query response:

Here is an example of invoking the reverse transformation, assuming createQuery() returns an IQuery graph structure:

IQuery query = createQuery();
InputStream inputStream = QueryInputTransformer.transform(query);

The following diagram depicts the various query and registration transformations, grouped by their parent module:

Registration/Deregistration Transformations

CMDBf specifies a registration service by which MDRs register items and relationships with the federating CMDB. The process for registration involves an XML-based transaction, similar to how the query service works. COSMOS provides a transformer to convert a registration request from XML to a POJO graph structure, and also provides the reverse of that transformation. It provides the same for three other registration-related constructs: registration responses, deregistration requests, and deregistration responses.

The following classes provide the transformation capability:

org.eclipse.cosmos.dc.cmdbf.services.registration.transform.RegistrationInputTransformer
org.eclipse.cosmos.dc.cmdbf.services.registration.transform.RegistrationOutputTransformer
org.eclipse.cosmos.dc.cmdbf.services.deregistration.transform.DeregistrationInputTransformer
org.eclipse.cosmos.dc.cmdbf.services.deregistration.transform.DeregistrationOutputTransformer

Here is an example usage of the register request transformers:

// Transform from XML to POJO
InputStream inputStream = new ByteArrayInputStream(example1().getBytes());
IRegisterRequest request = RegistrationInputTransformer.transform(inputStream);

// Transform from POJO to XML
InputStream inputStream = RegistrationInputTransformer.transform(createRegisterRequest());

The code for the transformations of IRegisterResponse, IDeregisterRequest, and IDeregisterResponse have the same kind of syntax.

The object graph for each of those interfaces very closely mirrors the structure of the XML. Please consult the API documentation for more information on these.

CMDBf Service Metadata

Section 6 of the CMDBf 1.0 specification describes metadata used to indicate the optional features provided by an implementation. The CMDBf services plug-in provides a module that assists in defining metadata for a query and/or a registration service. CMDBfMetadata.jar is used to convert from an XML metadata description to a POJO representation (and vice-versa). The example plug-in includes metadata samples for both services under the following package:

org.eclipse.cosmos.sample.cmdbf.services.metadata

See QueryMetadataLauncher.java and RegistrationMetadataLauncher.java for more details

Creating an MDR using COSMOS Framework

COSMOS provides a framework for constructing an MDR that is discoverable using an end point reference. The framework uses the query and registration modules to expose the operations as capabilities that can remotely be invoked. Read more about how to use the COSMOS framework at Constructing a Data Manager.

Visualizing Data from a Registered MDR

COSMOS allows adopters to visualize data graphically from an MDR using its web user interface. When you implement an MDR from your data source, your data can be visualized, much like the data from the COSMOS-provided MDR (i.e. Asset Repository). If you have followed the process described in COSMOS Programming Model, and your record data content is XML-compliant, the visualization code should work automatically.

The following snapshots display an overview of COSMOS web user interface.

More on this process is documented in Visualizing Data from a Registered Data Manager.

Additional Examples

There are a number of other resources you can refer to as examples. The testing plug-in contains example code for using the transformers and for constructing the query and registration artifacts using the provisional APIs. These JUnit tests can be retrieved from CVS under:

HEAD/org.eclipse.cosmos/tests/data-collection/org.eclipse.cosmos.dc.cmdbf.services.tests

Important note: The cmdbf test plug-in currently depends on the following three resource modeling plug-ins:

HEAD/org.eclipse.cosmos/resource-modeling/org.eclipse.cosmos.rm.validation
HEAD/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.validation.tests
HEAD/org.eclipse.cosmos/resource-modeling/org.eclipse.cosmos.rm.repository

Make sure the value of 'testTempDirectory' is modified to point to a valid temporary directory under the following file: org.eclipse.cosmos.dc.cmdbf.services.tests/src/org.eclipse.cosmos.dc.cmdbf.services.tests.common/CMDBfTestMessages.properties

In addition to the plug-ins above, the test plug-in also depends on the TPTP project. A copy of the TPTP project can be obtained from: [1].

Besides the CMDBf JUnit test plug-in, there is a fully implemented MDR available in CVS for the SML repository, so you can inspect how the custom handlers were implemented for that MDR in determining your implementation. The SML repository plug-in and its dependencies are displayed below:

HEAD/org.eclipse.cosmos/resource-modeling/org.eclipse.cosmos.rm.repository
HEAD/org.eclipse.cosmos/resource-modeling/org.eclipse.cosmos.rm.validation

The test plug-in that exercises the CMDBf query functionality of the SML repository is stored under the package org.eclipse.cosmos.rm.repository.internal.operations.cmdbf, contained in the plug-in:

HEAD/org.eclipse.cosmos/tests/resource-modeling/org.eclipse.cosmos.rm.repository.tests

If you still need further assistance, please contact us via Bugzilla, the COSMOS newsgroup, or the cosmos-dev email list, so we can help you in your efforts, and so we can make improvements to this document or the implementation as necessary.