Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Data Manager"
| Line 4: | Line 4: | ||
==Data Manager Diagram== | ==Data Manager Diagram== | ||
[[image:datamanager.jpg]] | [[image:datamanager.jpg]] | ||
| + | |||
| + | ==Data Manager== | ||
| + | The data manager is a component of the COSMOS framework. It is an adapter that plugs an existing data source into the COSMOS framework. It registers the data source with the COSMOS framework and handles with handshakes with the broker. It provides a query service for COSMOS clients. The query service understands CMDBf query or provides application specific WSDM capabilities for clients to gain access to data of the data source. It can make use of the data assembly to transform the output of the query to the desired output format (e.g. CMDBf response syntax). The request to the data manager may be in the form of a service request. (You may call this type of data manager a service manager.) The request can invoke a data assembly as a long running process that collects live statistical data from a server and write to a database. | ||
==Data assembly== | ==Data assembly== | ||
| Line 9: | Line 12: | ||
==Assembly component== | ==Assembly component== | ||
| − | An assembly is a software component that takes in an input data stream of a predefined structure, and output the processed data stream. Data processing may involve filtering, transforming input data into another format, or writing the data to database. An assembly component is reusable. It is agnostic of where input data come from, and who will consume its output. COSMOS may ship with some useful assembly components, e.g. CBE to WEF transformer. However, assembly components should be considered an extension of the data assembly framework, and is intended to be provided and customized by users of the COSMOS framework. | + | An assembly component is a software component that takes in an input data stream of a predefined structure, and output the processed data stream. Data processing may involve filtering, transforming input data into another format, or writing the data to database. An assembly component is reusable. It is agnostic of where input data come from, and who will consume its output. COSMOS may ship with some useful assembly components, e.g. CBE to WEF transformer. However, assembly components should be considered an extension of the data assembly framework, and is intended to be provided and customized by users of the COSMOS framework. |
| − | + | The blue boxes are “framework code” that comes with COSMOS download. The yellow boxes are extensions that users of the COSMOS framework need to provide. | |
| − | The | + | |
| + | '''Framework components: ''' | ||
| + | #CMDBf Query Service: If the data manager wants to be CMDBf compliant, it needs to understand CMDBf queries. COSMOS provide MDR interfaces via WSDM capabilities and utilities classes to access CMDBf query using Java APIs. | ||
| + | #Custom Query Service: provide application specific APIs (convenience API) to invoke some predefined queries with parameters. A data manager does not need to support both CMDBf query service and custom query service. | ||
| + | #Data manager control center: Code that handles the life cycle events of the data manager. For example, commands to start or stop the data manager. After it is started, it will contact the COSMOS broker or domain manager to register itself with the COSMOS system. It reads in a configuration file and use values in it during initialization. | ||
| + | #Data assembly: a framework for data transformation. Query adapter can hand data a data stream to the data assembly to transform data. | ||
| + | |||
| + | |||
| + | '''Extension code: ''' | ||
| + | #Implement a CMDBf query adapter (a.k.a. “data binding” in the IBM-CA slides) by coding against some public API of the data manager. The query adapter reads in values of the CMDBf query, and translates the query to a query language understood by the data source, or invokes some services of the data source that will return the information required by the query. | ||
| + | #Provide a configuration file for the data manager control center. Things to be configured include: | ||
| + | #*Class name or factory class of the query adapter: the implementation of the CMDBf query-to-native-query adapter | ||
| + | #*Path to data assembly config file: load the assembly on start up (if configured) | ||
| + | #*EPR of Data Manager (enough information for WSDM management clients to access the capabilities. | ||
| + | #*“Classification” of the data – an identifier of the data type, backed by a metadata definition. | ||
| + | #*Data source type | ||
| + | #*(other parameters required to register data manager with the broker) | ||
| + | #(optional) configure a data assembly by linking up some assembly components. | ||
| + | #*Assembly configuration file | ||
| + | #*Implement assembly components if necessary | ||
Revision as of 23:37, 29 August 2007
Contents
COSMOS System Diagram
Data Manager Diagram
Data Manager
The data manager is a component of the COSMOS framework. It is an adapter that plugs an existing data source into the COSMOS framework. It registers the data source with the COSMOS framework and handles with handshakes with the broker. It provides a query service for COSMOS clients. The query service understands CMDBf query or provides application specific WSDM capabilities for clients to gain access to data of the data source. It can make use of the data assembly to transform the output of the query to the desired output format (e.g. CMDBf response syntax). The request to the data manager may be in the form of a service request. (You may call this type of data manager a service manager.) The request can invoke a data assembly as a long running process that collects live statistical data from a server and write to a database.
Data assembly
A data assembly is a framework for data transformation. The framework can be configured declaratively in an XML file, by combining one or more assembly components together to execute the desired sequence of transformations to a data stream. The assembly is valid as long as the output of a component matches the input requirements of the next component in the assembly. While most use cases require a linear sequence of steps, it is also permissible for the assembly to be branched into multiple outputs, possibly multiplexed on certain conditions. This will make the assembly definition close to a workflow definition (a Compuware requirement).
Assembly component
An assembly component is a software component that takes in an input data stream of a predefined structure, and output the processed data stream. Data processing may involve filtering, transforming input data into another format, or writing the data to database. An assembly component is reusable. It is agnostic of where input data come from, and who will consume its output. COSMOS may ship with some useful assembly components, e.g. CBE to WEF transformer. However, assembly components should be considered an extension of the data assembly framework, and is intended to be provided and customized by users of the COSMOS framework.
The blue boxes are “framework code” that comes with COSMOS download. The yellow boxes are extensions that users of the COSMOS framework need to provide.
Framework components:
- CMDBf Query Service: If the data manager wants to be CMDBf compliant, it needs to understand CMDBf queries. COSMOS provide MDR interfaces via WSDM capabilities and utilities classes to access CMDBf query using Java APIs.
- Custom Query Service: provide application specific APIs (convenience API) to invoke some predefined queries with parameters. A data manager does not need to support both CMDBf query service and custom query service.
- Data manager control center: Code that handles the life cycle events of the data manager. For example, commands to start or stop the data manager. After it is started, it will contact the COSMOS broker or domain manager to register itself with the COSMOS system. It reads in a configuration file and use values in it during initialization.
- Data assembly: a framework for data transformation. Query adapter can hand data a data stream to the data assembly to transform data.
Extension code:
- Implement a CMDBf query adapter (a.k.a. “data binding” in the IBM-CA slides) by coding against some public API of the data manager. The query adapter reads in values of the CMDBf query, and translates the query to a query language understood by the data source, or invokes some services of the data source that will return the information required by the query.
- Provide a configuration file for the data manager control center. Things to be configured include:
- Class name or factory class of the query adapter: the implementation of the CMDBf query-to-native-query adapter
- Path to data assembly config file: load the assembly on start up (if configured)
- EPR of Data Manager (enough information for WSDM management clients to access the capabilities.
- “Classification” of the data – an identifier of the data type, backed by a metadata definition.
- Data source type
- (other parameters required to register data manager with the broker)
- (optional) configure a data assembly by linking up some assembly components.
- Assembly configuration file
- Implement assembly components if necessary