COSMOS Design 197867

Data Broker for COSMOS DC framework

Change History:

Name: Date: Revised Sections:

Jimmy Mohsin 7/25/2007 Initial version

Bill Muldoon 8/1/2007 Added content

Workload Estimation (required)

 Rough workload estimate in person weeks: 
 Process	Sizing	Names of people doing the work
 Design	1	MR 2 development team
 Code	.5
 Test	1
 Documentation	.2
 Build and infrastructure	.5	Bill Muldoon
 Code review & other committer work (e.g. check-in, contribution tracking) 
 if this is to be contributed by someone who is not a committer in the component	0	
 N/A - will be done by committer
 Total	3.2

Terminologies/Acronyms (required)

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term:

Term Definition

 Data Broker	This is the component where all the web services that share data register themselves
 Data Store	This is a physical software artifact that stores data, e.g. Oracle, a File System, etc
 Data Manager	Largely corporate implementation component that implements the SOA API for data exchange
 Service Broker	This is the component where all the web services that share functional behavior register themselves

• [mdw] Can we add these to the glossary?

Function:

• Retrieving data from the data source
• Associating a query endpoint with a collection of data
• Distinguish b/t query and collecting (pushing/pulling)
• Storing and maintaining a list of data sources
• Introspect the keyset definition
• Provide a cross reference to services. How does this differ from the first one?
• Registration of new keysets & datasets

Purpose (required)

This design document addresses COSMOS Bugzilla enhancement request 197867.

The Data Broker maintains a catalog of information about Data Managers according to a classification type.

The Data Managers can register and deregister themselves with the Data Broker.

The Client applications can query the Data Broker to locate the Data Managers.

Requirements (required)

The “Data Broker” does the following:

1. Processes registration and unregistration requests from Data Managers

2. Processes queries from the client to locate the Data Managers

Use Cases (required)

<Include a set of common use cases that your feature will introduce or impact>

Use case 1: Register a Data Manager

Use case 2: Query for a Data Manager

Use case 3: Deregister a Data Manager

Graphical Layout (optional)

Not applicable.

Class Diagram and Implementation Details

The Data Broker is implemented as a COSMOS DC query assembly with the following components:

1. Query Assembly definition. Refer to org.eclipse.cosmos.dc.sample.configurations\META-INF\cosmos\DataBroker.xml

 <?xml version="1.0" encoding="UTF-8"?>
 <context xmlns="http://www.eclipse.org/xmlns/cosmos/1.0"
 	xmlns:cosmos="http://www.eclipse.org/xmlns/cosmos/1.0"
 	xmlns:sample="http://www.eclipse.org/xmlns/sample/1.0"
 	cosmos:name="DataBroker" cosmos:direction="out">
 	<query cosmos:factory="org.eclipse.cosmos.dc.sample.components.query.DataBroker" cosmos:optimizable="true">
 		<sample:binding/>
 	</query>    
 </context>

2. Implementation Class, which is referenced by the Query Assembly definition. The implementation class maintains the persistent store. Refer to org.eclipse.cosmos.dc.sample.components\src\org\eclipse\cosmos\dc\sample\components\query\DataBroker.java

3. Capability file, which defines the API and interface of the implementation class (see the next section). Refer to org.eclipse.cosmos.dc.spec\src\org\eclipse\cosmos\dc\spec\capabilitiy\DataBrokerCapability.java

4. The persistent store is implemented as a new table in the COSMOS DC runtime derby database. Refer to org.eclipse.cosmos.dc.local.registry\persistence_setup\RegistrySchema.sql

 CREATE TABLE COSMOS.DATA_MANAGER (  
       NAME VARCHAR(50) NOT NULL ,
       ASSEMBLY VARCHAR(50) NOT NULL ,
     	CLASSIFICATION VARCHAR(50),
     	HOSTNAME VARCHAR(25) NOT NULL ,
     	RUNTIMEPORT VARCHAR(10) NOT NULL );
     
 ALTER TABLE COSMOS.DATA_MANAGER
   ADD CONSTRAINT COSMOS.DATA_MANAGER_PK Primary Key (
     ASSEMBLY, HOSTNAME, RUNTIMEPORT );

The Persistent Store is accessed using an ibatis sql map. Refer to org.eclipse.cosmos.dc.sample.components\src\org\eclipse\cosmos\dc\sample\components\persistence\sql\DataManager.xml

 <?xml version="1.0" encoding="UTF-8" ?>
 <!DOCTYPE sqlMap PUBLIC "-//ibatis.apache.org//DTD SQL Map 2.0//EN"
   "http://ibatis.apache.org/dtd/sql-map-2.dtd">
 <sqlMap namespace="StatisticalDataset">
   <typeAlias alias="DataManager" type="org.eclipse.cosmos.dc.sample.persistence.impl.DataManagerImpl"/>
   <statement id="getDataManagers" resultClass="DataManager" parameterClass="String">
     select
       NAME as name,
       ASSEMBLY as assembly,
       CLASSIFICATION as classification,
       HOSTNAME as hostname,
       RUNTIMEPORT as runtimeport
     FROM COSMOS.DATA_MANAGER
     WHERE CLASSIFICATION = #classification#
   </statement>
   <statement id="getAllDataManagers" resultClass="DataManager">
     select
       NAME as name,
       ASSEMBLY as assembly,
       CLASSIFICATION as classification,
       HOSTNAME as hostname,
       RUNTIMEPORT as runtimeport
     FROM COSMOS.DATA_MANAGER
   </statement>
   <insert id="addDataManager" parameterClass="DataManager">
     INSERT INTO COSMOS.DATA_MANAGER (NAME, ASSEMBLY, CLASSIFICATION, HOSTNAME, RUNTIMEPORT) 
     VALUES(#name#, #assembly#, #classification#, #hostname#, #runtimeport#)
   </insert>
   <delete id="deleteDataManager" parameterClass="String">
     DELETE from COSMOS.DATA_MANAGER 
       WHERE NAME = #name#
   </delete>
 </sqlMap>

Command extensions for the data broker.

Refer to org.eclipse.cosmos.dc.spec\src\org\eclipse\cosmos\dc\spec\ConsoleExtension.java

  osgi> broker
  Usage: 
      broker query <classification>
      broker register DataManagerName assembly.xml classification hostname hostPort
      broker deregister DataManagerName

Data Broker Client API.

Refer to the Client API document for details.

Class Diagrams (optional)

<This section is only required for features that are introducing APIs that will be leveraged by other components>

The Data Broker exposes this API through its capability definition:

         // Get the Data Managers
 	public IDataQueryResult getDataManagers( String classification ) throws Exception;
 	// register the Data Manager	
 	public boolean registerDataManager( String name, String assembly, String classification, String hostname, String runtimeport ) throws Exception;
 	// deregister the Data Manager
 	public boolean deregisterDataManager( String name) throws Exception	

Extension Points (optional)

<This section should only be included if new extension points are being introduced>

<Include any additional topics that will help in completing the implementation>

<Content related to the topic above>

Test Coverage (required)

<Include a description of the unit, functional, and system test cases that will be required to test the overall quality of the feature>

The following tests are performed by org.eclipse.cosmos.dc.tests\src\org\eclipse\cosmos\dc\tests\client\DataBrokerTests.java

1. Data Broker Client interface

2. Registration test

3. Query test

4. Deregistration test

Task Breakdown (required)

<Includes the individual tasks that need to be completed to meet the overall objective of the enhancement. A PERT chart is included to indicate the dependency of each task.>

1. Query Assembly

2. Interface class

3. Implementation class

4. Derby database and iBatis sql map

5. Command extensions

6. Test application