Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "COSMOS Design 220952"

(Broker)
(Broker)
 
(21 intermediate revisions by the same user not shown)
Line 58: Line 58:
 
=Design=
 
=Design=
 
==Broker==
 
==Broker==
'''Note:''' This section describe more features than what would be implemented in i10.  See the proposal at the end of the section for what will be implemented in i10.   
+
'''Note:''' This section describes more features than what would be implemented in i10.  See the proposal at the end of the section for what will be implemented in i10.   
* Remove the notion of management domain in i10, but we will have equivalent (or more) flexibility of providing a hierachical registry structure.  
+
* Remove the notion of management domain in i10, but we can have equivalent (or more) flexibility of providing a hierachical registry structure with the new design.  
 
* A broker is a general purpose web service registry.  It is not just designed for "data managers".
 
* A broker is a general purpose web service registry.  It is not just designed for "data managers".
 
* The COSMOS broker is a conceptually centralized registry for locating services.   
 
* The COSMOS broker is a conceptually centralized registry for locating services.   
Line 67: Line 67:
 
** a list of service groups: a "service group" is a logical collection of (related) web services
 
** a list of service groups: a "service group" is a logical collection of (related) web services
 
** a list of data managers: a "data manager" is a speicalized service group that includes the data manager service
 
** a list of data managers: a "data manager" is a speicalized service group that includes the data manager service
** a list of brokers:  This allows a hierarchy of brokers.  i.e. any broker can be a broker of brokers.  It is up to the deploy to decide how to organize the broker hierarchy.   
+
** a list of brokers:  This allows a hierarchy of brokers.  A broker can be a broker of brokers. (A broker is also a web service.) It is up to the deployer to decide how to organize the broker hierarchy.   
 
* Non-COSMOS MDR can be registered at the COSMOS broker.  As long as the MDR provides a query and/or registration service that comply to the CMDBf standard, the UI  can discover it and interact with it.   
 
* Non-COSMOS MDR can be registered at the COSMOS broker.  As long as the MDR provides a query and/or registration service that comply to the CMDBf standard, the UI  can discover it and interact with it.   
 
* The entry point of COSMOS is an EPR to a broker, from which the client can discover services, and other brokers.   
 
* The entry point of COSMOS is an EPR to a broker, from which the client can discover services, and other brokers.   
Line 76: Line 76:
 
|  +- service1
 
|  +- service1
 
|  +- service2
 
|  +- service2
+- service groups
+
+- groups
 
|  +- group1 (data manager)  
 
|  +- group1 (data manager)  
|  |  +- service1
+
|  |  +- service3
|  |  +- service2
+
|  |  +- service4
|  +- group1 (data manager)
+
|  +- group2 (data manager)
|    +- service1
+
|    +- service5
|    +- service2
+
|    +- service6
+- broker2 <- reference another broker
+
+- brokers
 +
  +- broker2 <- reference another broker
 +
  +- broker3 <- reference another broker
 
</pre>
 
</pre>
 +
* Example: (The set up for our end-to-end scenario)
 +
<pre>
 +
Broker
 +
+- groups
 +
  +- Example MDR
 +
  |  +- COSMOS Data Manager Service
 +
  |  +- CMDBf Query Service
 +
  +- SML MDR
 +
  |  +- COSMOS Data Manager Service
 +
  |  +- CMDBf Query Service
 +
  +- Sample Registration Service
 +
  |  +- COSMOS Data Manager Service
 +
  |  +- CMDBf Query Service
 +
  |  +- CMDBf Registration Service
 +
  +- Logging Data Manager
 +
  |  +- COSMOS Data Manager Service
 +
  |  +- Logging Service
 +
  +- Statistial Data Manager
 +
      +- COSMOS Data Manager Service
 +
      +- Stat Service
 +
</pre>
 +
 
* data structure at a broker: (COSMOS service registry schema)
 
* data structure at a broker: (COSMOS service registry schema)
 
<pre>
 
<pre>
 
<broker>
 
<broker>
   <serviceGroups>
+
   <groups>
     <serviceGroup>
+
     <group>
 
       <id></id>
 
       <id></id>
 
       <name></name>
 
       <name></name>
Line 101: Line 125:
 
         </service>+
 
         </service>+
 
       </services>
 
       </services>
     </serviceGroup>*
+
     </group>*
   </serviceGroups>?
+
   </groups>?
 
   <services>
 
   <services>
 
     <service>
 
     <service>
Line 111: Line 135:
 
     </service>*
 
     </service>*
 
   </services>?
 
   </services>?
 +
  <brokers>
 +
    <broker>
 +
      <epr></epr>
 +
    </broker>*
 +
  </brokers>?
 
</broker>
 
</broker>
 
</pre>   
 
</pre>   
Line 120: Line 149:
 
* Web service operations of the broker:
 
* Web service operations of the broker:
 
** registerService: request message payload is the service element
 
** registerService: request message payload is the service element
** '''registerServiceGroup: request message payload is the service group element'''
+
** '''registerServiceGroup: request message payload is the service group element (i10)'''
 
** deregisterService: Deregisters services that are not part of a service group. This operation will remove services from a service group.  Payload is EPR.  
 
** deregisterService: Deregisters services that are not part of a service group. This operation will remove services from a service group.  Payload is EPR.  
** '''deregisterServiceGroup: Deregister service group by group ID.'''
+
** '''deregisterServiceGroup: Deregister service group by group ID. (i10)'''
** getServices: get service by service namespace
+
** getServicesByNamespace: get service(s) given a service namespace (There may be more than one service with the same namespace because namespace is not a unique identifier.)
** getAllServices: get all services registered at the broker
+
** getAllServices: get all services registered at the broker, including services in service groups or not.
 
** getServiceGroup: get serviceGroup by group ID
 
** getServiceGroup: get serviceGroup by group ID
 
** getDataManager: same as getServiceGroup, but only return groups that include the data manager service
 
** getDataManager: same as getServiceGroup, but only return groups that include the data manager service
** '''getAllServiceGroups: get all service groups registered at the broker'''
+
** '''getAllServiceGroups: get all service groups registered at the broker (i10)''' <- The UI will use this operation.  The UI can treat each group as a "data manager", but this operation will also support non-COSMOS services. 
** '''getAllDataManagers: get all service groups registered at the broker'''
+
** getBrokers: get the list of brokers registered at this broker.
 
* Registration response data structure:  TBD
 
* Registration response data structure:  TBD
 
* Deregistration response data structure: TBD
 
* Deregistration response data structure: TBD
Line 136: Line 165:
 
** support WSA
 
** support WSA
  
==Service discovery service==  
+
==Service Discovery Service==  
 
* axis2 specific implementation
 
* axis2 specific implementation
* It helps finding service EPRs of axis2 web services deployed at a server. 
+
* COSMOS provides a web service that discovers all COSMOS data managers deployed on the same web server.   
* It helps finding data managers deployed at a server. 
+
i.e. all service groups that include a service with the namespace http://www.eclipse.org/cosmos/datamanager
* COSMOS provide a web service that discovers all COSMOS data managers deployed on the same web server.   
+
* The web service can also list out all Axis2 service groups available at the server
** the service will list out all service groups and their contents that include a service with the namespace http://www.eclipse.org/cosmos/datamanager
+
 
* operations:
 
* operations:
** getServiceGroups
+
** '''getServiceGroups'''
*** response payload: serviceGroup element
+
*** response payload: serviceGroups element
** getServices
+
** '''getDataManagers'''
 +
*** response payload: serviceGroups element
 +
** getServices (not supported in i10)
 
*** response payload: services element
 
*** response payload: services element
** getDataManagers
+
* operations responses use the COSMOS service registry schema.
*** response payload: serviceGroup element
+
* operation responses use the COSMOS service registry schema.
+
  
 
==Data Manager Service==
 
==Data Manager Service==
 
* *may* support auto-register function
 
* *may* support auto-register function
** will support this feature only if I find a solution to get service EPRs upon deployment of axis2 web services.  (looking for help...)
+
** will support this feature only if I find a solution to get service EPRs upon deployment of axis2 web services.   
 
** It is not a show-stopper.   
 
** It is not a show-stopper.   
 
* configures properties of the data manager in services.xml
 
* configures properties of the data manager in services.xml
Line 159: Line 187:
 
** getServices: return all details of the data manager.  Payload is the service group element
 
** getServices: return all details of the data manager.  Payload is the service group element
  
==Demo config program==
+
==Demo Config Program==
 
* Use service discovery service to find all data managers, and register them with the broker.
 
* Use service discovery service to find all data managers, and register them with the broker.
 
* This feature is very useful if we don't have auto-register feature.
 
* This feature is very useful if we don't have auto-register feature.

Latest revision as of 10:31, 1 April 2008

Broker to manage MDRs that do not support WSA and WSRF

Change History

Name: Date: Revised Sections:
Hubert Leung 03/04/2008
  • Initial version

Workload Estimation

Rough workload estimate in person weeks
Process Sizing Names of people doing the work
Design 0.2 Hubert Leung
Code 1.4 Hubert Leung
Test 0.6 Hubert Leung
Documentation 0.2 Hubert Leung
Build and infrastructure 0.2 Saurabh Dravid
Code review, etc.*
TOTAL 2.6

'* - includes other committer work (e.g. check-in, contribution tracking)

Purpose

This enhancement depends on enhancement 220949 - the support of stateless web services. The management domain and broker are services groups. They expect services to implement the WS-Addressing and WS-ResourceProperties standards. With the support of stateless web services, the broker will need to store addresses and properties of data managers that do not support WS-Addressing and WS-ResourceProperties standards.

Design

Broker

Note: This section describes more features than what would be implemented in i10. See the proposal at the end of the section for what will be implemented in i10.

  • Remove the notion of management domain in i10, but we can have equivalent (or more) flexibility of providing a hierachical registry structure with the new design.
  • A broker is a general purpose web service registry. It is not just designed for "data managers".
  • The COSMOS broker is a conceptually centralized registry for locating services.
  • The COSMOS broker can be a physically distributed network of brokers.
  • Each broker is a registry that consists of:
    • a list of web services: each web serivce entry has an EPR, a namespace, a display name and a description (optional)
    • a list of service groups: a "service group" is a logical collection of (related) web services
    • a list of data managers: a "data manager" is a speicalized service group that includes the data manager service
    • a list of brokers: This allows a hierarchy of brokers. A broker can be a broker of brokers. (A broker is also a web service.) It is up to the deployer to decide how to organize the broker hierarchy.
  • Non-COSMOS MDR can be registered at the COSMOS broker. As long as the MDR provides a query and/or registration service that comply to the CMDBf standard, the UI can discover it and interact with it.
  • The entry point of COSMOS is an EPR to a broker, from which the client can discover services, and other brokers.
  • We can have a "broker explorer" user interface that navigate the content of the COSMOS broker network. In i10, it is done using commmand line.
Broker1 <- entry point of system
+- services
|  +- service1
|  +- service2
+- groups
|  +- group1 (data manager) 
|  |  +- service3
|  |  +- service4
|  +- group2 (data manager)
|     +- service5
|     +- service6
+- brokers
   +- broker2 <- reference another broker
   +- broker3 <- reference another broker
  • Example: (The set up for our end-to-end scenario)
Broker
+- groups
   +- Example MDR
   |  +- COSMOS Data Manager Service
   |  +- CMDBf Query Service
   +- SML MDR
   |  +- COSMOS Data Manager Service
   |  +- CMDBf Query Service
   +- Sample Registration Service
   |  +- COSMOS Data Manager Service
   |  +- CMDBf Query Service
   |  +- CMDBf Registration Service
   +- Logging Data Manager
   |  +- COSMOS Data Manager Service
   |  +- Logging Service
   +- Statistial Data Manager
      +- COSMOS Data Manager Service
      +- Stat Service
  • data structure at a broker: (COSMOS service registry schema)
<broker>
  <groups>
    <group>
      <id></id>
      <name></name>
      <description></description>?
      <services>
        <service>
          <namespace></namespace>
          <endpoint></endpoint>
          <name></name>?
          <description></description>?
        </service>+
      </services>
    </group>*
  </groups>?
  <services>
    <service>
      <namespace></namespace>
      <endpoint></endpoint>
      <name></name>?
      <description></description>?
    </service>*
  </services>?
  <brokers>
    <broker>
      <epr></epr>
    </broker>*
  </brokers>?
</broker>
  • "Uniqueness" of services:
    • The namespace values are the identifiers of web services. However, it is not a unique identifier.
    • Usecase: The query service can be part of multiple service groups. Their namespace is the same, but EPRs are different. When searching for services by namespace, all EPRs tagged with that namespace will be returned.
    • EPR values are unique. That is, there is no two services in a broker with the same EPR.
  • Web service operations of the broker:
    • registerService: request message payload is the service element
    • registerServiceGroup: request message payload is the service group element (i10)
    • deregisterService: Deregisters services that are not part of a service group. This operation will remove services from a service group. Payload is EPR.
    • deregisterServiceGroup: Deregister service group by group ID. (i10)
    • getServicesByNamespace: get service(s) given a service namespace (There may be more than one service with the same namespace because namespace is not a unique identifier.)
    • getAllServices: get all services registered at the broker, including services in service groups or not.
    • getServiceGroup: get serviceGroup by group ID
    • getDataManager: same as getServiceGroup, but only return groups that include the data manager service
    • getAllServiceGroups: get all service groups registered at the broker (i10) <- The UI will use this operation. The UI can treat each group as a "data manager", but this operation will also support non-COSMOS services.
    • getBrokers: get the list of brokers registered at this broker.
  • Registration response data structure: TBD
  • Deregistration response data structure: TBD
  • Proposal: In version 1.0 (i10), we will only support the operations in bold, and the part of the data schema required for these operations. This is the scenario of having a single broker. The broker design can be extended to support the multi-broker configuration in the future. Since the UI in version 1.0 assumes there is only one broker and there is no urgent requirement for having multiple brokers, version 1.0 will only support the mimimum for completing the end-to-end scenario.
  • Other future extensions:
    • The schema is to be extended to support translation of translatable values like names and descriptions.
    • support WSA

Service Discovery Service

  • axis2 specific implementation
  • COSMOS provides a web service that discovers all COSMOS data managers deployed on the same web server.

i.e. all service groups that include a service with the namespace http://www.eclipse.org/cosmos/datamanager

  • The web service can also list out all Axis2 service groups available at the server
  • operations:
    • getServiceGroups
      • response payload: serviceGroups element
    • getDataManagers
      • response payload: serviceGroups element
    • getServices (not supported in i10)
      • response payload: services element
  • operations responses use the COSMOS service registry schema.

Data Manager Service

  • *may* support auto-register function
    • will support this feature only if I find a solution to get service EPRs upon deployment of axis2 web services.
    • It is not a show-stopper.
  • configures properties of the data manager in services.xml
  • operation:
    • getServices: return all details of the data manager. Payload is the service group element

Demo Config Program

  • Use service discovery service to find all data managers, and register them with the broker.
  • This feature is very useful if we don't have auto-register feature.

Open Issues/Questions

All reviewer feedback should go in the Talk page for 220952.

Back to the top