Jump to: navigation, search

Difference between revisions of "SMILA/Project Concepts/Generic Management And Monitoring Concept"

(Category Tree Construction / Registration)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Description ==
 
== Description ==
  
Now SMILA uses separate modules for Management and for Performance Counters but both uses JMX (jconsole) as a default presentation layer.
+
Currently SMILA uses separate modules for Management and for Performance Counters but both uses JMX/jconsole as a default presentation layer / UI.
  
During other components development it was found that both modules have issues.
+
During other SMILA components development it was found that both modules have issues.
  
This page is a concept about merging both modules into one generic module and to avoid issues found.
+
This page is a concept about merging both modules into one generic module and solving issues found.
  
 
== Discussion ==
 
== Discussion ==
Line 12: Line 12:
 
== Technical proposal ==
 
== Technical proposal ==
  
 
+
=== Current SMILA Modules ===
=== Performance Counters Module Architecture ===
+
==== Performance Counters Module Architecture ====
  
 
See [[SMILA/Project_Concepts/Performance_counters_API|Performance Counters API]]
 
See [[SMILA/Project_Concepts/Performance_counters_API|Performance Counters API]]
  
=== Management Module Architecture ===
+
==== Management Module Architecture ====
  
 
This chart shows the Management architecture:
 
This chart shows the Management architecture:
  
 
[[Image:ManagementRegistry.png]]
 
[[Image:ManagementRegistry.png]]
//TODO
 
  
  
=== Performance Counters Module Architecture issues===
+
=== Issues ===
 +
==== Performance Counters Module issues====
  
 
* its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
 
* its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
* its impossible to have few counters nested in one JMX leaf
+
* its impossible to have several counters nested in common JMX leaf
* its impossible to predict when counter will be initialized to start tracking because it initialized only on first usage
+
* its impossible to predict when counter will be initialized (to start tracking) because it initialized only on usage
  
=== Management Module Architecture issues ===
+
==== Management Module issues ====
  
 
* its impossible to group agents into tree
 
* its impossible to group agents into tree
  
=== Common issues  ===
+
==== Common issues  ====
 
* its impossible to mix performance counters and management agents methods nested in one JMX leaf
 
* its impossible to mix performance counters and management agents methods nested in one JMX leaf
  
 
=== Proposal ===
 
=== Proposal ===
Its suggested to use one registry from management module ( its more generic ) to use operations and counters.
+
==== Agent with Counters ====
 +
Its suggested to use one registry class to work with management operations and with counters. Its planned to upgrade management module registry and remove performance counters registry.
  
Description and  initialization of performance counters will be in the Management Agent implementation.
+
Description and  initialization of performance counters will be in the Agent implementation.
 
It will allow to group counters (also together with management operations).
 
It will allow to group counters (also together with management operations).
  
Line 51: Line 52:
  
 
   // some performance counter related to that module
 
   // some performance counter related to that module
   public PerformanceCounter getCounterOne();
+
   public Counter getCounterOne();
  
 
   // other performance counter related to that module
 
   // other performance counter related to that module
   public PerformanceCounter getCounterTwo();
+
   public Counter getCounterTwo();
 
   
 
   
 
}
 
}
 
</source>
 
</source>
  
But its also possible (and suggested) to separate them because dependencies. Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent
+
But its also possible (and suggested) to split agent into independent Management Agent and Counters Agent to avoid dependencies. Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent
  
 
<source lang="java">
 
<source lang="java">
Line 68: Line 69:
  
 
}
 
}
class MyCountersAgent implements Agent{
+
class MyMonitoringAgent implements Agent{
   public PerformanceCounter getCounterOne();
+
   public Counter getCounterOne();
   public PerformanceCounter getCounterTwo();
+
   public Counter getCounterTwo();
 
}
 
}
 
</source>
 
</source>
  
But how to use externally registered counters. For example Crawler Controller Agent register globally "Total" parameter. Each crawler should able to use it.
+
==== Usage ====
 +
But how to refer and use externally registered counters? For example, Crawler Controller Agent register globally "Total" counter. Each crawler instance should able to use it.
  
Mainly crawler should have access to CrawlerControllerAgent.
 
 
Its possible to pass CrawlerControllerAgent directly to crawler. Also if CrawlerControllerAgent was registered as service (its preferable way) its possible to find it by OSGi services registry. But also its possible to generate unique ID for crawler agent during registration and use it later.
 
Its possible to pass CrawlerControllerAgent directly to crawler. Also if CrawlerControllerAgent was registered as service (its preferable way) its possible to find it by OSGi services registry. But also its possible to generate unique ID for crawler agent during registration and use it later.
  
Line 84: Line 85:
  
 
       init(){
 
       init(){
         POC_AGENT_ID = ManagementRegistration.register(firstPocAgent)
+
         POC_AGENT_ID = Registry.register(firstPocAgent)
 
       }
 
       }
 
        
 
        
 
       dispose(){
 
       dispose(){
         ManagementRegistration.unregister(POC_AGENT_ID)
+
         Registry.unregister(POC_AGENT_ID)
 
       }
 
       }
 
   }
 
   }
Line 97: Line 98:
 
        
 
        
 
     usingTheFirstPocAgent(){
 
     usingTheFirstPocAgent(){
       FirstPocAgent firstPocAgent = ManagementRegistration.get(ServiceActivatorFirst.POC_AGENT_ID)
+
       FirstPocAgent firstPocAgent = Registry.get(ServiceActivatorFirst.POC_AGENT_ID)
 
       onePocAgent.getCounterOne().increment();
 
       onePocAgent.getCounterOne().increment();
 
     }
 
     }
Line 106: Line 107:
  
  
=== Category Tree Construction ===
+
==== Category Tree Construction / Registration ====
 
Its suggested to construct categories tree during registration
 
Its suggested to construct categories tree during registration
 
<source lang="java">
 
<source lang="java">
  ManagementCategory category = ManagementRegistration.getCategory("Crawler").getCategory("Administration");
+
 
 +
// manual categories ancestors iteration
 +
1.Option: Separation between Management and Monitoring
 +
  ManagementCategory category = ManagementRegistry.getCategory(ManagementRegistry.MANAGEMENTCATEGORY).getCategory("Crawler");
 +
2.Option: Management and Monitoring integrated
 +
ManagementCategory category = ManagementRegistry.getCategory("Crawler");
 +
 
  
 
  // first way to register
 
  // first way to register
  category.register(myAgent);
+
  category.register(agent);
 
  // the second way to register
 
  // the second way to register
  ManagementRegistration.register(category, agent);
+
  ManagementRegistry.register(category, agent);
 
+
// third way to register only a special Performance Counter (without an encapsulating agent-class) ## dynamically
 +
ManagementRegistry.register(category, performanceCounter);
 +
// third way to register only a some Performance Counters (without an encapsulating agent-class) ## dynamically
 +
ManagementRegistry.register(category, performanceCounterList);
  
 
  // unregister
 
  // unregister
  ManagementRegistration.unregister(agent);
+
  Registry.unregister(agent);
 
  // or
 
  // or
  category.unregister(myAgent);
+
  category.unregister(agent);
  
 
</source>
 
</source>
  
Also its suggested to assume that creation/removing of categories is done automatically by ManagementRegistration and ManagementCategory classes.
+
Also its suggested to assume that creation/removing of categories is done automatically by Registry and Category classes.
 +
 
 +
 
 +
Trees:
 +
1.Option: Separation between Management and Monitoring
 +
<source lang="java">
 +
SMILA
 +
- Management
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
- Monitoring
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
  - Crawlers
 +
    - FileSystemCrawler-HashID-DataSourceID
 +
    - FileSystemCrawler-HashID2-DataSourceID2
 +
    - WebSystemCrawler-HashID2-DataSourceID2
 +
  - Pipelines
 +
    - AddPipeline
 +
 +
</source>
 +
 
 +
2.Option: Management and Monitoring integrated
 +
<source lang="java">
 +
SMILA
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
  - Crawlers
 +
    - FileSystemCrawler-HashID-DataSourceID
 +
    - FileSystemCrawler-HashID2-DataSourceID2
 +
    - WebSystemCrawler-HashID2-DataSourceID2
 +
  - Pipelines
 +
    - AddPipeline
 +
 +
</source>
 +
 +
The Performance Counters are shown  under the JConsole Tab:Attributes and Management Functions are usable under the JConsole Tab:Operations.
 +
 
 +
Our suggestion is the second option.
  
 
[[Category:SMILA]]
 
[[Category:SMILA]]

Latest revision as of 12:39, 4 March 2009

Description

Currently SMILA uses separate modules for Management and for Performance Counters but both uses JMX/jconsole as a default presentation layer / UI.

During other SMILA components development it was found that both modules have issues.

This page is a concept about merging both modules into one generic module and solving issues found.

Discussion

Technical proposal

Current SMILA Modules

Performance Counters Module Architecture

See Performance Counters API

Management Module Architecture

This chart shows the Management architecture:

ManagementRegistry.png


Issues

Performance Counters Module issues

  • its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
  • its impossible to have several counters nested in common JMX leaf
  • its impossible to predict when counter will be initialized (to start tracking) because it initialized only on usage

Management Module issues

  • its impossible to group agents into tree

Common issues

  • its impossible to mix performance counters and management agents methods nested in one JMX leaf

Proposal

Agent with Counters

Its suggested to use one registry class to work with management operations and with counters. Its planned to upgrade management module registry and remove performance counters registry.

Description and initialization of performance counters will be in the Agent implementation. It will allow to group counters (also together with management operations).

class MyAgent implements Agent{
 
   // some management operation
   public  doSomething(args[]);
 
   // some performance counter related to that module
   public Counter getCounterOne();
 
   // other performance counter related to that module
   public Counter getCounterTwo();
 
}

But its also possible (and suggested) to split agent into independent Management Agent and Counters Agent to avoid dependencies. Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent

class MyManagementAgent implements Agent{
  ...
   public  doSomething(args[]);
  ....
 
}
class MyMonitoringAgent implements Agent{
   public Counter getCounterOne();
   public Counter getCounterTwo();
}

Usage

But how to refer and use externally registered counters? For example, Crawler Controller Agent register globally "Total" counter. Each crawler instance should able to use it.

Its possible to pass CrawlerControllerAgent directly to crawler. Also if CrawlerControllerAgent was registered as service (its preferable way) its possible to find it by OSGi services registry. But also its possible to generate unique ID for crawler agent during registration and use it later.

class ServiceActivatorFirst{
 
      init(){
        POC_AGENT_ID = Registry.register(firstPocAgent)
      }
 
      dispose(){
         Registry.unregister(POC_AGENT_ID)
      }
   }
 
   // in other class
 
    class TheSecondClass{
 
     usingTheFirstPocAgent(){
      FirstPocAgent firstPocAgent = Registry.get(ServiceActivatorFirst.POC_AGENT_ID)
      onePocAgent.getCounterOne().increment();
     }
 
    }


Category Tree Construction / Registration

Its suggested to construct categories tree during registration

 // manual categories ancestors iteration
 1.Option: Separation between Management and Monitoring
 ManagementCategory category = ManagementRegistry.getCategory(ManagementRegistry.MANAGEMENTCATEGORY).getCategory("Crawler");
 2.Option: Management and Monitoring integrated
 ManagementCategory category = ManagementRegistry.getCategory("Crawler");
 
 
 // first way to register
 category.register(agent);
 // the second way to register
 ManagementRegistry.register(category, agent);
 // third way to register only a special Performance Counter (without an encapsulating agent-class) ## dynamically
 ManagementRegistry.register(category, performanceCounter);
 // third way to register only a some Performance Counters (without an encapsulating agent-class) ## dynamically
 ManagementRegistry.register(category, performanceCounterList);
 
 // unregister
 Registry.unregister(agent);
 // or
 category.unregister(agent);

Also its suggested to assume that creation/removing of categories is done automatically by Registry and Category classes.


Trees:

1.Option: Separation between Management and Monitoring
 SMILA
 - Management
   - CrawlerController
   - LuceneService
   - DeltaIndexing
 - Monitoring
   - CrawlerController
   - LuceneService
   - DeltaIndexing
   - Crawlers
     - FileSystemCrawler-HashID-DataSourceID
     - FileSystemCrawler-HashID2-DataSourceID2
     - WebSystemCrawler-HashID2-DataSourceID2
   - Pipelines
     - AddPipeline
2.Option: Management and Monitoring integrated
 SMILA
   - CrawlerController
   - LuceneService
   - DeltaIndexing
   - Crawlers
     - FileSystemCrawler-HashID-DataSourceID
     - FileSystemCrawler-HashID2-DataSourceID2
     - WebSystemCrawler-HashID2-DataSourceID2
   - Pipelines
     - AddPipeline
The Performance Counters are shown  under the JConsole Tab:Attributes and Management Functions are usable under the JConsole Tab:Operations.

Our suggestion is the second option.