Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "SMILA/Project Concepts/Generic Management And Monitoring Concept"

(→‎Category Tree Construction / Registration)
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Description ==
 
== Description ==
  
−
Now SMILA uses separate modules for Management and for Performance Counters but both uses JMX (jconsole) as a default presentation layer.
+
Currently SMILA uses separate modules for Management and for Performance Counters but both uses JMX/jconsole as a default presentation layer / UI.
  
−
During other components development it was found that both modules have issues.
+
During other SMILA components development it was found that both modules have issues.
  
−
This page is a concept about merging both modules into one generic module and to avoid issues found.
+
This page is a concept about merging both modules into one generic module and solving issues found.
  
 
== Discussion ==
 
== Discussion ==
Line 12: Line 12:
 
== Technical proposal ==
 
== Technical proposal ==
  
−
 
+
=== Current SMILA Modules ===
−
=== Performance Counters Module Architecture issues===
+
==== Performance Counters Module Architecture ====
  
 
See [[SMILA/Project_Concepts/Performance_counters_API|Performance Counters API]]
 
See [[SMILA/Project_Concepts/Performance_counters_API|Performance Counters API]]
  
−
* its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
+
==== Management Module Architecture ====
−
* its impossible to have few counters nested in one JMX leaf
+
−
* its impossible to predict when counter will be initialized to start tracking because it initialized only on first usage
+
−
 
+
−
=== Management Module Architecture issues ===
+
  
 
This chart shows the Management architecture:
 
This chart shows the Management architecture:
  
 
[[Image:ManagementRegistry.png]]
 
[[Image:ManagementRegistry.png]]
−
//TODO
+
 
 +
 
 +
=== Issues ===
 +
==== Performance Counters Module issues====
 +
 
 +
* its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
 +
* its impossible to have several counters nested in common JMX leaf
 +
* its impossible to predict when counter will be initialized (to start tracking) because it initialized only on usage
 +
 
 +
==== Management Module issues ====
  
 
* its impossible to group agents into tree
 
* its impossible to group agents into tree
  
−
=== Common problems ===
+
==== Common issues  ====
 
* its impossible to mix performance counters and management agents methods nested in one JMX leaf
 
* its impossible to mix performance counters and management agents methods nested in one JMX leaf
  
 
=== Proposal ===
 
=== Proposal ===
−
Its suggested to use one registry from management module ( its more generic ) to use operations and counters.
+
==== Agent with Counters ====
 +
Its suggested to use one registry class to work with management operations and with counters. Its planned to upgrade management module registry and remove performance counters registry.
  
−
Description and  initialization of performance counters will be in the Management Agent implementation.
+
Description and  initialization of performance counters will be in the Agent implementation.
 
It will allow to group counters (also together with management operations).
 
It will allow to group counters (also together with management operations).
  
Line 46: Line 52:
  
 
   // some performance counter related to that module
 
   // some performance counter related to that module
−
   public PerformanceCounter getCounterOne();
+
   public Counter getCounterOne();
  
 
   // other performance counter related to that module
 
   // other performance counter related to that module
−
   public PerformanceCounter getCounterTwo();
+
   public Counter getCounterTwo();
 
   
 
   
 
}
 
}
 
</source>
 
</source>
  
−
But its also possible (and suggested) to separate them because dependencies.
+
But its also possible (and suggested) to split agent into independent Management Agent and Counters Agent to avoid dependencies. Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent
−
Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent
+
  
 
<source lang="java">
 
<source lang="java">
Line 64: Line 69:
  
 
}
 
}
−
class MyCountersAgent implements Agent{
+
class MyMonitoringAgent implements Agent{
−
   public PerformanceCounter getCounterOne();
+
   public Counter getCounterOne();
−
   public PerformanceCounter getCounterTwo();
+
   public Counter getCounterTwo();
 
}
 
}
 
</source>
 
</source>
  
 +
==== Usage ====
 +
But how to refer and use externally registered counters? For example, Crawler Controller Agent register globally "Total" counter. Each crawler instance should able to use it.
  
 +
Its possible to pass CrawlerControllerAgent directly to crawler. Also if CrawlerControllerAgent was registered as service (its preferable way) its possible to find it by OSGi services registry. But also its possible to generate unique ID for crawler agent during registration and use it later.
 +
 +
<source lang="java">
 +
 +
class ServiceActivatorFirst{
 +
 +
      init(){
 +
        POC_AGENT_ID = Registry.register(firstPocAgent)
 +
      }
 +
     
 +
      dispose(){
 +
        Registry.unregister(POC_AGENT_ID)
 +
      }
 +
  }
 +
 +
  // in other class
 +
 
 +
    class TheSecondClass{
 +
     
 +
    usingTheFirstPocAgent(){
 +
      FirstPocAgent firstPocAgent = Registry.get(ServiceActivatorFirst.POC_AGENT_ID)
 +
      onePocAgent.getCounterOne().increment();
 +
    }
 +
 +
    }
 +
 +
</source>
 +
 +
 +
==== Category Tree Construction / Registration ====
 +
Its suggested to construct categories tree during registration
 +
<source lang="java">
 +
 +
// manual categories ancestors iteration
 +
1.Option: Separation between Management and Monitoring
 +
ManagementCategory category = ManagementRegistry.getCategory(ManagementRegistry.MANAGEMENTCATEGORY).getCategory("Crawler");
 +
2.Option: Management and Monitoring integrated
 +
ManagementCategory category = ManagementRegistry.getCategory("Crawler");
 +
 +
 +
// first way to register
 +
category.register(agent);
 +
// the second way to register
 +
ManagementRegistry.register(category, agent);
 +
// third way to register only a special Performance Counter (without an encapsulating agent-class) ## dynamically
 +
ManagementRegistry.register(category, performanceCounter);
 +
// third way to register only a some Performance Counters (without an encapsulating agent-class) ## dynamically
 +
ManagementRegistry.register(category, performanceCounterList);
 +
 +
// unregister
 +
Registry.unregister(agent);
 +
// or
 +
category.unregister(agent);
 +
 +
</source>
 +
 +
Also its suggested to assume that creation/removing of categories is done automatically by Registry and Category  classes.
 +
 +
 +
Trees:
 +
1.Option: Separation between Management and Monitoring
 +
<source lang="java">
 +
SMILA
 +
- Management
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
- Monitoring
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
  - Crawlers
 +
    - FileSystemCrawler-HashID-DataSourceID
 +
    - FileSystemCrawler-HashID2-DataSourceID2
 +
    - WebSystemCrawler-HashID2-DataSourceID2
 +
  - Pipelines
 +
    - AddPipeline
 +
 +
</source>
 +
 +
2.Option: Management and Monitoring integrated
 +
<source lang="java">
 +
SMILA
 +
  - CrawlerController
 +
  - LuceneService
 +
  - DeltaIndexing
 +
  - Crawlers
 +
    - FileSystemCrawler-HashID-DataSourceID
 +
    - FileSystemCrawler-HashID2-DataSourceID2
 +
    - WebSystemCrawler-HashID2-DataSourceID2
 +
  - Pipelines
 +
    - AddPipeline
 +
 +
</source>
 +
 +
The Performance Counters are shown  under the JConsole Tab:Attributes and Management Functions are usable under the JConsole Tab:Operations.
  
 +
Our suggestion is the second option.
  
 
[[Category:SMILA]]
 
[[Category:SMILA]]

Latest revision as of 13:39, 4 March 2009

Description

Currently SMILA uses separate modules for Management and for Performance Counters but both uses JMX/jconsole as a default presentation layer / UI.

During other SMILA components development it was found that both modules have issues.

This page is a concept about merging both modules into one generic module and solving issues found.

Discussion

Technical proposal

Current SMILA Modules

Performance Counters Module Architecture

See Performance Counters API

Management Module Architecture

This chart shows the Management architecture:

ManagementRegistry.png


Issues

Performance Counters Module issues

  • its impossible to group counters into tree legally by API ( now its used "JMX specific names" hack to group them )
  • its impossible to have several counters nested in common JMX leaf
  • its impossible to predict when counter will be initialized (to start tracking) because it initialized only on usage

Management Module issues

  • its impossible to group agents into tree

Common issues

  • its impossible to mix performance counters and management agents methods nested in one JMX leaf

Proposal

Agent with Counters

Its suggested to use one registry class to work with management operations and with counters. Its planned to upgrade management module registry and remove performance counters registry.

Description and initialization of performance counters will be in the Agent implementation. It will allow to group counters (also together with management operations). 

class MyAgent implements Agent{
 
   // some management operation
   public  doSomething(args[]);
 
   // some performance counter related to that module
   public Counter getCounterOne();
 
   // other performance counter related to that module
   public Counter getCounterTwo();
 
}

But its also possible (and suggested) to split agent into independent Management Agent and Counters Agent to avoid dependencies. Mainly Management Agent using service and not used inside the service. And is visa versa for Counters Agent 

class MyManagementAgent implements Agent{
  ...
   public  doSomething(args[]);
  ....
 
}
class MyMonitoringAgent implements Agent{
   public Counter getCounterOne();
   public Counter getCounterTwo();
}

Usage

But how to refer and use externally registered counters? For example, Crawler Controller Agent register globally "Total" counter. Each crawler instance should able to use it.

Its possible to pass CrawlerControllerAgent directly to crawler. Also if CrawlerControllerAgent was registered as service (its preferable way) its possible to find it by OSGi services registry. But also its possible to generate unique ID for crawler agent during registration and use it later. 

class ServiceActivatorFirst{
 
      init(){
        POC_AGENT_ID = Registry.register(firstPocAgent)
      }
 
      dispose(){
         Registry.unregister(POC_AGENT_ID)
      }
   }
 
   // in other class
 
    class TheSecondClass{
 
     usingTheFirstPocAgent(){
      FirstPocAgent firstPocAgent = Registry.get(ServiceActivatorFirst.POC_AGENT_ID)
      onePocAgent.getCounterOne().increment();
     }
 
    }


Category Tree Construction / Registration

Its suggested to construct categories tree during registration 

 // manual categories ancestors iteration
 1.Option: Separation between Management and Monitoring
 ManagementCategory category = ManagementRegistry.getCategory(ManagementRegistry.MANAGEMENTCATEGORY).getCategory("Crawler");
 2.Option: Management and Monitoring integrated
 ManagementCategory category = ManagementRegistry.getCategory("Crawler");
 
 
 // first way to register
 category.register(agent);
 // the second way to register
 ManagementRegistry.register(category, agent);
 // third way to register only a special Performance Counter (without an encapsulating agent-class) ## dynamically
 ManagementRegistry.register(category, performanceCounter);
 // third way to register only a some Performance Counters (without an encapsulating agent-class) ## dynamically
 ManagementRegistry.register(category, performanceCounterList);
 
 // unregister
 Registry.unregister(agent);
 // or
 category.unregister(agent);

Also its suggested to assume that creation/removing of categories is done automatically by Registry and Category classes.


Trees: 

1.Option: Separation between Management and Monitoring

 SMILA
 - Management
   - CrawlerController
   - LuceneService
   - DeltaIndexing
 - Monitoring
   - CrawlerController
   - LuceneService
   - DeltaIndexing
   - Crawlers
     - FileSystemCrawler-HashID-DataSourceID
     - FileSystemCrawler-HashID2-DataSourceID2
     - WebSystemCrawler-HashID2-DataSourceID2
   - Pipelines
     - AddPipeline
2.Option: Management and Monitoring integrated

 SMILA
   - CrawlerController
   - LuceneService
   - DeltaIndexing
   - Crawlers
     - FileSystemCrawler-HashID-DataSourceID
     - FileSystemCrawler-HashID2-DataSourceID2
     - WebSystemCrawler-HashID2-DataSourceID2
   - Pipelines
     - AddPipeline
The Performance Counters are shown  under the JConsole Tab:Attributes and Management Functions are usable under the JConsole Tab:Operations.

Our suggestion is the second option.

Retrieved from "https://wiki.eclipse.org/index.php?title=SMILA/Project_Concepts/Generic_Management_And_Monitoring_Concept&oldid=142512"

This page was last modified 13:39, 4 March 2009 by Sebastian Voigt. Based on work by Ivan Churkin.

Back to the top