Difference between revisions of "SMILA/Documentation/AgentController"

From Eclipsepedia

Jump to: navigation, search
Line 1: Line 1:
 
== Overview ==
 
== Overview ==
  
The AgentController is a component that manages and monitors Agents.
+
The AgentController is a component that manages and monitors Agents. Whenever a new agent task is triggered (via <tt>startAgent()</tt>) a new instance of the used Agent is created and the agent object hash value is used a an id (called jobId) to identify records created by this agent instance. This jobid is set as an annotation on all records and will also visible on the agent instance in the JMX console.
  
 
== API ==
 
== API ==

Revision as of 05:11, 14 August 2009

Contents

Overview

The AgentController is a component that manages and monitors Agents. Whenever a new agent task is triggered (via startAgent()) a new instance of the used Agent is created and the agent object hash value is used a an id (called jobId) to identify records created by this agent instance. This jobid is set as an annotation on all records and will also visible on the agent instance in the JMX console.

API

AgentController provides two interfaces, one is used by management clients to start/stop agent instances, the other is used by Agents to execute callback methods on the AgentController itself, executing the ccommon processing logic.

/**
 * Management interface for the AgentController.
 */
public interface AgentController {
 
  /**
   * Starts an Agent using the given dataSourceId. This method creates a new Thread. If it is called for a dataSourceId
   * that is currently used by another agent a ConnectivityException is thrown. Returns the hashCode of the agent
   * instance used for performance counter.
   * 
   * @param dataSourceId
   *          the ID of the data source
   * @return - the hashcode of the agent instance as int value
   * @throws ConnectivityException
   *           if any error occurs
   */
  int startAgent(String dataSourceId) throws ConnectivityException;
 
  /**
   * Stops an active agent using the given dataSourceId.
   * 
   * @param dataSourceId
   *          the ID of the data source
   * @throws ConnectivityException
   *           if any error occurs
   */
  void stopAgent(String dataSourceId) throws ConnectivityException;
 
  /**
   * Checks if there are any active agents.
   * 
   * @return true if there are active agents, false otherwise
   * @throws ConnectivityException
   *           if any error occurs
   */
  boolean hasActiveAgents() throws ConnectivityException;
 
  /**
   * Returns a Collection of Strings containing the dataSourceIds of the currently active agents.
   * 
   * @return a Collection of Strings containing the dataSourceIds
   * @throws ConnectivityException
   *           if any error occurs
   */
  Collection<String> getActiveAgents() throws ConnectivityException;
 
  /**
   * returns the AgentController known Agents.
   * 
   * @return Collection with Strings
   */
  Collection<String> getAvailableAgents();
}
/**
 * Interface for callbacks on the AgentController. This interface is used by Agents to send add and delete requests and
 * to unregister an agent if a critical error occurred.
 */
public interface AgentControllerCallback extends ControllerCallback {
 
  /**
   * Add the given record.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param record
   *          the record to add
   * @param hash
   *          the hash value used for delta indexing
   */
  void add(final String sessionId, final DeltaIndexingType deltaIndexingType, final Record record, final String hash);
 
  /**
   * Delete the given id.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param id
   *          the id of the record to delete
   */
  void delete(final String sessionId, final DeltaIndexingType deltaIndexingType, final Id id);
 
  /**
   * Removes the Agent using the given DataSourceId from the list of active Agents.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param dataSourceId
   *          the ID of the data source used by the crawl
   */
  void unregister(final String sessionId, final DeltaIndexingType deltaIndexingType, final String dataSourceId);
}

Implementations

It is possible to provide different implementations for the AgentController interface. At the moment there is one implementation available.

org.eclipse.smila.connectivity.framework.impl

This bundle contains the default implementation of the AgentController interface.

The AgentController implements the general processing logic common for all types of Agents. Its interface is a pure management interface that can be accessed by its Java interface or its wrapping JMX interface. It has references to the following OSGi services:

  • ConnectivityManager
  • Agent ComponentFactory
  • ConfigurationManagement (t.b.d.)
  • CompoundManagement (t.b.d.)

Agent Factories register themselves at the AgentController. Each time an agent is started with a datasource for a specific type of agent, a new instance of that Agent type is created via the Agent ComponentFactory. This allows parallel watching of datasources with the same type (e.g. several rss feeds). Note that it is not possible to start muptiple agents on the same data source concurrently!


This chart shows the current AgentController processing logic for one agent run: AgentControllerProcessingLogic.png

  • the Agent is started, initializes DeltaIndexing for the data source by calling DeltaIndexingManager:init(String) and waits for events in a separate thread. One of the following events can occur:
    • ADD: a new or updated object on the datasource was detected. A record object is created. It is checked if the record was updated by calling DeltaIndexingManager:checkForUpdate(Id, String)
      • YES: the record is added to the Queue by calling ConnectivityManager:add(Record[]) and updated in the DeltaIndexingManager by calling DeltaIndexingManager:visit(Id, String, boolean)
      • NO: no actions are taken
    • DELETE: an object on the datasource was deleted. An Id object is created for the deleted object. This Id is deleted from both ConnectivityManager and DeltaIndexingManager by calling ConnectivityManager:delete(Id[])and DeltaIndexingManager:delete(Id[]).
    • STOP: the agent is stopped either via an external command or because some fatal errors occured
      • it finishes DeltaIndexing by calling DeltaIndexingManager:finish(String) and ends the thread

The processing logic will be enhanced when CompoundManagement is integrated.

Note

The exact logic depends on the settings of DeltaIndexing in the data source configuration. Depending on the configured value, delta indexing logic is executed fully, partially or not at all.


Configuration

There are no configuration options available for this bundle.

JMX interface

/**
 * The Interface AgentControllerAgent.
 */
public interface AgentControllerAgent {
 
  /**
   * Start agent.
   * 
   * @param dataSourceId
   *          the data source id
   * 
   * @return the string
   */
  String startAgent(final String dataSourceId);
 
  /**
   * Stop agent.
   * 
   * @param dataSourceId
   *          the data source id
   * 
   * @return the string
   */
  String stopAgent(final String dataSourceId);
 
  /**
   * Gets the active agents status.
   * 
   * @return the active agents status
   */
  String getActiveAgentTaskStatus();
 
  /**
   * Gets the active agents.
   * 
   * @return the active agents
   */
  String[] getActiveAgentTasks();
 
  /**
   * returns all Agents that have connected to the AgentController.
   * 
   * @return List with Strings of all available Agents
   */
  String[] getAvailableAgents();
 
}


Here is a screenshot of the AgentController in the JMX Console:

AgentControllerJMX.png