Jump to: navigation, search

Difference between revisions of "SMILA/Documentation/AgentController"

Line 68: Line 68:
 
  * to unregister an agent if a critical error occurred.
 
  * to unregister an agent if a critical error occurred.
 
  */
 
  */
public interface AgentControllerCallback {
+
public interface AgentControllerCallback extends ControllerCallback {
  
 
   /**
 
   /**
Line 75: Line 75:
 
   * @param sessionId
 
   * @param sessionId
 
   *          the delta indexing session Id
 
   *          the delta indexing session Id
 +
  * @param deltaIndexingType
 +
  *          the DeltaIndexingType
 
   * @param record
 
   * @param record
 
   *          the record to add
 
   *          the record to add
Line 80: Line 82:
 
   *          the hash value used for delta indexing
 
   *          the hash value used for delta indexing
 
   */
 
   */
   void add(final String sessionId, final Record record, final String hash);
+
   void add(final String sessionId, final DeltaIndexingType deltaIndexingType, final Record record, final String hash);
  
 
   /**
 
   /**
Line 87: Line 89:
 
   * @param sessionId
 
   * @param sessionId
 
   *          the delta indexing session Id
 
   *          the delta indexing session Id
 +
  * @param deltaIndexingType
 +
  *          the DeltaIndexingType
 
   * @param id
 
   * @param id
 
   *          the id of the record to delete
 
   *          the id of the record to delete
 
   */
 
   */
   void delete(final String sessionId, final Id id);
+
   void delete(final String sessionId, final DeltaIndexingType deltaIndexingType, final Id id);
  
 
   /**
 
   /**
Line 97: Line 101:
 
   * @param sessionId
 
   * @param sessionId
 
   *          the delta indexing session Id
 
   *          the delta indexing session Id
 +
  * @param deltaIndexingType
 +
  *          the DeltaIndexingType
 
   * @param dataSourceId
 
   * @param dataSourceId
 
   *          the ID of the data source used by the crawl
 
   *          the ID of the data source used by the crawl
 
   */
 
   */
   void unregister(final String sessionId, final String dataSourceId);
+
   void unregister(final String sessionId, final DeltaIndexingType deltaIndexingType, final String dataSourceId);
 
}
 
}
 
 
</source>
 
</source>
  

Revision as of 09:53, 10 June 2009

Overview

The AgentController is a component that manages and monitors Agents.

API

AgentController provides two interfaces, one is used by management clients to start/stop agent instances, the other is used by Agents to execute callback methods on the AgentController itself, executing the ccommon processing logic.

/**
 * Management interface for the AgentController.
 */
public interface AgentController {
 
  /**
   * Starts an Agent using the given dataSourceId. This method creates a new Thread. If it is called for a dataSourceId
   * that is currently used by another agent a ConnectivityException is thrown. Returns the hashCode of the agent
   * instance used for performance counter.
   * 
   * @param dataSourceId
   *          the ID of the data source
   * @return - the hashcode of the agent instance as int value
   * @throws ConnectivityException
   *           if any error occurs
   */
  int startAgent(String dataSourceId) throws ConnectivityException;
 
  /**
   * Stops an active agent using the given dataSourceId.
   * 
   * @param dataSourceId
   *          the ID of the data source
   * @throws ConnectivityException
   *           if any error occurs
   */
  void stopAgent(String dataSourceId) throws ConnectivityException;
 
  /**
   * Checks if there are any active agents.
   * 
   * @return true if there are active agents, false otherwise
   * @throws ConnectivityException
   *           if any error occurs
   */
  boolean hasActiveAgents() throws ConnectivityException;
 
  /**
   * Returns a Collection of Strings containing the dataSourceIds of the currently active agents.
   * 
   * @return a Collection of Strings containing the dataSourceIds
   * @throws ConnectivityException
   *           if any error occurs
   */
  Collection<String> getActiveAgents() throws ConnectivityException;
 
  /**
   * returns the AgentController known Agents.
   * 
   * @return Collection with Strings
   */
  Collection<String> getAvailableAgents();
}
/**
 * Interface for callbacks on the AgentController. This interface is used by Agents to send add and delete requests and
 * to unregister an agent if a critical error occurred.
 */
public interface AgentControllerCallback extends ControllerCallback {
 
  /**
   * Add the given record.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param record
   *          the record to add
   * @param hash
   *          the hash value used for delta indexing
   */
  void add(final String sessionId, final DeltaIndexingType deltaIndexingType, final Record record, final String hash);
 
  /**
   * Delete the given id.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param id
   *          the id of the record to delete
   */
  void delete(final String sessionId, final DeltaIndexingType deltaIndexingType, final Id id);
 
  /**
   * Removes the Agent using the given DataSourceId from the list of active Agents.
   * 
   * @param sessionId
   *          the delta indexing session Id
   * @param deltaIndexingType
   *          the DeltaIndexingType
   * @param dataSourceId
   *          the ID of the data source used by the crawl
   */
  void unregister(final String sessionId, final DeltaIndexingType deltaIndexingType, final String dataSourceId);
}

Implementations

It is possible to provide different implementations for the AgentController interface. At the moment there is one implementation available.

org.eclipse.smila.connectivity.framework.impl

This bundle contains the default implementation of the AgentController interface.

The AgentController implements the general processing logic common for all types of Agents. Its interface is a pure management interface that can be accessed by its Java interface or its wrapping JMX interface. It has references to the following OSGi services:

  • ConnectivityManager
  • Agent ComponentFactory
  • ConfigurationManagement (t.b.d.)
  • CompoundManagement (t.b.d.)

Agent Factories register themselves at the AgentController. Each time an agent is started with a datasource for a specific type of agent, a new instance of that Agent type is created via the Agent ComponentFactory. This allows parallel watching of datasources with the same type (e.g. several rss feeds). Note that it is not possible to start muptiple agents on the same data source concurrently!


This chart shows the current AgentController processing logic for one agent run: AgentControllerProcessingLogic.png

  • the Agent is started, initializes DeltaIndexing for the data source by calling DeltaIndexingManager:init(String) and waits for events in a separate thread. One of the following events can occur:
    • ADD: a new or updated object on the datasource was detected. A record object is created. It is checked if the record was updated by calling DeltaIndexingManager:checkForUpdate(Id, String)
      • YES: the record is added to the Queue by calling ConnectivityManager:add(Record[]) and updated in the DeltaIndexingManager by calling DeltaIndexingManager:visit(Id, String, boolean)
      • NO: no actions are taken
    • DELETE: an object on the datasource was deleted. An Id object is created for the deleted object. This Id is deleted from both ConnectivityManager and DeltaIndexingManager by calling ConnectivityManager:delete(Id[])and DeltaIndexingManager:delete(Id[]).
    • STOP: the agent is stopped either via an external command or because some fatal errors occured
      • it finishes DeltaIndexing by calling DeltaIndexingManager:finish(String) and ends the thread

The processing logic will be enhanced when CompoundManagement is integrated.

Note

The exact logic depends on the settings of DeltaIndexing in the data source configuration. Depending on the configured value, delta indexing logic is executed fully, partially or not at all.


Configuration

There are no configuration options available for this bundle.

JMX interface

/**
 * The Interface AgentControllerAgent.
 */
public interface AgentControllerAgent {
 
  /**
   * Start agent.
   * 
   * @param dataSourceId
   *          the data source id
   * 
   * @return the string
   */
  String startAgent(final String dataSourceId);
 
  /**
   * Stop agent.
   * 
   * @param dataSourceId
   *          the data source id
   * 
   * @return the string
   */
  String stopAgent(final String dataSourceId);
 
  /**
   * Gets the active agents status.
   * 
   * @return the active agents status
   */
  String getActiveAgentTaskStatus();
 
  /**
   * Gets the active agents.
   * 
   * @return the active agents
   */
  String[] getActiveAgentTasks();
 
  /**
   * returns all Agents that have connected to the AgentController.
   * 
   * @return List with Strings of all available Agents
   */
  String[] getAvailableAgents();
 
}


Here is a screenshot of the AgentController in the JMX Console:

AgentControllerJMX.png