Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

SMILA/Documentation/ConnectivityFramework

< SMILA‎ | Documentation
Revision as of 06:03, 17 April 2009 by Igor.novakovic.empolis.com (Talk | contribs) (Overview)

Overview

The Connectivity framework, as the name suggests, provides a framework to easily integrate data from external systems into SMILA. To access external data two kinds of components are supported: Agents and Crawlers. To integrate some new datasource type into SMILA just a new Agent or Crawler has to be implemented.

Architecture

Here is a short overview of the components of the ConnectivityFramework:

  • AgentController: The Agent Controller implements the general processing logic common for all types of Agents. It's service interface is used by Agents to trigger add/update/delete actions. This component is not yet implemented!
  • Agents: Agents monitor a data source for changes (add/update/delete) or are triggered by events (e.g. trigger in databases) and reports those changes to the AgentController. Agents are not yet implemented!
  • Crawler Controller: The Crawler Controller implements the general processing logic common for all types of Crawlers.It's service interface is used by clients (e.g. JMX console) to start/stop crawls. For more information see SMILA/Documentation/CrawlerController.
  • Crawlers: A Crawler crawls a data source (e.g. a filesystem or a website) and returns all found data objects. For more information see SMILA/Documentation/Crawler.
  • CompoundManagement: Provides extractors for certain MimeTypes (e.g. zip, chm) and handles the processing of compound objects. This component is not yet implemented.


In addition there are two components that are not part of the ConnectivityFramework, but that interact with it:

  • ConnectivityManager: The Connectivity Manager is the single point of entry for data in the SMILA. The Agent- and CrawlerController push the data through this component into the Queue. For more information see SMILA/Documentation/ConnectivityManager.
  • Configuration Management: This component is not yet implemented. It is designed to manage configurations for all kinds of services, e.g. DataSources for crawlers. At the moment all configurations have to be provided locally in the SMILA configuration folder.


The following chart shows the architecture of the Connectivity Framework with it's plugable components (Agents/Crawlers) and relationship to the SMILA entry point Connectivity Module. Connectivity Framework Architecture

The red labeled components are not yet implemented.


Configuration

There is no overall configuration for the framework. Check out the documentation of each framework component for detailed infomation.

Copyright © Eclipse Foundation, Inc. All Rights Reserved.