Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "SMILA/Documentation/Architecture Overview"

(Description)
(Architecture Overview)
Line 20: Line 20:
 
'''Hint:'''
 
'''Hint:'''
 
For initial architecture proposal please see the [[SMILA/Attic/Architecture Overview|archived version]].
 
For initial architecture proposal please see the [[SMILA/Attic/Architecture Overview|archived version]].
 +
 +
Original slides can be found here: [[Media:SMILA Architecture.zip|SMILA Architecture.zip]]
 +
  
 
----
 
----

Revision as of 10:21, 1 September 2011

This page describes the short overview of SMILA's current architecture.

Introduction

SMILA is a framework that runs on top of OSGi runtime and therefore follows its component model.

Architecture Overview

SMILA Architecture Overview.png

Description

This architecture overview depicts generally two processes: preprocessing and information retrieval.

Note: In case where SMILA is used for building a search application, we talk about indexing and search process.

The preprocessing process generally includes the interaction with the data source either by pushing (updated) data by agents or by pulling data by crawlers and pushing it into the system via the connectivity module. The information that could be pushed into the framework is in general document's metadata, content and diverse security relevant information i.e. access rights. All this information is being persisted in dedicated stores. Metadata and access rights are stored in Record store. Content i.e. (large) binary data is stored in the binary store. Beside these two storages, SMILA also offers Delta Indexing store for keeping information about visited objects/documents during a crawling of a data source. Ontology store is a dedicated store for persisting and managing ontologies. The Blackboard service represents a high level API for accessing record information. After a record has been stored a new processing message has been created and pushed into a specific queue by a router. On the other side of a queue listeners are consuming messages, initializing Blackboard and starting a BPEL engine which executes desired workflow. The workflow again is defined by the order of execution of some services either provided by the framework itself or implemented by application's developer. Since listeners can consume messages remotely the preprocessing can easily be spread and therefore parallelized across the whole cluster. Thus the message queue is the central framework component which enables horizontal scaling of the preprocessing process in the framework. Message queue also introduces asynchronous execution of the business logic.

The information retrieval provides a swift access to previously preprocessed and stored information. Since this process is synchronous there has to be some external component responsible for distributing the load and therefore enabling the horizontal scalability of the information retrieval process. The flexible definition and execution of application's business logic is provided here also by calling a BPEL engine with a desired workflow.

Hint: For initial architecture proposal please see the archived version.

Original slides can be found here: SMILA Architecture.zip



Component Documentation

For further up to date documentation of all implemented components please see:

Component Documentation

Back to the top