Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "SMILA/Documentation"

(Scripting)
(162 intermediate revisions by 20 users not shown)
Line 1: Line 1:
This page provides an overview an all available documentation.
+
== Basics ==
 +
* [[SMILA/5_Minutes_Tutorial|Installing and Running (5 Minutes Tutorial)]]
 +
* [[SMILA/Documentation/HowTo|HowTos]]
 +
* [[SMILA/Documentation/Architecture_Overview|Architecture]]
 +
* [[SMILA/Documentation/Default_configuration_workflow_overview|Overview of Default Configuration]]
 +
* [[SMILA/Documentation/Data_Model_and_Serialization_Formats|Data Model, XML, JSON, BON]]
 +
* [[SMILA/Documentation/Using_The_ReST_API|Using the REST API, REST Client]]
 +
* [[SMILA/Documentation/REST_API_Reference|REST API Reference]]
 +
* [[SMILA/Documentation/Enable Remote Access|Enabling Remote Access to SMILA]]
 +
 
 +
== Development Environment ==
 +
* [[SMILA/Documentation/HowTo/Howto_set_up_dev_environment|Setting up your Eclipse IDE for SMILA]]
 +
* [[SMILA/Documentation/HowTo/Howto_build_a_SMILA-Distribution|Building SMILA]]
 +
* [[SMILA/Documentation/HowTo/Create_a_bundle_(plug-in)|Creating new Components]]
 +
* [[SMILA/Documentation/HowTo/Create_a_test_bundle_(plug-in)|Testing new Components]]
 +
* [[SMILA/Documentation/HowTo/3rd_party_bundle|Adding Third Party Libraries to SMILA]]
 +
* [[SMILA/Documentation/Declarative_Services|Using OSGi Declarative Services]]
 +
* Extending the build process:
 +
** [[SMILA/Documentation/HowTo/How_to_integrate_new_bundle_into_build_process|Adding a new Component Bundle to the Build]]
 +
** [[SMILA/Documentation/HowTo/How_to_integrate_test_bundle_into_build_process|Adding a new Test Bundle to the Build]]
 +
 
 +
== Pipelines and Pipelets: Synchronous Workflows ==
 +
* [[SMILA/Documentation/Pipelets|What are Pipelines? What are Pipelets?]]
 +
* [[SMILA/Documentation/BPEL_Workflow_Processor|Configuring and Creating BPEL Pipelines]]
 +
** [[SMILA/Documentation/HowTo/How_to_filter_and_access_record_data_in_BPEL|Accessing SMILA records in BPEL activities]]
 +
* [[SMILA/BPEL_Designer|Using the SMILA BPEL Designer]]
 +
* ReST APIs
 +
** [[SMILA/Documentation/Processing/JSON_REST_API_for_pipelets|Getting Pipelet Descriptions]]
 +
** [[SMILA/Documentation/Processing/JSON_REST_API_for_BPEL_pipelines|Getting Pipeline Descriptions]]
 +
** [[SMILA/Documentation/Processing/JSON_REST_API_for_BPEL_pipelines|Creating, Editing, and Executing Pipelines]]
 +
* Basic Pipelets
 +
** [[SMILA/Documentation/Bundle org.eclipse.smila.processing.pipelets|Common Pipelets]]
 +
** [[SMILA/Documentation/Bundle org.eclipse.smila.processing.pipelets.xmlprocessing|XML Processing Pipelets]]
 +
** [[SMILA/Documentation/Bundle org.eclipse.smila.processing.pipelets.boilerpipe|Boilerpipe Pipelet (extract text from HTML content)]]
 +
** [[SMILA/Documentation/TikaPipelet|TikaPipelet (extract text from binary content)]]
 +
** [[SMILA/Documentation/JdbcLoggingPipelet|JdbcLoggingPipelet, JdbcFetcherPipelet, JdbcSelectPipelet (write/read to/from a database)]]
 +
** More special pipelets are provided by the components described below.
 +
* Developing new Pipelets
 +
** [[SMILA/Documentation/HowTo/How_to_write_a_Pipelet|Writing a Pipelet]]
 +
** [[SMILA/Documentation/Usage_of_Blackboard_Service|Using the Blackboard Service]]
 +
 
 +
== Scripting ==
 +
 
 +
{{note| Available since SMILA 1.3!}}
 +
 
 +
* [[SMILA/Documentation/Scripting|Scripting SMILA with JavaScript]]
 +
* [[SMILA/Documentation/Scripting/Debugging|Debugging JavaScripts running in SMILA]]
 +
 
 +
== Searching ==
 +
 
 +
* [[SMILA/Documentation/Search|Search Processing and APIs]]
 +
* [[SMILA/Documentation/Solr 3.5|Solr Integration: Configuration and Pipelets]]
 +
 
 +
== JobManager: Asynchronous Workflows ==
 +
* [[SMILA/Documentation/JobManager|What are Jobs and Tasks?]]
 +
** [[SMILA/Documentation/JobManagerFirstExample|JobManager Walk-Through]]
 +
* Creating Workflows and Jobs
 +
** [[SMILA/Documentation/DataObjectTypesAndBuckets|Data Object Types and Buckets]]
 +
** [[SMILA/Documentation/WorkerAndWorkflows|Workers and Workflows]]
 +
** [[SMILA/Documentation/JobDefinitions|Jobs]]
 +
** [[SMILA/Documentation/JobParameters|Job Parameters]]
 +
* [[SMILA/Documentation/JobRuns|Running and Monitoring Jobs]]
 +
* [[SMILA/Documentation/JobManagerConfiguration|Configuring the Job Manager]]
 +
* [[SMILA/Documentation/TaskManager|TaskManager: Asynchronous Scheduling of Tasks]]
 +
* Worker Reference
 +
** [[SMILA/Documentation/Bulkbuilder|Bulkbuilder worker]]
 +
** [[SMILA/Documentation/Worker/ScriptProcessorWorker|ScriptProcessor Worker]]
 +
** [[SMILA/Documentation/Worker/PipelineProcessorWorker|PipelineProcessor Worker]]
 +
** [[SMILA/Documentation/Worker/PipeletProcessorWorker|PipeletProcessor Worker]]
 +
** (see [[SMILA/Manual#Importing|Importing]] section for more workers)
 +
* Developing new Workers
 +
** [[SMILA/Documentation/WorkerManager|WorkerManager: Workers Made Easily]]
 +
** [[SMILA/Documentation/HowTo/How_to_write_a_Worker|How to Write a Worker]]
 +
** [[SMILA/Documentation/TaskGenerators|Task Generators]]
 +
 
 +
== Importing ==
 +
* [[SMILA/Documentation/Importing/Concept|Concepts, Workflow and Components]]
 +
** [[SMILA/Documentation/Importing/CompoundExtractorService|Compound Extractor Service]]
 +
* Reference of Import Workers
 +
**[[SMILA/Documentation/Importing/Crawler/File | FileCrawler and FileFetcher Worker]]
 +
**[[SMILA/Documentation/Importing/Crawler/Web | WebCrawler and WebFetcher Worker]]
 +
*** [[SMILA/Documentation/Importing/CrawlingMultipleStartURLs | Crawling multiple start URLs in one job run]]
 +
**[[SMILA/Documentation/Importing/Crawler/JDBC | JdbcCrawler and JdbcFetcher Worker]]
 +
**[[SMILA/Documentation/Importing/Crawler/Feed | FeedCrawler Worker]]
 +
**[[SMILA/Documentation/Importing/DeltaCheck | DeltaChecker Worker]]
 +
**[[SMILA/Documentation/Importing/UpdatePusher | UpdatePusher Worker]]
 +
* Developing new Import Workers
 +
** [[SMILA/Documentation/Importing/VisitedLinks | Using the VisitedLinks service]]
 +
** [[SMILA/Documentation/Importing/Crawler/Web#Internal_structure|Extending the WebCrawler worker]]
 +
** [[SMILA/Documentation/HowTo/How to add a new Data Source to the importing framework|Adding a Data Source to the SMILA Import Framework]]
 +
* Additionally
 +
** [[SMILA/Documentation/Importing/RemoteCrawling | Remote Crawling]]
 +
 
 +
== Embedded HTTP Server ==
 +
* [[SMILA/Documentation/JettyHttpServer|Configuring Jetty]]
 +
* [[SMILA/Documentation/JettyHttpServer#JSON_Handlers|Developing JSON ReST Handlers for SMILA]]
 +
 
 +
== Common Services ==
 +
* [[SMILA/Project Concepts/Simple configuration handler|Configuration Helper]]
 +
* [[SMILA/Documentation/Workspace Handling|Workspace Helper]]
 +
* [[SMILA/Documentation/Bundle_org.eclipse.smila.clusterconfig|ClusterConfig Service]]
 +
** [[SMILA/Documentation/Bundle_org.eclipse.smila.clusterconfig.simple|Simple Implementation]]
 +
* [[SMILA/Documentation/Bundle_org.eclipse.smila.zookeeper|Zookeeper Service]]
 +
* [[SMILA/Documentation/ObjectStore/Bundle_org.eclipse.smila.objectstore|ObjectStore]]
 +
** [[SMILA/Documentation/ObjectStore/Bundle_org.eclipse.smila.objectstore.filesystem|Filesystem Objectstore Implementation]]
 +
* [[SMILA/Documentation/Binary_Storage|BinaryStorage]]
 +
* [[SMILA/Documentation/Security|Processing Security Information]]
 +
** [[SMILA/Documentation/LDAPSecurityResolver|Implementation for using LDAP]]
 +
** [[SMILA/Documentation/SecurityProcessingServices|Example Pipelets]]
 +
* [[SMILA/Documentation/SesameOntologyManager|Ontology Processing with Sesame: Configuration and Pipelets]]
 +
* [[SMILA/Documentation/MimeTypeIdentifier|MimeTypeIdentifier]]
 +
* [[SMILA/Documentation/ParameterDefinition|Description of Worker and Pipelet Parameters]]
 +
* [[SMILA/Documentation/PublishingJAXWSWebservices|Publishing Web Services]]
 +
* [[SMILA/Documentation/General JPA Configuration in SMILA|General JPA Configuration in SMILA]]
 +
* [[SMILA/Documentation/SMILA_Versioning|SMILA Version Information]]
 +
* [[SMILA/Documentation/Adding_JDBC_Drivers|Adding JDBC Drivers]]
 +
 
 +
== Deprecated Components ==
 +
 
 +
** [[SMILA/Documentation/Management|JMX Management]]
 +
*** [[SMILA/Documentation/Management#JMX_Client|JMX Clients]]
 +
** [[SMILA/Documentation/Record_Storage|RecordStorage]]
  
* Connectivity Framework
 
** [[SMILA/Documentation/Web Crawler|Web Crawler]]
 
** ...
 
* Router / Filter / Listener Configuration
 
** [[SMILA/Documentation/QueueWorker|Queue Worker]]
 
*** [[SMILA/Documentation/QueueWorker/BrokerConnectionService|Broker Connection Service]]
 
*** [[SMILA/Documentation/QueueWorker/Router|Router]]
 
*** [[SMILA/Documentation/QueueWorker/Listener|Listener]]
 
** [[SMILA/Documentation/Record Filter|Record Filter]]
 
* Processing
 
** [[SMILA/Documentation/BPEL Workflow Processor|BPEL Workflow Processor]]
 
** [[SMILA/Documentation/Usage of Blackboard Service|Usage of Blackboard Service]]
 
** [[SMILA/Documentation/Pipelets and ProcessingServices|Pipelets and ProcessingServices]]
 
** Available Pipelets
 
*** [[SMILA/Documentation/Bundle org.eclipse.eilf.processing.pipelets|Bundle org.eclipse.eilf.processing.pipelets]]
 
*** [[SMILA/Documentation/Bundle org.eclipse.eilf.processing.pipelets.xmlprocessing|Bundle org.eclipse.eilf.processing.pipelets.xmlprocessing]]
 
*** [[SMILA/Documentation/AperturePipelet|AperturePipelet]]
 
*** ...
 
** Available ProcessingServices
 
*** [[SMILA/Documentation/LuceneIndexService|LuceneIndexService]]
 
*** [[SMILA/Documentation/ApertureMimeTypeIdentifier|ApertureMimeTypeIdentifier]]
 
*** ...
 
* XML Storage
 
** [[SMILA/Documentation/XMLDataStorage|XML data storage]]
 
** [[SMILA/Documentation/XML storage|XML storage]]
 
** [[SMILA/Documentation/XQJ Implementation|XQJ Implementation]]
 
* Utilities
 
** [[SMILA/Documentation/Workspace Handling|Workspace Handling]]
 
** ...
 
* Concerning Third Party Software
 
** [[SMILA/Documentation/SNAPSHOT versions - which and why|SNAPSHOT versions - which and why]]
 
  
 
[[Category:SMILA]]
 
[[Category:SMILA]]

Revision as of 09:03, 6 November 2014

Basics

Development Environment

Pipelines and Pipelets: Synchronous Workflows

Scripting

Note.png
Available since SMILA 1.3!


Searching

JobManager: Asynchronous Workflows

Importing

Embedded HTTP Server

Common Services

Deprecated Components

Back to the top