Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Documentation"
< SMILA
m (→Pipelines and Pipelets: Synchronous Workflows) |
m (→JobManager: Asynchronous Workflows) |
||
| (One intermediate revision by the same user not shown) | |||
| Line 49: | Line 49: | ||
** [[SMILA/Documentation/JobManagerFirstExample|JobManager Walk-Through]] | ** [[SMILA/Documentation/JobManagerFirstExample|JobManager Walk-Through]] | ||
* Creating Workflows and Jobs | * Creating Workflows and Jobs | ||
| − | ** [[SMILA/Documentation/DataObjectTypesAndBuckets| | + | ** [[SMILA/Documentation/DataObjectTypesAndBuckets|Data Object Types and Buckets]] |
| − | ** [[SMILA/Documentation/WorkerAndWorkflows| | + | ** [[SMILA/Documentation/WorkerAndWorkflows|Workers and Workflows]] |
| − | ** [[SMILA/Documentation/JobDefinitions| | + | ** [[SMILA/Documentation/JobDefinitions|Jobs]] |
| − | ** [[SMILA/Documentation/JobParameters| | + | ** [[SMILA/Documentation/JobParameters|Job Parameters]] |
* [[SMILA/Documentation/JobRuns|Running and Monitoring Jobs]] | * [[SMILA/Documentation/JobRuns|Running and Monitoring Jobs]] | ||
* [[SMILA/Documentation/JobManagerConfiguration|Configuring the Job Manager]] | * [[SMILA/Documentation/JobManagerConfiguration|Configuring the Job Manager]] | ||
| Line 60: | Line 60: | ||
** [[SMILA/Documentation/Worker/PipelineProcessorWorker|PipelineProcesor Worker]] | ** [[SMILA/Documentation/Worker/PipelineProcessorWorker|PipelineProcesor Worker]] | ||
** [[SMILA/Documentation/Worker/PipeletProcessorWorker|PipeletProcessor Worker]] | ** [[SMILA/Documentation/Worker/PipeletProcessorWorker|PipeletProcessor Worker]] | ||
| − | ** | + | ** (see [[SMILA/Manual#Importing|Importing]] section for more workers) |
* Developing new Workers | * Developing new Workers | ||
** [[SMILA/Documentation/WorkerManager|WorkerManager: Workers Made Easily]] | ** [[SMILA/Documentation/WorkerManager|WorkerManager: Workers Made Easily]] | ||
| Line 81: | Line 81: | ||
** [[SMILA/Documentation/HowTo/How to add a new Data Source to the importing framework|Adding a Data Source to the SMILA Import Framework]] | ** [[SMILA/Documentation/HowTo/How to add a new Data Source to the importing framework|Adding a Data Source to the SMILA Import Framework]] | ||
* Additionally | * Additionally | ||
| + | ** [[SMILA/Documentation/Importing/RemoteCrawling | Remote Crawling]] | ||
** [[SMILA/Documentation/Importing/CrawlingMultipleStartURLs | Crawling multiple start URL in one job run]] | ** [[SMILA/Documentation/Importing/CrawlingMultipleStartURLs | Crawling multiple start URL in one job run]] | ||
Revision as of 07:30, 19 July 2013
Contents
Basics
- Installing and Running
- HowTos
- Architecture
- Overview of Default Configuration
- Data Model, XML, JSON, BON
- Using the REST API, REST Client
- REST API Reference
- Enabling Remote Access to SMILA
Development Environment
- Setting up your Eclipse IDE for SMILA
- Building SMILA
- Creating new Components
- Testing new Components
- Adding Third Party Libraries to SMILA
- Using OSGi Declarative Services
- Extending the build process:
Pipelines and Pipelets: Synchronous Workflows
- What are Pipelines? What are Pipelets?
- Configuring and Creating BPEL Pipelines
- Using the SMILA BPEL Designer
- ReST APIs
- Basic Pipelets
- Common Pipelets
- XML Processing Pipelets
- Boilerpipe Pipelet (extract text from HTML content)
- TikaPipelet (extract text from binary content)
- JdbcLoggingPipelet (log to a database)
- More special pipelets are provided by the components described below.
- Developing new Pipelets
Searching
JobManager: Asynchronous Workflows
- What are Jobs and Tasks?
- Creating Workflows and Jobs
- Running and Monitoring Jobs
- Configuring the Job Manager
- TaskManager: Asynchronous Scheduling of Tasks
- Worker Reference
- Bulkbuilder worker
- PipelineProcesor Worker
- PipeletProcessor Worker
- (see Importing section for more workers)
- Developing new Workers
Importing
- Concepts, Workflow and Components
- Reference of Import Workers
- Developing new Import Workers
- Additionally
Embedded HTTP Server
Common Services
- Configuration Helper
- Workspace Helper
- ClusterConfig Service
- Zookeeper Service
- ObjectStore
- BinaryStorage
- Processing Security Information
- Ontology Processing with Sesame: Configuration and Pipelets
- MimeTypeIdentifier
- Description of Worker and Pipelet Parameters
- Publishing Web Services
- General JPA Configuration in SMILA
- SMILA Version Information