Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Documentation"
< SMILA
m (→Importing) |
m (→Importing) |
||
| Line 63: | Line 63: | ||
== Importing == | == Importing == | ||
* [[SMILA/Documentation/Importing/Concept|Import Concepts]] | * [[SMILA/Documentation/Importing/Concept|Import Concepts]] | ||
| − | * | + | * Reference of Import Workers |
**[[SMILA/Documentation/Importing/Crawler/File | FileCrawler and FileFetcher Worker]] | **[[SMILA/Documentation/Importing/Crawler/File | FileCrawler and FileFetcher Worker]] | ||
**[[SMILA/Documentation/Importing/Crawler/Web | WebCrawler and WebFetcher Worker]] | **[[SMILA/Documentation/Importing/Crawler/Web | WebCrawler and WebFetcher Worker]] | ||
**[[SMILA/Documentation/Importing/DeltaCheck | DeltaChecker Worker]] | **[[SMILA/Documentation/Importing/DeltaCheck | DeltaChecker Worker]] | ||
**[[SMILA/Documentation/Importing/UpdatePusher | UpdatePusher Worker]] | **[[SMILA/Documentation/Importing/UpdatePusher | UpdatePusher Worker]] | ||
| − | * Developing | + | * Developing new Import Workers |
** [[SMILA/Documentation/Importing/VisitedLinks | VisitedLinks service]] | ** [[SMILA/Documentation/Importing/VisitedLinks | VisitedLinks service]] | ||
** [[SMILA/Documentation/Importing/Crawler/Web#Internal_structure|Extending the WebCrawler worker]] | ** [[SMILA/Documentation/Importing/Crawler/Web#Internal_structure|Extending the WebCrawler worker]] | ||
| − | ** [[SMILA/Documentation/HowTo/How to add a new Data Source to the importing framework|Adding a Data Source to the SMILA | + | ** [[SMILA/Documentation/HowTo/How to add a new Data Source to the importing framework|Adding a Data Source to the SMILA Import Framework]] |
== The SMILA HTTP Server == | == The SMILA HTTP Server == | ||
Revision as of 12:46, 25 January 2012
Contents
Basics
- Architecture
- Overview of Default Configuration
- Data Model, XML, JSON, BON
- Using the ReST API
- REST API Reference
- Enabling Remote Access to SMILA
The SMILA Development Environment
- Setting up your Eclipse IDE for SMILA
- Building SMILA
- Creating new Components
- Testing new Components
- Adding Third Party Libraries to SMILA
- Using OSGi Declarative Services
- Extending the build process:
Pipelines and Pipelets: Synchronous Workflows of Java components
- What are Pipelines? What are Pipelets?
- Configuring and Creating BPEL Pipelines
- Using the SMILA BPEL Designer
- ReST APIs
- Basic Pipelets
- Common Pipelets in Bundle org.eclipse.smila.processing.pipelets
- XML Processing Pipelets in Bundle org.eclipse.smila.processing.pipelets.xmlprocessing
- More special pipelets are provided by the components described below.
- Developing new Pipelets
Using SMILA for Search
JobManager: Asynchronous Workflows
- What are Jobs and Tasks?
- Creating Workflows and Jobs
- Running and Monitoring Jobs
- Configuring the Job Manager
- TaskManager: Asynchronous Scheduling of Tasks
- Worker Reference
- Bulkbuilder worker
- PipelineProcesor Worker
- PipeletProcessor Worker
- See Importing below for more workers
- Developing Workers
Importing
- Import Concepts
- Reference of Import Workers
- Developing new Import Workers
The SMILA HTTP Server
Common Services
- Configuration Helper
- Workspace Helper
- ClusterConfig Service
- ObjectStore
- BinaryStorage
- Processing Security Information
- Ontology Processing with Sesame: Configuration and Pipelets
- MimeTypeIdentifier
- Publishing Web Services
- General JPA Configuration in SMILA