Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Manual"
< SMILA
(→Common Services) |
(→Importing) |
||
| Line 53: | Line 53: | ||
== Importing == | == Importing == | ||
| − | + | * [[SMILA/Documentation/Importing/Concept|Concepts: Crawler, Fetcher, Delta, VisitedLinks]] | |
| − | + | * Importing Workers reference | |
| − | + | **[[SMILA/Documentation/Importing/Crawler/File | FileCrawler worker]] | |
| − | + | **[[SMILA/Documentation/Importing/Crawler/Web | WebCrawler worker]] | |
| − | + | **[[SMILA/Documentation/Importing/DeltaCheck | DeltaCheck worker]] | |
| − | + | **[[SMILA/Documentation/Importing/UpdatePusher | UpdatePusher worker]] | |
| − | + | * Developing Importing Workers | |
| − | + | ** [[SMILA/Documentation/Importing/VisitedLinks | VisitedLinks service]] | |
| − | + | ** [[SMILA/Documentation/Importing/Crawler/Web#Internal_structure|Extending the WebCrawler worker]] | |
== The SMILA HTTP Server == | == The SMILA HTTP Server == | ||
Revision as of 10:18, 19 January 2012
Contents
Basics
- Architecture
- Data Model, XML, JSON, BON
- Using the ReST API
- REST API Reference
The SMILA Development Environment
- Setup your Eclipse IDE for SMILA
- Build SMILA
- Creating new components
- Testing new components
- Adding Third Party libraries to SMILA
- Extending the build process:
Pipelines and Pipelets: Synchronous Workflows of Java components
- What are Pipelets?
- Configuration and Creating BPEL Pipelines
- Using the SMILA BPEL Designer|
- Basic Pipelets
- Common pipelets: bundle org.eclipse.smila.processing.pipelets
- Common XML pipelets: Bundle org.eclipse.smila.processing.pipelets.xmlprocessing
- More special pipelets are provided by components described below.
- Developing Pipelets
Using SMILA for Search
JobManager: Asynchronous Workflows
- Concepts: Jobs and Tasks
- Creating Workflows and Jobs
- Running and Monitoring Jobs
- JobManager Configuration
- TaskManager: Asynchronous scheduling of tasks
- Workers Reference
- Bulkbuilder
- PipelineProcesor
- PipeletProcessor
- See SMILA/Manual#Importing below for more workers
- Developing Workers
- WorkerManager: Workers made easy
- Developing Workers
Importing
- Concepts: Crawler, Fetcher, Delta, VisitedLinks
- Importing Workers reference
- Developing Importing Workers
The SMILA HTTP Server
Common Services
- Configuration Helper
- Workspace Helper
- ClusterConfig service
- ObjectStore
- BinaryStorage
- Processing security information
- Ontology processing with Sesame
- MimeTypeIdentifier
- Publishing Web Services