Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

SMILA/Specifications/2 Nodes

< SMILA‎ | Specifications
Revision as of 11:34, 14 November 2008 by Dhazin.brox.de (Talk | contribs)

Use case

  1. XmlStorage is not started
  2. First node runs CrawlerController, Crawlers and DeltaIndexing
  3. Second node runs Broker, Processing and Indexing

Configuration

First node

Here is bundles configuration for starting SMILA on the first node, where crawling jobs will run. Please note that probably some bundles that are not really needed for crawling, currently must be started to satisfy dependencies of other bundles (this must be fixed of course)

osgi.bundles=org.eclipse.core.runtime@:start, \
org.eclipse.equinox.common@2:start, \
org.eclipse.update.configurator@3:start, \
org.eclipse.equinox.util@2:start, \
org.eclipse.equinox.ds@3:start, \
#com.sleepycat.dbxml@4:start, \
org.eclipse.smila.utils@3:start, \
org.eclipse.smila.management@3:start, \
org.eclipse.smila.connectivity.deltaindexing@4:start, \
org.eclipse.smila.management.jmx@4:start, \
org.eclipse.smila.management.crawlercontroller@4:start, \
#org.eclipse.smila.connectivity.queue.broker.main@4:start, \
org.eclipse.smila.xmlstorage@4:start, \
#org.eclipse.smila.xmlstorage.bdb@4:start, \
org.eclipse.smila.processing@4:start, \
org.eclipse.smila.processing.bpel@4:start, \
#org.eclipse.smila.processing.pipelets.mimetype@4:start, \
#org.eclipse.smila.lucene@4:start, \
org.eclipse.smila.connectivity.framework@4:start, \
org.eclipse.smila.connectivity.impl@4:start, \
org.eclipse.smila.connectivity.framework.crawler.filesystem@4:start, \
org.eclipse.smila.connectivity.framework.crawler.web@4:start, \
org.eclipse.smila.connectivity.queue.worker@4:start, \
org.eclipse.smila.binarystorage@4:start, \
org.eclipse.smila.blackboard@4:start, \
#org.eclipse.smila.tomcat@4:start

First node doesn't need Listener too, so Listener rules must be outcommented into ListenerConfig.xml

For the Router, the 'no-filter' must be applied to records:

 <Rule Name="Default ADD Rule">
   <Condition>Operation='ADD'</Condition>
   <Task BlackboardSync="true"> 
     <Fetch Filter="no-filter"/>
     <Send BrokerId="broker1" Queue="SMILA.connectivity">
     </Send>
   </Task>
 </Rule>
 
 
 <Rule Name="Default DELETE Rule">
   <Condition>Operation='DELETE'</Condition>
   <Task BlackboardSync="true">
     <Fetch Filter="no-filter"/>
     <Send BrokerId="broker1" Queue="SMILA.connectivity">
     </Send>
   </Task>
 </Rule>

Broker conection must be configured properly in the ConnectionConfig.xml in case when Broker runs on remote computer:

 <ConnectionConfig Id="broker1">
   <URL>tcp://localhost:61616</URL>
   <User>any</User>
   <Password>any</Password>
   <ConnectionFactory>org.apache.activemq.ActiveMQConnectionFactory</ConnectionFactory>
 </ConnectionConfig>

Separate nodes will provide separate management functionality, so JMX port must be different on both nodes. It can be changed in the SMILA.ini:

-Dcom.sun.management.jmxremote.port=10001

Second node

Here is the second node bundles configuration:

osgi.bundles=org.eclipse.core.runtime@:start, \
org.eclipse.equinox.common@2:start, \
org.eclipse.update.configurator@3:start, \
org.eclipse.equinox.util@2:start, \
org.eclipse.equinox.ds@3:start, \
#com.sleepycat.dbxml@4:start, \
org.eclipse.smila.utils@3:start, \
org.eclipse.smila.management@3:start, \
#org.eclipse.smila.connectivity.deltaindexing@4:start, \
org.eclipse.smila.management.jmx@4:start, \
org.eclipse.smila.management.crawlercontroller@4:start, \
org.eclipse.smila.connectivity.queue.broker.main@4:start, \
org.eclipse.smila.xmlstorage@4:start, \
#org.eclipse.smila.xmlstorage.bdb@4:start, \
org.eclipse.smila.processing@4:start, \
org.eclipse.smila.processing.bpel@4:start, \
org.eclipse.smila.processing.pipelets.mimetype@4:start, \
org.eclipse.smila.lucene@4:start, \
org.eclipse.smila.connectivity.framework@4:start, \
org.eclipse.smila.connectivity.impl@4:start, \
#org.eclipse.smila.connectivity.framework.crawler.filesystem@4:start, \
#org.eclipse.smila.connectivity.framework.crawler.web@4:start, \
org.eclipse.smila.connectivity.queue.worker@4:start, \
org.eclipse.smila.binarystorage@4:start, \
org.eclipse.smila.blackboard@4:start, \
org.eclipse.smila.tomcat@4:start

It must use 'no-filter' as a Fetch-filter too and JMX port that differs from the first node.

Back to the top