Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Specifications/2 Nodes"
< SMILA | Specifications
m |
|||
| Line 5: | Line 5: | ||
== Configuration == | == Configuration == | ||
| + | |||
| + | === Both nodes === | ||
| + | Both nodes must use the same binary storage configuration to have access to records attachments. | ||
| + | To configure binary storage, open file ''configuration/org.eclipse.smila.binarystorage/configuration/binarystorageconfig.properties '', uncomment property ''path'' and set it to some existing path that will be accessible by both SMILA instances, for example path=e:/storage. | ||
| + | |||
=== First node === | === First node === | ||
| Line 38: | Line 43: | ||
First node doesn't need Listener too, so Listener rules must be outcommented into ListenerConfig.xml | First node doesn't need Listener too, so Listener rules must be outcommented into ListenerConfig.xml | ||
| − | + | Filters are not needed for the Router, so comment or remove Fetch element. | |
<Rule Name="Default ADD Rule"> | <Rule Name="Default ADD Rule"> | ||
<Condition>Operation='ADD'</Condition> | <Condition>Operation='ADD'</Condition> | ||
<Task BlackboardSync="true"> | <Task BlackboardSync="true"> | ||
| − | <Fetch Filter="no-filter"/> | + | <!-- Fetch Filter="no-filter"/ --> |
<Send BrokerId="broker1" Queue="SMILA.connectivity"> | <Send BrokerId="broker1" Queue="SMILA.connectivity"> | ||
<!--<SetProperty Name="doSomething">no</SetProperty>--> | <!--<SetProperty Name="doSomething">no</SetProperty>--> | ||
| Line 54: | Line 59: | ||
<Condition>Operation='DELETE'</Condition> | <Condition>Operation='DELETE'</Condition> | ||
<Task BlackboardSync="true"> | <Task BlackboardSync="true"> | ||
| − | <Fetch Filter="no-filter"/> | + | <!-- Fetch Filter="no-filter"/ --> |
<Send BrokerId="broker1" Queue="SMILA.connectivity"> | <Send BrokerId="broker1" Queue="SMILA.connectivity"> | ||
<!--<SetProperty Name="doSomething">no</SetProperty>--> | <!--<SetProperty Name="doSomething">no</SetProperty>--> | ||
| Line 104: | Line 109: | ||
org.eclipse.smila.tomcat@4:start | org.eclipse.smila.tomcat@4:start | ||
| − | It must use | + | It must use JMX port that differs from the first node, and must NOT use filters in Listener configuration too. |
Revision as of 06:59, 17 November 2008
Use case
- XmlStorage is not started
- First node runs CrawlerController, Crawlers and DeltaIndexing
- Second node runs Broker, Processing and Indexing
Configuration
Both nodes
Both nodes must use the same binary storage configuration to have access to records attachments. To configure binary storage, open file configuration/org.eclipse.smila.binarystorage/configuration/binarystorageconfig.properties , uncomment property path and set it to some existing path that will be accessible by both SMILA instances, for example path=e:/storage.
First node
Here is bundles configuration for starting SMILA on the first node, where crawling jobs will run. Please note that probably some bundles that are not really needed for crawling, currently must be started to satisfy dependencies of other bundles (this must be fixed of course)
osgi.bundles=org.eclipse.core.runtime@:start, \ org.eclipse.equinox.common@2:start, \ org.eclipse.update.configurator@3:start, \ org.eclipse.equinox.util@2:start, \ org.eclipse.equinox.ds@3:start, \ #com.sleepycat.dbxml@4:start, \ org.eclipse.smila.utils@3:start, \ org.eclipse.smila.management@3:start, \ org.eclipse.smila.connectivity.deltaindexing@4:start, \ org.eclipse.smila.management.jmx@4:start, \ org.eclipse.smila.management.crawlercontroller@4:start, \ #org.eclipse.smila.connectivity.queue.broker.main@4:start, \ org.eclipse.smila.xmlstorage@4:start, \ #org.eclipse.smila.xmlstorage.bdb@4:start, \ org.eclipse.smila.processing@4:start, \ org.eclipse.smila.processing.bpel@4:start, \ #org.eclipse.smila.processing.pipelets.mimetype@4:start, \ #org.eclipse.smila.lucene@4:start, \ org.eclipse.smila.connectivity.framework@4:start, \ org.eclipse.smila.connectivity.impl@4:start, \ org.eclipse.smila.connectivity.framework.crawler.filesystem@4:start, \ org.eclipse.smila.connectivity.framework.crawler.web@4:start, \ org.eclipse.smila.connectivity.queue.worker@4:start, \ org.eclipse.smila.binarystorage@4:start, \ org.eclipse.smila.blackboard@4:start, \ #org.eclipse.smila.tomcat@4:start
First node doesn't need Listener too, so Listener rules must be outcommented into ListenerConfig.xml
Filters are not needed for the Router, so comment or remove Fetch element.
<Rule Name="Default ADD Rule">
<Condition>Operation='ADD'</Condition>
<Task BlackboardSync="true">
<Send BrokerId="broker1" Queue="SMILA.connectivity">
</Send>
</Task>
</Rule>
<Rule Name="Default DELETE Rule">
<Condition>Operation='DELETE'</Condition>
<Task BlackboardSync="true">
<Send BrokerId="broker1" Queue="SMILA.connectivity">
</Send>
</Task>
</Rule>
Broker conection must be configured properly in the ConnectionConfig.xml in case when Broker runs on remote computer:
<ConnectionConfig Id="broker1"> <URL>tcp://localhost:61616</URL> <User>any</User> <Password>any</Password> <ConnectionFactory>org.apache.activemq.ActiveMQConnectionFactory</ConnectionFactory> </ConnectionConfig>
Separate nodes will provide separate management functionality, so JMX port must be different on both nodes. It can be changed in the SMILA.ini:
-Dcom.sun.management.jmxremote.port=10001
Second node
Here is the second node bundles configuration:
osgi.bundles=org.eclipse.core.runtime@:start, \ org.eclipse.equinox.common@2:start, \ org.eclipse.update.configurator@3:start, \ org.eclipse.equinox.util@2:start, \ org.eclipse.equinox.ds@3:start, \ #com.sleepycat.dbxml@4:start, \ org.eclipse.smila.utils@3:start, \ org.eclipse.smila.management@3:start, \ #org.eclipse.smila.connectivity.deltaindexing@4:start, \ org.eclipse.smila.management.jmx@4:start, \ org.eclipse.smila.management.crawlercontroller@4:start, \ org.eclipse.smila.connectivity.queue.broker.main@4:start, \ org.eclipse.smila.xmlstorage@4:start, \ #org.eclipse.smila.xmlstorage.bdb@4:start, \ org.eclipse.smila.processing@4:start, \ org.eclipse.smila.processing.bpel@4:start, \ org.eclipse.smila.processing.pipelets.mimetype@4:start, \ org.eclipse.smila.lucene@4:start, \ org.eclipse.smila.connectivity.framework@4:start, \ org.eclipse.smila.connectivity.impl@4:start, \ #org.eclipse.smila.connectivity.framework.crawler.filesystem@4:start, \ #org.eclipse.smila.connectivity.framework.crawler.web@4:start, \ org.eclipse.smila.connectivity.queue.worker@4:start, \ org.eclipse.smila.binarystorage@4:start, \ org.eclipse.smila.blackboard@4:start, \ org.eclipse.smila.tomcat@4:start
It must use JMX port that differs from the first node, and must NOT use filters in Listener configuration too.