Skip to main content
Jump to: navigation, search

Difference between revisions of "SMILA/Documentation/LuceneIndexPipelet"

(Bundle: org.eclipse.smila.lucene.LuceneIndexService)
(Configuration files)
Line 18: Line 18:
 
==== Configuration files ====
 
==== Configuration files ====
  
* <tt>configuration/com.brox.anyfinder.datadictionary/DataDictionary.xml</tt>
+
* <tt>configuration/org.eclipse.smila.search.datadictionary/DataDictionary.xml</tt>
 
Here the Lucene index structure and the search template are configured. It is possible to define more than one index here. The index to work on is set by the Annotation "indexName". The defined "FieldNo" are referenced in LuceneAttributeMapping.xml and LuceneAttachmentMapping.xml. For more information about configuration of DataDictionary.xml see Anyfinder documentation.
 
Here the Lucene index structure and the search template are configured. It is possible to define more than one index here. The index to work on is set by the Annotation "indexName". The defined "FieldNo" are referenced in LuceneAttributeMapping.xml and LuceneAttachmentMapping.xml. For more information about configuration of DataDictionary.xml see Anyfinder documentation.
  
Line 24: Line 24:
 
* <tt>configuration/org.eclipse.smila.lucene/Mappings.xml</tt>
 
* <tt>configuration/org.eclipse.smila.lucene/Mappings.xml</tt>
 
Here a mapping of attribute and attachment names to Lucene "FieldNo" (defined in DataDictionary.xml) is configured. It is possible to define mappings for multiple indexes in this file, using the same "indexName" as in file DataDictionary.xml.
 
Here a mapping of attribute and attachment names to Lucene "FieldNo" (defined in DataDictionary.xml) is configured. It is possible to define mappings for multiple indexes in this file, using the same "indexName" as in file DataDictionary.xml.
 
 
  
 
==== Example ====
 
==== Example ====

Revision as of 06:22, 13 November 2008

Bundle: org.eclipse.smila.lucene.LuceneIndexService

Description

This ProcessingService is used to index SMILA records in a Lucene document index. It supports adding, updating and deleting of records.

Configuration

Annotations

The LuceneIndexService uses the Annotation org.eclipse.smila.lucene.LuceneIndexService on records to decide how to handle a record. It supports the following required values.

Name Value Description
indexName a String the name of the index to work on
executionMode ADD or DELETE ADD - add or update the record, DELETE - delete the record from the index

Configuration files

  • configuration/org.eclipse.smila.search.datadictionary/DataDictionary.xml

Here the Lucene index structure and the search template are configured. It is possible to define more than one index here. The index to work on is set by the Annotation "indexName". The defined "FieldNo" are referenced in LuceneAttributeMapping.xml and LuceneAttachmentMapping.xml. For more information about configuration of DataDictionary.xml see Anyfinder documentation.


  • configuration/org.eclipse.smila.lucene/Mappings.xml

Here a mapping of attribute and attachment names to Lucene "FieldNo" (defined in DataDictionary.xml) is configured. It is possible to define mappings for multiple indexes in this file, using the same "indexName" as in file DataDictionary.xml.

Example

The following example was used in the SMILA example application to index records delivered by Filesystem- and WebCrawler.

addpipeline.bpel

...
<extensionActivity name="invokeLuceneService">
    <proc:invokeService>
        <proc:service name="LuceneIndexService" />
        <proc:variables input="request" output="request" />
        <proc:setAnnotations>
            <rec:An n="org.eclipse.smila.lucene.LuceneIndexService">
                <rec:V n="indexName">test_index</rec:V>
                <rec:V n="executionMode">ADD</rec:V>
            </rec:An>
        </proc:setAnnotations>
    </proc:invokeService>
</extensionActivity>
...

DataDictionary.xml

<?xml version="1.0" encoding="UTF-8"?>
<AnyFinderDataDictionary xmlns="http://www.anyfinder.de/DataDictionary" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.anyfinder.de/DataDictionary ../xml/AnyFinderDataDictionary.xsd">
  <Index Name="test_index">
    <Connection xmlns="http://www.anyfinder.de/DataDictionary/Connection" MaxConnections="5"/>
    <IndexStructure xmlns="http://www.anyfinder.de/IndexStructure" Name="test_index">
      <Analyzer ClassName="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
      <IndexField FieldNo="5" IndexValue="true" Name="Title" StoreText="true" Tokenize="true" Type="Text"/>
      <IndexField FieldNo="4" IndexValue="true" Name="Url" StoreText="true" Tokenize="false" Type="Text">
        <Analyzer ClassName="org.apache.lucene.analysis.WhitespaceAnalyzer"/>
      </IndexField>
      <IndexField FieldNo="3" IndexValue="true" Name="Date" StoreText="true" Tokenize="false" Type="Text"/>
      <IndexField FieldNo="2" IndexValue="true" Name="Path" StoreText="true" Tokenize="true" Type="Text"/>
      <IndexField FieldNo="1" IndexValue="true" Name="Filename" StoreText="true" Tokenize="true" Type="Text"/>
      <IndexField FieldNo="0" IndexValue="true" Name="Content" StoreText="true" Tokenize="true" Type="Text"/>
    </IndexStructure>
    <Result>
      <Field FieldNo="0" Name="ID"/>
    </Result>
    <Configuration xmlns="http://www.anyfinder.de/DataDictionary/Configuration" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.anyfinder.de/DataDictionary/Configuration ../xml/DataDictionaryConfiguration.xsd">
      <DefaultConfig>
        <Field FieldNo="5">
          <FieldConfig Constraint="optional" Weight="1" xsi:type="FTText">
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="OR" Tolerance="exact"/>
          </FieldConfig>
        </Field>
        <Field FieldNo="4">
          <FieldConfig Constraint="optional" Weight="1" xsi:type="FTText">
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="OR" Tolerance="exact"/>
          </FieldConfig>
        </Field>
        <Field FieldNo="3">
          <FieldConfig Constraint="optional" Weight="1" xsi:type="FTText">
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="OR" Tolerance="exact"/>
          </FieldConfig>
        </Field>
        <Field FieldNo="2">
          <FieldConfig Constraint="optional" Weight="1" xsi:type="FTText">
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="OR" Tolerance="exact"/>
          </FieldConfig>
        </Field>
        <Field FieldNo="1">
          <FieldConfig Constraint="optional" Weight="1" xsi:type="FTText">
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="OR" Tolerance="exact"/>
          </FieldConfig>
        </Field>
        <Field FieldNo="0">
          <FieldConfig Constraint="required" Weight="1" xsi:type="FTText">
            <NodeTransformer xmlns="http://www.anyfinder.de/Search/ParameterObjects" Name="urn:ExtendedNodeTransformer">
              <ParameterSet xmlns="http://www.brox.de/ParameterSet"/>
            </NodeTransformer>
            <Parameter xmlns="http://www.anyfinder.de/Search/TextField" Operator="AND" Tolerance="exact"/>
          </FieldConfig>
        </Field>
      </DefaultConfig>
      <Result Name="">
        <ResultField FieldNo="5" Name="Title"/>
        <ResultField FieldNo="4" Name="Url"/>
        <ResultField FieldNo="3" Name="Date"/>
        <ResultField FieldNo="2" Name="Path"/>
        <ResultField FieldNo="1" Name="Filename"/>
      </Result>
      <HighlightingResult Name="">
        <HighlightingResultField FieldNo="0" Name="Content" xsi:type="HLTextField">
          <HighlightingTransformer Name="urn:Sentence">
            <ParameterSet xmlns="http://www.brox.de/ParameterSet">
              <Parameter Name="MaxLength" xsi:type="Integer">
                <Value>300</Value>
              </Parameter>
              <Parameter Name="MaxHLElements" xsi:type="Integer">
                <Value>999</Value>
              </Parameter>
              <Parameter Name="MaxSucceedingCharacters" xsi:type="Integer">
                <Value>30</Value>
              </Parameter>
              <Parameter Name="SucceedingCharacters" xsi:type="String">
                <Value>...</Value>
              </Parameter>
              <Parameter Name="SortAlgorithm" xsi:type="String">
                <Value>Occurrence</Value>
              </Parameter>
              <Parameter Name="TextHandling" xsi:type="String">
                <Value>ReturnSnipplet</Value>
              </Parameter>
            </ParameterSet>
          </HighlightingTransformer>
          <HighlightingParameter xmlns="http://www.anyfinder.de/DataDictionary/Configuration/TextHighlighting"/>
        </HighlightingResultField>
      </HighlightingResult>
    </Configuration>
  </Index>
  <Index Name="another_index">
    ...
  </Index>
</AnyFinderDataDictionary>

Mappings.xml

<?xml version="1.0" encoding="utf-8" ?>
<Mappings xmlns="http://www.eclipse.org/smila/lucene">
    <Mapping indexName="test_index">
        <Attributes>
            <Attribute name="Filename" fieldNo="1" />
            <Attribute name="Path" fieldNo="2" />    
            <Attribute name="LastModifiedDate" fieldNo="3" />
            <Attribute name="Url" fieldNo="4" />
            <Attribute name="Title" fieldNo="5" />
        </Attributes>
        <Attachments>
            <Attachment name="Text" fieldNo="0" />
        </Attachments>
    </Mapping>
    <Mapping indexName="another_index">
        <Attributes>
        ...
        </Attributes>
        <Attachments>
        ...
        </Attachments>
    </Mapping>
</Mappings>

Back to the top