|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| − | This page illustrates all steps that need to be performed in order to integrate the HelloWorld web service as a [[SMILA/Glossary#P|pipelet]] in SMILA. For general information on how to integrate components and add functionality to SMILA refer to [[SMILA/Howto_integrate_a_component_in_SMILA| How to integrate a component in SMILA]].
| + | #REDIRECT [[SMILA/Development_Guidelines/How_to_integrate_the_HelloWorld_webservice_as_a_Pipelet]] |
| − | | + | |
| − | == Preparations ==
| + | |
| − | | + | |
| − | It may be helpful to first take a look at the SMILA [[SMILA/Development_Guidelines|Development guidelines]] as many topics that are beyond the scope of this tutorial are illustrated there.
| + | |
| − | | + | |
| − | === Create new bundle ===
| + | |
| − | | + | |
| − | * Create a new bundle that should contain your pipelet. Follow the instructions on [[SMILA/Development_Guidelines/Create_a_bundle_%28plug-in%29|How to create a bundle]] and use the following settings:
| + | |
| − | <pre>
| + | |
| − | Project name: org.eclipse.smila.sample.pipelet
| + | |
| − | Plug-in ID: org.eclipse.smila.sample.pipelet
| + | |
| − | Plug-in Version: 1.0.0
| + | |
| − | Plug-in Name: Sample Pipelet Bundle
| + | |
| − | Plug-in Provider: your name or company
| + | |
| − | </pre>
| + | |
| − | * Edit the file <tt>META-INF/MANIFEST.MF</tt> and add the following import-package dependencies as those are required to implement the basic functionalities of your pipelet:
| + | |
| − | <pre>
| + | |
| − | Import-Package: org.apache.commons.logging;version="1.1.1",
| + | |
| − | org.eclipse.smila.blackboard;version="0.8.0",
| + | |
| − | org.eclipse.smila.datamodel;version="0.8.0",
| + | |
| − | org.eclipse.smila.processing;version="0.8.0"
| + | |
| − | </pre>
| + | |
| − | * To make sure that the <tt>PipeletTrackerService</tt> detects your new pipelet, add the following line to the file <tt>META-INF/MANIFEST.MF</tt>. This registers the class that will implement your SMILA pipelet:
| + | |
| − | <pre>
| + | |
| − | SMILA-Pipelets: org.eclipse.smila.sample.pipelet.HelloWorldPipelet
| + | |
| − | </pre>
| + | |
| − | | + | |
| − | === Create Java classes from WSDL using Axis2 ===
| + | |
| − | | + | |
| − | * Install Axis2 1.4.1: Download from http://ws.apache.org/axis2/download/1_4_1/download.cgi and unpack into any directory.
| + | |
| − | * Open a shell in the Axis2 directory and execute <tt>wsdl2java</tt> similar to this example - replace the WSDL-URL with that of the Webservice you want to use after <tt>-uri</tt>, change the package name after <tt>-p</tt> and the output directory after <tt>-o</tt>:
| + | |
| − | <source lang="text">
| + | |
| − | bin\wsdl2java -uri http://localhost:8081/axis2/services/HelloWorld.HelloWorldImplPort?wsdl
| + | |
| − | -d xmlbeans -p org.eclipse.smila.sample.helloworld -s -o helloworld-ws
| + | |
| − | </source>
| + | |
| − | : This creates two folders inside <tt>helloworld-ws</tt>: <tt>src</tt> and <tt>resources</tt>.
| + | |
| − | : If you do not want to run the generator inside the Axis2 installation you must set an environment variable <tt>AXIS2_HOME</tt> to the Axis2 installation directory.
| + | |
| − | * Add Import-Package declarations with minimum versions as available in your target platform (they will be set automatically if you use the Manifest editor's Dependencies tab to add them). To run this example at least these are needed (with valid versions at the time of writing):
| + | |
| − | <source lang="text">
| + | |
| − | javax.xml.stream;version="1.0.1",
| + | |
| − | org.apache.axiom.om;version="1.2.7",
| + | |
| − | org.apache.axiom.om.impl;version="1.2.7",
| + | |
| − | org.apache.axiom.om.impl.llom;version="1.2.7",
| + | |
| − | org.apache.axiom.soap;version="1.2.7",
| + | |
| − | org.apache.axis2;version="1.4.1",
| + | |
| − | org.apache.axis2.addressing;version="1.4.1",
| + | |
| − | org.apache.axis2.client;version="1.4.1",
| + | |
| − | org.apache.axis2.context;version="1.4.1",
| + | |
| − | org.apache.axis2.description;version="1.4.1",
| + | |
| − | org.apache.axis2.transport;version="1.4.1",
| + | |
| − | org.apache.axis2.transport.http;version="1.4.1",
| + | |
| − | org.apache.axis2.wsdl;version="1.4.1",
| + | |
| − | org.apache.xmlbeans;version="2.3.0",
| + | |
| − | org.apache.xmlbeans.impl.schema;version="2.3.0",
| + | |
| − | org.apache.xmlbeans.impl.values;version="2.3.0",
| + | |
| − | org.apache.xmlbeans.xml.stream;version="2.3.0"
| + | |
| − | </source>
| + | |
| − | : You will not get compile errors if the import for <tt>org.apache.xmlbeans.impl.schema</tt> is missing, but it is needed during runtime.
| + | |
| − | : For more complex webservices, additional imports may be required. Check the imported generated client code for compile errors.
| + | |
| − | * Create a source folder <tt>code/gen</tt> in your bundle and move the '''content''' of the generated <tt>src</tt> folder into it.
| + | |
| − | * Create a folder <tt>lib</tt> in your bundle, create a zip file from the '''content''' of the generated <tt>resources</tt> folder, change the suffix to <tt>jar</tt> and move it to <tt>lib</tt>. Refresh the bundle in your Eclipse workspace, and add this jar to the Bundle-Classpath of your bundle (Manifest editor, tab Runtime, Classpath setting).
| + | |
| − | | + | |
| − | | + | |
| − | == Implementation ==
| + | |
| − | | + | |
| − | * Create the package <tt>org.eclipse.smila.sample.pipelet</tt> and the Java class <tt>HelloWorldPipelet</tt>.
| + | |
| − | * Use the following code as a template for your new class. It contains empty method bodies and a reference to the logger. In the following we are going to gradually replace the comments in this file by the corresponding code snippets. For your convenience you may also download the complete zipped source file from [[Media:HelloWorldPipelet.zip|HelloWorldPipelet.zip]].
| + | |
| − | | + | |
| − | <source lang="Java">
| + | |
| − | package org.eclipse.smila.sample.pipelet
| + | |
| − |
| + | |
| − | import org.apache.axis2.transport.http.HTTPConstants;
| + | |
| − | import org.apache.commons.logging.Log;
| + | |
| − | import org.apache.commons.logging.LogFactory;
| + | |
| − | import org.eclipse.smila.blackboard.Blackboard;
| + | |
| − | import org.eclipse.smila.datamodel.AnyMap;
| + | |
| − | import org.eclipse.smila.processing.ProcessingException;
| + | |
| − | import org.eclipse.smila.processing.SimplePipelet;
| + | |
| − | | + | |
| − | import com.empolis.smila.sample.helloworld.HelloWorldStub;
| + | |
| − | | + | |
| − | import demo.hw.server.SayHi;
| + | |
| − | import demo.hw.server.SayHiDocument;
| + | |
| − | import demo.hw.server.SayHiResponse;
| + | |
| − | import demo.hw.server.SayHiResponseDocument;
| + | |
| − |
| + | |
| − | public class HelloWorldPipelet implements Pipelet {
| + | |
| − |
| + | |
| − | // additional member variables
| + | |
| − | private final Log _log = LogFactory.getLog(HelloWorldPipelet.class);
| + | |
| − | | + | |
| − | public HelloWorldPipelet(){
| + | |
| − | }
| + | |
| − |
| + | |
| − | public void configure(AnyMap configuration) throws ProcessingException {
| + | |
| − | // read the configuration properties
| + | |
| − | }
| + | |
| − |
| + | |
| − | public String[] process(Blackboard blackboard, String[] recordIds) throws ProcessingException {
| + | |
| − | // process the recordIds and create a result
| + | |
| − | return null;
| + | |
| − | }
| + | |
| − | }
| + | |
| − | </source>
| + | |
| − | | + | |
| − | === Read PipeletConfiguration ===
| + | |
| − | * First let's create two member variables that store the names of the input and output attributes as well as string constants for the property names used in the configuration. Replace the comment "<tt>// additional member variables </tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | private final String PROP_IN_ATT_NAME= "IN_ATT_NAME";
| + | |
| − | private final String PROP_OUT_ATT_NAME= "OUT_ATT_NAME";
| + | |
| − | | + | |
| − | private String _inAttName;
| + | |
| − | private String _outAttName;
| + | |
| − | </source>
| + | |
| − | | + | |
| − | * Then we are going to fill those members with the attribute names provided by the <tt>PipeletConfiguration</tt> in method <tt>configure(AnyMap configuration)</tt>. The method <tt>getPropertyFirstValueNotNull(String)</tt> will check that the value of the property is not null. If it is null a <tt>ProcessingException</tt> will be thrown. In addition we should ensure, that the provided string is not empty or consists of whitespaces only. Replace the comment "<tt>// read the configuration properties</tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | _inAttName = configuration.getStringValue(PROP_IN_ATT_NAME);
| + | |
| − | if (_inAttName == null || _inAttName.trim().length() == 0) {
| + | |
| − | throw new ProcessingException("Property " + PROP_IN_ATT_NAME + " must not be an empty String");
| + | |
| − | }
| + | |
| − | | + | |
| − | _outAttName = configuration.getStringValue(PROP_OUT_ATT_NAME);
| + | |
| − | if (_outAttName == null || _outAttName.trim().length() == 0) {
| + | |
| − | throw new ProcessingException("Property " + PROP_OUT_ATT_NAME + " must not be an empty String");
| + | |
| − | }
| + | |
| − | </source>
| + | |
| − | '''Note''': Of course it is also possible to store the configuration map in a member variable and access the properties as needed in the <tt>process(Blackboard blackboard, String[] recordIds)</tt> method.
| + | |
| − | | + | |
| − | === Process IDs and implement exception handling ===
| + | |
| − | The method <tt>process(Blackboard blackboard, String[] recordIds)</tt> has two parameters:
| + | |
| − | * a reference to the [[SMILA/Glossary#B|blackboard service]] that allows access on [[SMILA/Glossary#R|records]] and
| + | |
| − | * a list of record IDs to process.
| + | |
| − | The HelloWorld pipelet should therefore iterate over the IDs in the parameter <tt>recordIds</tt>, get the required data from the record identified by the ID, process this data, and store the result in the record. Let's place a <tt>try ... catch()</tt> block in the <tt>for</tt> loop to ensure that errors do only interrupt the processing of the current ID. The comments in the code serve as placeholders for the functionality described in the following sections. At the end we return the unmodified input parameter <tt>recordIds</tt> as the result of the pipelet. Replace the comment "<tt>// process the recordIds and create a result</tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | for (String id : recordIds) {
| + | |
| − | try {
| + | |
| − | // Read Input Data
| + | |
| − | | + | |
| − | // Process Input Data
| + | |
| − | | + | |
| − | // Write Output Data
| + | |
| − | | + | |
| − | } catch (final Exception ex) {
| + | |
| − | if (_log.isErrorEnabled()) {
| + | |
| − | _log.error("error during execution of HelloWorldPipelet with record " + id, ex);
| + | |
| − | }
| + | |
| − | }
| + | |
| − | } // for
| + | |
| − | | + | |
| − | return recordIds;
| + | |
| − | </source>
| + | |
| − | '''Note''': Most of the time the return value of a pipelet is the unmodified input parameter <tt>recordIds</tt>. However, in some cases a pipelet may filter record IDs or even create new records. Then the return value has to be adopted appropriately.
| + | |
| − | | + | |
| − | === Read input data ===
| + | |
| − | Now we want to read the data of the attribute with the name stored in <tt>_inAttName</tt>. Therefore we first have to create a <tt>Path</tt> object with the attribute's name. Before accessing the literal value we check if the record contains an attribute with the given <tt>Path</tt>. In this tutorial we know that the value of the attribute is a string value, so we directly access the value by calling the method <tt>getStringValue()</tt>.
| + | |
| − | Replace the comment "<tt>// Read Input Data</tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | String inputValue = "";
| + | |
| − | AnyMap record = blackboard.getMetadata(id);
| + | |
| − | inputValue = record.getStringValue(_inAttName);
| + | |
| − | </source>
| + | |
| − | '''Note''': Accessing attribute values can be achieved more generically. Therefore you have to check what data type a certain literal contains using the method <tt>getValueType()</tt>. Then you can use the appropriate getter method to access the raw data.
| + | |
| − | | + | |
| − | === Process input data ===
| + | |
| − | Now we will call the HelloWorld web service with the parameter <tt>inputValue</tt> and store the result in variable <tt>outputValue</tt>. Therefore we use the classes generated from WSDL by Axis2. The HelloWorld web service will return a String message in the format <tt>"Hello "</tt> + the content of variable <tt>inputValue</tt>. Replace the comment "<tt>// Process Input Data</tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | HelloWorldStub ws = new HelloWorldStub("http://localhost:8081/axis2/services/HelloWorld.HelloWorldImplPort");
| + | |
| − | ws._getServiceClient().getOptions().setProperty(HTTPConstants.CHUNKED, Boolean.FALSE);
| + | |
| − | SayHiDocument sayHiDoc = SayHiDocument.Factory.newInstance();
| + | |
| − | SayHi sayHi = sayHiDoc.addNewSayHi();
| + | |
| − | sayHi.setArg0(inputValue);
| + | |
| − | SayHiResponseDocument respDoc = ws.sayHi(sayHiDoc);
| + | |
| − | SayHiResponse response = respDoc.getSayHiResponse();
| + | |
| − | String outputValue = response.getReturn();
| + | |
| − | </source>
| + | |
| − | | + | |
| − | === Write output data ===
| + | |
| − | Finally, we want to store the content of the variable <tt>outputValue</tt> in the record attribute with the name contained in variable <tt>_outAttName</tt>. Therefore we have to create a new <tt>Literal</tt> object and set its value. Then we only need to set this <tt>Literal</tt> for the current ID on the black board.
| + | |
| − | Replace the comment "<tt>// Write Output Data</tt>" with the following code snippet.
| + | |
| − | <source lang="Java">
| + | |
| − | record.put(_outAttName, outputValue);
| + | |
| − | </source>
| + | |
| − | '''Note''': The method <tt>commit(Id)</tt> of the blackboard service does not need to be called in each pipelet as it is automatically called at the end of the [[SMILA/Glossary/#p|pipeline]].
| + | |
| − | | + | |
| − | == Configuration and invocation in BPEL ==
| + | |
| − | In this tutorial we will integrate the HelloWorld pipelet in the SMILA indexing process just before the record is stored in the Lucene index. With this configuration the input for the HelloWorld pipelet will be read from attribute ''Title'' and the modified output will be stored in the same attribute, overwriting the previous value.
| + | |
| − | * Edit the file <tt>configuration/org.eclipse.smila.processing.bpel/pipelines/addpipeline.bpel</tt> and add the following right between the <tt><extensionActivity name="convertDocument"></tt> and the <tt><extensionActivity name="invokeLuceneService"></tt> section.
| + | |
| − | <source lang="XML">
| + | |
| − | <extensionActivity>
| + | |
| − | <proc:invokePipelet name="invokeHelloWorldPipelet">
| + | |
| − | <proc:pipelet class="org.eclipse.smila.sample.pipelet.HelloWorldPipelet" />
| + | |
| − | <proc:variables input="request" output="request" />
| + | |
| − | <proc:configuration>
| + | |
| − | <rec:Value name="IN_ATT_NAME">Title</rec:Value>
| + | |
| − | <rec:Value name="OUT_ATT_NAME">Title</rec:Value>
| + | |
| − | </proc:Property>
| + | |
| − | </proc:configuration>
| + | |
| − | </proc:invokePipelet>
| + | |
| − | </extensionActivity>
| + | |
| − | </source>
| + | |
| − | | + | |
| − | == Build and Test your pipelet ==
| + | |
| − | | + | |
| − | Depending on the SMILA distribution you are using you have different options how to build and test your pipelet:
| + | |
| − | | + | |
| − | === Source Code Distribution ===
| + | |
| − | You can simply integrate your pipelet in the SMILA build process. Refer to the instructions on [[SMILA/Development_Guidelines/How_to_integrate_new_bundle_into_build_process|How to integrate a new bundle into build process]] for details. You can build your SMILA application and run it as usual.
| + | |
| − | | + | |
| − | In addition to building the SMILA application you can also directly run SMILA within your eclipse IDE. To test your pipelet, you have to include the bundle in the OSGi launch configuration:
| + | |
| − | * Open ''Run > Open Run Dialog''.
| + | |
| − | * In the left window select ''OSGi Framework > SMILA''.
| + | |
| − | * In the right window expand ''Workspace'' and select ''org.eclipse.smila.sample.pipelet''.
| + | |
| − | * Set the ''Default Auto-Start'' option to ''true''.
| + | |
| − | * Click the ''Apply'' button.
| + | |
| − | * Launch SMILA by clicking the ''Run'' button.
| + | |
| − | | + | |
| − | === Binary Distribution ===
| + | |
| − | | + | |
| − | To test your pipelet you have to add it as a plugin to your SMILA installation.
| + | |
| − | | + | |
| − | * export your pipelet as a plugin using eclipse IDE wizards. Refer to the instructions on [[SMILA/Development_Guidelines/How_to_export_a_bundle|How to export a bundle]] for a step by step description.
| + | |
| − | * Copy your plugin to the directory <tt>%SMILA_HOME%/plugins</tt>.
| + | |
| − | * Add the following XML snippet to the file <tt>%SMILA_HOME%/features/org.eclipse.smila.feature_1.0.0/feature.xml</tt>:
| + | |
| − | <code lang="XML">
| + | |
| − | <plugin
| + | |
| − | id="org.eclipse.smila.sample.pipelet"
| + | |
| − | download-size="0"
| + | |
| − | install-size="0"
| + | |
| − | version="1.0.0"
| + | |
| − | unpack="false"/>
| + | |
| − | </code>
| + | |
| − | * Launch SMILA by starting <tt>eclipse.exe</tt>.
| + | |
| − | | + | |
| − | If SMILA is running, you can start a crawling job as described in [[SMILA/Development_Guidelines#Run_and_manage_the_connectivity_framework|Run and manage the connectivity framework]] beginning at step 5.
| + | |
| − | While crawling your data source you can already search for indexed documents. Open your browser, navigate to [http://localhost:8080/SMILA/search http://localhost:8080/SMILA/search] and execute a query. In the result table take a look at the attribute '''Title'''. Every '''Title''' should now have the suffix <tt>"modified by HelloWorldPipelet"</tt>, as this was added by the pipelet.
| + | |
| − | | + | |
| − | === Troubleshooting ===
| + | |
| − | If there are any problems please take a look at the log files <tt>SMILA.log</tt> and <tt>/workspace/.metadata/.log</tt> and feel free to ask for support at the [http://www.eclipse.org/smila/newsgroup.php SMILA Newsgroup].
| + | |
| − | | + | |
| − | [[Category:SMILA]]
| + | |
