Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Triquetrum/Kepler"
(Status) |
(Notes about creating plugins and features.) |
||
Line 163: | Line 163: | ||
Kepler provides a hadoop project that could be helpful here! | Kepler provides a hadoop project that could be helpful here! | ||
− | Solution: Right click on the org.kepler.triquetrum.ddpDemo project -> Build Path -> Configure Build Path and add the following package | + | Solution: Right click on the org.kepler.triquetrum.ddpDemo project -> Build Path -> Configure Build Path and add the following package dependencies |
* ddp-common | * ddp-common | ||
* hadoop | * hadoop | ||
+ | |||
+ | == Packaging the plugin(s) for your RCP editor == | ||
+ | See [[Triquetrum/Extending_Triquetrum#Packaging_the_plugin.28s.29_for_your_RCP_editor | Packaging the plugin(s) for your RCP editor]] | ||
+ | |||
+ | Create a simple (non-Java plugin): File -> New Plug-in Project. | ||
+ | * Name: org.kepler.triquetrum.ddp.palette | ||
+ | * Uncheck Create a Java Project | ||
+ | * Click Next, then Finish | ||
+ | |||
+ | Right click on the newly created project, select File -> New -> File and create plugin.xml. | ||
+ | |||
+ | plugin.xml should look like: | ||
+ | <pre> | ||
+ | <?xml version="1.0" encoding="UTF-8"?> | ||
+ | <?eclipse version="3.4"?> | ||
+ | <plugin> | ||
+ | <extension | ||
+ | name="Triquetrum palette ExecutionChoice extension" | ||
+ | point="org.eclipse.triquetrum.workflow.editor.paletteContribution"> | ||
+ | <group displayName="Examples"> | ||
+ | <entry | ||
+ | class="org.kepler.ddp.actor.ExecutionChoice" | ||
+ | displayName="ExecutionChoice" | ||
+ | type="Actor"> | ||
+ | </entry> | ||
+ | </group> | ||
+ | </extension> | ||
+ | </plugin> | ||
+ | </pre> | ||
+ | |||
+ | MANIFEST.MF should look like: | ||
+ | <pre> | ||
+ | Manifest-Version: 1.0 | ||
+ | Bundle-ManifestVersion: 2 | ||
+ | Bundle-Name: Palette entry for ExecutionChoice | ||
+ | Bundle-SymbolicName: org.kepler.triquetrum.ddp.palette;singleton:=true | ||
+ | Bundle-Version: 1.0.0.qualifier | ||
+ | Require-Bundle: org.eclipse.triquetrum.workflow.editor;bundle-version="1.0.0" | ||
+ | </pre> | ||
+ | |||
+ | == Implementing_an_actor_feature == | ||
+ | For details, see [[Triquetrum/Extending_Triquetrum#Implementing_an_actor_feature]]. | ||
+ | |||
+ | # File -> New -> Feature Project | ||
+ | ## Name: org.kepler.triquetrum.ddp.feature | ||
+ | ## Finish | ||
+ | |||
+ | feature.xml should contain: | ||
+ | <pre> | ||
+ | <?xml version="1.0" encoding="UTF-8"?> | ||
+ | <feature | ||
+ | id="org.kepler.triquetrum.ddp.feature" | ||
+ | label="Feature" | ||
+ | version="1.0.0.qualifier"> | ||
+ | |||
+ | <requires> | ||
+ | <import plugin="org.eclipse.ui"/> | ||
+ | <import plugin="org.eclipse.core.runtime"/> | ||
+ | <import plugin="org.apache.commons.io"/> | ||
+ | <import plugin="org.apache.commons.logging"/> | ||
+ | <import plugin="org.ptolemy.commons"/> | ||
+ | <import plugin="ptolemy.gui"/> | ||
+ | <import plugin="ptolemy.core"/> | ||
+ | <import plugin="ptolemy.moml"/> | ||
+ | <import plugin="ptolemy.domains.sdf"/> | ||
+ | <import plugin="org.eclipse.triquetrum.workflow.editor" version="1.0.0" match="greaterOrEqual"/> | ||
+ | </requires> | ||
+ | |||
+ | <plugin | ||
+ | id="org.kepler.triquetrum.ddp" | ||
+ | download-size="0" | ||
+ | install-size="0" | ||
+ | version="0.0.0" | ||
+ | unpack="false"/> | ||
+ | |||
+ | <plugin | ||
+ | id="org.kepler.triquetrum.ddp.palette" | ||
+ | download-size="0" | ||
+ | install-size="0" | ||
+ | version="0.0.0"/> | ||
+ | |||
+ | </feature> | ||
+ | </pre> | ||
+ | |||
+ | == Creating the Feature == | ||
+ | For details, see [[Triquetrum/Extending_Triquetrum#Installing_the_feature_in_your_workbench]]. | ||
+ | |||
+ | == Using the actor == | ||
+ | |||
+ | Erwin suggested that running an update from a Triquetrum launched from another Eclipse is complex and that it would be better to [https://hudson.eclipse.org/triquetrum/job/triquetrum/lastSuccessfulBuild/artifact/org.eclipse.triquetrum.repository/target/products/ Download] a Triquetrum and run it. | ||
Revision as of 14:29, 7 June 2016
This page is a work in progress.
The goal of this page is to illustrate how to use Kepler Scientific Workflow System Actors in Triquetrum.
Note that the Kepler Scientific Workflow System is not the same as the Kepler release of Eclipse. In this page, the term Kepler is used to mean the Kepler Scientific Workflow System.
Contents
Kepler
The Kepler Scientific Workflow System has actors that we would like to use in Triquetrum. In this case, we would like to use Kepler's Distributed Data-Parallel (DDP) actors.
For a brief overview of DDP, see:
- Ilkay Altintas, Daniel Crawl, Jianwu Wang. A Distributed Data-Parallel Execution Framework in the Kepler Scientific Workflow System. October 16, 2015. Presented at the Eleventh Biennial Ptolemy Miniconference, Berkeley.
The biggest issue with the integration is that Kepler's classes are not set up for OSGi. Triquetrum/Extending Triquetrum discusses how to set up the packages.
Status
As of June 7, 2016, porting the Kepler ExecutionChoice actor to Triquetrum is partially complete. The primary issue is that the ExecutionChoice code uses the Swing-based BasicGraphFrame code that is not present in Triquetrum. The next step would be to refactor ExecutionChoice to follow the ptolemy.actor.lib.hoc.Case example where the UI is in ptolemy.vergil.modal.CaseGraphFrame and CaseGraphTableau.
The current work is in checked in to the keplerTriquetrum repository. See Triquetrum/Kepler Triquetrum Repository
The instructions below partially cover how the repository was created.
Kepler Installation
- See Kepler and Eclipse for how to download and configure Kepler. To get the DDP work, we use the biokepler configuration, so:
mkdir kepler cd kepler svn co https://code.kepler-project.org/code/kepler/trunk/modules/build-area cd build-area/ ant change-to -Dsuite=biokepler ant clean-cache ant eclipse
- Import the Kepler projects into a new workspace that is not the same as where Triquetrum is set up
- Create a Kepler Run configuration an run it:
- project: biokepler
- main class: org.kepler.Kepler
Kepler Fails to start: ProvKAREntryHandler$Factory
If, during startup, we get this message:
java.lang.ClassNotFoundException: org.kepler.kar.handlers.ProvKAREntryHandler$Factory at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
The solution is to select use the biokepler project, not the kepler project.
Run the DDP demo
Kepler will start up. Close the splash window and under Components, double click on DDP and then wordcount-execution-choice.xml. Below is a screen shot of the initial Kepler window:
The wordcount demo has multiple ways of counting words. The default way is to use Unix commands.
Below is a run that counts words using Unix commands:
To use the other ways of counting words, double click on the Word Count actor and change the Choice parameter.
Our first goal is to get Kepler running and count words using Unix commands.
Creating a new Kepler Bundle
- Create the plug-in,
- In Eclipse: File -> New Plug-in Project
- In the dialog, set the Project Name: to org.kepler.triquetrum.ddp
- Click Next, then Finish
- In the org.kepler.triquetrum.ddpDemo package, update MANIFEST.MF:
Require-Bundle: org.eclipse.ui, org.eclipse.core.runtime, org.apache.commons.io, org.apache.commons.logging, ptolemy.gui, ptolemy.core, ptolemy.moml, ptolemy.domains.sdf
- Create the p2.inf file (See Creating an actor bundle)
- Open the org.eclipse.triquetrum.workflow.actor.plot project, click on META-INF, highlight the p2.inf file and select copy
- In the org.kepler.triquetrum.ddp project click on META-INF and paste
- The p2.inf file will be created.
Create ExecutionChoice.java
- In the org.kepler.triquetrum.ddp project, right click on the src icon and click New -> Class
- Change the package to org.kepler.ddp.actor
- We will update the enclosing type later
- Use File -> Open to open the kepler/ddp-common/src/org/kepler/ddp/actor/ExecutionChoice.java file, copy and paste it in to the newly created file
- There will be lots of errors.
Fix the compilation errors in ExecutionChoice.java
Below are the steps taken to fix the compilation errors in ExecutionChoice.java
The import org.apache.commons.io.FileUtils cannot be found
The error:
import org.apache.commons.io.FilenameUtils;
Solution: Add the following to org.kepler.triquetrum.ddp MANIFEST.MF under Require-Bundle:
org.apache.commons.io,
Various Kepler classes are missing
In ExecutionChoice, these imports are failing:
import org.kepler.build.modules.Module; import org.kepler.build.modules.ModuleTree; import org.kepler.ddp.gui.ExecutionChoiceEditorFactory; import org.kepler.ddp.gui.ExecutionChoiceEditorPane; import org.kepler.provenance.ProvenanceRecorder; import org.kepler.reporting.ReportingListener
The solution is to copy them over. An alternative would be to use the Eclipse projects for these Kepler projects.
(cd ~/src/kepler/build-area/src; \ tar -cf - org/kepler/build/modules/{Module,ModuleTree}.java) \ | (cd ~/src/workspaceTriq03JunB/org.kepler.triquetrum.ddp/src/; tar -xvf -) (cd ~/src/kepler/ddp-common/src; \ tar -cf - org/kepler/ddp/gui/ExecutionChoiceEditor{Factory,Pane}.java) \ | (cd ~/src/workspaceTriq03JunB/org.kepler.triquetrum.ddp/src/; tar -xvf -) (cd ~/src/kepler/provenance/src; \ tar -cf - org/kepler/provenance/ProvenanceRecorder.java) \ | (cd ~/src/workspaceTriq03JunB/org.kepler.triquetrum.ddp/src/; tar -xvf -) (cd ~/src/kepler/reporting/src; \ tar -cf - org/kepler/reporting/ReportingListener.java) \ | (cd ~/src/workspaceTriq03JunB/org.kepler.triquetrum.ddp/src/; tar -xvf -)
Various Ptolemy Classes are Missing
DDFDirector, various actor.hoc, KeplerDocumentationAttribute and LibraryAttribute classes are missing, so we copy them over.
FIXME: These should be added as a bundle to Triquetrum.
(cd $PTII; \ tar -cf - ptolemy/domains/ddf/kernel/{DDFDirector,ActorEnablingStatus}.java \ ptolemy/actor/util/DFUtilities.java \ ptolemy/actor/lib/hoc/{Case,CaseDirector,MultiCompositeActor,MultiCompositePort,Refinement,RefinementPort}.java \ ptolemy/actor/util/Time.java ptolemy/math/ExtendedMath.java \ ptolemy/vergil/basic/{DocAttribute,KeplerDocumentationAttribute}.java \ ptolemy/moml/LibraryAttribute.java) \ | (cd ~/src/workspaceTriq03JunB/org.kepler.triquetrum.ddp-common/src/; tar -xvf -)
Hadoop
The error is:
Description Resource Path Location Type Configuration cannot be resolved to a type DDPDataSink.java /org.kepler.triquetrum.ddpDemo/src/org/kepler/ddp/actor/pattern line 258 Java Problem
The issue is that these imports in DDPDataSink are not present
import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.FileUtil; import org.apache.hadoop.fs.Path;
One solution would be to import the jars from ~/KeplerData/workflows/module/hadoop/tools/share/hadoop/
A better solution would be to find a hadoop Eclipse bundle upon which we could depend.
Kepler provides a hadoop project that could be helpful here!
Solution: Right click on the org.kepler.triquetrum.ddpDemo project -> Build Path -> Configure Build Path and add the following package dependencies
- ddp-common
- hadoop
Packaging the plugin(s) for your RCP editor
See Packaging the plugin(s) for your RCP editor
Create a simple (non-Java plugin): File -> New Plug-in Project.
- Name: org.kepler.triquetrum.ddp.palette
- Uncheck Create a Java Project
- Click Next, then Finish
Right click on the newly created project, select File -> New -> File and create plugin.xml.
plugin.xml should look like:
<?xml version="1.0" encoding="UTF-8"?> <?eclipse version="3.4"?> <plugin> <extension name="Triquetrum palette ExecutionChoice extension" point="org.eclipse.triquetrum.workflow.editor.paletteContribution"> <group displayName="Examples"> <entry class="org.kepler.ddp.actor.ExecutionChoice" displayName="ExecutionChoice" type="Actor"> </entry> </group> </extension> </plugin>
MANIFEST.MF should look like:
Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Palette entry for ExecutionChoice Bundle-SymbolicName: org.kepler.triquetrum.ddp.palette;singleton:=true Bundle-Version: 1.0.0.qualifier Require-Bundle: org.eclipse.triquetrum.workflow.editor;bundle-version="1.0.0"
Implementing_an_actor_feature
For details, see Triquetrum/Extending_Triquetrum#Implementing_an_actor_feature.
- File -> New -> Feature Project
- Name: org.kepler.triquetrum.ddp.feature
- Finish
feature.xml should contain:
<?xml version="1.0" encoding="UTF-8"?> <feature id="org.kepler.triquetrum.ddp.feature" label="Feature" version="1.0.0.qualifier"> <requires> <import plugin="org.eclipse.ui"/> <import plugin="org.eclipse.core.runtime"/> <import plugin="org.apache.commons.io"/> <import plugin="org.apache.commons.logging"/> <import plugin="org.ptolemy.commons"/> <import plugin="ptolemy.gui"/> <import plugin="ptolemy.core"/> <import plugin="ptolemy.moml"/> <import plugin="ptolemy.domains.sdf"/> <import plugin="org.eclipse.triquetrum.workflow.editor" version="1.0.0" match="greaterOrEqual"/> </requires> <plugin id="org.kepler.triquetrum.ddp" download-size="0" install-size="0" version="0.0.0" unpack="false"/> <plugin id="org.kepler.triquetrum.ddp.palette" download-size="0" install-size="0" version="0.0.0"/> </feature>
Creating the Feature
For details, see Triquetrum/Extending_Triquetrum#Installing_the_feature_in_your_workbench.
Using the actor
Erwin suggested that running an update from a Triquetrum launched from another Eclipse is complex and that it would be better to Download a Triquetrum and run it.