Triquetrum/Kepler

This page is a work in progress.

The goal of this page is to illustrate how to use Kepler Scientific Workflow System Actors in Triquetrum.

Kepler

The Kepler Scientific Workflow System has actors that we would like to use in Triquetrum. In this case, we would like to use Kepler's Distributed Data-Parallel (DDP) actors.

For a brief overview of DDP, see:

• Ilkay Altintas, Daniel Crawl, Jianwu Wang. A Distributed Data-Parallel Execution Framework in the Kepler Scientific Workflow System. October 16, 2015. Presented at the Eleventh Biennial Ptolemy Miniconference, Berkeley.

The biggest issue with the integration is that Kepler's classes are not set up for OSGi. Triquetrum/Extending Triquetrum discusses how to set up the packages.

Kepler Installation

1. See Kepler and Eclipse for how to download and configure Kepler.

Run the DDP demo

We will run Kepler from within Eclipse later, but first, let's see the demo we want to run. In the command line, run

 cd kepler/build-area
 ant run

Kepler will start up. Close the splash window and under Components, double click on DDP and then wordcount-execution-choice.xml. Below is a screen shot of the initial Kepler window:

The wordcount demo has multiple ways of counting words. The default way is to use Unix commands.

Below is a run that counts words using Unix commands:

File:File:KeplerWorkflowSystemLocalExecution.png

To use the other ways of counting words, double click on the Word Count actor and change the Choice parameter.

Our first goal is to get Kepler running and count words using Unix commands.

Kepler and Eclipse

1. After running "ant eclipse", import the Kepler projects into the same workspace as to where Triquetrum is set up
2. Create a Kepler Run configuration an run it

Kepler Fails to start

During startup, we get this message:

 java.lang.ClassNotFoundException: org.kepler.kar.handlers.ProvKAREntryHandler$Factory

at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.kepler.kar.KAREntryHandlerFactory.registerKAREntryHandlers(KAREntryHandlerFactory.java:94) at org.kepler.objectmanager.cache.CacheManager.initKAREntryHandlers(CacheManager.java:240) at org.kepler.objectmanager.cache.CacheManager.getInstance(CacheManager.java:163) at org.kepler.icon.ComponentEntityConfig.tryToAssignIconBySemanticType(ComponentEntityConfig.java:283) at org.kepler.icon.ComponentEntityConfig.addSVGIconTo(ComponentEntityConfig.java:158) at org.kepler.objectmanager.library.LibraryManager.createAndAddTreeItem(LibraryManager.java:513) at org.kepler.objectmanager.library.LibraryGenerator.generate(LibraryGenerator.java:113) at org.kepler.objectmanager.library.LibraryManager.buildLibrary(LibraryManager.java:439) at org.kepler.moml.KARLibraryBuilder.buildLibrary(KARLibraryBuilder.java:78) at ptolemy.actor.gui.UserActorLibrary.openLibrary(UserActorLibrary.java:220) at ptolemy.actor.gui.UserActorLibrary.openUserLibrary(UserActorLibrary.java:139) at ptolemy.vergil.VergilApplication._createDefaultConfiguration(VergilApplication.java:387) at ptolemy.vergil.VergilApplication._createEmptyConfiguration(VergilApplication.java:405) at ptolemy.actor.gui.ConfigurationApplication._parseArgs(ConfigurationApplication.java:1398) at ptolemy.vergil.VergilApplication._parseArgs(VergilApplication.java:536) at ptolemy.actor.gui.ConfigurationApplication.<init>(ConfigurationApplication.java:276) at ptolemy.actor.gui.MoMLApplication.<init>(MoMLApplication.java:100) at ptolemy.vergil.VergilApplication.<init>(VergilApplication.java:129) at ptolemy.vergil.VergilApplication.<init>(VergilApplication.java:112) at ptolemy.vergil.VergilApplication$1.run(VergilApplication.java:274) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:726) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Clicking OK brings up the Kepler UI, but we cannot open DDP -> wordcount-execution-choice.xml:

 ptolemy.kernel.util.IllegalActionException: Cannot find class: org.kepler.ddp.actor.ExecutionChoice. In Ptolemy,  
   classes are typically Java .class files. Entities like actors may instead be defined within a .xml file.  In any
   case, the class was not found. If the class uses a third party package, then the class would be present only if the
   third party package was found at compile time.  It may be necessary to upgrade Java or install the third party
   package, reconfigure and recompile.
 Because:
 -- /Users/cxh/KeplerData/workflows/module/ddp/demos/org/kepler/ddp/actor/ExecutionChoice.xml (No such file or directory)
 -- XML file not found relative to classpath.
 -- /Users/cxh/src/kepler/kepler/org/kepler/ddp/actor/ExecutionChoice.xml
 /Users/cxh/src/kepler/kepler/org/kepler/ddp/actor/ExecutionChoice.xml (No such file or directory)
  in file:/Users/cxh/KeplerData/workflows/module/ddp/demos/wordcount-execution-choice.xml at line 212 and column 64

at ptolemy.moml.MoMLParser._createEntity(MoMLParser.java:4291) at ptolemy.moml.MoMLParser.startElement(MoMLParser.java:2837) at com.microstar.xml.XmlParser.parseElement(XmlParser.java:921) at com.microstar.xml.XmlParser.parseContent(XmlParser.java:1104) at com.microstar.xml.XmlParser.parseElement(XmlParser.java:924) at com.microstar.xml.XmlParser.parseDocument(XmlParser.java:481)

Resources

• Bug 84: Test out integrating Kepler code into Triquetrum: Use DDP/Hadoop as driver.
• Triquetrum/Extending Triquetrum