Difference between revisions of "Triquetrum/CapeCode"

Latest revision as of 16:08, 1 February 2017

CapeCode

This intro to be corrected/extended by Christoper or someone else with real knowledge on CapeCode.

CapeCode is a project of the UC Berkeley CHESS group, who are also the originators of Ptolemy II. CapeCode is related to the domain of IoT, implementing assemblies of accessors to interact with connected devices in a well-defined way, reusing formal concepts from Ptolemy II's actor system.

Supporting CapeCode actors in Triquetrum's editor

As in other cases where we want to extend Triquetrum with extra actors, adding CapeCode actors consists of 2 main actions :

• providing the actor implementations in a way that is accessible by the editor and execution runtime
• defining editor palette contributions for them

The CapeCode actors that we consider here are JSAccessor actors, whose concrete behaviour is implemented in JavaScript. Adding CapeCode actors in a Triquetrum installation then requires at least the following :

• having the JSAccessor implementation (+dependencies) in the Triquetrum platform and being able to locate and load the JavaScript files that define the actor implementations. This is tricky business, especially when we want to remain compatible with native (non-OSGi) Ptolemy II approaches to load files from its class path.
• being able to read actor library definitions from CapeCode hosts and files and to add them to the editor palette. Ideally this should work without needing to restart the editor for each change in the CapeCode configuration.

Jar Files

Cape Code needs a number of jar files. To get a complete list:

 cd $PTII
 make echo_classpath_jars JARS=CAPECODE_JNLP_JARS | sed "s@$PTII/@@g" | awk -F : '{for(i=1;i<=NF;i++) print $i}'

The jar files fall in to the these categories

1. jar files that contain actors that must be added to the palette
   1. ptolemy.actor.lib.jjs.JavaScript and org.terraswarm.accessor.JSAccessor are the two primary actors
      1. ptolemy/actor/lib/jjs/demo/demo.jar
      2. ptolemy/actor/lib/jjs/jjs.jar
      3. ptolemy/actor/lib/jjs/modules/demo.jar
      4. ptolemy/actor/lib/jjs/modules/modules.jar
   2. There are a number of other actors that are in the configuration that need to be made available. Some of these actors are possibly in packages that are already in Triquetrum, which could cause trouble. We also need to include the demos.
      1. org/ptolemy/machineLearning/machineLearning.jar
      2. org/ptolemy/optimization/optimization.jar
      3. org/ptolemy/ssm/ssm.jar
      4. org/terraswarm/accessor/accessor.jar
      5. org/terraswarm/accessor/accessors/accessors.jar
      6. org/terraswarm/accessor/demo/demo.jar
      7. org/terraswarm/ros/demo/demo.jar
      8. ptolemy/actor/lib/colt/colt.jar
      9. ptolemy/actor/lib/conversions/json/json.jar
      10. ptolemy/actor/lib/database/database.jar
      11. ptolemy/actor/lib/io/comm/comm.jar
      12. ptolemy/actor/lib/io/comm/demo/demo.jar
      13. ptolemy/actor/parameters/demo/demo.jar
   3. Below are lower priority actors
      1. org/ptolemy/opencv/opencv.jar
      2. org/ptolemy/ptango/ptango.jar
      3. ptolemy/actor/lib/mail/mail.jar
      4. ptolemy/actor/lib/database/mysql-connector-java-5.1.6-bin.jar - check the license of this
      5. ptolemy/cg/cgAccessor.jar
      6. ptolemy/domains/domains.jar
      7. ptolemy/domains/space/space.jar
      8. ptolemy/matlab/demo/demo.jar
      9. ptolemy/matlab/matlab.jar
   4. Below are documentation files that need to be handled. These are lower priority.
      1. doc/codeDoc.jar
      2. doc/codeDocCapeCode.jar
      3. doc/docConfig.jar
   5. ptolemy/actor/lib/js/JavaScript.java uses the Mozilla JavaScript interpreter instead of the Nashorn JavaScript interpreter. these files are lower priority and can probably be removed from Cape Code.
      1. lib/js.jar
      2. ptolemy/actor/lib/js/demo/demo.jar
      3. ptolemy/actor/lib/js/js.jar
2. jar files that contain third party dependencies. These files are in lib/. Not all of the jar files listed are required for a useful Triquetrum
   1. ptolemy.actor.lib.jjs.VertxHelperBase requires a number of vertx and other packages. For these, we need to either add them as official third party Eclipse dependencies or provide them on the non-Eclipse p2 site at https://chess.eecs.berkeley.edu/ptexternal/src/p2/
      1. lib/hazelcast-3.5.2.jar
      2. lib/jackson-annotations-2.6.0.jar
      3. lib/jackson-core-2.6.1.jar
      4. lib/jackson-databind-2.6.1.jar
      5. lib/netty-buffer-4.0.33.Final.jar
      6. lib/netty-codec-4.0.33.Final.jar
      7. lib/netty-codec-http-4.0.33.Final.jar
      8. lib/netty-common-4.0.33.Final.jar
      9. lib/netty-handler-4.0.33.Final.jar
      10. lib/netty-transport-4.0.33.Final.jar
      11. lib/org.apache.oltu.oauth2.client-1.0.1-SNAPSHOT.jar
      12. lib/org.apache.oltu.oauth2.common-1.0.1-SNAPSHOT.jar
      13. lib/socketio.jar
      14. lib/vertx-codegen-3.2.0.jar
      15. lib/vertx-core-3.2.0.jar
      16. lib/vertx-hazelcast-3.2.0.jar
      17. lib/vertx-lang-js-3.2.0.jar
      18. lib/vertx-service-proxy-3.2.0.jar
      19. lib/vertx-web-3.2.0.jar
   2. There are other 3rd party dependencies
      1. com/cureos/cureos.jar - FIXME: need to check
      2. com/jhlabs/jhlabs.jar - FIXME: need to check
      3. edu/umich/eecs/april/april.jar - needed for the April Tags recognition accessor
      4. lib/bridj-0.6.2.jar - FIXME: need to check
      5. lib/gdp-0.7-2.jar - Needed for the GDP Accessor
      6. lib/jna-4.0.0-variadic.jar - Used by the GDP Accessor
      7. lib/ptcolt.jar - Used by the Colt random number actors.
      8. lib/ptliblicenses.jar - this contains all the licenses for the jars in $PTII/lib
      9. lib/slf4j-api-1.7.13.jar - FIXME: need to check
      10. lib/slf4j-nop-1.7.13.jar - FIXME: need to check
      11. lib/smack.jar - FIXME: need to check
      12. lib/smackx.jar - FIXME: need to check
      13. lib/webcam-capture-0.3.10.jar - Webcam capture
      14. lib/xbjlib-1.1.0.nrjavaserial.jar - FIXME: need to check
      15. org/json/json.jar - Definitely needed
   3. Below are some lower importance dependencies
      1. lib/javax.mail.jar - Used by the mail actor
      2. lib/jsoup-1.8.2.jar - Used for exporting models to html
      3. lib/matlab.jar - Used by the Matlab actor, requires shared libraries
      4. lib/nrjavaserial-3.11.0.devel.jar - Needed by the XBee actor
      5. lib/opencv-310.jar - Used by one OpenCV demo
      6. org/ptolemy/opencv/opencv.jar
3. jar files for functionality not needed in Triquetrum
   1. lib/PDFRenderer.jar
   2. lib/diva.jar
   3. lib/kieler.jar - Used for automatic layout
   4. lib/rsyntaxtextarea-2.6.0-SNAPSHOT.jar
   5. ptolemy/actor/gui/jnlp/CapeCodeApplication.jar
   6. ptolemy/ptsupport.jar - includes actor.lib and other common classes
   7. ptolemy/vergil/basic/export/export.jar
   8. ptolemy/vergil/basic/export/html/html.jar
   9. ptolemy/vergil/basic/export/html/jsoup/jsoup.jar
   10. ptolemy/vergil/basic/export/web/web.jar
   11. ptolemy/vergil/basic/imprt/accessor/accessor.jar
   12. ptolemy/vergil/basic/layout/layout.jar
   13. ptolemy/vergil/pdfrenderer/pdfrenderer.jar
   14. ptolemy/vergil/vergil.jar

Testing Jar Files

Below is a sample run, where all the jar files are in /tmp/jars. The output may change slightly over time, the main thing is that the numbers 1 through 4 appear.

bash-3.2$ java -classpath `cat /tmp/jars` ptolemy.actor.lib.jjs.NashornAccessorHostApplication -timeout 5000 test/auto/RampJSDisplay.js
commonHost.js: processCommandLineArguments(): Setting timeout to stop after 5000 ms.
Instantiated accessor RampJSDisplay with class /Users/cxh/ptII/org/terraswarm/accessor/accessors/web/test/au\
to/RampJSDisplay.js
1
2
3
4
commonHost.js: processCommandLineArguments(): Maximum time reached. Calling stopAllAccessors().
commonHost.js: invoking wrapup() for accessor: RampJSDisplay
All initialized accessors have wrapped up. Terminating.
bash-3.2$ 

In the text below we'll describe an approach to support CapeCode contributions to the editor palette.

Loading palette contributions dynamically

The traditional approach to define the graphical editor's palette is via extensions in the plugin.xml files of one or more palette configuration bundles.

This fits well with standard Eclipse practices but has some disadvantages as well, such as :

• Palette contents must be known & defined upfront,
• by someone able to develop Eclipse plugins,
• that must be built and then installed in the (RCP) application,
• which even requires a restart to activate the changes/additions.

But this approach does not fit the requirements for CapeCode where actor library configurations are managed in JSON files, hosted on remote locations. We don't want to replicate such configurations in plugin.xml files, nor having to go through complete development/build/install cycles to add or maintain CapeCode libraries.

So an alternative mechanism to add palette contributions has been put in place, to have the possibility to reuse CapeCode's existing library definition files. It is based on a provider interface (in org.eclipse.triquetrum.workflow.editor.palette.spi):

 public interface PaletteEntryProvider {
   /**
    *
    * @return the palette entries from this provider, as configuration elements
    */
   IConfigurationElement[] getPaletteEntries();
 }

org.eclipse.triquetrum.workflow.editor.palette.spi.PaletteConfigurationElement is a plain implementation of IConfigurationElement that provides all properties of palette entries to match the features that are also supported in the extension point schema.

To load an accessor library, the provider interface can be implemented based on CapeCode's org.terraswarm.accessor.AccessorLibrary. An example AccessorPaletteEntryProvider implementation can be found on https://github.com/erwindl0/triquetrum-capecode.

The implementation constructs an AccessorLibrary instance from the library json definition :

 CompositeEntity ce = new CompositeEntity();
 ce.setName("CapeCode configuration");
 AccessorLibrary accessorLibrary = new AccessorLibrary(ce, "Accessors");
 accessorLibrary.configure(null, URI.createPlatformPluginURI("org.eclipse.triquetrum.ptolemy.capecode.palette",true).toString() + "/src/main/resources", null);
 accessorLibrary.populate();

The library reads the JSON file from a URI and creates accessors for each entry in there. The test file contains one simple accessor :

 [
   "BaseA.js"
 ]

Each library entry then gets transformed in an IConfigurationElement as follows :

 public IConfigurationElement[] getPaletteEntries() {
   try {
     AccessorLibrary accessorLibrary = buildAccessorLibrary();
     List<IConfigurationElement> results = new ArrayList<>();
     for (Entity<?> accessor : (List<Entity<?>>) accessorLibrary.entityList()) {
       Map<String, String> attributes = new HashMap<>();
       attributes.put(CLASS, accessor.getClass().getName()); // (1)
       attributes.put(DISPLAY_NAME, accessor.getDisplayName()); // (2)
       attributes.put(ICON, "icons/time_obj.gif"); // (3)
       attributes.put(TYPE, "Actor"); // (4)
       PaletteConfigurationElement pce = new PaletteConfigurationElement("entry", "org.eclipse.triquetrum.ptolemy.capecode.palette", attributes );
       results.add(pce);
     }
     return results.toArray(new IConfigurationElement[0]);
   } catch (IllegalActionException | NameDuplicationException | MalformedURLException e) {
     LOGGER.error("Error obtaining palette entries for Acessors library", e);
     return new IConfigurationElement[] {};
   }
 }

Following properties are set (constants from static imports):

1. CLASS : the java class name of the accessor, always JSAccessor
2. DISPLAY_NAME : the name to be displayed in the palette
3. ICON : this is not supported as such in the accessor library json files it seems.
4. TYPE : always Actor

The provider implementation can then be registered in an extension, i.o. needing to statically define each individual entry in there :

 <plugin>
  <extension
        point="org.eclipse.triquetrum.workflow.editor.paletteContribution">
     <group
           displayName="CapeCode"
           provider="org.eclipse.triquetrum.ptolemy.capecode.palette.AccessorPaletteEntryProvider">
     </group>
  </extension>
 </plugin>

And the results is something like :

Hacks

The whole thing is still just a proof of concept! A fully functional implementation is not possible until the incompatibilities are resolved for loading the JavaScript files, between Ptolemy II's native approach and an OSGi runtime's requirements. To get the editor loading an accessor, all kinds of dirty hacks were done...

• Mocked the setup in a local json file.
• Ptolemy sources for jjs were extracted in the osgi-2.0 branch in the accessors project and adapted to get the files loading upto the point needed to get the editor working.
• All dependencies (netty, vert.x, jackson, org.terraswarm.accessor, ...) were added by hand in the dvp workspace. Details can be communicated/improved on request.

See the related issue 18 on GitHub for more details.