View source for Triquetrum/Ptolemy/Porting

This page covers how to port Ptolemy actors to Triquetrum.  

''This page is not yet completed.''

== See Also ==
* [[Triquetrum/Extending_Triquetrum]] - A good place to start if you are not familiar with RCP and features.
* [[Triquetrum/Kepler]] - Porting Kepler actors to Triquetrum.
* [https://github.com/eclipse/triquetrum/issues/90 Bug #90 Document the process for porting Ptolemy actors to Triquetrum]

== IP Issues ==
All jar files that are to be shipped with an official release must be vetted for IP issues, see [https://wiki.eclipse.org/Development_Resources/Contribution_Questionnaire#Third_Party_Libraries Contribution Questionnaire->Third Party Libraries]].

Actors that are checked in to the Ptolemy II ptII svn repository are considered to be third party libraries.

A large subset of the Ptolemy II actors have been reviewed as part of Eclipse IP process.  The reviews themselves are not public, see [https://github.com/eclipse/triquetrum/issues/70 Issue #70: Open Ipzilla issues for Triquetrum third party code].

== OSGi split-package constraints ==
One limitation with OSGi is that a Java package is ideally only exported by one OSGi bundle. Or more correctly : it's tricky to have good control on what happens when a bundle imports a package when that package is exported by 2 provider bundles, with a same version. Especially when the contents of the package are different in those providers.

This means that actors in the ptolemy.actor.lib package that are not in source bundle that has passed the Eclipse Contribution Questionnaire (CQ) IP process must be put into another package, or added to the existing ptolemy.actor.lib bundle in ptII svn osgi-2-0 repository and a new CQ created.  

In general we do not want to request IP review very often, so if the package is already in a bundle that has been reviewed, then we would like to be able to for instance provide extra bundles that can be installed from an update site. To avoid the package name conflict, this would involve renaming the concerned package. See also [https://github.com/eclipse/triquetrum/issues/113 Bug 113: Need a way for OSGi bundles to add to the MoML Filters].

At the end of this page, an approach is described to combine this approach while still maintaining compatibility with existing models that use such extra actors, i.e. using the original fully-qualified actor class names.

== Overview ==
Porting Ptolemy actors requires the following steps
# Create an actor bundle
# Add Required Projects
# Implementing an actor
# Providing the actor class to the runtime
# Packaging the plugin(s) for your RCP editor

One key element is that because of IP issues, the Ptolemy actor OSGI jar files are not checked in to the Eclipse Triquetrum repository.  Instead they are checked in elsewhere (''FIXME: where?'')

On this page, we import the [http://ptolemy.eecs.berkeley.edu/ptolemyII/ptII10.0/ptII10.0.1/doc/codeDoc/ptolemy/actor/lib/hoc/package-summary.html Ptolemy II Higher Order Components (hoc) actors and directors] like the Case actor.

== Create an actor bundle ==
See [[Triquetrum/Extending_Triquetrum#Creating_an_actor_bundle | Extending Triquetrum - Creating an Actor Bundle]]

''TBD - Describe how to set up the bundle for the ptolemy.actor.lib.hoc classes''

== Add Required Projects ==
See [[Triquetrum/Extending_Triquetrum#Added_Required_Projects | Extending Triquetrum - Added Required Projects]]

== Implementing an actor ==
See [[Triquetrum/Extending_Triquetrum#Implementing_an_actor | Extending Triquetrum - Implementing an actor]]

The code for the actors is already present in the ptII tree, what needs to happen is that we need to create an OSGi jar file that contains those classes.

=== Check out the ptII osgi-2-0 branch ===
See [[Triquetrum/Extending_Triquetrum#Providing_the_actor_class_to_the_runtime | Extending Triquetrum - Providing the actor class to the runtime]]

Because of IP issues, the Ptolemy actor OSGi jar files are not checked in to the Eclipse Triquetrum repository.  Instead they are checked in to the regular Ptolemy II repo at https://github.com/icyphy/ptII

  git clone https://github.com/icyphy/ptII
  git checkout remotes/origin/origin/osgi-2-0
Or
  git clone https://github.com/icyphy/ptII --depth 1 --branch origin/osgi-2-0 --single-branch osgi-2-0

Note, the above checks out the head of the osgi-2-0 branch,  To check out the latest version that contains the approved sources that have gone through the Contribution Questionnaire (CQ), use

  git checkout   remotes/origin/origin/osgi-2-0-cq-1

See [https://github.com/eclipse/triquetrum/issues/70 Bug #70: Open Ipzilla issues for Triquetrum third party code]

==== Updating the osgi-2-0 branch and creating a new branch for cq ====

# Verify that Triquetrum works as is with Eclipse Oxygen
# Verify that Triquetrum works as is with Eclipse 2019-06
# Create a new branch called osgi-2-0-cq-2 that will be what will be submitted for cq.  osgi-2-0-cq-2 will be based on osgi-2-0-cq-1
# Update osgi-2-0-cq-2 with any changes that are present in osgi-2-0.  Is there a magic git command that will do this?
# Create a p2 site using osgi-2-0-cq-2 and test Triquetrum under Eclipse 2019-06
# Update the osgi-2-0 repo by doing diffs with the current ptII head.  Be sure not to add the README.txt, makefiles and package.html files that are present in the ptII head. Is there a magic git command that will do this?
# Update the osgi-2-0-cq-2 repo with diffs from osgi-2-0. Is there a magic git command that will do this?
# Create a p2 site using osgi-2-0-cq-2 and test Triquetrum under Eclipse 2019-06
# Create a diff for each cq'd osgi-2-0 feature/package/or-what-ever-the-word is
# Submit new cqs
# Profit!

=== Set up a jar file for the actors ===
''TBD - describe the edits necessary to the osgi branch''

''To do: Create a script that does this automatically''

=== Publish the OSGi bundle for the actors ===
''TBD - where?''
We probably should create a p2 site and project somewhere outside of the Eclipse repos that is writable by the Triquetrum contributors.  One possibility is to use the https://chess.eecs.berkeley.edu/triq.  There is a website there and we can add users that need to update the location.

== Packaging the plugin(s) for your RCP editor ==
See [[Triquetrum/Extending_Triquetrum#Packaging_the_plugin.28s.29_for_your_RCP_editor | Extending Triquetrum - Packaging the plugin(s) for your RCP Editor]]


=== Using preexisting *Icon.xml files ===
If you are adding Ptolemy II actors to Triquetrum, they may optionally have *Icon.xml files that can also be used in Triquetrum's editor to render customized actor icons.
These files are in fact Ptolemy MOML files, defining icons as an assembly of specific types of VisibleAttributes.

To collect icon files from a Ptolemy II source tree, you can follow a procedure as follows:
<blockquote>
Create /tmp/pj with:

  grep class ~/src/triquetrum-master/git/triquetrum/plugins/editor/org.eclipse.triquetrum.workflow.editor.palette/plugin.xml| grep actor.lib | awk -F \" '{print $(NF-1)}' | sed 's@\.@/@g' | awk '{print $1 ".java"}' | sort | uniq > /tmp/pj

Create a list of possible *Icon.xml files:

  sed 's@.java@Icon.xml@' /tmp/pj > /tmp/xj

Create a list of actual *Icon.xml files:

  (cd $PTII; ls -1 `cat /tmp/xj`) > /tmp/xj2

Copy the *Icon.xml files:

  (cd $PTII; cp `cat /tmp/xj2` /Users/cxh/src/triquetrum-master/git/triquetrum/plugins/editor org.eclipse.triquetrum.workflow.editor.palette/icons)
</blockquote>

We prefer to keep the Triquetrum editor palette unchanged, and to add extra content via extra palette contribution plugins.
So you may want to replace all references to org.eclipse.triquetrum.workflow.editor.palette by the actual plugin name of your actor contribution!

Once the actor classes and their icons are available, they must be listed in a palette contribution as extensions in a plugin.xml, with entries similar to:

    <entry
       class="ptolemy.actor.lib.CurrentDate"
       displayName="CurrentDate"
       icon="icons/CurrentDateIcon.xml"
       iconType="ptolemy"
       type="Actor">
    </entry>

For this kind of icon files, the iconType must be '''ptolemy'''!

=== Generating svg icon files ===
Another approach in Ptolemy to manage actor icons is by defining SVG fragments in the actor source code. These can be extracted as SVG files with a procedure as follows:
<blockquote>
Create /tmp/ij
(cd $PTII; cat /tmp/pj | xargs grep _iconDescription) | awk -F : '{print $1}' | sed /> /tmp/ij

If /tmp/ij contains that names of the new actors that have _iconDescription attributes, then use the following to create /tmp/doit:

  (cd $PTII; cat /tmp/pj | xargs grep _iconDescription) | awk -F : '{print $1}' | sed 's@/@.@g' | sed 's@\.java@@' | awk '{ print "set filename [string range " $1 " [expr [string last . " $1 "] + 1] [string length " $1 "]]; set a [java::new " $1 " $c $filename]; set i [java::cast ptolemy.kernel.util.ConfigurableAttribute [$a getAttribute _iconDescription]]; set f [open ${filename}Icon.svg w ]; puts $f [$i getExpression]; close $f"}' >& /tmp/doit

Edit /tmp/doit and add the following to the top

  set workspace [java::new ptolemy.kernel.util.Workspace]                                                                      
  set c [java::new ptolemy.kernel.CompositeEntity $workspace]

Invoke with $PTII/bin/ptjacl /tmp/doit
</blockquote>

The svg files will be created in the current directory. Then copy over the the *.svg files to the icons/ folder in your palette contribution plugin.
Then edit plugins.xml and create entries like:

      <entry
          class="ptolemy.actor.lib.CurrentMicrostep"
          displayName="CurrentMicrostep"
          icon="icons/CurrentMicrostepIcon.svg"
          iconType="svg"
          type="Actor">
      </entry>

For SVG icon files, the iconType must be '''svg'''!

=== Icon adaptation requirements ===
The existing icon definitions often need some small manual changes to get them to render in the right way in Triquetrum's graphical editor.
We list the typical interventions that are needed below.

==== Ptolemy MOML icon files ====
TextAttributes must get smaller fonts in Triquetrum. Exact location coordinates may need to be adjusted for correct positioning.
E.g. in AbsoluteValueIcon, the Annotation element (a TextAttribute) has a font size of 28 and a location of [-18.0, -17.0].
For Triquetrum this was changed to a font size of 20 and location [-18.0, -18.0]

The font Serif used in Ptolemy icons was replaced by Pt Serif in Triquetrum.

Some icon files refer to images that are retrievable on the classpath in Ptolemy using an expression like e.g. "$CLASSPATH/ptolemy/actor/lib/AccumulatorIcon.png".
In Triquetrum those contained image files should also be stored in the palette contribution plugin, e.g. also in the icons/ folder.
But to load them in an OSGi context, the path should be changed to e.g. "platform:/plugin/org.ptolemy.triquetrum.editor.palette/icons/AccumulatorIcon.png".

Some PtolemyII icon files use a location of class "ptolemy.kernel.util.RelativeLocation". This is an old package name for RelativeLocation, that is now in ptolemy.vergil.basic.RelativeLocation it seems. The corresponding MomlFilter to maintain compatibility with deprecated class names is not available in Triquetrum. Luckily it seems none of the icons encountered so far actually configyre a location relative to another elemnt, so we can simply replace them by standard "ptolemy.kernel.util.Location" types which work well.

Exact alignment between icon elements is not identical in both graphical environments. This could be related to the limitation of having integer pixel counts as x and y coordinates, whereas Ptolemy's diva framework seems to be able to work with doubles. This impacts text positions but also other types of icon elements (e.g. LineAttributes).

For complex cases it may be easier to regroup e.g. text elements. This has been done in e.g ArrayAverageIcon.xml and similar icons. In the original icon definition elements are declared for separate text parts like "i" and "= 1". In Triquetrum we've grouped them as one "i = 1" to avoid vertical alignment problems.

There are following remaining issues that can not yet be solved just by changing some values in icon files:
* No support in Triquetrum yet for something like AttributeValueAttribute, i.e. where the icon dynamically shows parameter expressions.
* There is a thing called a selector in  ConfigurationSelectIcon.xml ConfigurationSwitchIcon.xml. In Triquetrum the MomlParser raises an error for that, so we removed the dynamic color selections.
* CurrentDateIcon renders nicely in Ptolemy in an unexpected manner. Line lengths of 1 are rendered with lengths of 3 or 4, top border has no gaps etc.
The same lengths are rendered "correctly" in Triquetrum, e.g. with a length of 1, but this results in a not-nice visual result.

==== SVG icon files ====
The SVG icon files need less manipulation from the SVG fragments in the actor code. These are the typical things needed:
* there are missing spaces before style attributes in many SVG files (probably an issue in the extraction code decribed above?). E.g. EqualsIcon.svg : add space before style attr of text element.
* SVG icons often have element locations with negative coordinates. In such cases the leading <svg> element should be changed to e.g. : <svg x="-15" y="-15" style="overflow: visible">
(this example is for the converter icon)
* by default borders are not drawn for SVG icons, you need to set a style with stroke:black  to draw them. In Ptolemy diva this seems not needed, so we have added lots of stroke:black  styles. See e.g. https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/stroke for more info about expected default.

== Solving the ptolemy.actor.lib package naming issue ==
Ptolemy II provides a huge list of actors in a single package, ptolemy.actor.lib. And the policy seems to be to keep on extending the list of actors in there. As explained above already, this does not fit well with the standard approach for OSGi bundles and contained packages, if one does not want to keep on modifying the ptolemy.actor.lib bundle contained in the Triquetrum distribution.

Here we describe a possible approach to work with this situation. 

The requirements are :
* do not reuse a package name across 2 or more actor bundles : rename the packages
* support parsing and running existing models that point to the original actor class & package names : provide a mechanism for handling aliases

=== Creating a ptolemy.actor.lib2 ===
As an example we're going to create an extra bundle to add a couple of actors that are not yet in the default ptolemy.actor.lib bundle :
# ptolemy.actor.lib.Bernoulli
# ptolemy.actor.lib.BooleanMultiplexor
# ptolemy.actor.lib.BooleanSelect
# ptolemy.actor.lib.BooleanSwitch

We start by creating a new bundle ptolemy.actor.lib2, copying the actor code in there and renaming their package to ptolemy.actor.lib2.
Follow the steps described above to setup the bundle manifest etc.

=== Implementing aliases ===
As in all actor bundles, we must define a ModelElementClassProvider ( see [[Triquetrum/Extending_Triquetrum#Providing_the_actor_class_to_the_runtime | Extending Triquetrum - Providing the actor class to the runtime]] ). This provides a spot to implement aliases, as it determines which actor class is provided for a requested class name.

An example provider implementation can be :

 public class AliasModelElementClassProvider implements ModelElementClassProvider {
 
  /**
   * Creates a provider that does not care about class versions,
   * i.e. it will only check on class names to check if it can provide a requested class.
   *
   * @param knownClasses
   */
  public AliasModelElementClassProvider(Map<String,Class<? extends NamedObj>> knownClasses) {
    this(null, knownClasses);
  }
 
  /**
   * Creates a provider that cares about class versions,
   * i.e. it will check on class names and on the requested version to check if it can provide a requested class.
   *
   * @param version if null, the provider will not care about versions
   * @param knownClasses
   */
  public AliasModelElementClassProvider(VersionSpecification version, Map<String,Class<? extends NamedObj>> knownClasses) {
    this._knownClasses = knownClasses;
    this._versionSpec = version;
  }
 
  public Class<? extends NamedObj> getClass(String className, VersionSpecification versionSpec) throws ClassNotFoundException {
    if (_versionSpec != null && _versionSpec.compareTo(versionSpec)<0) {
      throw new ClassNotFoundException(className + " " + versionSpec);
    } else {
      for (Map.Entry<String, Class<? extends NamedObj>> actorEntry : _knownClasses.entrySet()) {
        if (actorEntry.getKey().equals(className)) {
          return actorEntry.getValue();
        }
      }
      throw new ClassNotFoundException(className);
    }
  }
 
  // private stuff
 
  private VersionSpecification _versionSpec;
  private Map<String,Class<? extends NamedObj>> _knownClasses;
 }

and this can be registered by the activator as follows, defining the aliases :

 public void start(BundleContext context) throws Exception {
  Version bundleVersion = context.getBundle().getVersion();
  VersionSpecification providerVersion = new ThreeDigitVersionSpecification(
      bundleVersion.getMajor(),
      bundleVersion.getMinor(),
      bundleVersion.getMicro(),
      bundleVersion.getQualifier());
 
  Map<String, Class<? extends NamedObj>> actorAliases = new HashMap<>();
    actorAliases.put("ptolemy.actor.lib.Bernoulli", Bernoulli.class);
    actorAliases.put("ptolemy.actor.lib.BooleanMultiplexor", BooleanMultiplexor.class);
    actorAliases.put("ptolemy.actor.lib.BooleanSelect", BooleanSelect.class);
    actorAliases.put("ptolemy.actor.lib.BooleanSwitch", BooleanSwitch.class);
 
  _apSvcReg = context.registerService(ModelElementClassProvider.class.getName(),
        new AliasModelElementClassProvider(providerVersion, actorAliases),
        null);
 }