Revision as of 07:50, 23 March 2016

e(fx)clipse provides a SmartCode-Editing Framework who can be embedded in fairly any application (not only OSGi)

Integration into OSGi

Setup

Before adding the source you need to configure your application to support the smart code editor. To make that happen you need to add the following bundle to a feature or create a new one:

• org.eclipse.fx.code.editor
• org.eclipse.fx.code.editor.configuration
• org.eclipse.fx.code.editor.configuration.text
• org.eclipse.fx.code.editor.configuration.text.e4
• org.eclipse.fx.code.editor.configuration.text.fx
• org.eclipse.fx.code.editor.e4
• org.eclipse.fx.code.editor.fx
• org.eclipse.fx.code.editor.fx.e4

Basic Control with Syntax Highlighting

Basics

To make it easier to define syntax highlighting for any language the smart-code framework uses unlike the Eclipse IDE a declarative language named ldef.

The first step when integrating a syntax highlighting editor into your application is to create a file ending with .ldef (eg java.ldef, ...).

package my.plugin
 
js {
  partitioning {
    partition __dftl_partition_content_type
    partition __js_single_line_comment
    partition __js_multi_line_comment
    partition __js_string
    partition __js_regex
    rule {
      single_line __js_single_line_comment  "//"  => ''
      multi_line __js_multi_line_comment    "/*"  => "*/"
      single_line __js_string               "'"   => "'" escaped by "\\"
      single_line __js_string               '"'   => '"' escaped by "\\"
      single_line __js_regex                '/'   => '/' escaped by "\\"
    }
  }
  lexical_highlighting {
    rule __dftl_partition_content_type whitespace javawhitespace {
      default js_default
      js_operator {
        character [ 
          ';', '.', '=', '/', '\\', '+', '-', '*', 
          '<', '>', ':', '?', '!', ',', '|', '&', '^', '%', '~' 
        ]
      }
      js_bracket {
        character [ '(', ')', '{', '}', '[', ']' ]
      }
      js_keyword {
        keywords [
          "break", "case", "catch", "continue",
          "debugger","default",	"delete", "do",
          "else", "finally", "for", "function",
          "if", "in", "instanceof", "new",
          "return", "switch", "this", "throw",
          "try", "typeof", "var", "void",
          "while", "with" 
        ]
      }
      js_constant {
        keywords [ "true", "false", "undefined" ]
      }
      js_number {
        pattern "\\d" containing "[\\d|\\.]"
      }
    }
    rule __js_single_line_comment {
      default js_doc_default
    }
    rule __js_multi_line_comment {
      default js_doc_default
    }
    rule __js_string {
      default js_string
    }
    rule __js_regex {
      default js_string
    }
    token_def {
      js_default "-source-editor-code";
      js_operator "-source-editor-operator";
      js_bracket "-source-editor-bracket";
      js_keyword "-source-editor-keyword" bold;
      js_doc_default "-source-editor-doc";
      js_string "-source-editor-string";
      js_constant "-source-editor-keyword" bold;
      js_number "-source-editor-number";
    }
  }
}

As the smart code editor framework is built on top of the EclipseText-Framework you notice that the DSL aligns heavily to the concepts you find there:

• Documents are first split in partitions (eg source-code, documentation, strings, ...)
• Single Partitions are then split in tokens like constants, reserved words, ...
• Tokens are then associated with styling information (color, font-weight,...)

Once you created an ldef-File for your favorite language the Eclipse-Tooling will generate a total of 3 files in the src-gen folder:

• $language$.json: Configuration data loaded at runtime and used to configure Eclipse Text
• $language$.css: Styleing information when using JavaFX as the renderer
• $language$-swt-style.json: Styleing information when using SWT as the renderer (we are not looking into this feature in this document)

Dependencies

To integrate a smart editor into your application you need to add dependencies to the following bundles your MANIFEST.MF:

• org.eclipse.fx.code.editor
• org.eclipse.fx.code.editor.configuration
• org.eclipse.fx.code.editor.configuration.text
• org.eclipse.fx.ui.services

Or if you prefer the following package-imports:

• org.eclipse.fx.code.editor
• org.eclipse.fx.code.editor.configuration
• org.eclipse.fx.code.editor.configuration.text
• org.eclipse.fx.code.editor.services
• org.eclipse.fx.ui.services.theme

Editor Registration

From the DSL we generated various artifacts are generated and we need to register them in the framework:

• $language$.json file is registered through a org.eclipse.fx.code.editor.configuration.text.ConfigurationModelProvider OSGi-Service-Component
• $language$.css file is registered through a org.eclipse.fx.ui.services.theme.Stylesheet OSGi-Service-Component

Samples for JavaScript could look like this:

ConfigurationModelProvider-Component

@Component
public class JavaScriptConfigurationProvider implements ConfigurationModelProvider {
  @Override
  public boolean applies(Input<?> input) {
    return ((URIProvider)input).getURI().endsWith(".js");
  }
 
  @Override
  public LanguageDef getModel(Input<?> input) {
    try(InputStream in = getClass().getResourceAsStream("js.json");
        Reader r = new InputStreamReader(in)) {
      return EditorGModel.create().createObject(r);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }
}

Stylesheet-Component

@Component
public class JavaScriptStylesheet implements Stylesheet {
  @Override
  public boolean appliesToTheme(Theme t) {
    return true;
  }
 
  @Override
  public URL getURL(Theme t) {
    try {
      return new URL("platform:/plugin/my.plugin/my/plugin/js.css");
    } catch (MalformedURLException e) {
      throw new RuntimeException(e);
    }
  }
}

Syntax Highlighting

If you remember the ldef file we shown above we defined the token styles like this:

token_def {
  js_default "-source-editor-code";
  js_operator "-source-editor-operator";
  js_bracket "-source-editor-bracket";
  js_keyword "-source-editor-keyword" bold;
  js_doc_default "-source-editor-doc";
  js_string "-source-editor-string";
  js_constant "-source-editor-keyword" bold;
  js_number "-source-editor-number";
  }

The first segment in between "" is the foreground color to be used but we are not directly putting the color values (eg #ff0000) there but color references who can be defined in another css. This allows us to easily ship them with themes eg a dark theme most likely would change the colors dramatically, ... .

All e(fx)clipse applications have at least one theme with a basic css all than you have to do is to change that to define color constants for the above values.

The perfect place is the .root-selector:

.root {
  -source-editor-code: rgb(0, 0, 0);
  -source-editor-operator: rgb(0, 0, 0);
  -source-editor-bracket: rgb(0, 0, 0);
  -source-editor-keyword: rgb(127, 0, 85);
  -source-editor-string:  rgb(42, 0, 255);
  -source-editor-number:  #6c83c4;
  -source-editor-doc: rgb(63, 127, 95);
}

The above color specs provide you a eclipse like syntax highlighting.

Editor visual customization

Syntax colors are defined as shown above but to customize other areas of your code area the following CSS will be of great help to you:

• .styled-text-area .invisible-char:
  selector for a specialized node which is a subtype of javafx.scene.text.Text and additionally allows to set an -fx-content property for the text displayed
• .styled-text-area .invisible-char.space:
  selector to configure what text to display for spaces
• .styled-text-area .invisible-char.tab:
  selector to configure what text to display for tabs
• .styled-text-area .invisible-char.enter:
  selector to configure what text
• .source-viewer.styled-text-area .selection-marker:
  selector to configure how a selected area is display most likely you want to customize the -fx-background-color - the javafx-Node is of type javafx.scene.layout.Region
• .styled-text-area .list-view:
  the content area of the text editor which is of type javafx.scene.layout.Region
• .styled-text-area .current-line:
  selector to configure how the current cursor line is displayed - the javafx-Node is of type javafx.scene.layout.Region
• .styled-text-area .line-ruler-text:
  selector to configure how the text in the line ruler is displayed
• .styled-text-area .text-caret:
  selector to configure how the text-caret is rendered - the javafx-Node is of type javafx.scene.shape.Shape
• .styled-text-hover .errors:
  selector to configure how the error hover popup is displayed
• .styled-text-hover .warnings:
  selector to configure how the warnings hover popup is displayed
• .styled-text-hover .infos:
  selector to configure how the infos hover popup is displayed
• .styled-text-hover .docs:
  selector to configure how the doc hover popup is displayed
• .styled-text-hover .others:
  selector to configure how the others hover popup is displayed
• .styled-text-hover .context-info:
  selector to configure how the warnings hover popup is displayed

Open an editor

Using the editor opener service

The smart code framework by default publishes an org.eclipse.fx.code.editor.services.EditorOpener-Service who can be retrieved via injection and used to open an editor based on an URI like this

class MyComponent {
  @Inject
  EditorOpener opener;
 
  private void openFile(File f) {
    opener.openEditor( f.toURI().toURL().toExternalForm() );
  } 
}

The default implementation of the opener function will search the window/perspective for a MElementContainer tagged with editorContainer, so if you want to use that service you need to update your application model and tag eg one of your MPartStacks with it.

Doing it yourself

If you don't want to use the EditorOpener-Service but do things yourself all you need to do is to create an MPart instance like this:

EModelService modelService = // ...;
MPart part = modelService.createModelElement(MPart.class);
part.setCloseable(true);
part.setLabel("MySample.js");
part.setContributionURI("bundleclass://org.eclipse.fx.code.editor.fx/org.eclipse.fx.code.editor.fx.TextEditor");
part.getPersistedState().put(org.eclipse.fx.code.editor.Constants.DOCUMENT_URL, uri);
part.getTags().add(EPartService.REMOVE_ON_HIDE_TAG);
// ...

Dirty State Tracking & Save

As we don't want to force you into predefined start-state and save strategy we don't push support to your application by default but require you to handle that yourself. What we provide are to base implementation:

• An addon who does the dirty state tracking org.eclipse.fx.code.editor.e4.addons.DirtyStateTrackingAddon
• A handler who saves the file a dirty file org.eclipse.fx.code.editor.e4.handlers.SaveFile

Adding support for Autocomplete Features

First of providing auto-completion is an optional feature you can provide for your language. Adding support is done with 2 distinct services:

• list of proposals: org.eclipse.fx.code.editor.services.ProposalComputer
• (optional) presentation of proposals: org.eclipse.fx.code.editor.fx.services.CompletionProposalPresenter

To register both services you need to push the following services to the OSGi-Service-Registry:

• org.eclipse.fx.code.editor.services.ProposalComputerTypeProvider
• org.eclipse.fx.code.editor.fx.services.CompletionProposalPresenterTypeProvider

For a potential JavaScript-Editor the providers could look like this:

@Component
public class JSProposalComputerTypeProvider implements ProposalComputerTypeProvider {
  @Override
  public boolean test(Input<?> input) {
    return input instanceof SourceFileInput && ((SourceFileInput)input).getURI().endsWith(".js");
  }
 
  @Override
  public Class<? extends ProposalComputer> getType(Input<?> input) {
    return JSProposalComputer.class;
  }
}

@Component
public class JSCompletionProposalPresenterTypeProvider implements CompletionProposalPresenterTypeProvider {
  @Override
  public Class<? extends CompletionProposalPresenter> getType(Input<?> s) {
    return JSCompletionProposalPresenter.class;
  }
  @Override
  public boolean test(Input<?> input) {
    return input instanceof SourceFileInput && ((SourceFileInput)input).getURI().endsWith(".js");
  }
}

The cool thing about JSProposalComputer & JSCompletionProposalPresenter is that they are created per editor instance and so you can access all informations available for an editor through Eclipse DI.

The computer is responsible to talk to your language framework to detect what auto-completions are available (eg methods, ...)

public class JSProposalComputer implements ProposalComputer {
  private Logger logger;
  @Inject
  JSProposalComputer(@Log Logger logger /* more stuff */) {
    this.logger = logger;
  }
 
  public CompletableFuture<List<CompletionProposal>> compute(ProposalContext context) {
    // ... produce a JSCompletionProposal
  }
}

You can take a look at how the dart-language tooling does it at DartProposalComputer.java

The presenter on the other hand takes the proposal objects and translates them into text and graphic informations shown by the proposal dialog.

public class JSCompletionProposalPresenter implements CompletionProposalPresenter {
  private final GraphicsLoader graphicsLoader;
 
  @Inject
  public DartCompletionProposalPresenter(GraphicsLoader graphicsLoader) {
    this.graphicsLoader = graphicsLoader;
  }
 
  @Override
  public ICompletionProposal createProposal(CompletionProposal proposal) {
    JSCompletionProposal p = (JSCompletionProposal) proposal;
    String label = p.getProposalText();
    Node imageNode = p.getJSType().equals("method") ? new ImageView("method.png") : new ImageView("field.png");
    // ...
    return new FXCompletionProposal( proposal, label, () -> imageNode, null, null );
  }
}

You can take a look at how the dart-language tooling does it at DartCompletionProposalPresenter.java

Adding support for Error-Markers

Errors, Warnings, ... are presented called Annotations and there are 2 ways to present them in the editor:

• as part of the line-ruler on the left
• as part of the text-content eg underlining the text

Providing annotations

The first and mandatory step to show annotations in an editor is to provide an instance org.eclipse.jface.text.source.AnnotationModel for your language. The model keeps track of org.eclipse.jface.text.source.Annotation.

To instruct the framework to build such a model you need to register an org.eclipse.fx.code.editor.services.AnnotationModelTypeProvider following the same steps you used for the auto-complete:

@Component
public class JSAnnotationModelTypeProvider implements AnnotationModelTypeProvider {
  @Override
  public boolean test(Input<?> input) {
    return input instanceof SourceFileInput && ((SourceFileInput)input).getURI().endsWith(".js");
  }
 
  @Override
  public Class<? extends IAnnotationModel> getType(Input<?> input) {
    return JSAnnotationModel.class;
  }
}

public class JSAnnotationModel extends AnnotationModel {
  private final ThreadSynchronize synchronize;
 
  @Inject
  public JSAnnotationModel(Input<?> input, ThreadSynchronize synchronize) {
     // observe the input and when there are compile errors update
     // the annotations
  }
}

Presenting annotations

Once there's an annotation model for your language you can present it in the UI by providing an org.eclipse.fx.text.ui.source.AnnotationPresenter which is following the scheme you used for all other services already.

You register an OSGi-Service of type org.eclipse.fx.code.editor.fx.services.AnnotationPresenterTypeProvider similar to this:

@Component
public class JSAnnotationPresenterTypeProvider implements AnnotationPresenterTypeProvider {
  @Override
  public Class<? extends AnnotationPresenter> getType(Input<?> s) {
    return JSAnnotationPresenter.class;
  }
 
  @Override
  public boolean test(Input<?> input) {
    return input instanceof SourceFileInput && ((SourceFileInput)input).getURI().endsWith(".js");
  }
}

Presenting annotations in line ruler

The annotation presenter distinguishes between 2 presentation types:

• org.eclipse.fx.text.ui.source.ILineRulerAnnotationPresenter
• org.eclipse.fx.text.ui.source.ITextAnnotationPresenter

To show annotations in the line ruler you need to implement the first one.

Presenting annotations as text-overlays

To provide text-markers all you need to do is to provide an implementation of ITextAnnotationPresenter

public class TextMarker implements ITextAnnotationPresenter {
  @Override
  public boolean isApplicable(Annotation annotation) {
    return annotation instanceof DartAnnotation;
  }
 
  @Override
  public Node createNode() {
    return new Region();
  }
 
  @Override
  public void updateNode(Node node, Annotation annotation) {
    Region r = (Region) node;
 
    JSAnnotation a = (JSAnnotation) annotation;
 
    Color c;
    switch (a.getError().getSeverity()) {
      case ERROR: c = Color.RED; break;
      case WARNING: c = Color.DARKORANGE; break;
      default:
      case INFO: c = Color.BLANCHEDALMOND; break;
    }
    r.setBorder(new Border(new BorderStroke(c, BorderStrokeStyle.SOLID, CornerRadii.EMPTY, new BorderWidths(0, 0, 1.5, 0))));
  }
 
  @Override
  public String toString() {
    return "JSTextMarkerAP@" + hashCode();
  }
}

Code navigation

Predefined annotations like Linenumbers, ...

There are currently 2 predefined control annotations found in org.eclipse.fx.text.ui.Feature:

• SHOW_LINE_NUMBERS: showing a ruler on the left with linenumbers (on by default)
• SHOW_HIDDEN_SYMBOLS: displaying informations about whitespace characters like tab, line-breaks (off by default)

Features are provided by SourceViewerConfiguration#getFeatures() : SetProperty<Feature> and modifying the provided Set instance - adding/removeing features - updates the editor bound to it.

Using preferences

When using the framework in an e4 application where DefaultSourceViewerConfiguration one can also use the preference system to set the features and restore them the current feature settings are stored with

• key: Constants.PREFERENCE_KEY_EDITOR_FEATURE
• nodePath: Constants.PREFERENCE_NODE_PATH

or expressed through the e(fx)clipse DI-Addons

@Inject
@Preference(key=Constants.PREFERENCE_KEY_EDITOR_FEATURE,nodePath=Constants.PREFERENCE_NODE_PATH) 
Set<Feature> featureSet;

or if you want to publish

@Inject
@Preference(key=Constants.PREFERENCE_KEY_EDITOR_FEATURE,nodePath=Constants.PREFERENCE_NODE_PATH) 
Property<Set<Feature>> featureSet;

As toggling features is a standard task there's a default handler implementation (org.eclipse.fx.code.editor.fx.handlers.ToggleEditorFeature) available for this task which you can register in your e4xmi-File. The command-parameter you need to pass is named feature and you pass in the feature name as a string.

Tabs and Spaces

Tab Advance

By default a tab (\t) character advances in the text 4 characters. If you want to customize this you need make use of the Preference-System with:

• key: Constants.PREFERENCE_TAB_ADVANCE
• nodePath: Constants.PREFERENCE_NODE_PATH

So if you want to modify the tab-advance for the editor you can use the e(fx)clipse DI-Extensions like this:

@Inject
@Preference(key=Constants.PREFERENCE_TAB_ADVANCE,nodePath=Constants.PREFERENCE_NODE_PATH)
Property<Integer> tabAdvancePreference;

As updateing the tabAdvance is a standard task there's a handler implementation available named org.eclipse.fx.code.editor.fx.handlers.SetTabAdvance who accepts a command-parameter named tabAdvance.

Tabs instead of white spaces

By default the control inserts a tab (\t) character if you press the tab-key but if you want to insert a certain amount of spaces instead you can configure the editor to act like that by toggleing the preference with:

• key: Constants.PREFERENCE_SPACES_FOR_TAB
• nodePath: Constants.PREFERENCE_NODE_PATH

As toggling the preference is a standard task there's handler implementation available named org.eclipse.fx.code.editor.fx.handlers.ToggleInsertSpacesForTab

The InputContext

The org.eclipse.fx.code.editor.InputContext is a way to group inputs in a logical context (comparable to a project in your eclipse instance) by default all Input instances are grouped in a default-context but you are free to provide your own logic by registering a service of type org.eclipse.fx.code.editor.services.InputContextProvider in the service registry.

Dynamic Lexical Highlightings

There are chances that you want to color things like library functions in your UI but those can not be preconfigured because eg they depend on what libraries you have in your application, version of your language you develop against, ...

The code editing framework has support to dynamically add/remove lexical highlighting rules (even to an already opened editor). The service API responsible to provide rules dynamically is org.eclipse.fx.code.editor.configuration.text.DynamicScannerRuleCalculator. To register this service you need to push an instance of org.eclipse.fx.code.editor.configuration.text.DynamicScannerRuleCalculatorTypeProvider.

Sample

The most complete sample showing the complete framework with all its features in action can be found at github implementing a Dart-Code-Editor