Revision as of 09:57, 9 November 2015

JSDT Renewal

Principles

The new architecture should be based on principals which will allow JSDT to meet the following fundamental requirements:

• Fast (Performance is the key requirement. If we choose between performance and some advanced feature, performance always wins)
• Stable
• Leveraging source solutions elsewhere
• Extendable

The functionality may be limited to the basics: Syntax highlighting, very basic code completion and validation, debugger. The main goal for Eclipse Neon is to provide a platform which meets the requirements above and not to focus on any advanced functionality. JSDT should not prevent users from developing (performance issues, false positive validation problems, etc) and help when possible.

Current Problems

• JavaScript Parser
  • No ECMAScript 2015 support
  • Problems with corner syntax cases on the current specification
  • AST model does not cover ES 2015
  • Inference Engine is built-in
  • Validation is built-in
  • Validation only covers syntax errors found through parser.

• JavaScript Editor
  • JavaScript Code Scanner does not support ES2015

• Validation
  • No extension points for adding 3rd party validation

• Debugging
  • Debugger does not support debugging with current browsers and JavaScript runtimes

Discussion for changes

Backwards compatability

API compatability

Feature compatability

Replace (or develop) a ES2015 compliant JavaScript Parser

See also 481706

Use typescript language services

Pros:

• Provides ES6 parsing
• Strong backing
• Enables typescript support in the future
• Inference is built-in

Cons:

• Requires node.js to run
• Tends to get slow on large projects due to inference
• Inference is not as successful as JavaScript targeted engines
• limits 3rd party integration with services bundled in.

Use acorn(or esprima) parser

Pros:

• Provides ESTree AST format used in many tools
• Used by Orion project

Cons:

• Requires node.js to run
• AST tree needs to be converted to a Java model

Use Shift-ast parser

Pros:

• Implementation in Java
• Provides a Java AST model
• Performance

Cons:

• non-tolerant parsing. Parsing stops on the first syntax error.
• AST model is not compatible to ESTree needs conversion
• Fairly new project

Build ES6 support to JDST parser

Pros:

• The easiest alternate to integrate.

Cons:

• Parser is originally developed for Java requires an update to even existing rules
• The way to update the parser is fragile
• Requires JSDT to continue with maintenance of the parser

AST Model update

We need to update AST model to include ES6 features. Unfortunately, JSDT (similar to JDT) has 2 AST models. Depending on the parser strategy chosen we may introduce a 3rd one.

Proposal 1

1. Update internal AST model
2. Deprecate/remove org.eclipse.wst.jsdt.core.dom AST model.

Proposal 2

1. Introduce a new AST model (shift-ast and/or estree compliant)
2. Update internal AST model but deprecate it.
3. Remove org.eclipse.wst.jsdt.core.dom AST model.

JSDT architecture proposal

• JS compiler (for the file)
  • Parses the file
  • Infers
  • Creates the model of the file and store it in JS model.
• JS model
  • Provides API for clients:
    • Code completion
    • Validation
    • Refactoring
    • Navigation
    • Debugger
    • …
  • Think about the structure of JS model. It could be a set (or tree?) of individual components/models each of them representing a *.js file or a lib. It should has a predefined contexts/libs (browser, etc.) + ability to add/remove individual js models to it.
• JS Editor
  • When the editor is opened it triggers JS compiler on the file
  • JS Source Editor configuration provides:
    • Presentation reconciler for highlighting
    • JS Code Assist for code completion
    • Hyper-link navigation
    • Outline
  • As-you-type validation. Transform it to a Structure Source Editor? SSE provides as-you-type validation out of the box but we need to investigate if the current as-you-type validation in SSE has any big issues preventing use it as-it-is for JS files. Anyway we need an as-you-type validation via SSE mechanism or our own validator pluged-in via a dedicated reconciler.
• HTML Editor
  • Should support the main functionality of JS Editor for JS code withing an HTML file.
• Validation
  • Build time validation can be plugged-in via the WST validation extension point.
• Debugger
• ...

Principal changes comparing to the current architecture

• Kill the builder. Compilation (for the file) is triggered when the editor is opened or changed in the editor. It means we won't have an up-to-date model for entire project. We will have fragments. Each fragment represents a *.js file opened in the editor or a predefined lib/context. The main advantage of this approach is performance and scalability. Should work on huge projects as fast as on small ones.
• Rely on third-party parsers/compilers which are being actively driven by the community so we will get the latest standards (ECMA6 etc) supported in the tooling. Plus the ecosystem of extensions (jQuery, Angular, etc).
• Do we need JSDT nature?