JSDT/JSDTRenewalArchitecture

JSDT Renewal

Principles

The new architecture should be based on principals which will allow JSDT to meet the following fundamental requirements:

• Fast (Performance is the key requirement. If we choose between performance and some advanced feature, performance always wins)
• Stable
• Leveraging source solutions elsewhere
• Extendable

The functionality may be limited to the basics: Syntax highlighting, very basic code completion and validation, debugger. The main goal for Eclipse Neon is to provide a platform which meets the requirements above and not to focus on any advanced functionality. JSDT should not prevent users from developing (performance issues, false positive validation problems, etc) and help when possible.

Backwards compatibility

API compatibility

As a general rule, all APIs are subject to be deprecated for Neon. We will not maintain deprecated APIs on Neon and the removal of deprecated APIs can happen as early as Neon+1

We will have workspace compatibility for the Neon release.

Feature compatibility

Only workspace compatibility is guaranteed. We reserve the right to discontinue/replace features.

Current Problems

• JavaScript Parser
  • No ECMAScript 2015 support
  • Problems with corner syntax cases on the current specification
  • AST model does not cover ES 2015
  • Inference Engine is built-in
  • Validation is built-in
  • Validation only covers syntax errors found through parser.

• JavaScript Editor
  • JavaScript Code Scanner does not support ES2015

• Validation
  • No extension points for adding 3rd party validation

• Debugging
  • Debugger does not support debugging with current browsers and JavaScript runtimes

Proposal/Discussion for changes

Replace (or develop) a ES2015 compliant JavaScript Parser

See also 481706

Use typescript language services

Pros:

• Provides ES6 parsing
• Strong backing
• Enables typescript support in the future
• Inference is built-in

Cons:

• Requires node.js to run
• Tends to get slow on large projects due to inference
• Inference is not as successful as JavaScript targeted engines
• limits 3rd party integration with services bundled in.

Use acorn(or esprima) parser

Pros:

• Provides ESTree AST format used in many tools
• Used by Orion project
• Parser plugins to support other formats (.jsx etc)

Cons:

• Requires node.js to run
• AST tree needs to be converted to a Java model

Use Shift-ast parser

Pros:

• Implementation in Java
• Provides a detailed Java AST model
• Performance

Cons:

• Lack of tolerant parsing. Parsing stops on the first syntax error.
• AST model is not compatible to ESTree needs conversion
• Fairly new project
• SourceLocation implementatation was not complete at version 2.1.1

Build ES6 support to JDST parser

Pros:

• The easiest alternate to integrate.

Cons:

• Parser is originally developed for Java requires an update for even existing rules
• The mechanics to update the parser is fragile
• Requires JSDT project to continue with maintenance of the parser

AST Model update

We need to update AST model to include ES6 features. Unfortunately, JSDT (similar to JDT) has 2 AST models. Depending on the parser strategy chosen we may introduce a 3rd one.

Proposal 1

1. Update internal AST model
2. Deprecate/remove org.eclipse.wst.jsdt.core.dom AST model.

Proposal 2

1. Introduce a new AST model (shift-ast and/or estree compliant)
2. Update internal AST model but deprecate it.
3. Remove org.eclipse.wst.jsdt.core.dom AST model.

Committed Changes

Debugging

See 479466 The plan is to use chromedevtools. Follow up early work

Validation

See 477031

1. Remove anything related to validation from Parser except basic syntax validation.
2. Utilize "org.eclipse.wst.sse.ui.extensions.sourcevalidation" for as-you-type validation
3. Utilize "org.eclipse.wst.validation.validator" for batch validation

Principal changes comparing to the current architecture

• Rely on third-party parsers/compilers which are being actively driven by the community so we will get the latest standards (ECMA6 etc) supported in the tooling. Plus the ecosystem of extensions (jQuery, Angular, etc).