JSDT/JSDTRenewalArchitecture

JSDT Renewal

Principles

The new architecture should be based on principals which will allow JSDT to meet the following fundamental requirements:

• Fast (Performance is the key requirement. If we choose between performance and some advanced feature, performance always wins)
• Stable
• Leveraging source solutions elsewhere
• Extendable

The functionality may be limited to the basics: Syntax highlighting, very basic code completion and validation, debugger. The main goal for Eclipse Neon is to provide a platform which meets the requirements above and not to focus on any advanced functionality. JSDT should not prevent users from developing (performance issues, false positive validation problems, etc) and help when possible.

Current Problems

• JavaScript Parser
  • No ECMAScript 2015 support
  • Problems with corner syntax cases on the current specification
  • AST model does not cover ES 2015
  • Inference Engine is built-in
  • Validation is built-in
  • Validation only covers syntax errors found through parser.

• JavaScript Editor
  • JavaScript Code Scanner does not support ES2015

• Validation
  • No extension points for adding 3rd party validation

• Debugging
  • Debugger does not support debugging with current browsers and JavaScript runtimes

Proposal/Discussion for changes

Replace (or develop) a ES2015 compliant JavaScript Parser

See also 481706

Use typescript language services

Pros:

• Provides ES6 parsing
• Strong backing
• Enables typescript support in the future
• Inference is built-in

Cons:

• Requires node.js to run
• Tends to get slow on large projects due to inference
• Inference is not as successful as JavaScript targeted engines
• limits 3rd party integration with services bundled in.

Use acorn(or esprima) parser

Pros:

• Provides ESTree AST format used in many tools
• Used by Orion project
• Parser plugins to support other formats (.jsx etc)

Cons:

• Requires node.js to run
• AST tree needs to be converted to a Java model

Use Shift-ast parser

Pros:

• Implementation in Java
• Provides a detailed Java AST model
• Performance

Cons:

• Lack of tolerant parsing. Parsing stops on the first syntax error.
• AST model is not compatible to ESTree needs conversion
• Fairly new project
• SourceLocation implementatation was not complete at version 2.1.1

Build ES6 support to JDST parser

Pros:

• The easiest alternate to integrate.

Cons:

• Parser is originally developed for Java requires an update to even existing rules
• The way to update the parser is fragile
• Requires JSDT to continue with maintenance of the parser

AST Model update

We need to update AST model to include ES6 features. Unfortunately, JSDT (similar to JDT) has 2 AST models. Depending on the parser strategy chosen we may introduce a 3rd one.

Proposal 1

1. Update internal AST model
2. Deprecate/remove org.eclipse.wst.jsdt.core.dom AST model.

Proposal 2

1. Introduce a new AST model (shift-ast and/or estree compliant)
2. Update internal AST model but deprecate it.
3. Remove org.eclipse.wst.jsdt.core.dom AST model.

Validation

1. Remove anything related to validation from Parser except basic syntax validation.
2. Utilize "org.eclipse.wst.sse.ui.extensions.sourcevalidation" for as-you-type validation
3. Utilize "org.eclipse.wst.validation.validator" for batch validation

Committed Changes

Debugging

See 479466 The plan is to use chromedevtools. Follow up early work

Principal changes comparing to the current architecture

• Kill the builder. Compilation (for the file) is triggered when the editor is opened or changed in the editor. It means we won't have an up-to-date model for entire project. We will have fragments. Each fragment represents a *.js file opened in the editor or a predefined lib/context. The main advantage of this approach is performance and scalability. Should work on huge projects as fast as on small ones.
• Rely on third-party parsers/compilers which are being actively driven by the community so we will get the latest standards (ECMA6 etc) supported in the tooling. Plus the ecosystem of extensions (jQuery, Angular, etc).
• Do we need JSDT nature?