Jump to: navigation, search

Difference between revisions of "A guide to building a DLTK-based language IDE"

(Summary)
(Introduction)
Line 5: Line 5:
 
== Introduction ==
 
== Introduction ==
  
* Python, Ruby, Tcl IDEs can be used as examples
+
There are three ways in DLTK to contribute language-specific things via extension points:
* to contribute language-specific things we use extension points:
+
* language toolkit – parsers, checkers, selection, completion engines, etc.
** language toolkit – parsers, checkers, selection, completion engines, etc.
+
* UI language toolkit – label providers, and so on.
** UI language toolkit – label providers, and so on.
+
* other extension points, like documentation providers, view filters, etc.
 +
 
 +
Lets make simple example IDE and see how these mechanisms could be used. For language lets select Python.
  
 
== Building required IDE things ==
 
== Building required IDE things ==

Revision as of 07:03, 2 April 2007

Summary

This article describes the steps one is to take when building an integrated development environment (IDE) for a dynamically-typed language using the Eclipse Dynamic Languages Toolkit (DLTK).

Introduction

There are three ways in DLTK to contribute language-specific things via extension points:

  • language toolkit – parsers, checkers, selection, completion engines, etc.
  • UI language toolkit – label providers, and so on.
  • other extension points, like documentation providers, view filters, etc.

Lets make simple example IDE and see how these mechanisms could be used. For language lets select Python.

Building required IDE things

  • Most non-UI, language-specific things are provided to DLTK by a language toolkit. It could be added to project by using org.eclipse.dltk.language extension point.
  • Defining of this extension point require definition of Eclipse project nature.
  • For DLTK start working required one more thing, is SourceElementParser. This parsers build source module content. So it could be accessible from DLTK. (From script explorer for example).

Lets make example project and define language toolkit and nature and empty SourceElementParser. For language lets select Python.

Creating core plugin

To create core plugin lets select do New->Plug-in Project: set name org.eclipse.dltk.examples.python.

Core create.png

After project created, we need to setup dependencies for:

  • org.eclipse.dltk.core – core of DLTK.
  • org.eclipse.core.filesystem – Eclipse file system.
  • org.eclipse.core.resources – Eclipse Resources, Workspace.

Dependencies.png

After that we could start adding nature, toolkit and source element parser.

Eclipse Nature

In extensions tab we need to create org.eclipse.core.resource.natures extension point and define it ID: org.eclipse.dltk.examples.python.core.nature, and create nature class PythonNature which extends ScriptNature class. ScriptNature class has all required stuff for nature management. It also setups incremental builder for project.


package org.eclipse.dltk.examples.python.core;

import org.eclipse.dltk.core.ScriptNature;

public class PythonNature extends ScriptNature
{
    public static final String NATURE_ID = PythonPlugin.PLUGIN_ID + ".nature";
}

Nature.png

Creating a simple Language Toolkit

In extensions tab we need to create org.eclipse.dltk.core.languege extension point and define nature id( we just created it): org.eclipse.dltk.examples.python.nature and language toolkit class, which implements IDLTKLanguageToolkit, lets name it PythonLanguageToolkit. Language toolkit could extend AbstractLanguageToolkit class for some basic implementation. Most methods could be left stubs.

Dltk-guide1-lang toolkit.png

Creating source element parser

Lets create class PythonSourceElementParser which implements ISourceElementpParser interface. This class used to build model for source modules.

public class PythonSourceElementParser implements ISourceElementParser
{
 
 private ISourceElementRequestor fRequestor = null;
 private IProblemReporter fReporter = null;
 
 public PythonSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter ) {
    this.fRequestor = requestor;    
    this.fReporter = problemReporter;
 }
 
 public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {    
     String content = new String( contents );
     // create a parser, that gives us an AST, for example it could be antlr-based parser
     PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
     // fetch AST
     ModuleDeclaration moduleDeclaration = sourceParser.parse( content );                
     return moduleDeclaration;
 }
 public void setRequirestor( ISourceElementRequestor requestor ) {
     this.fRequestor = requestor;
 }
}

In PythonLanguageToolkit we need to implement createSourceElementParser function. In function we need to create new PythonSourceElementParser.

public ISourceElementParser createSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter, Map options) { 
    return new PythonSourceElementParser(requestor, problemReporter);
}

For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).

Extending Source Element parser to build correct model

For building model we provide ISourceElementRequestor interface which is passed to the SourceElementParser when building content of source module. It works as visitor. SourceElementParser should call methods to define model elements. For now it could be types, methods, fields, package declarations. As result assuming that such visitor are called PythonSourceElementRequestor, we’ll have following code of the SourceElementParser:

public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {    
   String content = new String( contents );
   // create a parser, that gives us an AST, for example it could be antlr-based parser
   PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
   // fetch AST
   ModuleDeclaration moduleDeclaration = sourceParser.parse( content );                
   // traverse fetched AST with a visitor, that reports model element 
   // to given ISourceElementRequestor
   PythonSourceElementRequestor requestor = new PythonSourceElementRequestor( this.fRequestor );        
   try {
       moduleDeclaration.traverse( requestor );
   } catch( Exception e ) {
       e.printStackTrace( );
   }
   return moduleDeclaration;
}

As you can notice, it is possible to use any kind of parser with DLTK. For our python implementation we use ANTLR parser, for JavaScript RHINO, for Ruby JRuby.

After defining correct source element parser we could see module contents in script explorer. Also model with all project information could be accessible via creation of IDLTKProject, by calling:

IProject proj = ResourcesPlugin.getWorkspace().getRoot().getProject(“myproject”)
IDLTKProject project = DLTKCore.create( project );

Now, even without any UI code we could see results. Let’s create project, manually fix .project file by adding nature: org.eclipse.dltk.examples.python.core.nature

Then we run Eclipse, enable ScriptExplorer view and see following:

Dltk ide guide result.png

Adding basic user stuff

To start working with language we need project creation wizard and editor. Outline view also could be useful.

First, let’s create a separate UI plugin, for example org.eclipse.dltk.examples.python.ui. Immediately after that, we add following dependencies (they will be used in future): org.eclipse.dltk.ui, org.eclipse.dltk.core, org.eclipse.dltk.launching, org.eclipse.ui.ide, org.eclipse.core.resources, and plugin that we created above: org.eclipse.dltk.examples.python.

Project creation wizard

For creating projects with associated nature, and configured buildpath, we need to create PythonProjectCreationWizard, which extends NewElementWizard from DLTK Core. You can look into example’s sources for a code, it’s just about a 100 lines of code. Basically you just have to set correct labels there, add pages(ProjectWizardFirstPage and ProjectWizardSecondPage classes from core) and implement following method:

public IModelElement getCreatedElement() {
    return DLTKCore.create(fFirstPage.getProjectHandle());
}

Also, ProjectWizardSecondPage requires a BuildpathsBlock object. We just extend BuildpathsBlock as following:

public class PythonBuildPathsBlock extends BuildpathsBlock { 
 public PythonBuildPathsBlock(IRunnableContext runnableContext, 
  IStatusChangeListener context, int pageToShow, boolean useNewPage,   
  IWorkbenchPreferenceContainer pageContainer) { 
     super(runnableContext, context, pageToShow, useNewPage, pageContainer); 
 }
 
 protected IPreferenceStore getPreferenceStore() {
     return PythonUI.getDefault().getPreferenceStore();
 }
 
 protected boolean supportZips() {
     return true;
 }
}

That’s all. Now we can register our wizard using org.eclipse.ui.newWizards extension point and see results:

Dltk ide guide wizard.png

Dltk ide guide wizard result1.png

Dltk ide guide wizard result2.png

Editor

Let’s create class PythonEditor, using ScriptEditor as base. It has abstract methods for creating folding structure provider, outline page, etc. In most cases it’s enough to leave them returning null. After that we just register editor through org.eclipse.editors extension point. Of course, you can use your own editor, but ScriptEditor base has lot’s of things, that are already implemented. Also revealing model elements in editor and other stuff would work only with a ScriptEditor.

Outline page, folding and more

Editor class has methods doCreateOutlinePage() and getFoldingStructureProvider(). Using base classes ScriptOutlinePage and AbstractASTFoldingStructureProvider we can very easily fetch outline view and folding, based on AST. Their implementations would be about half-screen of code.

Now we can look at the result:

Dltk ide guide finish.png

Other free UI

You also could add other UI like preferences, dialog pages(search, for ex.) and other.

Results

After doing this things, we have basic IDE with some free features:

  1. Structured source model, with delta management.
  2. Indexed source code. (By default all declarations are indexed). Fast search engine.
  3. Many free UI, like ScriptExplorer, Outline, Search and so on.