Jump to: navigation, search

Difference between revisions of "A guide to building a DLTK-based language IDE"

m (Creating source element parser)
(Creating source element parser)
Line 91: Line 91:
 
}
 
}
 
</pre></code>
 
</pre></code>
We need to specify org.eclipse.dltk.core.sourceParsers extension point. Extension point schema almost same to language toolkit schema.
+
We need to specify org.eclipse.dltk.core.sourceElementParsers extension point. Extension point schema almost same to language toolkit schema.
  
 
For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).
 
For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).

Revision as of 03:38, 9 July 2007

Summary

This article describes the steps one is to take when building an integrated development environment (IDE) for a dynamically-typed language using the Eclipse Dynamic Languages Toolkit (DLTK).

Introduction

There are three ways in DLTK to contribute language-specific things via extension points:

  • language toolkit – parsers, checkers, selection, completion engines, etc.
  • UI language toolkit – label providers, and so on.
  • other extension points, like documentation providers, view filters, etc.

Lets make simple example IDE and see how these mechanisms could be used. For language lets select Python.

You'll need DLTK Core SDK available in your workspace.

Building required IDE things

Most non-UI language-specific things are provided to DLTK by a language toolkit. It's most important thing in DLTK and language specific code interaction. Language toolkit could be added to project by using org.eclipse.dltk.language extension point, and requires definition of Eclipse project nature. Primary thing that language toolkit should provide is a source element parser. This parser builds a source module content, so it could be accessible from DLTK (from script explorer for example). Other required thing is a source module validation. Basically it could be just checking for a file name extension, in more complex cases it could be checks for file headers and more. UI stuff could be extended gradually as required.

Now for a start we will make example plug-in project, define language toolkit, nature and implement source element parser for a Python language.

Creating core plugin

To create core plugin lets select do New->Plug-in Project: set name org.eclipse.dltk.examples.python.core

Core create.png

After project created, we need to setup dependencies for:

  • org.eclipse.dltk.core – core of DLTK.
  • org.eclipse.core.filesystem – Eclipse file system.
  • org.eclipse.core.resources – Eclipse Resources, Workspace.

Dependencies.png

After that we could start adding nature, toolkit and source element parser.

Eclipse Nature

In extensions tab we need to create org.eclipse.core.resource.natures extension point and define it ID: org.eclipse.dltk.examples.python.core.nature, and create nature class PythonNature which extends ScriptNature class. ScriptNature class has all required stuff for nature management. It also setups incremental builder for project.


package org.eclipse.dltk.examples.python.core;

import org.eclipse.dltk.core.ScriptNature;

public class PythonNature extends ScriptNature
{
    public static final String NATURE_ID = PythonPlugin.PLUGIN_ID + ".nature";
}

Nature.png

Creating a simple Language Toolkit

In extensions tab we need to create org.eclipse.dltk.core.language extension point and define nature id( we just created it): org.eclipse.dltk.examples.python.nature and language toolkit class, which implements IDLTKLanguageToolkit, lets name it PythonLanguageToolkit. Language toolkit could extend AbstractLanguageToolkit class for some basic implementation. Most methods could be left stubs.

Dltk-guide1-lang toolkit.png

Also you should specify org.eclipse.dltk.ui.language extension with UI stuff.

Creating source element parser

Lets create class PythonSourceElementParser which implements ISourceElementParser interface. This class used to build model for source modules (The required IProblemReportercan be imported from org.eclipse.dltk.compiler.problem.IProblemReporter).

public class PythonSourceElementParser implements ISourceElementParser
{
 
 private ISourceElementRequestor fRequestor = null;
 private IProblemReporter fReporter = null;
 
 public PythonSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter ) {
    this.fRequestor = requestor;    
    this.fReporter = problemReporter;
 }
 
 public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {    
     String content = new String( contents );
     // create a parser, that gives us an AST, for example it could be antlr-based parser
     PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
     // fetch AST
     ModuleDeclaration moduleDeclaration = sourceParser.parse( content );                
     return moduleDeclaration;
 }
 public void setRequirestor( ISourceElementRequestor requestor ) {
     this.fRequestor = requestor;
 }
}

We need to specify org.eclipse.dltk.core.sourceElementParsers extension point. Extension point schema almost same to language toolkit schema.

For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).

Extending Source Element parser to build correct model

For building model we provide ISourceElementRequestor interface which is passed to the SourceElementParser when building content of source module. It works as visitor. SourceElementParser should call methods to define model elements. For now it could be types, methods, fields, package declarations. As result assuming that such visitor are called PythonSourceElementRequestor, we’ll have following code of the SourceElementParser:

public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {    
   String content = new String( contents );
   // create a parser, that gives us an AST, for example it could be antlr-based parser
   PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
   // fetch AST
   ModuleDeclaration moduleDeclaration = sourceParser.parse( content );                
   // traverse fetched AST with a visitor, that reports model element 
   // to given ISourceElementRequestor
   PythonSourceElementRequestor requestor = new PythonSourceElementRequestor( this.fRequestor );        
   try {
       moduleDeclaration.traverse( requestor );
   } catch( Exception e ) {
       e.printStackTrace( );
   }
   return moduleDeclaration;
}

As you can notice, it is possible to use any kind of parser with DLTK. For our python implementation we use ANTLR parser, for JavaScript RHINO, for Ruby JRuby.

After defining correct source element parser we could see module contents in script explorer. Also model with all project information could be accessible via creation of IDLTKProject, by calling:

IProject proj = ResourcesPlugin.getWorkspace().getRoot().getProject(“myproject”)
IDLTKProject project = DLTKCore.create( project );

Now, even without any UI code we could see results. Let’s create project, manually fix .project file by adding nature: org.eclipse.dltk.examples.python.core.nature

Then we run Eclipse, enable ScriptExplorer view and see the following:

Dltk ide guide result.png

Adding basic user stuff

To start working with language we need project creation wizard and editor. Outline view also could be useful.

First, let’s create a separate UI plugin, for example org.eclipse.dltk.examples.python.ui. Immediately after that, we add following dependencies (they will be used in future): org.eclipse.dltk.ui, org.eclipse.dltk.core, org.eclipse.dltk.launching, org.eclipse.ui.ide, org.eclipse.core.resources, and plugin that we created above: org.eclipse.dltk.examples.python.

Project creation wizard

For creating projects with associated nature, and configured buildpath, we need to create PythonProjectCreationWizard, which extends NewElementWizard from DLTK Core. You can look into example’s sources for a code, it’s just about a 100 lines of code. Basically you just have to set correct labels there, add pages(ProjectWizardFirstPage and ProjectWizardSecondPage classes from core) and implement following method:

public IModelElement getCreatedElement() {
    return DLTKCore.create(fFirstPage.getProjectHandle());
}

Also, ProjectWizardSecondPage requires a BuildpathsBlock object. We just extend BuildpathsBlock as following:

public class PythonBuildPathsBlock extends BuildpathsBlock { 
 public PythonBuildPathsBlock(IRunnableContext runnableContext, 
  IStatusChangeListener context, int pageToShow, boolean useNewPage,   
  IWorkbenchPreferenceContainer pageContainer) { 
     super(runnableContext, context, pageToShow, useNewPage, pageContainer); 
 }
 
 protected IPreferenceStore getPreferenceStore() {
     return PythonUI.getDefault().getPreferenceStore();
 }
 
 protected boolean supportZips() {
     return true;
 }
}

That’s all. Now we can register our wizard using org.eclipse.ui.newWizards extension point and see results:

Dltk ide guide wizard.png

Dltk ide guide wizard result1.png

Dltk ide guide wizard result2.png

Editor

Let’s create class PythonEditor, using ScriptEditor as base. It has abstract methods for creating folding structure provider, outline page, etc. In most cases it’s enough to leave them returning null. After that we just register editor through org.eclipse.editors extension point. Of course, you can use your own editor, but ScriptEditor base has lot’s of things, that are already implemented. Also revealing model elements in editor and other stuff would work only with a ScriptEditor.

Outline page, folding and more

Editor class has methods doCreateOutlinePage() and getFoldingStructureProvider(). Using base classes ScriptOutlinePage and AbstractASTFoldingStructureProvider we can very easily fetch outline view and folding, based on AST. Their implementations would be about half-screen of code.

Now we can look at the result:

Dltk ide guide finish.png

Other free UI

You also could add other UI like preferences, dialog pages(search, for ex.) and other.

Results

After doing this things, we have basic IDE with some free features:

  1. Structured source model, with delta management.
  2. Indexed source code. (By default all declarations are indexed). Fast search engine.
  3. Many free UI, like ScriptExplorer, Outline, Search and so on.


Example code

Example project sources could be downloaded from http://us.xored.com/~haiodo/org.eclipse.dltk.examples.python.zip

Requirements:
1. Latest DLTK 1.0 integration build required.
2. Eclipse 3.3M6 or later required.