Jump to: navigation, search

Difference between revisions of "A guide to building a DLTK-based language IDE"

 
(32 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Summary ==  
+
== Summary ==
  
This article describes the steps one is to take when building an integrated development environment (IDE) for a dynamically-typed language using the Eclipse Dynamic Language Toolkit (DLTK).
+
This tutorial describes the set of steps to create an integrated development environment (IDE) for a dynamically-typed language using Eclipse Dynamic Languages Toolkit (DLTK). In this tutorial we will make a simple IDE for Python language.
  
== Introduction ==
+
== Skeleton ==
  
* Python, Ruby, Tcl IDEs can be used as examples
+
At this step we'll build a skeleton for our IDE.
* to contribute language-specific things we use extension points:
+
Skeleton includes empty structure parser, content type, declaration of python language to DLTK.
** language toolkit – parsers, checkers, selection, completion engines, etc.
+
And we also create Python Project wizards, and support of python interpreter.
** UI language toolkit – label providers, and so on.
+
  
== Building required IDE things ==
+
[[DLTK IDE Guide:Step 1 Skeleton|Step 1: Skeleton]]
  
* Most non-UI, language-specific things are provided to DLTK by a language toolkit. It could be added to project by using org.eclipse.dltk.language extension point.
+
== Towards an Editor ==
* Defining of this extension point require definition of Eclipse project nature.
+
* For DLTK start working required one more thing, is SourceElementParser. This parsers build source module content. So it could be accessible from DLTK. (From script explorer for example).
+
  
Lets make example project and define language toolkit and nature and empty SourceElementParser. For language lets select Python.
+
At this step we'll add source code parser to the project and implement source code editor with syntax highlighting.
  
== Creating core plugin==
+
As a result our IDE will receive basic source editing and project navigation capabilities.
 +
This would include Code Outline, Editor with code folding and syntax highlighting, editor preference pages and Source module structure shown in the Script Explorer.
  
To create core plugin lets select do New->Plug-in Project: set name org.eclipse.dltk.examples.python.
+
[[DLTK IDE Guide:Step 2. Towards an Editor|Step 2. Towards an Editor]]
  
[[Image:core_create.png]]
+
== Towards an IDE ==
  
After project created, we need to setup dependencies for:
+
The step will show how to add search capabilities (search for Declarations, for References), implement "Open Type" action and "Goto Declaration" feature. We'll add basic code assistance (completion on keywords, and code templates support).
  
* org.eclipse.dltk.core – core of DLTK.
+
[[DLTK IDE Guide:Step 3. Towards an IDE|Step 3. Towards an IDE]]
* org.eclipse.core.filesystem – Eclipse file system.
+
* org.eclipse.core.resources – Eclipse Resources, Workspace.
+
  
[[Image:dependencies.png]]
+
[[Category:DLTK]]
 
+
After that we could start adding nature, toolkit and source element parser.
+
 
+
== Eclipse Nature ==
+
 
+
In extensions tab we need to create org.eclipse.core.resource.natures extension point and define it ID: org.eclipse.dltk.examples.python.core.nature, and create nature class PythonNature which extends ScriptNature class. ScriptNature class has all required stuff for nature management. It also setups incremental builder for project.
+
 
+
 
+
package org.eclipse.dltk.examples.python.core;
+
+
import org.eclipse.dltk.core.ScriptNature;
+
+
public class PythonNature extends ScriptNature
+
{
+
    public static final String NATURE_ID = PythonPlugin.PLUGIN_ID + ".nature";
+
}
+
 
+
[[Image:nature.png]]
+
 
+
== Creating a simple Language Toolkit ==
+
 
+
In extensions tab we need to create org.eclipse.dltk.core.languege extension point and define nature id( we just created it): org.eclipse.dltk.examples.python.nature and language toolkit class, which implements IDLTKLanguageToolkit, lets name it PythonLanguageToolkit. Language toolkit could extend AbstractLanguageToolkit class for some basic implementation. Most methods could be left stubs.
+
 
+
[[Image:lang_toolkit.png]]
+
 
+
== Creating source element parser ==
+
 
+
Lets create class PythonSourceElementParser which implements ISourceElementpParser interface. This class used to build model for source modules.
+
 
+
<code><pre>
+
public class PythonSourceElementParser implements ISourceElementParser
+
{
+
+
private ISourceElementRequestor fRequestor = null;
+
private IProblemReporter fReporter = null;
+
+
public PythonSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter ) {
+
    this.fRequestor = requestor;   
+
    this.fReporter = problemReporter;
+
}
+
+
public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {   
+
    String content = new String( contents );
+
    // create a parser, that gives us an AST, for example it could be antlr-based parser
+
    PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
+
    // fetch AST
+
    ModuleDeclaration moduleDeclaration = sourceParser.parse( content );               
+
    return moduleDeclaration;
+
}
+
public void setRequirestor( ISourceElementRequestor requestor ) {
+
    this.fRequestor = requestor;
+
}
+
}
+
</pre></code>
+
 
+
In PythonLanguageToolkit we need to implement createSourceElementParser function. In function we need to create new PythonSourceElementParser.
+
 
+
public ISourceElementParser createSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter, Map options) {
+
    return new PythonSourceElementParser(requestor, problemReporter);
+
}
+
 
+
For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).
+
 
+
== Extending Source Element parser to build correct model ==
+
 
+
For building model we provide ISourceElementRequestor interface which is passed to the SourceElementParser when building content of source module. It works as visitor. SourceElementParser should call methods to define model elements. For now it could be types, methods, fields, package declarations. As result assuming that such visitor are called PythonSourceElementRequestor, we’ll have following code of the SourceElementParser:
+
 
+
public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {   
+
    String content = new String( contents );
+
    // create a parser, that gives us an AST, for example it could be antlr-based parser
+
    PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
+
    // fetch AST
+
    ModuleDeclaration moduleDeclaration = sourceParser.parse( content );               
+
    // traverse fetched AST with a visitor, that reports model element
+
    // to given ISourceElementRequestor
+
    PythonSourceElementRequestor requestor = new PythonSourceElementRequestor( this.fRequestor );       
+
    try {
+
        moduleDeclaration.traverse( requestor );
+
    } catch( Exception e ) {
+
        e.printStackTrace( );
+
    }
+
    return moduleDeclaration;
+
}
+
 
+
As you can notice, it is possible to use any kind of parser with DLTK. For our python implementation we use ANTLR parser, for JavaScript RHINO, for Ruby JRuby.
+
 
+
After defining correct source element parser we could see module contents in script explorer. Also model with all project information could be accessible via creation of IDLTKProject, by calling:
+
 
+
IProject proj = ResourcesPlugin.getWorkspace().getRoot().getProject(“myproject”)
+
IDLTKProject project = DLTKCore.create( project );
+
 
+
Now, even without any UI code we could see results. Let’s create project, manually fix .project file by adding nature:
+
org.eclipse.dltk.examples.python.core.nature
+
 
+
Then we run Eclipse, enable ScriptExplorer view and see following:
+
 
+
== Adding basic user stuff ==
+
 
+
To start working with language we need project creation wizard and editor. Outline view also could be useful.
+
 
+
First, let’s create a separate UI plugin, for example org.eclipse.dltk.examples.python.ui. Immediately after that, we add following dependencies (they will be used in future): org.eclipse.dltk.ui, org.eclipse.dltk.core, org.eclipse.dltk.launching, org.eclipse.ui.ide, org.eclipse.core.resources, and plugin that we created above: org.eclipse.dltk.examples.python.
+
 
+
== Project creation wizard ==
+
 
+
For creating projects with associated nature, and configured buildpath, we need to create PythonProjectCreationWizard, which extends NewElementWizard from DLTK Core. You can look into example’s sources for a code, it’s just about a 100 lines of code. Basically you just have to set correct labels there, add pages(ProjectWizardFirstPage and ProjectWizardSecondPage classes from core) and implement following method:
+
 
+
public IModelElement getCreatedElement() {
+
    return DLTKCore.create(fFirstPage.getProjectHandle());
+
}
+
 
+
Also, ProjectWizardSecondPage requires a BuildpathsBlock object. We just extend BuildpathsBlock as following:
+
 
+
public class PythonBuildPathsBlock extends BuildpathsBlock {
+
public PythonBuildPathsBlock(IRunnableContext runnableContext, IStatusChangeListener context, int pageToShow, boolean useNewPage,  IWorkbenchPreferenceContainer pageContainer) { super(runnableContext, context, pageToShow, useNewPage, pageContainer); }
+
+
protected IPreferenceStore getPreferenceStore() {
+
    return PythonUI.getDefault().getPreferenceStore();
+
}
+
+
protected boolean supportZips() {
+
    return true;
+
}
+
}
+
 
+
That’s all. Now we can register our wizard using org.eclipse.ui.newWizards extension point and see results:
+
 
+
== Editor ==
+
 
+
Let’s create class PythonEditor, using ScriptEditor as base. It has abstract methods for creating folding structure provider, outline page, etc. In most cases it’s enough to leave them returning null.
+
After that we just register editor through org.eclipse.editors extension point. Of course, you can use your own editor, but ScriptEditor base has lot’s of things, that are already implemented. Also revealing model elements in editor and other stuff would work only with a ScriptEditor.
+
Outline page, folding and more
+
 
+
Editor class has methods doCreateOutlinePage() and getFoldingStructureProvider(). Using base classes ScriptOutlinePage and AbstractASTFoldingStructureProvider we can very easily fetch outline view and folding, based on AST. Their implementations would be about half-screen of code.
+
 
+
Now we can look at the result:
+
 
+
== Over free UI ==
+
 
+
You also could add other UI like preferences, dialog pages(search, for ex.) and other.
+
 
+
== Results ==
+
 
+
After doing this things, we have basic IDE with some free features:
+
# Structured source model, with delta management.
+
# Indexed source code. (By default all declarations are indexed). Fast search engine.
+
# Many free UI, like ScriptExplorer, Outline, Search and so on.
+

Latest revision as of 05:34, 6 May 2008

Summary

This tutorial describes the set of steps to create an integrated development environment (IDE) for a dynamically-typed language using Eclipse Dynamic Languages Toolkit (DLTK). In this tutorial we will make a simple IDE for Python language.

Skeleton

At this step we'll build a skeleton for our IDE. Skeleton includes empty structure parser, content type, declaration of python language to DLTK. And we also create Python Project wizards, and support of python interpreter.

Step 1: Skeleton

Towards an Editor

At this step we'll add source code parser to the project and implement source code editor with syntax highlighting.

As a result our IDE will receive basic source editing and project navigation capabilities. This would include Code Outline, Editor with code folding and syntax highlighting, editor preference pages and Source module structure shown in the Script Explorer.

Step 2. Towards an Editor

Towards an IDE

The step will show how to add search capabilities (search for Declarations, for References), implement "Open Type" action and "Goto Declaration" feature. We'll add basic code assistance (completion on keywords, and code templates support).

Step 3. Towards an IDE