Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "A guide to building a DLTK-based language IDE"

m (Broken link is detected, so I've googled for an alternative one)
 
(2 intermediate revisions by one other user not shown)
Line 1: Line 1:
== Summary ==  
+
== Summary ==
  
This article describes the steps one is to take when building an integrated development environment (IDE) for a dynamically-typed language using the Eclipse [[Dynamic Languages Toolkit]] (DLTK).
+
This tutorial describes the set of steps to create an integrated development environment (IDE) for a dynamically-typed language using Eclipse Dynamic Languages Toolkit (DLTK). In this tutorial we will make a simple IDE for Python language.
  
== Introduction ==
+
== Skeleton ==
  
There are three ways in DLTK to contribute language-specific things via extension points:
+
At this step we'll build a skeleton for our IDE.
* language toolkit – parsers, checkers, selection, completion engines, etc.
+
Skeleton includes empty structure parser, content type, declaration of python language to DLTK.
* UI language toolkit – label providers, and so on.
+
And we also create Python Project wizards, and support of python interpreter.
* other extension points, like documentation providers, view filters, etc.
+
  
Lets make simple example IDE and see how these mechanisms could be used. For language lets select Python.
+
[[DLTK IDE Guide:Step 1 Skeleton|Step 1: Skeleton]]
  
You'll need DLTK Core SDK available in your workspace.
+
== Towards an Editor ==
  
== Building required IDE things ==
+
At this step we'll add source code parser to the project and implement source code editor with syntax highlighting.
  
Most non-UI language-specific things are provided to DLTK by a language toolkit. It's most important thing in DLTK and language specific code interaction. Language toolkit could be added to project by using org.eclipse.dltk.language extension point, and requires definition of Eclipse project nature.  
+
As a result our IDE will receive basic source editing and project navigation capabilities.
Primary thing that language toolkit should provide is a source element parser. This parser builds a source module content, so it could be accessible from DLTK (from script explorer for example). Other required thing is a source module validation. Basically it could be just checking for a file name extension, in more complex cases it could be checks for file headers and more.
+
This would include Code Outline, Editor with code folding and syntax highlighting, editor preference pages and Source module structure shown in the Script Explorer.
UI stuff could be extended gradually as required.
+
+
Now for a start we will make example plug-in project, define language toolkit, nature and implement source element parser for a Python language.
+
  
== Creating core plugin==
+
[[DLTK IDE Guide:Step 2. Towards an Editor|Step 2. Towards an Editor]]
  
To create core plugin lets select do New->Plug-in Project: set name org.eclipse.dltk.examples.python.core
+
== Towards an IDE ==
  
[[Image:core_create.png]]
+
The step will show how to add search capabilities (search for Declarations, for References), implement "Open Type" action and "Goto Declaration" feature. We'll add basic code assistance (completion on keywords, and code templates support).
  
After project created, we need to setup dependencies for:
+
[[DLTK IDE Guide:Step 3. Towards an IDE|Step 3. Towards an IDE]]
  
* org.eclipse.dltk.core – core of DLTK.
+
[[Category:DLTK]]
* org.eclipse.core.filesystem – Eclipse file system.
+
* org.eclipse.core.resources – Eclipse Resources, Workspace.
+
 
+
[[Image:dependencies.png]]
+
 
+
After that we could start adding nature, toolkit and source element parser.
+
 
+
== Eclipse Nature ==
+
 
+
In extensions tab we need to create an extension to the org.eclipse.core.resource.natures extension point and define it ID: org.eclipse.dltk.examples.python.core.nature, and create nature class PythonNature which extends ScriptNature class. ScriptNature class has all required stuff for nature management. It also sets up incremental builder for project.
+
 
+
 
+
package org.eclipse.dltk.examples.python.core;
+
+
import org.eclipse.dltk.core.ScriptNature;
+
+
public class PythonNature extends ScriptNature
+
{
+
    public static final String NATURE_ID = PythonPlugin.PLUGIN_ID + ".nature";
+
}
+
 
+
[[Image:nature.png]]
+
 
+
== Creating a simple Language Toolkit ==
+
 
+
In extensions tab we need to create org.eclipse.dltk.core.language extension point and define nature id( we just created it): org.eclipse.dltk.examples.python.nature and language toolkit class, which implements IDLTKLanguageToolkit, lets name it PythonLanguageToolkit. Language toolkit could extend AbstractLanguageToolkit class for some basic implementation. Most methods could be left stubs.
+
 
+
[[Image:Dltk-guide1-lang toolkit.png]]
+
 
+
Also you should specify org.eclipse.dltk.ui.language extension with UI stuff.
+
 
+
== Creating source element parser ==
+
 
+
Lets create class PythonSourceElementParser which implements ISourceElementParser interface. This class used to build model for source modules ''(The required IProblemReporter can be imported from org.eclipse.dltk.compiler.problem.IProblemReporter)''.
+
 
+
<code><pre>
+
public class PythonSourceElementParser implements ISourceElementParser
+
{
+
+
private ISourceElementRequestor fRequestor = null;
+
private IProblemReporter fReporter = null;
+
+
public PythonSourceElementParser( ISourceElementRequestor requestor, IProblemReporter problemReporter ) {
+
    this.fRequestor = requestor;   
+
    this.fReporter = problemReporter;
+
}
+
+
public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {   
+
    String content = new String( contents );
+
    // create a parser, that gives us an AST, for example it could be antlr-based parser
+
    PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
+
    // fetch AST
+
    ModuleDeclaration moduleDeclaration = sourceParser.parse( content );               
+
    return moduleDeclaration;
+
}
+
public void setRequirestor( ISourceElementRequestor requestor ) {
+
    this.fRequestor = requestor;
+
}
+
}
+
</pre></code>
+
We need to specify org.eclipse.dltk.core.sourceElementParsers extension point. Extension point schema almost same to language toolkit schema.
+
 
+
For now we have almost all required for start working. We could Run Eclipse with our code, create project, setup nature, and Script Explorer will show source folders, and so on. Index will look for sources and index all content returned from source element parser( it returns empty for this time ).
+
 
+
== Extending Source Element parser to build correct model ==
+
+
For building model we provide ISourceElementRequestor interface which is passed to the SourceElementParser when building content of source module. It works as visitor. SourceElementParser should call methods to define model elements. For now it could be types, methods, fields, package declarations. As result assuming that such visitor are called PythonSourceElementRequestor, we’ll have following code of the SourceElementParser:
+
 
+
public ModuleDeclaration parseSourceModule( char[] contents, ISourceModuleInfo astCashe ) {   
+
    String content = new String( contents );
+
    // create a parser, that gives us an AST, for example it could be antlr-based parser
+
    PythonSourceParser sourceParser = new PythonSourceParser(this.fReporter);
+
    // fetch AST
+
    ModuleDeclaration moduleDeclaration = sourceParser.parse( content );               
+
    // traverse fetched AST with a visitor, that reports model element
+
    // to given ISourceElementRequestor
+
    PythonSourceElementRequestor requestor = new PythonSourceElementRequestor( this.fRequestor );       
+
    try {
+
        moduleDeclaration.traverse( requestor );
+
    } catch( Exception e ) {
+
        e.printStackTrace( );
+
    }
+
    return moduleDeclaration;
+
}
+
 
+
As you can notice, it is possible to use any kind of parser with DLTK. For our python implementation we use ANTLR parser, for JavaScript RHINO, for Ruby JRuby.
+
 
+
Now, we could run Eclipse Application with "org.eclipse.ui.ide.workbench" specified and see some results.
+
After Eclipse is started, we need to create new empty project and modify .project file.:
+
 
+
We need to add project nature: org.eclipse.dltk.examples.python.core.nature.
+
 
+
'.project' file contents
+
 
+
<?xml version="1.0" encoding="UTF-8"?>
+
<projectDescription>
+
<name>myPython</name>
+
<comment></comment>
+
<projects>
+
</projects>
+
<buildSpec>
+
<buildCommand>
+
<name>org.eclipse.dltk.core.scriptbuilder</name>
+
<arguments>
+
</arguments>
+
</buildCommand>
+
</buildSpec>
+
<natures>
+
<nature>org.eclipse.dltk.examples.python.core.nature</nature>
+
</natures>
+
</projectDescription>
+
 
+
 
+
To view project and module structure we need to ipen ScriptExplorer view:
+
 
+
[[Image:dltk_ide_guide_result.png]]
+
 
+
== Adding basic user stuff ==
+
 
+
To start working with language we need project creation wizard and editor. Outline view also could be useful.
+
 
+
First, let’s create a separate UI plugin, for example org.eclipse.dltk.examples.python.ui. Immediately after that, we add following dependencies (they will be used in future): org.eclipse.dltk.ui, org.eclipse.dltk.core, org.eclipse.dltk.launching, org.eclipse.ui.ide, org.eclipse.core.resources, and plugin that we created above: org.eclipse.dltk.examples.python.
+
 
+
== Project creation wizard ==
+
 
+
For creating projects with associated nature, and configured buildpath, we need to create PythonProjectCreationWizard, which extends NewElementWizard from DLTK Core. You can look into example’s sources for a code, it’s just about a 100 lines of code. Basically you just have to set correct labels there, add pages(ProjectWizardFirstPage and ProjectWizardSecondPage classes from core) and implement following method:
+
 
+
public IModelElement getCreatedElement() {
+
    return DLTKCore.create(fFirstPage.getProjectHandle());
+
}
+
 
+
Also, ProjectWizardSecondPage requires a BuildpathsBlock object. We just extend BuildpathsBlock as following:
+
 
+
<code><pre>
+
public class PythonBuildPathsBlock extends BuildpathsBlock {
+
public PythonBuildPathsBlock(IRunnableContext runnableContext,
+
  IStatusChangeListener context, int pageToShow, boolean useNewPage, 
+
  IWorkbenchPreferenceContainer pageContainer) {
+
    super(runnableContext, context, pageToShow, useNewPage, pageContainer);
+
}
+
+
protected IPreferenceStore getPreferenceStore() {
+
    return PythonUI.getDefault().getPreferenceStore();
+
}
+
+
protected boolean supportZips() {
+
    return true;
+
}
+
}
+
</pre></code>
+
 
+
That’s all. Now we can register our wizard using org.eclipse.ui.newWizards extension point and see results:
+
 
+
[[Image:dltk_ide_guide_wizard.png]]
+
 
+
[[Image:dltk_ide_guide_wizard_result1.png]]
+
 
+
[[Image:dltk_ide_guide_wizard_result2.png]]
+
 
+
== Editor ==
+
 
+
Let’s create class PythonEditor, using ScriptEditor as base. It has abstract methods for creating folding structure provider, outline page, etc. In most cases it’s enough to leave them returning null.
+
After that we just register editor through org.eclipse.editors extension point. Of course, you can use your own editor, but ScriptEditor base has lot’s of things, that are already implemented. Also revealing model elements in editor and other stuff would work only with a ScriptEditor.
+
 
+
== Outline page, folding and more ==
+
 
+
Editor class has methods doCreateOutlinePage() and getFoldingStructureProvider(). Using base classes ScriptOutlinePage and AbstractASTFoldingStructureProvider we can very easily fetch outline view and folding, based on AST. Their implementations would be about half-screen of code.
+
 
+
Now we can look at the result:
+
 
+
[[Image:dltk_ide_guide_finish.png]]
+
 
+
== Other free UI ==
+
 
+
You also could add other UI like preferences, dialog pages(search, for ex.) and other.
+
 
+
== Results ==
+
 
+
After doing this things, we have basic IDE with some free features:
+
# Structured source model, with delta management.
+
# Indexed source code. (By default all declarations are indexed). Fast search engine.
+
# Many free UI, like ScriptExplorer, Outline, Search and so on.
+
 
+
 
+
== Example code ==
+
 
+
Example project sources could be downloaded from [http://us.xored.com/~fourdman/dltk-python-example.zip http://us.xored.com/~fourdman/dltk-python-example.zip] or [http://us.xored.com/~haiodo/org.eclipse.dltk.examples.python.zip http://us.xored.com/~haiodo/org.eclipse.dltk.examples.python.zip]
+
 
+
'''Requirements:'''<br/>
+
1. Latest DLTK 1.0 integration build required.<br/>
+
2. Eclipse 3.3M6 or later required.<br/>
+

Latest revision as of 06:34, 6 May 2008

Summary

This tutorial describes the set of steps to create an integrated development environment (IDE) for a dynamically-typed language using Eclipse Dynamic Languages Toolkit (DLTK). In this tutorial we will make a simple IDE for Python language.

Skeleton

At this step we'll build a skeleton for our IDE. Skeleton includes empty structure parser, content type, declaration of python language to DLTK. And we also create Python Project wizards, and support of python interpreter.

Step 1: Skeleton

Towards an Editor

At this step we'll add source code parser to the project and implement source code editor with syntax highlighting.

As a result our IDE will receive basic source editing and project navigation capabilities. This would include Code Outline, Editor with code folding and syntax highlighting, editor preference pages and Source module structure shown in the Script Explorer.

Step 2. Towards an Editor

Towards an IDE

The step will show how to add search capabilities (search for Declarations, for References), implement "Open Type" action and "Goto Declaration" feature. We'll add basic code assistance (completion on keywords, and code templates support).

Step 3. Towards an IDE

Copyright © Eclipse Foundation, Inc. All Rights Reserved.