PTP/photran/automated ui tests

Overview

Photran's core components (parser, refactoring engine, etc.) are tested pretty extensively using JUnit, but the UI is still manually tested, and that has led to a least a couple of regressions.  SWTBot seems to provide good automated GUI testing for Eclipse and might be appropriate for automating some UI tests.

The following sections of the Photran User's Guide correspond to features that need automated tests:

• User's Guide, Section 3 - Creating new projects
• User's Guide, Section 5.5.4 - Comment/uncomment
• Advanced Features Guide, Section 1 - Enabling advanced features, setting include paths, setting module paths
• Advanced Features Guide, Section 2 - Advanced editing features (can these be tested automatically?)
• Advanced Features Guide, Section 3.1 - Open Declaration

Considerations

A student who has worked with SWTBot provided the following information:

In short, I do recommend SWTBot, it just takes a little getting used to (then again, what framework doesn't?).

SWTBot's most obvious deficit is its lack of documentation. However, I think its API is designed well enough that it should be pretty intuitive after examining some source code for already-written tests.

A roadblock I ran into early on is figuring out the standard terms for each widget (e.g. a "Shell" is a dialog box).

Also, most of the widgets can be accessed by their labels (in the case of "Shells", the text on their title bar), but some widgets do not have such labels. You must access them by their "index", which needs to be found by trial and error and can be hit or miss.

For example, the only way to access the text on the Text widget for the number of failures in the JUnit View is to do something like...

  bot.viewByTitle("JUnit").setFocus();
  String numFailures = bot.text(2).getText();

Another problem I had was not being able to access Shells until they were open and SWTBot not waiting for Shells to close before continuing execution. This could lead to timing issues and code being executed before the workspace is ready. I got around this by writing helper functions such as waitTillShellIsOpen(String shellName) and waitTillShellIsClosed(String shellName).

Then again, any of my issues could have been already addressed in SWTBot, it's just hard to know because the documentation is so sparse.

In any case, with some patience, SWTBot is pretty usable and, in my opinion, is mature enough to be used.

Functionality Without Automated Tests

Not all of this can be tested with SWTBot, but some of it can.

User's Guide sections:

1.3 Installation 3 Starting a New Project 5.1 Configuring Source Forms 5.2 Fixed Form Line Width 5.5.4 Comment/Uncomment 5.5.7.1 Horizontal Rules vs. Folding 5.5.7.2 Setting Colors for Syntax Highlighting 6.3 Error Parsers

Advanced Features sections:

1.1 Enabling Advanced Features and Setting Paths 2 Advanced Editing Features 3 Search and Navigation