Difference between revisions of "PTP/designs/Resource Manager Configuration"

Latest revision as of 07:50, 5 February 2014

PTP 6.0.0 Changes

A number of new features and changes have been included in the 6.0.0 release.

The buttonId attribute on widgets is now global. In the previous version button IDs were local to each tab. This allows a button on one tab to change the state of a control on a different tab;
The control-state element now supports obtaining state by comparing an attribute value to a string;
The saveValueTo attribute on widgets has been renamed to simply attribute. This is to reflect the fact that an widget can read from and save to an attribute.

PTP 5.0.1 Changes

Aside from some significant bug fixes, there are a number of new features with the 5.0.1 release.

The widget component has been split up into three components: widget, button-group, and browse;
action push-button functionality has been added;
control-state elements have been added to all UI control descriptors;
The launch tab import controller type has been modified (largely for maintaining unformity of implementation) to subclass the dynamic controller;
Ability to inspect invisible discovered (excluded) properties and attributes via a "View" button (see note) has been added;
Preferences have been added to help with the debugging of stream tokenizers;
Resource Manager "environment" handling has been totally rewritten in order to support the exclusion of properties associated with invisible or disabled widgets, and to allow for the restoration of their values when re-enabled.

In addition, some tweaking of the SWT "knobs" was necessary to get the XML to reflect more closely the behavior of the Java classes (especially in terms of defaults).

Introduction

The JAXB Resource Manager plug-ins allow you to launch and monitor applications on local or remote resources using resource managers which are configured from an XML file via JAXB ( javax.xml.bind ) technology.

There are two main motivations for providing this class of resource managers:

To allow for maximum adaptability. Often job schedulers (PBS, LSF, LoadLeveler, etc.) or interactive runtime systems (OpenMPI, PE, SLURM, etc.) are set up by system administrators in special or non-standard ways which make it difficult to use a generic tool. The configuration file allows a user or community of users to fit the resource manager to a class of systems, to a single host, or even to special application usage.
Building the resource manager and its UI presentation from an XML configuration means that in most cases no special Java coding is necessary. Users should be able to accommodate new systems, at least on the client-end, without writing and loading additional Eclipse plugins. (The only qualification here is that the monitoring component also support that type of scheduler or runtime; see the following paragraphs.)

Additional considerations in designing a generically configurable resource manager were to partition the client functionality so as to eliminate the need for special server-side proxies and to scale more successfully in the updating of job and resource information.

To this end, JAXB resource managers now consist of two components, a "control", which governs the configuration, launch and cancellation of individual jobs entirely from the client end, and a "monitor", which displays job status as well as global information about the HPC resource. In most cases, the monitor will be a pre-built type provided by the PTP distribution, implemented using LLview. Since LLview already supports a good number of the standard scheduler types, adding a new resource manager type will normally entail only the specific configuration of its control part. The default mode of usage is that the client stages the necessary LLview components (mostly Perl scripts) automatically, but in a future release, the monitor will also be capable of connecting directly to a system-wide (web-based) deployment.

The following is a guide to the resource manager XML definition. Those interested only in using the JAXB resource managers already provided with the PTP distribution should consult the User pages under the relevant scheduler (currently only the PBS resource managers are JAXB-configurable; please see the PBS Res Mgr help).

Configuring/Customizing the Resource Manager

A simple description of creating a custom configuration for a resource manager based on an existing configuration is described on the PTP FAQ page item for customizing a resource manager

We provide here an introductory tutorial slide set demonstrating some simple additions to a pre-existing resource manager definition file.

Tutorial Slides - JAXBDemo.pdf

Example .xml definitions, including the demo-example.xml used in this tutorial, are available here: Demo XML (PTP 5.0.0); further examples which work with the upcoming 5.0.1 release: Demo XML (PTP 5.0.1).

Preferences

The preference options are aids for debugging configuration changes. The first has to do with altering the default behavior of the resource manager, which is to cache its XML definition file; this can be overridden to reread the file every time you start the Resource Manager. This way, if you make changes, they will automatically be picked up without having to reboot the application or recreate the Resource Manager.

The other options are useful if you are writing a new stream tokenizer for a command; checking all four options usually produces quite a bit of output, so in that case it may be advisable to log the results to a file rather than stdout (at any rate the latter would be available only if PTP were running as an application within the Eclipse IDE platform, which is not the case for those modifying the XML simply from within the PTP package installation workspace).

The JAXB Configurable Resource Manager XML Schema

The JAXB Resource Manager is model-driven ; this means that its functioning and appearance are determined by a set of definitions provided via an XML file. What follows is a detailed explanation of the XML schema (XSD) governing the resource manager XML definition.

The `resource-manager-data` Element

The top-level of the definition tree is contained within the resource-manager-data element. It consists of three elements: site-data, control-data and monitor-data. In addition, a resource manager should be given a name using the name attribute, which sufficiently distinguishes it from others of a similar type; e.g., pbs-torque-v_2.3.7_abe is specific to an installation on the host abe, ll-v_4.0 suits all installations of LoadLeveler version 4, etc.

The site-data element provides an optional place to set fixed or default remote site information. The connection strings are URIs which are specific to the PTP RemoteServices definitions. The scheme for these URIs will usually name the specific remote service (e.g, rse: or remotetools: ; local is simply file: ). The host name and port given here will appear as defaults in the resource manager selection wizard when you create a new connection.

The control-data element is used to define the control actions of the resource manager. The top-level control elements include properties and attributes, files to be staged, job script to be generated (if any), commands specific to the resource manager, and the layout of the Launch Tab.

The monitor-data element is used to define the monitoring actions of the resource manager. The top-level monitor elements include properties, drivers for specifying paths to remote commands, and as element for specifying new monitoring workflows.

The `control-data` Element

The majority of the XML definition is given over to the set-up of the resource manager control. One can think of this section as having four subdivisions:

Configuration Variable Definitions (the Environment)
Files and Scripts
External Commands and their Stream Parsers
UI Configuration (Launch Tab)

We will look at these each in turn.

Resource Manager Environment

The resource manager implementation constructs a variable map from the defined properties and attributes which serves as the resource manager "environment". Some elements directly reference these by name; a text/string representation of property or attribute fields can be obtained via the Eclipse variable resolver syntax, with the namespace prefix set to ptp_rm: ${ptp_rm:name#fieldName} , e.g., ${ptp_rm:queues#value} (see further below on the specific fields for properties and attributes).

All properties and attributes defined in the configuration are included in the map. The following hard-coded properties are also added at runtime:

os.name

Operating system name of the (remote) connection. For example, given results from the "uname" command:

Linux
AIX
Mac OS X - if results equal "Darwin" then results from "sw_vers -productName"
everything else - results from "uname" command

os.version

Operating system version of the (remote) connection. For example:

For Linux - results from "uname -r" such as "2.6.32-279.2.1.el6.x86_64"
For AIX - results from "oslevel" such as "7.1.0.0"
For Mac OS X - results from "sw_vers -productVersion" such as "10.8.3"
For everything else - "unknown"

os.arch

Machine architecture of the (remote) connection. For example:

For Linux - results from "uname -m" such as "x86_64"
For AIX - if results from "uname -p" equals "powerpc"
- then if "prtconf -k" contains "64-bit" then "ppc64" else "ppc"
- else the result from "uname -p"
For Mac OS X - if results from "uname -m" equals "i386"
- then if results from "sysctl -n hw.optional.x86_64" equals "1" then "x86_64" else the results from "uname -m"
- else the results from "uname -m"
For everything else - "unknown"

file.separator

File separator character of the (remote) connection. Hardcoded "/" (forward slash).

path.separator

Path separator character of the (remote) connection. Hardcoded ":" (colon).

line.separator

Line separator character of the (remote) connection. Hardcoded "\n" (new-line).

control.user.name

User name of the (remote) connection used by the resource manager.

control.address

Address (hostname) of the resource manager connection.

control.working.dir

Working directory of the resource manager connection. Usually the user's home directory.

executablePath

The path of the executable to be submitted or launched by the resource manager. This path is only valid on the target system.

executableDir

The directory containing the executable on the target system. Only valid on the target system.

progArgs

The arguments to be passed to the executable prior to launch.

directory

The working directory specified in the launch configuration Arguments tab.

In addition, there are a number of special properties that are required so that stdout and stderr from a scheduled job can to be delivered to the client. These properties should be included in the resource manager property set.

control.queue.name: Should be linked to an attribute or property containing the destination queue.
stdout_remote_path: Can be linked to an attribute or property containing a path, otherwise a default path will be used.
stderr_remote_path: Can be linked to an attribute or property containing a path, otherwise a default path will be used.

These properties may be linked (see below) to other attributes to furnish the resource manager with an implementation-independent variable for the queue and for any remote output files produced in connection with the job submission.

The `property` and `attribute` Elements

A property is any variable necessary for the functioning of the resource manager. Properties often (but not necessarily) are not visible. The value for properties can be any primitive type, or lists or maps of strings.

The distinction between attribute and property is not hard and fast, but the intention is to provide a way of differentiating between external, system-determined variables (attributes) and variables only required by the PTP implementation itself. There are several other elements and fields on the attribute, e.g., for type-specific defaults (min/max, choice). Generally, if the variable is exposed to the user, who should also be provided with a description or instructions for use, then it should be defined as an attribute.

The visible attribute is a way of indicating to the resource manager that the user will not be directly changing the value of the property or attribute via the user interface. Certain widgets (such as the attribute viewer ) check this to see if the property or attribute should be included automatically in its list.

BEST PRACTICE: VISIBLE vs INVISIBLE

Always mark properties or attributes which are the targets of attribute (see below) as visible="true"; if this value is false, this variable will always be passed to the configuration/environment of the resource manager; selection/exclusion of sets of properties or attributes pertains only to those marked visible.

status is an optional field for capturing the validity of the attribute at runtime or for a particular installation, if such information is made available by the system. The readOnly property can be defined here, but more often will be enforced directly on the widget which is connected to the property or attribute. min and max give the boundaries for integer values, usually represented by spinner widgets. translateBooleanAs applies to boolean values which should be represented as strings other than "true" and "false"; use a comma-delimited pair to indicate the string equivalent for the boolean; e.g., "YES,NO" would mean true yields the first string and false the second.

description is meant to be a brief one-sentence definition; most of the time detailed information will go into the tooltip element. choice is a predetermined, fixed set of values to choose from; these are comma-delimited and will appear as the items of a combo-box. items-from instead links the combo items (choice) to another property or attribute value (whose type must be java.util.Collection ).

Property and Attribute Values

The untyped value element on properties and attributes is for internal use only; to give a predefined (primitive) value, use the default element along with the type attribute.

link-value-to works as follows: the property or attribute gets its value from the property or attribute linked to, unless that property or attribute has an undefined (empty) value; in the latter case, any default defined on the present property or attribute will be used. This feature is primarily intended for defining dependent properties or attributes whose values are not directly modified by the user.

Attributes also carry optional validators which allow you to enforce syntactic rules, numerical ranges (each with one or both boundaries; these are OR'd) or file-system semantics on the value when the user enters text.

The `regex` and `file-match` Elements

The validator makes use of the regex type (also used in the parser or tokenizer; see below), along with a special file-match type. A regex can be used to match by setting the expression field; to split the segment into tokens on the basis of the expression, set the split field to true (this applies only to the match type discussed below). The expression should follow the grammar in java.util.regex ; the flags are those made available in the java.util.regex.Pattern class (consult the Javadoc for explanations):

CASE_INSENSITIVE
MULTILINE
DOTALL
UNICODE_CASE
CANON_EQ
LITERAL
COMMENTS
UNIX_LINES

These can be OR'd in the usual manner. NOTE: when using the regex on the match type, the expression can contain variable references to be resolved in the environment.

file-match exports the attributes pertaining to org.eclipse.core.filesystem.IFileInfo ; efsAttributes is an OR'd string of the EFS constants:

ATTRIBUTE_READ_ONLY
ATTRIBUTE_IMMUTABLE
ATTRIBUTE_OWNER_READ
ATTRIBUTE_OWNER_WRITE
ATTRIBUTE_OWNER_EXECUTE
ATTRIBUTE_GROUP_READ
ATTRIBUTE_GROUP_WRITE
ATTRIBUTE_GROUP_EXECUTE
ATTRIBUTE_OTHER_READ
ATTRIBUTE_OTHER_WRITE
ATTRIBUTE_OTHER_EXECUTE
ATTRIBUTE_EXECUTABLE
ATTRIBUTE_ARCHIVE
ATTRIBUTE_HIDDEN
ATTRIBUTE_SYMLINK
ATTRIBUTE_LINK_TARGET

The lastModified... fields require the format yyyy/MM/dd HH:mm:ss .

The `managed-file` Element

By "managed file" is meant a local file required by the executable but which may not be present on the host on which the job will run; hence, these files may either be external or may actually be generated from the resource manager environment in conjunction with the job submission, but in any case need to be copied to that host just prior to it. The script file used in connection with scheduler (batch) jobs is a special category of the managed-file type, and will be discussed in the next sub-section.

Managed files are added to the definition in groups determined by their shared staging location, which is a path relative to the working directory of the connection. (The batch script is by default staged to .eclipsesettings in the user home directory, since it is by default temporary and deleted after the submit call returns; however, one can use the file-staging-location on the script element to set the path explicitly.) The file itself can be set to be deleted after submission (deleteTargetAfterUse; this is currently not implemented in the general case) and if the file is actually generated by the resource manager, the local temporary copy can be deleted or retained (deleteSourceAfterUse). In the latter case, one can also distinguish multiple copies of the generated file by setting uniqueIdPrefix to true.

The managed file definition allows one of three possibilities as to specifying content.

If it is an external file or one edited perhaps through the workspace, one can use the path element to indicate location; this can be either a hard-coded path or a reference to a property or attribute value.
The contents element is a string which literally represents the file contents. The resolveContents attribute in this case is used to indicate whether to pass this string through the variable resolver, substituting any references to property or attribute values it may contain. However, caution should be exercised here in that this string should not contain "${...}" sequences which do not actually refer to Eclipse variables (such as batch script variables), or the resolution will fail. If resolveContents is set to false, you can still provide this text element with a single reference to a property or attribute, in which case the value of that property or attribute will be used as is, without further dereferencing.
The preferred, and most flexible way to represent the contents of a file to be generated, however, is to use the line type which is discussed below in connection with the script type.

When the submit call is executed, managed files are written out, if so indicated, and then their target paths are determined as follows:

If the path element is used, the target path becomes staging directory / name of pre-existing file;
If the content or line elements are used, the target path becomes staging directory/ [uniqueIdPrefix]managed-file-name);
A property is placed in the environment whose name is the managed-file name and whose value is this target path.

The `script` Element

A resource manager definition for a scheduler system such as PBS or LoadLeveler can be instrumented to work without a batch script (by setting command-line flags or passing all the job control information as environment variables, for instance), but most frequently the use of a script allows more flexibility in configuring the job (interactive managers of course have no need of a script).

If the script is specified in the XML definition, its path is automatically added to the list of managed files to be staged to the appropriate directory (by default .eclipsesettings, or whereever indicated by the file-staging-location element), and so there is no need to include a script entry explicitly under the managed-files element. As with the managed-file , deleteAfterSubmit indicates that the script target should not be retained (this is the default behavior); unlike the managed-file , however, the local copy of the generated script is always deleted. A reserved property, managed_file_for_script , should be used to reference the script's path on the target resource; for instance, in the PBS submit command:

   <arg>qsub</arg> <arg>${ptp_rm:managed_file_for_script#value}</arg>

Note: If the import tab (see below) is used to provide an external or workspace edited batch script to the run, nothing extra need be done in the definition XML, as the presence of any external path for the script is handled under the covers.

insertEnvironmentAfter is a line number indicating where in the script to add any extra environment variables set through the Environment tab provided as part of the Run Configuration wizard. This way the user has control over whether these should overwrite previously defined environment variable values.

The `line` Element

The line element was mentioned above in connection with managed file content; it is essentially a wrapper around a series of whitespace separated arg elements, all of which are placed on a single line (that is, the group is terminated by a line separator).

The `arg` Element

The arg element is used for script and managed file content as well as in the definition of commands. Its text element can contain variable references to be resolved against the environment, but also allows for the entire string to be treated as a literal by setting resolve to false (default is true). This is useful inasmuch as it allows for the presence of batch-type variables (e.g., ${HOME}) which should be resolved by the remote shell and not inside the Eclipse client.

The default behavior of the argument resolver is not to write out or include arguments whose value is undefined (either null or zero-length).

The isUndefinedIfMatches attribute affords more nuanced control over whether an argument should be so eliminated. If the argument references property or attribute values, but also has text-literal segments, a regex can be provided to define what an "empty" argument would be in this case. For instance, if an argument flag should not appear when the value it precedes is an empty string, one could write:

 
   <arg isUndefinedIfMatches="-f">-f ${ptp_rm:flag#value}</arg>

For the purposes of matching, trailing whitespace is trimmed from the resolved argument, so there is no need to specify this as part of the regex used to match.

"Just-in-time" resolution of @jobId and `managed-file` paths

@jobId is a special property name designating the runtime id for a job instance. In the lifecycle of the run/launch (submit) call, this value begins as an internally generated unique id which then is swapped for the id returned by the scheduler or runtime system.

The @jobId, along with the target paths for managed-file elements, are not known at configuration time (i.e., before the user hits "Run"). While the former is made visible to the parsers and the returned status object of the submit command, neither is in the scope of (available for reference in) other managed files or the script element, because these latter elements are generated just prior to the actual submission.

If the script needs to refer to the @jobId, it must do so via the variable made available by the particular scheduler it is written for. An example of how to reference the target path of a managed-file inside the script is included in the tutorial slides ; this essentially involves defining a variable in the submission command's environment, with a reference to the managed-file path property as its value, and then using this environment variable inside the script .

The `command` Element

Commands are system calls, either to a local or remote OS, depending on the connection defined for the resource manager and denotes a [UNIX-type] system call made on the resource manager's remote (target) connection (PTP does not generally support execution on Windows systems). The command is always configured and executed through a Java API (the "process builder") which bottoms out in a "sh -c" command.

The start-up-command and shut-down-command are arbitrary commands to be run (serially in order) when the resource manager is started or stopped.

The submit commands are those used to launch jobs. Currently a configuration may have only either batch or an interactive commands. Normal (run) mode is specified using the submit-batch or submit-interactive commands. Debug mode is specified using the -debug versions of these commands. In the future we may allow all batch and interactive commands to coexist in a single configuration.

The get-job-status is a user-initiated (on-demand) request to refresh the status information for a submission. Normal (polled) updates, on the other hand, are the responsibility of the monitor component. The status command nevertheless needs to be implemented in most cases, as it will be called internally just after submission.

The terminate-job command is used to terminate a previously launched batch or interactive job. Note: if the submission type is interactive, the terminate-job command usually does not need to be implemented, as the process termination will be handled internally. However, in some cases (such as PBS -I) which require the interactive job to run as a pseudo-terminal, one may need this command in order to force its termination externally.

The remaining -job commands are operations which can be executed on jobs submitted to job schedulers (batch-systems) and do not apply to resource managers which connect to interactive runtime-systems such as OpenMPI or PE.

The button-action is an arbitrary command associated with a button exposed through the Launch Tab (see further below).

There are a number of attributes that can be specified on the command-type element:

directory (defaults to the "home" or working directory of the remote control connection, i.e. the control.working.dir property)

Specifies the location on the remote systemto where the command will be executed

redirectStderr (default false)

Merge both the output and error streams onto stdout

streamBufferLimit (not currently used)

Used to specify the buffer sizes on the stream readers

replaceEnvironment (default false)

Specifies if the environment set on the command should entirely replace the shell environment (the default behavior is to append the command environment)

ignoreExitStatus (default false)

Specifies that no error should be thrown in the case of non-zero exit of the command

flags

An OR'd string of three possible values:

NONE (default)
ALLOCATE_PTY (allocates a pseudo-terminal)
FORWARD_X11

waitForId (default false)

Indicates to the resource manager that the output stream for the command is being parsed for an id which will appear as a property in the environment during the command execution, and that the execution should not return until it sees this id. Very often the submit commands will have these semantics.

keepOpen (default false)

Indicates that the command should be held open for potentially repeated redirection of input (see next paragraph). There can only be one such process open inside the resource manager at a time (these semantics will usually pertain to interactive managers using a pseudo-terminal mode to control job submission). NOTE: if keepOpen is false, then the internal command will not complete until the remote process has finished.

BEST PRACTICE: WAIT-FOR-ID

As previously described, there are two basic types of commands, (uu)id commands which are related to a specific job, and generic (non-uuid) commands. submit- and -job- commands fall into the former category; those in the latter category are usually simple interactive commands, and their output is not redirected to the console, but uuid-commands have several subclasses:

COMMAND TYPE WAIT FOR ID TOKENIZER STATES

batch yes SUBMITTED

interactive no

interactive yes RUNNING

interactive, open no

interactive, open yes (SUBMITTED,) RUNNING

interactive, open, pseudo-terminal no

interactive, open, pseudo-terminal yes (SUBMITTED,) RUNNING

This table shows the various combinations that are feasible between command type and waiting states. Simple interactive commands, whether held open for further input or not, can wait or not wait, but in the case of waiting, they should be provided with a stream tokenizer which recognizes and sets the RUNNING state. Batch jobs, on the other hand, will usually have a tokenizer which recognizes and sets SUBMITTED. In the case of some open-process jobs (e.g., qsub -I for PBS, which also requires a pseudo-terminal to connect remotely), there will usually be a pause before the terminal job is actually scheduled; in order to allow the user to see that the job has been accepted and is pending, the tokenizer needs to set both SUBMITTED and RUNNING states, the latter when the job has actually started.

The `arg` Element

The first argument of the command must be the name/path of the executable only. Its own arguments can then be subsequently specified by an arbitrary number of arg elements. See above for more details. More than one argument can be specified using an arg type.

The `input` Element

The input element is similar to the arg element but is directed to the input stream of the command. If the keepOpen attribute is true, a check will be made to see if an open process already exists (which is also alive), which will then be used; otherwise, the arguments are executed, then the input arguments are given to the process. With an open command/process, the input arguments can be fed repeatedly to the same process. This allows, for instance, for continuous testing of an interactive job in the same interactive session.

The `environment` Type

The environment element can be used to pass strings containing properties or attributes to the target system as environment variables. The name attribute is used to specify the name of the environment variable that will be set on the target system. The value attribute specifies the string value of the environment variable. This string is resolved against the variable map, so can contain property or attribute references.

This element can be used in conjunction with the replaceEnvironment attribute on the command.

Stream Parsers

One can attach parsers (which we also refer to as tokenizers) to the stdout and stderr streams of any command element using the stdout-parser and stderr-parser elements respectively. The redirect-parser element can be used to parse a combined stdout and stderr stream. This allows output to be captured and used to side-effect existing properties or attributes, or to generate new ones on the fly. While the parser is not completely general, it is capable of a wide range of tasks which would typically be required in the handling of output from batch and runtime systems. We here explain the main elements ( target, match, test ) used by the tokenizer; one should further consult the tokenizer examples demonstrating various usage scenarios.

The built-in tokenizer can read the stream in two different ways. If delim is provided, the stream is split using the indicated value. The string should only be one character in length (escaped or non-escaped). Provision is made for the '\r\n' (Windows) two-character delimiter internally; in this case the delimiter should be set to "\r" (however, as already mentioned, PTP does not generally guarantee that system calls will work on Windows). Setting includeDelim means that the delimiter will appear as the last char on the returned stream segment.

The second way to read from the stream is to provide a maxMatchLen size; what this indicates is that whatever substring needs to be found on the stream will not exceed this length. The stream is then read in swatches of maxMatchLen, with the internal buffer set to twice this size, so that each successive read shifts the buffer to the "left" by one length. This guarantees that all such substrings will eventually be matched.

Sometimes a sort of "look-ahead" paradigm is necessary. For instance, one may need to match a segment or segments whose position is defined from the end of the output, but you do not know in advance the actual stream length. In this case, one can opt to read until the end of the stream (all="true"), retaining only the last N buffer-lengths or delimited segments, as indicated by the save field. When the parser reaches the end of the stream, it will then apply the various targets to each saved segment in order.

The applyToAll attribute is discussed further under target . The exit-on element indicates that the tokenizer should quit immediately when it encounters this pattern; exit-after indicates that the tokenizer should quit when it encounters this pattern, but should first apply the current segment to its targets.

The `type` Element

The type element will most commonly not be set, meaning the built-in parser will be used; however, it is possible to implement a custom parser as a contribution to the org.eclipse.ptp.rm.jaxb.core.streamParserTokenizer extension point, in which case this element should be set to its extension id. Note that the extension requires the class to implement org.eclipse.ptp.rm.jaxb.control.internal.IStreamParserTokenizer , which is a Runnable interface with an initialize method which passes in any job id plus the current environment map; the details of such a parser's implementation are not, however, configured from the XML document.

The `target` Element

A tokenizer may be given any number of target elements. The target denotes a particular value (object) currently in, or to be written to, the environment, which will be side-effected on the basis of the result of the tokenization. A target in turn contains match elements and test elements; the former are run as part of the stream processing; the latter are run after the stream processing has been completed. The optional else element is applied only if there are no other tests defined or if none of the defined tests succeed).

The target object is either to be constructed at match time, or it pre-exists in the environment. If constructed, the type field is used to indicate whether the object is a property or an attribute type; otherwise, ref points to the name of the property or attribute in the environment (recall that for the runtime job identifier, @jobId is used).

Note: when new targets are constructed, there is a merge operation at the end of tokenization which attempts to combine objects into a single instance identified by their name attribute. This assumes that such names will be unique and that any other values to be set on the object which are not explicitly bound in some way to that name via the match pattern will appear on the stream before a new name does (see ex. 5 in tokenizer examples). The default behavior of this merge is that it will fail if two objects with the same name but differing values are generated by the parsing. (This excludes add and put operations which create a list or map; in these cases, the two collections or maps will be combined into one. This does NOT work, however, for append actions.) To allow duplicates, set the allowOverwrites to true; in this case, successive duplicates simply replace the preceding object.

The default behavior of the tokenizer read-match sequence is as follows:

read from the stream either a set number of chars or until the delimiter is found;
for each target:
- for each match:
  - if the match is positive, process each of the actions, and go to (1).

Only one qualifying target is processed for any given segment read, and for the given target, the first pattern matched is the one processed for its actions. This is basically the "OR" semantics of normal logic programming; hence the implementer must be careful to arrange the matches inside a target in such a way that the more specific match patterns precede the more general.

Three boolean fields allow you to modify this behavior.

The applyToAll field on the tokenizer-type element means take the unmatched part of the read stream and pass it to the next target, even if there was a previous match; this allows you to capture more than one regex pattern per stream segment (see ex. 6 in tokenizer examples).
The matchAll field on the target-type element means do not try to match an already matched expression until all the others are matched (i.e., a logical AND instead of OR governs the set of matches at successive calls to the target match operation); this allows one to use, for instance, .* repeatedly but set different fields of the object with the resulting match (see ex. 5 in tokenizer examples).
The moveToTop field on the match-type element indicates to the tokenizer that the matched target be promoted to first position in the list of targets. This is useful when there is an ordering which expects types of attributes or properties to be grouped in sequence on the stream (see ex. 4 in tokenizer examples).

When a match is found, the set of action types it contains are all applied.

The `match` Element

Each of these types corresponds to an action to be taken on the indicated field of the target object.

`set`	sets the value of that field
`append`	adds to a string buffer, whose string value will be set on the field
`add`	adds to a list to which the value of that field will be set
`put`	places a key-value pair in a map to which the value of that field will be set
`throw`	throws an exception and (optionally) also sets the value of the field

The actions listed here all have entry-type children, either single ( set, throw ) or potentially multiple. All of these except throw also allow you to force the creation of a new object (forceNewObject) each time it is applied; the new object then replaces the current one for successive actions in the match.

The `entry` Element

This value-abstraction allows one to set key (for maps) and value as literals or references to other properties or attributes to be resolved in the current environment; to reference the matched segment parts one sets keyIndex and valueIndex if the regex was used to split the segment; otherwise, keyGroup and valueGroup refer to the capture group of the regex pattern, with group 0 referring to the entire match.

The `test` Element

As mentioned above, the test-type elements are all run after the tokenization has reached the end of the stream. This class of actions is useful for setting values based on other values produced during tokenization. A test is one or more comparison operations plus a set of actions to apply to the target fields in the case of either success or failure (the "else" element); see ex. 3 or the "get-job-status" example in tokenizer examples.

The op attribute can be one of the following comparisons:

EQ	:	equals
LT	:	less than
LE	:	less than or equal to
GT	:	greater than
GE	:	greater than or equal to

When the operation is set to one of these, it is expected that the two value elements will be used. As usual, these elements can be literals or can contain variables to be resolved into a string type; #FIELD refers to the value of the given field on the current target; the strings will be converted in conformity with the inferred (primitive) type of the comparison. The else element also pertains to comparison tests; the actions listed there will be taken upon failure of the comparison.

The op attribute can also be a logical operator [AND, OR, NOT], in which case the embedded test object should be used; these can be nested to an arbitrary depth, but of course must bottom out in a comparison operation.

Tokenizer Examples

See: Tokenizer Example XML.

Example	Description
1	output is a list of line-separated queue names to be assigned to the known property "available-queues"
2	output is to be searched for its final line which should contain a job id of the form "[digits].[chars]"
3	indeterminate number and order of lines containing parts of attribute definitions, but each line bearing a distinct id (e.g., openMPI attribute discovery)
4	indeterminate number of definitions, but grouped by caption; use of moveToTop to promote the target to the top of the list when the caption appears
5	similar to 4, but without delimiter (implicit ordering)
6	similar to 4, but with indeterminate type order and using buffer + DOTALL \| UNIX_LINES
7	indeterminate number of property definitions, but on single line
8	looking for values interspersed in the stream but which will not exceed 32 chars
9	successive names/values, in order, but staggered
10	forced merge
11	exit-on
12	exit-after
13	get-job-status (use of tests)

The `pre-launch-cmd` and `post-launch-cmd` Elements

It is sometimes necessary to execute commands just prior to and just after submitting the job. This is particularly the case for -debug commands, where the debugger needs cleanup and/or other types of actions to launch correctly.

The pre-launch-cmd and pre-launch-cmdelements only allow simple command execution. Stream parsers and other command elements are not available.

Available attributes include:

exec: Specifies the command to be launched along with any arguments. Property and attribute substitution is performed on this string.
directory: Specifies the directory in which the command will be executed. If not supplied, the main command directory will be used.
ignoreExitStatus: Allows the exit status of the command to be ignored. If not set, and the command exits with a non-zero exit status, the entire submission will be aborted.
wait: Specifies that the submission should not proceed until the command has completed. Should only be used if the command completes in a reasonable time.

The `launch-tab` Element

The UI component which needs to be configured in conjunction with the resource manager control is the Launch Tab, which is actually the main area of the Resources Tab, one of the six tabs comprising the Run Configuration wizard or the seven tabs comprising the Debug Configuration wizard. The purpose of this component is to allow the user to set up the job by providing values relating to resource requirements or to the properties of the job in general. In other words, it is in this component that the user can set the values on the attribute elements making up the configurable environment as defined in the XML.

Specifying the composition and layout of this component can be tricky and tedious because it deals with widgets defined by the Eclipse JFace library and with their underlying implementation in SWT . We intend in the near future to provide some standard "templates" for typical widget groups that can be cut and pasted into the XML definition. Some of this can already be achieved by importing into the workspace a provided definition (such as for the PBS resource manager) and modifying it or borrowing from it. This is, in fact, the approach demostrated in the tutorial.

Throughout the following, it may generally be assumed that fields such as style, foreground, background, etc., take string equivalents of the corresponding SWT constants (e.g., "SWT.LEFT", "SWT.VERTICAL", "SWT.NONE") which can be OR'd (using "|") wherever they would be in Java code. We will not exhaustively specify here which constants pertain to which widgets, because in most cases we have tried to reflect very closely the names of the SWT object types and fields, and this information can be found by consulting the Javadocs relative to the widgets in question.

As seen from this top-level specification, the Launch Tab can have any number of dynamic tab-controllers, "dynamic" meaning configurable from the XML, which will discuss in more detail in the following paragraphs.

There is also a fixed import controller type (implemented as an extension/subclass of the dynamic controller, in order to provide uniform handling of the environment); this component allows you to browse the workspace to set a custom script as the one to run. Its main text area widget is read-only; any editing of the script must be done through the workspace editor. The browse is local only, so remote scripts should first be imported into a local project in the workspace (using the remote systems import wizard), and then captured by this controller's browse button.

The configurable part of the import tab is an optional attribute viewer for selecting the variables from the environment which you wish to pass to the launch command along with the script. The values set here should correspond to whatever the script contains; they are set for the benefit of the internal consistency of the PTP client, and do not override those hard-coded in script. For an example, see the Import PBS Script tab.

The `tab-controller` Element

The dynamic controllers belong to the tab-controller-type , which contains an arbitrary number of tab-folder, composite, widget, browse, button-group, action, or viewer elements; as will be seen shortly, the first two, which are "containers", are potentially recursive elements, so that one can nest widgets in the usual manner. For all intents and purposes, the tab-controller-type is the same as a composite , but without the group attribute and with the extra field for controlling shared widgets.

The includeWidgetValuesFrom field is important. By default, it is set to empty, meaning that the subset of the total resource manager environment represented by the controller's widgets is all that will be passed to the actual job configuration when the job is launched from this tab. This behavior can be changed by giving a comma-delimited list of the title values for other controllers. In this case, the configuration for the job, when the "Run" button is pressed with this controller being the visible one, will also include all the properties or attributes valid for the listed but currently invisible controllers.

Property/Attribute Names vs Values

While this field controls the inclusion or exclusion of attribute and property names in the configuration, their values are always shared across the controllers (there is only one environment); thus if a variable value is changed on one controller, and this variable is also exposed on another, the value for that variable will also change on the other controller when it becomes visible; similarly, if the user switches off a controller to another having only a subset of the first controller's variables, the variables not represented on the second controller will become undefined or return to their defaults.

Note that the union of valid names is not transitively closed; that is, if Tab1 has includeWidgetValuesFrom="Tab2", and Tab2 similarly has includeWidgetValuesFrom="Tab3", Tab1 will get only the valid values local to Tab2. Note also, that if there are names which intersect with strictly local widget targets, the local settings (visibility, selection, enablement) still determine whether they belong in the set of valid property/attribute names.

The showViewConfig and showViewExcluded fields allow control over whether the "View Configuration" and "View Excluded" buttons appear at the bottom of the tab (the "View Script" button appears if the configuration contains a script element). showViewConfig is true by default; this button brings up a scrolling dialog which allows you to inspect the current configuration (that is, the one relevant to the visible controller). showViewExcluded is by default false, and should only be set if the resource manager processes attributes or properties discovered by a command at run time; this dialog allows you to see which ones have been marked invisible; unlike invisible predefined properties or attributes, which are always included, invisible discovered properties or attributes are always excluded from the configuration and thus do not persist across resource manager sessions.

The `tab-folders` and `tab-items` Elements

The tab-folder is a container which looks like a file-folder, with tabs running across the top or bottom (style includes SWT.TOP, SWT.BOTTOM). Each tab label corresponds to a tab-item which provides the actual visible content. The tooltip pertains to the entire folder.

The tab-item is another all-purpose "container" in which all widget types can be embedded, but distinguished from the composite in that it can only be the child of a tab-folder and that it can have a tooltip (which appears on the tab).

As with composite (see below), the tab folder and tab item take layout and layout-data elements; these closely reflect their SWT objects and types; for further explanation we refer the reader once again to the relevant Javadocs.

The `layout` Element

The `layout-data` Element

The `composite` Element

The main container, of course, is the composite; its sub-type, the group (group="true"), allows for decoration and a title, while the simple composite acts as a widget holder and has no visible characteristics, though it does have a layout. As is evident, composites can be contained in composites and tab-items, and tab-folders can be contained in composites.

The `control-state` Element

It should be evident that all the XML elements corresponding to UI SWT control types have been given control-state children.

The purpose of these elements is to provide an optional way of setting the enabled and/or visible states of the control on the basis of the state of button (checkbox/radio) selection. One can define two such elements per control, using one of the show/hide or enable/disable pairs. Each of these is associated with a control-state-rule:

The basic rule is defined by its attributes:

the id of the button control to check;
whether its selected state should be true or false.

Rules can be nested to an arbitrary depth to form clauses governed by the logical operators.

A SelectionListener is created for this target which subscribes to all the controls in the rule; when it receives an event, it evaluates the rule, and if the result is true, takes the associated action on this target.

When one wishes to wire two or more such controls in this way, the "trigger" button control must have an id unique to the resource manager configuration. This will be assigned via the buttonId element on the widget corresponding to the button (true also for the buttons nested in a button-group ).

Here is an example which sets the enabled and visible properties of both a text field and a related checkbox:

               <widget type="checkbox" buttonId="002" title="Host list:" attribute="checked002">
                  <layout-data>
                     <grid-data horizontalAlign="SWT.LEFT" verticalAlign="SWT.TOP"/>
                  </layout-data>
                  <control-state>
                     <show-if button="001" selected="true"/>
                  </control-state>
               </widget>
               <widget type="text" style="SWT.V_SCROLL | SWT.BORDER" attribute="host_list">
                  <layout-data>
                     <grid-data horizontalAlign="SWT.FILL" verticalAlign="SWT.FILL" horizontalSpan="2" grabExcessHorizontal="true"
                        grabExcessVertical="true"/>
                  </layout-data>
                  <control-state>
                     <show-if button="001" selected="true"/>
                     <enable-if>
                        <and>
                           <rule button="001" selected="true"/>
                           <rule button="002" selected="true"/>
                        </and>
                     </enable-if>
                  </control-state>
               </widget>

Note also that this implicit dependency graph is checked for cycles; the building of the UI tab will be aborted if such a cycle is discovered.

BEST PRACTICE: CONTROL STATE

An invisible property should be assigned to each checkbox which controls the state of other widgets (e.g., checked002 in the example above); this will allow you to restore the state of the tab correctly when it is rebuilt or re-initialized.

The `widget` Element

We now come to the five non-container widget elements with functional characteristics. The first of these is the widget , which comprises a number of simple widget types.

The type of the widget can be one of the following:

`label`	fixed text for display
`text`	area for entering/editing text
`checkbox`	corresponds to a boolean choice
`radiobutton`	usually used in connection with `button-group`
`spinner`	provides a continuous range of integer values
`combo`	select from a list of items

Not all attributes and elements of the widget descriptor are applicable to all types.

title	used with `checkbox` and `radiobutton` widgets for the button label.
style	carries the `SWT` style constants appropriate to the widget type.
readOnly	applies to `text`, `spinner` and `combo` widgets; the text area in this case is not editable.
attribute	applies to all widgets except `label`; the name of the property or attribute whose value should be set on the basis of the widget's "selected" value or text Note: use of this attribute is mutually exclusive with the `fixed-` and `dynamic-text` elements.
`items-from`	applies to the `combo` widget; indicates a property or attribute value (of type `java.util.Collection`) to use to populate the items. Note: if a combo widget saves its value to an attribute and the present element on the widget descriptor is not set, the attribute will be checked for non-empty `choice` or `items-from` values, which then will be used to populate the combo items; the `items-from` field on the widget is for added flexibility (for instance, linking up a target property with another property providing the list of choices).
`fixed-text`	applies to the `label` and (read-only) `text` widgets; text is resolved once and remains constant thereafter.
`dynamic-text`	applies only to (read-only) `text` widgets; text is resolved each time there is an update of a value from the Launch Tab.

Widget foreground, background (color) and font are in principle configurable, though the font setting may not always work. The tooltip will appear on the widget whereever it usually does.

The `button-group` Element

The button group provides an explicit selection of a single value based on a mutually exclusive set of button labels. Just as with the widget , one uses attribute to indicate what property or attribute value to set. The button elements, which can be arranged horizontally or vertically using the style attribute (SWT.HORIZONTAL, SWT.VERTICAL), will be widget elements whose type is either checkbox or radiobutton (if a different widget type is given, an error will occur); the attribute attribute on the individual button will be ignored. Radio buttons are "sticky" in the sense that once they are selected, they cannot be "unchecked" until a different button in the group is selected. It is thus advisable always to provide a default value for the property or attribute associated with the group value if radio buttons are used. If check boxes are used, they can be individually checked on and off (one at a time), thus allowing for the value of the group to be potentially empty.

Below is an example of a button group using radio buttons, disposed vertically.

The `browse` Element

A special widget-pair -- a text widget followed by a push button -- allowing the user to browse directories or files from either the local or remote file system. Aside from the fields shared with the widget type, we note the following:

directory	limit the `browse` widget to directories.
uri	have the `browse` widget set a URI as the text (default is false, in which case the text is an absolute path).
localOnly	limit the `browse` widget to the local file system (otherwise, the choice of local or remote is enabled).

layout and control-state are specified as belonging to either the text or the button. Style applies only to the text widget.

The `action-button` Element

A push button which is associated with a button-action command. The button-action specifies the name of a command ( action ) defined in the XML; this can be one of the start-up or shut-down commands, or a special button-action command (note that it cannot, however, be a (uu)id command, as the Launch Tab is not coupled to any submitted jobs); the button-action-type also indicates whether the variable affected by the command should be cleared prior to the command execution, and whether the launch tab should be refreshed (default is true) afterward.

The `attribute-viewer` Element

The last of the widget types, the attribute-viewer , displays visible properties and attributes in a table or tree viewer, with the value columns as editable widgets. The type of cell editor is determined implicitly here by the type of the property or attribute; the presence of a combo editor is determined by an attribute whose choice or items-from element is defined. (This is the only viable possibility for a combo in the table, as there would also need to be a target to save the selected item to, viz., the value of this same attribute.)

Each entry in the viewer is provided with a checkbox to indicate which attributes have been chosen (and thus marked valid). A checkbox is also placed above the viewer to allow for filtering the unchecked items. The viewer outputs its values to the environment. The name of the viewer is also stored against a string of all selected values, constructed from a template (further discussion at the end of this section).

The viewer type can be either "table" or "tree"; in the case of the tree, the type, default, status and description fields of the attribute or property are displayed as collapsible child rows hanging off the main name entry. Either table or tree can have up to six columns whose names/types are based on the following property and attribute fields:

Name

Value

Type

Status

Default

Description

These can appear in any order. There is a slight difference in how these columns are treated, depending upon whether tree or table is chosen as type. For table type, the value of the field simply appears in the corresponding column:

In the tree type, the name field displays all fields except value as children; value always appears as a separate column on the same row as the name (it contains the cell editor). If the viewer also has the column corresponding to the child field, that field value appears in that column;

Otherwise, the child fields display their value in the value column along with the actual value field:

Tooltips for the column appear when hovering over the column header. (A caveat on the foreground and background settings for this type: they are exposed here but may not work as expected: these are currently applied to the column through the cell label provider; foreground does not change the font color but applies the color to the entire cell.)

sort is done on the name field/column. This is a toggle sort, such that alternate clicks reverse the sort direction. headerVisible and linesVisible control the look of the table. tooltipEnabled enables column-viewer label provider tooltip support. The tooltip defined for an attribute will appear on the row of the table or the parent node of the tree. The attribute description will also appear as a tooltip on the description columns if they exist.

Control over the actual contents of the viewer is achieved through the viewer-items-type ; one can here explicitly list properties or attributes for inclusion using the include element, or one can set one or both of the all attributes; in the latter case, attributes and properties can be excluded from the set using exclude . The initialCheckAll field, which defaults to true, indicates that if the attribute or property is new or undefined from a previous session, it will appear as checked; turn this off if you want new attributes or properties to appear at first as unchecked.

A property bearing the name given to this widget is written to the environment at each update/refresh; the value of the property is a string formed from the selected (checked) rows of the table using the template-type (the value element). The pattern can be any literal string embedding @name and @value tags as placeholders for the row's name and value; for instance, '@name="@value"' would generate a string of name-value assignments. The default separator between patterned segments is a single space.

Below is a fuller example of the tree version of the attribute viewer. The text area above it displays the current value of its templated pattern string. The child rows reporting status, default, etc., cannot be selected or unselected (only the parent row can). Hovering over the name (here walltime) displays the tooltip.

The `monitor-data` Element

At present, the monitor component has less elements to define; this may change in future releases, when certain hard-coded features of the driver may be handed over to the client for configuration.

This component can be furnished with its own set of properties (currently unused). What is necessary to set at present is only the schedulerType attribute. It specifies the target system's job scheduler. Currently supported batch systems are COBALT_BG, GRIDENGINE, LL, LL_BG, LSF, OPENMPI, PBS, PE, SLURM, SLURM_ALPS, TORQUE and TORQUE_ALPS. These names can be used as attribute values for schedulerType. refreshFrequencyInSeconds defaults to 60, but this can be changed according to the needs of the user. Be aware that too low a setting will probably not work, as the command on an average sized system will take upwards of five seconds to complete (XML is being streamed to the driver and a diff is sent back to the client ).

If the driver element is configured, then the default behavior, which is to stage over the necessary driver scripts to the .eclipsesettings directory in the user's home, is overridden by connecting to a pre-existent installation. This can be specified either using the url , or a combination of the name , path and args elements.

@@ Line 1: / Line 1: @@
-= The JAXB Configurable Resource Manager =
+== PTP 6.0.0 Changes ==
-<br>
+	<p>A number of new
+		features and changes have been included in the 6.0.0 release.</p>
+	<ol>
+		<li>The <code>buttonId</code> attribute on widgets is now global. In the previous version
+                       button IDs were local to each tab. This allows a button on one tab to change the state
+                       of a control on a different tab;</li>
+		<li>The <code>control-state</code> element now supports obtaining state by comparing
+                       an attribute value to a string;</li>
+		<li>The <code>saveValueTo</code> attribute on widgets has been renamed to simply
+                       <code>attribute</code>. This is to reflect the fact that an widget can read from and
+                       save to an attribute.</li>
+	</ol>
+	<hr>
+== PTP 5.0.1 Changes ==
+	<p>Aside from some significant bug fixes, there are a number of new
+		features with the 5.0.1 release.</p>
+	<ol>
+		<li>The <code>widget</code> component has been split up into
+			three components: [[#Widget|<code>widget</code>]],
+			[[#Button Group|<code>button-group</code>]], and
+			[[#Browse|<code>browse</code>]];</li>
+		<li>[[#Action Button|<code>action</code>]] push-button
+			functionality has been added;</li>
+		<li>[[#Control State Wiring|<code>control-state</code>]]
+			elements have been added to all UI control descriptors;</li>
+		<li>The launch tab [[#4. The Launch Tab|<code>import</code>]]
+			controller type has been modified (largely for maintaining unformity
+			of implementation) to subclass the <code>dynamic</code> controller;</li>
+		<li>Ability to inspect invisible discovered (excluded) properties
+			and attributes via a "View" button (see [[#Tab Controller|note]])
+			has been added;</li>
+                <li>[[#Preferences|Preferences]] have been added to
+			help with the debugging of [[#Stream Parsers|stream
+				tokenizers]];</li>
+		<li>Resource Manager "environment" handling has been totally
+			rewritten in order to support the exclusion of properties associated
+			with invisible or disabled widgets, and to allow for the restoration
+			of their values when re-enabled.</li>
+	</ol>
+	<p>In addition, some tweaking of the SWT "knobs" was necessary to
+		get the XML to reflect more closely the behavior of the Java classes
+		(especially in terms of defaults).</p>
+	<hr>
 == Introduction ==
 <p>
-	The JAXB Resource Manager plug-ins allow you to launch and monitor
+   The JAXB Resource Manager plug-ins allow you to launch and monitor
-	applications on local or remote resources using resource managers which
+   applications on local or remote resources using resource managers which
-	are configured from an XML file via JAXB (
+   are configured from an XML file via JAXB (
-	<code>javax.xml.bind</code>
+   <code>javax.xml.bind</code>
-	) technology.<br>
+   ) technology.<br>
 </p>
 <p>There are two main motivations for providing this class of
-	resource managers:</p>
+   resource managers:</p>
 <ol>
-	<li>To allow for maximum configurability. Often job schedulers
+   <li>To allow for maximum adaptability. Often job schedulers
-		(PBS, LSF, LoadLeveler, etc.) or interactive runtime systems (OpenMPI,
+      (PBS, LSF, LoadLeveler, etc.) or interactive runtime systems (OpenMPI,
-		PE, SLURM, etc.) are set up by system administrators in special or
+      PE, SLURM, etc.) are set up by system administrators in special or
-		non-standard ways which make it difficult to use a generic tool. The
+      non-standard ways which make it difficult to use a generic tool. The
-		configuration file allows a user or community of users to fit the
+      configuration file allows a user or community of users to fit the
-		resource manager to a class of systems, to a single host, or even to
+      resource manager to a class of systems, to a single host, or even to
-		special application usage.</li>
+      special application usage.</li>
-	<li>Building the resource manager and its UI presentation from an
+   <li>Building the resource manager and its UI presentation from an
-		XML configuration means that in most cases no special Java coding is
+      XML configuration means that in most cases no special Java coding is
-		necessary. Users should be able to accommodate new systems, at least
+      necessary. Users should be able to accommodate new systems, at least
-		on the client-end, without writing and loading additional Eclipse
+      on the client-end, without writing and loading additional Eclipse
-		plugins. (The only qualification here is that the monitoring component
+      plugins. (The only qualification here is that the monitoring component
-		also support that type of scheduler or runtime; see the following
+      also support that type of scheduler or runtime; see the following
-		paragraphs.)</li>
+      paragraphs.)</li>
 </ol>
 <p>Additional considerations in designing a generically configurable
-	resource manager were to partition the client functionality so as to
+   resource manager were to partition the client functionality so as to
-	eliminate the need for special server-side proxies and to scale more
+   eliminate the need for special server-side proxies and to scale more
-	successfully in the updating of job and resource information.</p>
+   successfully in the updating of job and resource information.</p>
 <p>
-	To this end, JAXB resource managers now consist of two components, a
+   To this end, JAXB resource managers now consist of two components, a
-	"control", which governs the configuration, launch and cancellation of
+   "control", which governs the configuration, launch and cancellation of
-	individual jobs entirely from the client end, and a "monitor", which
+   individual jobs entirely from the client end, and a "monitor", which
-	displays job status as well as global information about the HPC
+   displays job status as well as global information about the HPC
-	resource. In most cases, the monitor will be a pre-built type provided
+   resource. In most cases, the monitor will be a pre-built type provided
-	by the PTP distribution, implemented using
+   by the PTP distribution, implemented using
-	[[http://www2.fz-juelich.de/jsc/llview|LLview]]. Since <b>LLview</b>
+   [http://www2.fz-juelich.de/jsc/llview LLview]. Since <b>LLview</b>
-	already supports a good number of the standard scheduler types, adding
+   already supports a good number of the standard scheduler types, adding
-	a new resource manager type will normally entail only the specific
+   a new resource manager type will normally entail only the specific
-	configuration of its control part. The default mode of usage is that
+   configuration of its control part. The default mode of usage is that
-	the client stages the necessary LLview components (mostly Perl scripts)
+   the client stages the necessary LLview components (mostly Perl scripts)
-	automatically, but in a future release, the monitor will also be
+   automatically, but in a future release, the monitor will also be
-	capable of connecting directly to a system-wide (web-based) deployment.
+   capable of connecting directly to a system-wide (web-based) deployment.
 </p>
 <p>
-	<b>The following is a guide to the resource manager XML definition.
+   <b>The following is a guide to the resource manager XML definition.
-		Those interested only in using the JAXB resource managers already
+      Those interested only in using the JAXB resource managers already
-		provided with the PTP distribution should consult the User pages under
+      provided with the PTP distribution should consult the User pages under
-		the relevant scheduler (currently only the PBS resource managers are
+      the relevant scheduler (currently only the PBS resource managers are
-		JAXB-configurable).</b>
+      JAXB-configurable; please see the [http://help.eclipse.org/indigo/topic/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/toc.html   PBS Res Mgr help]).</b>
 </p>
-<hr>
-== [[JAXBDemo.pdf|Configuring/Customizing the Resource Manager]] ==
+== Configuring/Customizing the Resource Manager  ==
+A simple description of creating a custom configuration for a resource manager based on an existing configuration is described on the [[PTP/FAQ#Q:_How_do_I_customize_the_new_.28JAXB.29_Configurable_Resource_Manager.3F  | PTP FAQ page item for customizing a resource manager]]
-<p>We provide here an introductory tutorial slide set demonstrating
+We provide here an introductory tutorial slide set demonstrating some simple additions to a pre-existing resource manager definition file.
-	some simple additions to a pre-existing resource manager definition
-	file.</p>
-== [[The JAXB Configurable Resource Manager XML Schema]] ==
+[[Media:JAXBDemo.pdf|'''Tutorial Slides - JAXBDemo.pdf''']] <br>
+Example .xml definitions, including the <code> demo-example.xml </code> used in this tutorial, are available here: [[Media:JAXBexamples_5.0.0.zip|'''Demo XML (PTP 5.0.0)''']]; further examples which work with the upcoming 5.0.1 release: [[Media:JAXBExamples_5.0.1.1.zip|'''Demo XML (PTP 5.0.1)''']].
+=== Preferences ===
+       <p>The preference options are aids for debugging configuration
+		changes. The first has to do with altering the default behavior of the
+		resource manager, which is to cache its XML definition file; this can
+		be overridden to reread the file every time you start the Resource
+		Manager. This way, if you make changes, they will automatically be
+		picked up without having to reboot the application or recreate the
+		Resource Manager.
+	</p>
+	<p>The other options are useful if you are writing a new stream
+		tokenizer for a command; checking all four options usually produces
+		quite a bit of output, so in that case it may be advisable to log the
+		results to a file rather than stdout (at any rate the latter would be
+		available only if PTP were running as an application within the
+		Eclipse IDE platform, which is not the case for those modifying the
+		XML simply from within the PTP package installation workspace).</p>
+<br>
+[[Image:00Preferences.png]]
+<br>
+<hr>
+== The JAXB Configurable Resource Manager XML Schema ==
 <p>
-	The JAXB Resource Manager is <b><i>model-driven</i> </b>; this means
+   The JAXB Resource Manager is <b><i>model-driven</i> </b>; this means
-	that its functioning and appearance are determined by a set of
+   that its functioning and appearance are determined by a set of
-	definitions provided via an XML file. What follows is a detailed
+   definitions provided via an XML file. What follows is a detailed
-	explanation of the [[PTP/designs/resource manager xsd| schema]] governing the resource manager XML
+   explanation of the <b>[[PTP/designs/resource_manager_xsd#XSD| XML schema (XSD)]]</b> governing the resource manager XML
-	definition.
+   definition.
 </p>
-=== The Resource Manager Data Type ===
+=== The <code>resource-manager-data</code> Element ===
 <br>
 [[Image:01rm-data.jpeg]]
 <p>
-	The top-level of the definition tree consists of three elements: <b>site-</b>,
+   The top-level of the definition tree is contained within the <b>resource-manager-data</b> element.
-	<b>control-</b> and <b>monitor-data</b>. In addition, a resource
+   It consists of three elements: <b>site-data</b>,
-	manager should be given a name which sufficiently distinguishes it from
+   <b>control-data</b> and <b>monitor-data</b>. In addition, a resource
-	others of a similar type; e.g., <i>pbs-torque-v_2.3.7_abe</i> is
+   manager should be given a name using the <b>name</b> attribute,
-	specific to an installation on the host abe, <i>ll-v_4.0</i> suits all
+   which sufficiently distinguishes it from
-	installations of LoadLeveler version 4, etc.
+   others of a similar type; e.g., <i>pbs-torque-v_2.3.7_abe</i> is
+   specific to an installation on the host abe, <i>ll-v_4.0</i> suits all
+   installations of LoadLeveler version 4, etc.
 </p>
 <p>
-	The <b>site-data</b> element provides an optional place to set fixed or
+   The <b>site-data</b> element provides an optional place to set fixed or
-	default remote site information. The connection strings are URIs which
+   default remote site information. The connection strings are URIs which
-	are specific to the PTP RemoteServices definitions. The scheme for
+   are specific to the PTP RemoteServices definitions. The scheme for
-	these URIs will usually name the specific remote service (e.g,
+   these URIs will usually name the specific remote service (e.g,
-	<code>rse:</code>
+   <code>rse:</code>
-	or
+   or
-	<code>remotetools:</code>
+   <code>remotetools:</code>
-	; local is simply
+   ; local is simply
-	<code>file:</code>
+   <code>file:</code>
-	). The host name and port given here will appear as defaults in the
+   ). The host name and port given here will appear as defaults in the
-	resource manager selection wizard when you create a new connection.
+   resource manager selection wizard when you create a new connection.
 </p>
 <p>
-	The principal section of the schema is devoted to defining the resource
+   The <b>control-data</b> element is used to define the control actions of
-	manager's <b>control</b> part. The top-level control elements include
+   the resource manager. The top-level control elements include
-	properties and attributes, files to be staged, job script to be
+   properties and attributes, files to be staged, job script to be
-	generated (if any), commands specific to the resource manager, and the
+   generated (if any), commands specific to the resource manager, and the
-	layout of the Launch Tab.
+   layout of the Launch Tab.
 </p>
-==== Resource Manager Environment ====
 <p>
-	The resource manager implementation constructs a variable map from the
+   The <b>monitor-data</b> element is used to define the monitoring actions
-	defined properties and attributes which serves as the resource manager
+   of the resource manager. The top-level monitor elements
-	"environment". Some elements directly reference these by name; a
+   include properties, drivers for specifying paths to remote commands,
-	text/string representation of property or attribute fields can be
+   and as element for specifying new monitoring workflows.
-	obtained via the Eclipse variable resolver syntax, with the namespace
-	prefix set to <i>ptp_rm</i>:
-	<code>
-		${ptp_rm:name#<i>fieldName</i>}
-	</code>
-	, e.g.,
-	<code>${ptp_rm:queues#value}</code>
-	(see further [[#1. Property and Attribute Types]] on the specific fields
-	for properties and attributes).
-</p>
-<p>
-	All properties and attributes defined in the configuration are mapped.
-	The following hard-coded properties are also added at runtime: <br>
-	<br>
-	<code>control.user.name</code>
-	<br>
-	<code>control.address</code>
-	<br>
-	<code>control.working.dir</code>
-	<br>
-	<code>executablePath</code>
-	<br>
-	<code>progArgs</code>
-	<br>
-	<code>directory</code>
-	<br> <br> In addition, these properties may be linked (see
-	[[1. Property and Attribute Types]]) to other attributes to furnish the
-	resource manager with an implementation-independent variable for the
-	queue and for any remote output files produced in connection with the
-	job submission. <br> <br>
-	<code>control.queue.name</code>
-	<br>
-	<code>stdout_remote_path</code>
-	<br>
-	<code>stderr_remote_path</code>
-	<br>
 </p>
-==== Resource Manager Command Types ====
+=== The <code>control-data</code> Element ===
-<p>
-	<i>Commands</i> are system calls, either to a local or remote OS,
-	depending on the connection defined for the resource manager. The <b>start-up-</b>
-	and <b>shut-down-command</b>s are arbitrary commands to be run
-	(serially in order) when the resource manager is started or stopped.
-	The <b>submit</b> commands are those used to launch jobs. Currently a
-	configuration may have only a batch or an interactive mode. Thus it may
-	have only two submission modes, a run and a debug, for the given type.
-	In the future we may allow all four to coexist in a single
-	configuration. <b>get-job-status</b> is a user-initiated (on-demand)
-	request to refresh the status information for a submission. Normal
-	(polled) updates, on the other hand, are the responsibility of the
-	<code>monitor</code>
-	component. The status command nevertheless needs to be implemented in
-	most cases, as it will be called internally just after submission. The
-	remaining "-job" commands are operations which can be executed on
-	submitted jobs; with the exception of <i>terminate</i>, the rest of
-	these have to do with schedulers (batch-systems) and do not apply to
-	resource managers which connect to interactive runtime-systems such as
-	OpenMPI or PE. <b>Note</b>: if the submission type is interactive, the
-	<b>terminate-job</b> command usually does not need to be implemented,
-	as the process termination will be handled internally. However, in some
-	cases (such as PBS -I) which require the interactive job to run as a
-	pseudo-terminal, one may need this command in order to force its
-	termination externally. Finally, a <b>button-action</b> is an arbitrary
-	command associated with a button exposed through the
-	<code>Launch Tab</code>
-	(see further [[Widget]]).
-</p>
-=== The Control Data Type ===
 <br>
 [[Image:02control.jpeg]]
 <p>The majority of the XML definition is given over to the set-up of
-	the resource manager control. One can think of this section as having
+   the resource manager control. One can think of this section as having
-	four subdivisions:</p>
+   four subdivisions:</p>
 <ol>
-	<li>Configuration Variable Definitions (the <i>Environment</i>)</li>
+   <li>Configuration Variable Definitions (the <i>Environment</i>)</li>
-	<li>Files and Scripts</li>
+   <li>Files and Scripts</li>
-	<li>External Commands and their Stream Parsers</li>
+   <li>External Commands and their Stream Parsers</li>
-	<li>UI Configuration (Launch Tab)</li>
+   <li>UI Configuration (Launch Tab)</li>
 </ol>
 <p>We will look at these each in turn.</p>
-==== 1. Property and Attribute Types ====
+==== Resource Manager Environment ====
+The resource manager implementation constructs a variable map from the
+defined properties and attributes which serves as the resource manager
+"environment". Some elements directly reference these by name; a
+text/string representation of property or attribute fields can be
+obtained via the Eclipse variable resolver syntax, with the namespace
+prefix set to <i>ptp_rm</i>:
+<code>${ptp_rm:name#<i>fieldName</i>}</code>
+, e.g.,
+<code>${ptp_rm:queues#value}</code>
+(see further [[#1. Property and Attribute Types | below]] on the specific fields
+for properties and attributes).
+All properties and attributes defined in the configuration are included in the map.
+The following hard-coded properties are also added at runtime:
+<dl>
+<dt>os.name
+<dd>Operating system name of the (remote) connection.
+For example, given results from the "uname" command:
+<ul>
+<li>Linux</li>
+<li>AIX</li>
+<li>Mac OS X - if results equal "Darwin" then results from "sw_vers -productName"</li>
+<li>everything else - results from "uname" command</li>
+</ul>
+<dt>os.version
+<dd>Operating system version of the (remote) connection.
+For example:
+<ul>
+<li>For Linux - results from "uname -r" such as "2.6.32-279.2.1.el6.x86_64"</li>
+<li>For AIX - results from "oslevel" such as "7.1.0.0"</li>
+<li>For Mac OS X - results from "sw_vers -productVersion" such as "10.8.3"</li>
+<li>For everything else - "unknown"</li>
+</ul>
+<dt>os.arch
+<dd>Machine architecture of the (remote) connection.
+For example:
+<ul>
+<li>For Linux - results from "uname -m" such as "x86_64"</li>
+<li>For AIX - if results from "uname -p" equals "powerpc"
+<ul style="list-style: none;">
+<li>then if "prtconf -k" contains "64-bit" then "ppc64" else "ppc"</li>
+<li>else the result from "uname -p"</li>
+</ul>
+</li>
+<li>For Mac OS X - if results from "uname -m" equals "i386"
+<ul style="list-style: none;">
+<li>then if results from "sysctl -n hw.optional.x86_64" equals "1" then "x86_64" else the results from "uname -m"</li>
+<li>else the results from "uname -m"</li>
+</ul>
+</li>
+<li>For everything else - "unknown"</li>
+</ul>
+<dt>file.separator
+<dd>File separator character of the (remote) connection. Hardcoded "/" (forward slash).
+<dt>path.separator
+<dd>Path separator character of the (remote) connection. Hardcoded ":" (colon).
+<dt>line.separator
+<dd>Line separator character of the (remote) connection. Hardcoded "\n" (new-line).
+<dt>control.user.name
+<dd>User name of the (remote) connection used by the resource manager.
+<dt>control.address
+<dd>Address (hostname) of the resource manager connection.
+<dt>control.working.dir
+<dd>Working directory of the resource manager connection. Usually the user's home directory.
+<dt>executablePath
+<dd>The path of the executable to be submitted or launched by the resource manager. This path is only valid on the target system.
+<dt>executableDir
+<dd>The directory containing the executable on the target system. Only valid on the target system.
+<dt>progArgs
+<dd>The arguments to be passed to the executable prior to launch.
+<dt>directory
+<dd>The working directory specified in the launch configuration Arguments tab.
+</dl>
+In addition, there are a number of special properties that are required so that <code>stdout</code>
+and <code>stderr</code> from a scheduled job can to be delivered to the client. These properties
+should be included in the resource manager property set.
+<dl>
+<dt>control.queue.name
+<dd>Should be linked to an attribute or property containing the destination queue.
+<dt>stdout_remote_path
+<dd>Can be linked to an attribute or property containing a path, otherwise a default path will be used.
+<dt>stderr_remote_path
+<dd>Can be linked to an attribute or property containing a path, otherwise a default path will be used.
+</dl>
+These properties may be linked (see [[#1. Property and Attribute Types | below]]) to other attributes to furnish the
+resource manager with an implementation-independent variable for the
+queue and for any remote output files produced in connection with the job submission.
+==== The <code>property</code> and <code>attribute</code> Elements ====
 <p>
-	A property is any variable necessary for the functioning of the
+   A property is any variable necessary for the functioning of the
-	resource manager. Properties often (but not necessarily) are not
+   resource manager. Properties often (but not necessarily) are not
-	visible. The value for properties can be any primitive type, or lists
+   visible. The value for properties can be any primitive type, or lists
-	or maps of strings. If
+   or maps of strings.
-	<code>stdout</code>
-	and
-	<code>stderr</code>
-	from a scheduled job is to be delivered to the client, the properties <i>stdout_remote_path</i>
-	and <i>stderr_remote_path</i> should be included in the resource
-	manager property set. Simarly, the native designation for the selected
-	queue, if used, should be linked to <i>control.queue.name</i>.
 </p>
 <p>The distinction between attribute and property is not hard and
-	fast, but the intention is to provide a way of differentiating between
+   fast, but the intention is to provide a way of differentiating between
-	external, system-determined variables (attributes) and variables only
+   external, system-determined variables (attributes) and variables only
-	required by the PTP implementation itself. There are several other
+   required by the PTP implementation itself. There are several other
-	elements and fields on the attribute, e.g., for type-specific defaults
+   elements and fields on the attribute, e.g., for type-specific defaults
-	(min/max, choice). Generally, if the variable is exposed to the user,
+   (min/max, choice). Generally, if the variable is exposed to the user,
-	who should also be provided with a description or instructions for use,
+   who should also be provided with a description or instructions for use,
-	then it should be defined as an attribute.</p>
+   then it should be defined as an attribute.</p>
 [[Image:040attribute.jpeg]]
 <p>
-	The <i>visible</i> attribute is a way of indicating to the resource
+		The <i>visible</i> attribute is a way of indicating to the resource
-	manager that the user will not be directly changing the value of the
+		manager that the user will not be directly changing the value of the
-	property or attribute via the user interface. Certain widgets (such as
+		property or attribute via the user interface. Certain widgets (such as
-	the
+		the
-	<code>attribute viewer</code>
+		<code>attribute viewer</code>
-	) check this to see if the property or attribute should be included
+		) check this to see if the property or attribute should be included
-	automatically in its list. <i>status</i> is an optional field for
+		automatically in its list.
-	capturing the validity of the attribute at runtime or for a particular
+	</p>
-	installation, if such information is made available by the system. The
-	<i>readOnly</i> property can be defined here, but more often will be
+	<hr>
-	enforced directly on the widget which is connected to the property or
+<blockquote>
-	attribute. <i>min</i> and <i>max</i> give the boundaries for integer
+	<b>BEST PRACTICE: VISIBLE vs INVISIBLE</b>
-	values, usually represented by spinner widgets.
+	<p>
-</p>
+		Always mark properties or attributes which are the targets of <i>attribute</i>
+		(see [[#Widget |below]]) as <i>visible</i>="true"; if this value is
+		false, this variable will <i>always</i> be passed to the
+		configuration/environment of the resource manager; selection/exclusion
+		of sets of properties or attributes pertains only to those marked
+		visible.
+	</p>
+</blockquote>
+	<hr>
+	<p>
+		<i>status</i> is an optional field for capturing the validity of the
+		attribute at runtime or for a particular installation, if such
+		information is made available by the system. The <i>readOnly</i>
+		property can be defined here, but more often will be enforced directly
+		on the widget which is connected to the property or attribute. <i>min</i>
+		and <i>max</i> give the boundaries for integer values, usually
+		represented by spinner widgets. <i>translateBooleanAs</i> applies to
+		boolean values which should be represented as strings other than
+		"true" and "false"; use a comma-delimited pair to indicate the string
+		equivalent for the boolean; e.g., "YES,NO" would mean true yields the first
+		string and false the second.
+	</p>
 <p>
-	<code>description</code>
+   <code>description</code>
-	is meant to be a brief one-sentence definition; most of the time
+   is meant to be a brief one-sentence definition; most of the time
-	detailed information will go into the
+   detailed information will go into the
-	<code>tooltip</code>
+   <code>tooltip</code>
-	element.
+   element.
-	<code>choice</code>
+   <code>choice</code>
-	is a predetermined, fixed set of values to choose from; these are
+   is a predetermined, fixed set of values to choose from; these are
-	comma-delimited and will appear as the items of a combo-box.
+   comma-delimited and will appear as the items of a combo-box.
-	<code>items-from</code>
+   <code>items-from</code>
-	instead links the combo items (choice) to another property or attribute
+   instead links the combo items (choice) to another property or attribute
-	value (whose type must be
+   value (whose type must be
-	<code>java.util.Collection</code>
+   <code>java.util.Collection</code>
-	).
+   ).
 </p>
@@ Line 268: / Line 376: @@
 <p>
-	The untyped
+   The untyped
-	<code>value</code>
+   <code>value</code>
-	element on properties and attributes is for internal use only; to give
+   element on properties and attributes is for internal use only; to give
-	a predefined (primitive) value, use the
+   a predefined (primitive) value, use the
-	<code>default</code>
+   <code>default</code>
-	element along with the <i>type</i> attribute.
+   element along with the <i>type</i> attribute.
 </p>
 <p>
-	<code>link-value-to</code>
+   <code>link-value-to</code>
-	works as follows: the property or attribute gets its value from the
+   works as follows: the property or attribute gets its value from the
-	property or attributed linked to, unless that property or attribute has
+   property or attribute linked to, unless that property or attribute has
-	an undefined (empty) value; in the latter case, any default defined on
+   an undefined (empty) value; in the latter case, any default defined on
-	the present property or attribute will be used. This feature is
+   the present property or attribute will be used. This feature is
-	primarily intended for defining dependent properties or attributes
+   primarily intended for defining dependent properties or attributes
-	whose values are not directly modified by the user.
+   whose values are not directly modified by the user.
 </p>
 <p>Attributes also carry optional validators which allow you to
-	enforce syntactic rules or file-system semantics on the value when the
+		enforce syntactic rules, numerical ranges (each with one or both
-	user enters text.</p>
+		boundaries; these are OR'd) or file-system semantics on the value when
+		the user enters text.</p>
 [[Image:041validator.jpeg]]
 <br>
-===== Regex and File Match Types =====
+===== The <code>regex</code> and <code>file-match</code> Elements =====
 <p>
-	The validator makes use of the
+   The validator makes use of the
-	<code>regex</code>
+   <code>regex</code>
-	type (also used in the parser or tokenizer; see [[Stream Parsers]]),
+   type (also used in the parser or tokenizer; see [[#Stream Parsers | below]]),
-	along with a special
+   along with a special
-	<code>file-match</code>
+   <code>file-match</code>
-	type. A regex can be used to match by setting the <i>expression</i>
+   type. A regex can be used to match by setting the <i>expression</i>
-	field; to split the segment into tokens on the basis of the expression,
+   field; to split the segment into tokens on the basis of the expression,
-	set the <i>split</i> field to true (this applies only to the
+   set the <i>split</i> field to true (this applies only to the
-	<code>match</code>
+   <code>match</code>
-	type discussed [[Match Type]]). The expression should follow the
+   type discussed [[#Match Type | below]]). The expression should follow the
-	grammar in
+   grammar in
-	<code>java.util.regex</code>
+   <code>java.util.regex</code>
-	; the <i>flags</i> are those made available in the
+   ; the <i>flags</i> are those made available in the
-	<code>java.util.regex.Pattern</code>
+   <code>java.util.regex.Pattern</code>
-	class (consult the Javadoc for explanations):
+   class (consult the Javadoc for explanations):
 </p>
 <ul>
-	<li>CASE_INSENSITIVE</li>
+   <li>CASE_INSENSITIVE</li>
-	<li>MULTILINE</li>
+   <li>MULTILINE</li>
-	<li>DOTALL</li>
+   <li>DOTALL</li>
-	<li>UNICODE_CASE</li>
+   <li>UNICODE_CASE</li>
-	<li>CANON_EQ</li>
+   <li>CANON_EQ</li>
-	<li>LITERAL</li>
+   <li>LITERAL</li>
-	<li>COMMENTS</li>
+   <li>COMMENTS</li>
-	<li>UNIX_LINES</li>
+   <li>UNIX_LINES</li>
 </ul>
-<p>These can be OR'd in the usual manner.</p>
+	<p>
+		These can be OR'd in the usual manner. <b>NOTE:</b> when using the
+		regex on the [[#Match Type|<code>match</code>]] type, the expression can contain variable references to be resolved
+		in the environment.
+	</p>
 <p>
-	<code>file-match</code>
+   <code>file-match</code>
-	exports the attributes pertaining to
+   exports the attributes pertaining to
-	<code>org.eclipse.core.filesystem.IFileInfo</code>
+   <code>org.eclipse.core.filesystem.IFileInfo</code>
-	; <i>efsAttributes</i> is an OR'd string of the EFS constants:
+   ; <i>efsAttributes</i> is an OR'd string of the EFS constants:
 </p>
 <ul>
-	<li>ATTRIBUTE_READ_ONLY</li>
+   <li>ATTRIBUTE_READ_ONLY</li>
-	<li>ATTRIBUTE_IMMUTABLE</li>
+   <li>ATTRIBUTE_IMMUTABLE</li>
-	<li>ATTRIBUTE_OWNER_READ</li>
+   <li>ATTRIBUTE_OWNER_READ</li>
-	<li>ATTRIBUTE_OWNER_WRITE</li>
+   <li>ATTRIBUTE_OWNER_WRITE</li>
-	<li>ATTRIBUTE_OWNER_EXECUTE</li>
+   <li>ATTRIBUTE_OWNER_EXECUTE</li>
-	<li>ATTRIBUTE_GROUP_READ</li>
+   <li>ATTRIBUTE_GROUP_READ</li>
-	<li>ATTRIBUTE_GROUP_WRITE</li>
+   <li>ATTRIBUTE_GROUP_WRITE</li>
-	<li>ATTRIBUTE_GROUP_EXECUTE</li>
+   <li>ATTRIBUTE_GROUP_EXECUTE</li>
-	<li>ATTRIBUTE_OTHER_READ</li>
+   <li>ATTRIBUTE_OTHER_READ</li>
-	<li>ATTRIBUTE_OTHER_WRITE</li>
+   <li>ATTRIBUTE_OTHER_WRITE</li>
-	<li>ATTRIBUTE_OTHER_EXECUTE</li>
+   <li>ATTRIBUTE_OTHER_EXECUTE</li>
-	<li>ATTRIBUTE_EXECUTABLE</li>
+   <li>ATTRIBUTE_EXECUTABLE</li>
-	<li>ATTRIBUTE_ARCHIVE</li>
+   <li>ATTRIBUTE_ARCHIVE</li>
-	<li>ATTRIBUTE_HIDDEN</li>
+   <li>ATTRIBUTE_HIDDEN</li>
-	<li>ATTRIBUTE_SYMLINK</li>
+   <li>ATTRIBUTE_SYMLINK</li>
-	<li>ATTRIBUTE_LINK_TARGET</li>
+   <li>ATTRIBUTE_LINK_TARGET</li>
 </ul>
 <p>
-	The <i>lastModified...</i> fields require the format
+   The <i>lastModified...</i> fields require the format
-	<code>yyyy/MM/dd HH:mm:ss</code>
+   <code>yyyy/MM/dd HH:mm:ss</code>
-	.
+   .
 </p>
-==== 2. Managed Files ====
+==== The <code>managed-file</code> Element ====
 <p>
-	By "managed file" is meant a local file required by the executable but
+   By "managed file" is meant a local file required by the executable but
-	which may not automatically be present on the host on which the job
+   which may not be present on the host on which the job
-	will run; hence, these files may either be external or may actually be
+   will run; hence, these files may either be external or may actually be
-	generated from the resource manager environment in conjunction with the
+   generated from the resource manager environment in conjunction with the
-	job submission, but in any case need to be copied to that host just
+   job submission, but in any case need to be copied to that host just
-	prior to it. The
+   prior to it. The
-	<code>script</code>
+   <code>script</code>
-	file used in connection with scheduler (batch) jobs is a special
+   file used in connection with scheduler (batch) jobs is a special
-	category of the
+   category of the
-	<code>managed-file</code>
+   <code>managed-file</code>
-	type, and will be discussed in the next sub-section.
+   type, and will be discussed in the next sub-section.
 </p>
 [[Image:05managed-files.jpeg]]
-<p>
+<p>Managed files are added to the definition in groups determined by
-	Managed files are added to the definition in groups determined by their
+		their shared staging location, which is a path relative to the working
-	shared staging location, which is a path relative to the working
+		directory of the connection. (The batch script is by default staged to
-	directory of the connection. (The batch script is normally staged to <i>
+		<i> .eclipsesettings</i> in the user home directory, since it is by
-		.eclipsesettings</i> in the user home directory, since it is by default
+		default temporary and deleted after the submit call returns; however,
-	temporary and deleted after the submit call returns.) The file itself
+		one can use the
-	can be set to be deleted after submission (<i>deleteTargetAfterUse</i>;
+		<code>file-staging-location</code>
-	this is currently not implemented in the general case) and if the file
+		on the
-	is actually generated by the resource manager, the local temporary copy
+		<code>script</code>
-	can be deleted or retained (<i>deleteSourceAfterUse</i>). In the latter
+		element to set the path explicitly.) The file itself can be set to be
-	case, one can also distinguish multiple copies of the generated file by
+		deleted after submission (<i>deleteTargetAfterUse</i>; this is
-	setting <i>uniqueIdPrefix</i> to true.
+		currently not implemented in the general case) and if the file is
-</p>
+		actually generated by the resource manager, the local temporary copy
+		can be deleted or retained (<i>deleteSourceAfterUse</i>). In the
+		latter case, one can also distinguish multiple copies of the generated
+		file by setting <i>uniqueIdPrefix</i> to true.</p>
 <p>The managed file definition allows one of three possibilities as
-	to specifying content.</p>
+   to specifying content.</p>
 <ol>
-	<li>If it is an external file or one edited perhaps through the
+   <li>If it is an external file or one edited perhaps through the
-		workspace, one can use the <code>path</code> element to indicate
+      workspace, one can use the <code>path</code> element to indicate
-		location; this can be either a hard-coded path or a reference to a
+      location; this can be either a hard-coded path or a reference to a
-		property or attribute value.</li>
+      property or attribute value.</li>
-	<li>The <code>contents</code> element is a string which literally
+   <li>The <code>contents</code> element is a string which literally
-		represents the file contents. The <i>resolveContents</i> attribute in
+      represents the file contents. The <i>resolveContents</i> attribute in
-		this case is used to indicate whether to pass this string through the
+      this case is used to indicate whether to pass this string through the
-		variable resolver, substituting any references to property or
+      variable resolver, substituting any references to property or
-		attribute values it may contain. However, caution should be exercised
+      attribute values it may contain. However, caution should be exercised
-		here in that this string should not contain "${...}" sequences which
+      here in that this string should not contain "${...}" sequences which
-		do not actually refer to Eclipse variables (such as batch script
+      do not actually refer to Eclipse variables (such as batch script
-		variables), or the resolution will fail. If <i>resolveContents</i> is
+      variables), or the resolution will fail. If <i>resolveContents</i> is
-		set to false, you can still provide this text element with a single
+      set to false, you can still provide this text element with a single
-		reference to a property or attribute, in which case the value of that
+      reference to a property or attribute, in which case the value of that
-		property or attribute will be used <i>as is</i>, without further
+      property or attribute will be used <i>as is</i>, without further
-		dereferencing.</li>
+      dereferencing.</li>
-	<li>The preferred, and most flexible way to represent the contents
+   <li>The preferred, and most flexible way to represent the contents
-		of a file to be generated, however, is to use the <code>line</code>
+      of a file to be generated, however, is to use the <code>line</code>
-		type which is discussed [[line and arg types]] in connection with the
+      type which is discussed [[#line and arg types | below]] in connection with the
-		<code>script</code> type.</li>
+      <code>script</code> type.</li>
 </ol>
 <p>When the submit call is executed, managed files are written out,
-	if so indicated, and then their target paths are determined as follows:</p>
+   if so indicated, and then their target paths are determined as follows:</p>
 <ul>
-	<li>If the <code>path</code> element is used, the target path
+   <li>If the <code>path</code> element is used, the target path
-		becomes <i>staging directory / name of pre-existing file</i>;</li>
+      becomes <i>staging directory / name of pre-existing file</i>;</li>
-	<li>If the <code>content</code> or <code>line</code> elements are
+   <li>If the <code>content</code> or <code>line</code> elements are
-		used, the target path becomes <i>staging directory/
+      used, the target path becomes <i>staging directory/
-			[uniqueIdPrefix]managed-file-name</i>);</li>
+         [uniqueIdPrefix]managed-file-name</i>);</li>
-	<li>A property is placed in the environment whose <i>name</i> is
+   <li>A property is placed in the environment whose <i>name</i> is
-		the managed-file <i>name</i> and whose <i>value</i> is this target
+      the managed-file <i>name</i> and whose <i>value</i> is this target
-		path.</li>
+      path.</li>
 </ul>
-===== The (Batch) Script Element =====
+===== The <code>script</code> Element =====
 <p>A resource manager definition for a scheduler system such as PBS
-	or LoadLeveler can be instrumented to work without a batch script (by
+   or LoadLeveler can be instrumented to work without a batch script (by
-	setting command-line flags or passing all the job control information
+   setting command-line flags or passing all the job control information
-	as environment variables, for instance), but most frequently the use of
+   as environment variables, for instance), but most frequently the use of
-	a script allows more flexibility in configuring the job (interactive
+   a script allows more flexibility in configuring the job (interactive
-	managers of course have no need of a script).</p>
+   managers of course have no need of a script).</p>
 [[Image:06script.jpeg]]
 <p>
-	If the script is specified in the XML definition, its path is
+		If the script is specified in the XML definition, its path is
-	automatically added to the list of managed files to be staged to the <i>.eclipsesettings</i>
+		automatically added to the list of managed files to be staged to the
-	directory, and so there is no need to include a script entry explicitly
+		appropriate directory (by default <i>.eclipsesettings</i>, or
-	under the
+		whereever indicated by the
-	<code>managed-files</code>
+		<code>file-staging-location</code>
-	element. As with the
+		element), and so there is no need to include a script entry explicitly
-	<code>managed-file</code>
+		under the
-	, <i>deleteAfterSubmit</i> indicates that the script target should not
+		<code>managed-files</code>
-	be retained (this is the default behavior); unlike the
+		element. As with the
-	<code>managed-file</code>
+		<code>managed-file</code>
-	, however, the local copy of the generated script is always deleted. A
+		, <i>deleteAfterSubmit</i> indicates that the script target should not
-	reserved property,
+		be retained (this is the default behavior); unlike the
-	<code>managed_file_for_script</code>
+		<code>managed-file</code>
-	, should be used to reference the script's path on the target resource;
+		, however, the local copy of the generated script is always deleted. A
-	for instance, in the PBS submit command:
+		reserved property,
-</p>
+		<code>managed_file_for_script</code>
+		, should be used to reference the script's path on the target
+		resource; for instance, in the PBS submit command:
+	</p>
-<code>
+<source lang="xml">
-	&lt;arg&gt;qsub&lt;/arg&gt;<br>
+   <arg>qsub</arg> <arg>${ptp_rm:managed_file_for_script#value}</arg>
-	&lt;arg&gt;${ptp_rm:managed_file_for_script#value}&lt;/arg&gt;
+</source>
-</code>
 <br>
 <p>
-	<b>Note</b>: If the
+   <b>Note</b>: If the
-	<code>import</code>
+   <code>import</code>
-	tab (see [[4. The Launch Tab]]) is used to provide an external or
+   tab (see [[#4. The Launch Tab | below]]) is used to provide an external or
-	workspace edited batch script to the run, nothing extra need be done in
+   workspace edited batch script to the run, nothing extra need be done in
-	the definition XML, as the presence of any external path for the script
+   the definition XML, as the presence of any external path for the script
-	is handled under the covers.
+   is handled under the covers.
 </p>
 <p>
-	<i>insertEnvironmentAfter</i> is a line number indicating where in the
+   <i>insertEnvironmentAfter</i> is a line number indicating where in the
-	script to add any extra environment variables set through the <b>Environment</b>
+   script to add any extra environment variables set through the <b>Environment</b>
-	tab provided as part of the <b>Run Configuration</b> wizard. This way
+   tab provided as part of the <b>Run Configuration</b> wizard. This way
-	the user has control over whether these should overwrite previously
+   the user has control over whether these should overwrite previously
-	defined environment variable values.
+   defined environment variable values.
 </p>
 <p></p>
-=====
+===== The <code>line</code> Element  =====
-<code>line</code>
-and
-<code>arg</code>
-types =====
 <p>
-	The
+   The
-	<code>line</code>
+   <code>line</code>
-	element was mentioned [[Files">above</a> in connection with managed file
+   element was mentioned [[#2. Managed Files|above]] in connection with managed file
-	content; it is essentially a wrapper around a series of
+   content; it is essentially a wrapper around a series of whitespace separated
-	<code>arg</code>
+   <code>arg</code>
-	elements, all of which are placed on a single line (that is, the group
+   elements, all of which are placed on a single line (that is, the group
-	is terminated by a line separator).
+   is terminated by a line separator).
 </p>
 [[Image:07line-arg.jpeg]]
+===== The <code>arg</code> Element =====
 <p>
-	The
+   The
-	<code>arg</code>
+   <code>arg</code>
-	element is used for script and managed file content as well as in the
+   element is used for script and managed file content as well as in the
-	definition of commands. Its text element can contain variable
+   definition of commands. Its text element can contain variable
-	references to be resolved against the environment, but also allows for
+   references to be resolved against the environment, but also allows for
-	the entire string to be treated as a literal by setting <i>resolve</i>
+   the entire string to be treated as a literal by setting <i>resolve</i>
-	to false (default is true). This is useful in the context of the
+   to false (default is true). This is useful inasmuch as it allows for the presence of batch-type
-	<code>script</code>
+   variables (e.g., ${HOME}) which should be resolved by the remote shell
-	definition inasmuch as it allows for the presence of batch-type
+   and not inside the Eclipse client.
-	variables (e.g., ${HOME}) which should be resolved by the remote shell
-	and not inside the Eclipse client.
 </p>
 <p>
-	The default behavior of the argument resolver is not to write out or
+   The default behavior of the argument resolver is not to write out or
-	include arguments whose value is undefined (either
+   include arguments whose value is undefined (either
-	<code>null</code>
+   <code>null</code>
-	or zero-length). <i>isUndefinedIfMatches</i> affords more nuanced
+   or zero-length).
-	control over whether an argument should be so eliminated. If the
-	argument references property or attribute values, but also has
-	text-literal segments, a regex can be provided to define what an
-	"empty" argument would be in this case. For instance, if an argument
-	flag should not appear when the value it precedes is an empty string,
-	one could write:
 </p>
+<p>
+   The <b>isUndefinedIfMatches</b> attribute affords more nuanced
+   control over whether an argument should be so eliminated. If the
+   argument references property or attribute values, but also has
+   text-literal segments, a regex can be provided to define what an
+   "empty" argument would be in this case. For instance, if an argument
+   flag should not appear when the value it precedes is an empty string,
+   one could write:
+</p>
+<source lang="xml">
+   <arg isUndefinedIfMatches="-f">-f ${ptp_rm:flag#value}</arg>
+</source>
-<code> &lt;arg isUndefinedIfMatches="-f"&gt;-f
-	${ptp_rm:flag#value}&lt;/arg&gt; </code>
 <br>
 <p>For the purposes of matching, trailing whitespace is trimmed from
-	the resolved argument, so there is no need to specify this as part of
+   the resolved argument, so there is no need to specify this as part of
-	the regex used to match.</p>
+   the regex used to match.</p>
-===== "Just-in-time" resolution of
+===== "Just-in-time" resolution of <i>@jobId</i> and <code>managed-file</code> paths =====
-<i>@jobId</i>
-and
-<code>managed-file</code>
-paths =====
 <p>
-	<i>@jobId</i> is a special property name designating the runtime id for
+   <i>@jobId</i> is a special property name designating the runtime id for
-	a job instance. In the lifecycle of the run/launch (submit) call, this
+   a job instance. In the lifecycle of the run/launch (submit) call, this
-	value begins as an internally generated unique id which then is swapped
+   value begins as an internally generated unique id which then is swapped
-	for the id returned by the scheduler or runtime system.
+   for the id returned by the scheduler or runtime system.
 </p>
 <p>
-	The <i>@jobId</i>, along with the target paths for
+   The <i>@jobId</i>, along with the target paths for
-	<code>managed-file</code>
+   <code>managed-file</code>
-	elements, are not known at configuration time (i.e., before the user
+   elements, are not known at configuration time (i.e., before the user
-	hits "Run"). While the former is made visible to the parsers and the
+   hits "Run"). While the former is made visible to the parsers and the
-	returned status object of the submit command, neither is available for
+   returned status object of the submit command, neither is in the scope of (available for reference in) other managed files or the
-	reference in other managed files or in the
+   <code>script</code>
-	<code>script</code>
+   element, because these latter elements are generated just prior to the actual
-	element, because these elements are generated prior to the actual
+   submission.
-	submission.
 </p>
 <p>
-	If the
+   If the
-	<code>script</code>
+   <code>script</code>
-	needs to refer to the <i>@jobId</i>, it must do so via the variable
+   needs to refer to the <i>@jobId</i>, it must do so via the variable
-	made available by the particular scheduler it is written for. An
+   made available by the particular scheduler it is written for. An
-	example of how to reference the target path of a
+   example of how to reference the target path of a
-	<code>managed-file</code>
+   <code>managed-file</code>
-	inside the
+   inside the
-	<code>script</code>
+   <code>script</code>
-	is included in the [[JAXBDemo.pdf|tutorial slides]]; this essentially
+   is included in the [[Media:JAXBDemo.pdf|tutorial slides ]]; this essentially
-	involves defining a variable in the submission command's environment,
+   involves defining a variable in the submission command's environment,
-	with a reference to the
+   with a reference to the
-	<code>managed-file</code>
+   <code>managed-file</code>
-	path property as its value, and then using this environment variable
+   path property as its value, and then using this environment variable
-	inside the
+   inside the
-	<code>script</code>
+   <code>script</code>
-	.
+   .
 </p>
-==== 3. Commands ====
-<p>
-	As mentioned [[Resource Manager Command Types]], the resource manager
-	<code>command</code>
-	denotes a [UNIX-type] system call made on the resource manager's remote
-	(target) connection (PTP does not generally support execution on
-	Windows systems). The command is always configured and executed through
-	a Java API (the "process builder") which bottoms out in a
-	<code>bash -c</code>
-	command. The first argument of the command thus must be the name/path
-	of the executable only. Its own arguments can then be subsequently
-	specified by an arbitrary number of
-	<code>arg</code>
-	elements.
-</p>
-[[Image:08command.jpeg]]
+==== The <code>command</code> Element ====
 <p>
-	There are a number of important attributes on the
+   <i>Commands</i> are system calls, either to a local or remote OS,
-	<code>command-type</code>
+   depending on the connection defined for the resource manager and
-	element which bear detailed discussion here. <i>directory</i> of course
+   denotes a [UNIX-type] system call made on the resource manager's remote
-	refers to where the command will be executed; also as expected, <i>redirectStderr</i>
+   (target) connection (PTP does not generally support execution on
-	means that both output and error streams are sent back on
+   Windows systems). The command is always configured and executed through
-	<code>stdout</code>
+   a Java API (the "process builder") which bottoms out in a
-	. One can, if necessary, tune the buffer sizes on the stream readers
+   "<code>sh -c</code>"
-	via the <i>streamBufferLimit</i>. If the environment set on the command
+   command.
-	should entirely replace the shell environment (the default behavior is
-	to append the command environment), set <i>replaceEnvironment</i> to
-	true.
 </p>
 <p>
-	If you do not wish an error to be thrown in the case of non-zero exit
+   The <b>start-up-command</b>
-	of the command, set <i>ignoreExitStatus</i> to true. The <i>flags</i>
+   and <b>shut-down-command</b> are arbitrary commands to be run
-	attribute is an OR'd string of three possible values:
+   (serially in order) when the resource manager is started or stopped.
+</p>
+<p>
+   The <b>submit</b> commands are those used to launch jobs. Currently a
+   configuration may have only either batch or an interactive commands. Normal
+   (run) mode is specified using the <b>submit-batch</b> or <b>submit-interactive</b>
+   commands. Debug mode is specified using the <b>-debug</b> versions of these
+   commands.
+   In the future we may allow all batch and interactive commands to coexist in a single
+   configuration.
+</p>
+<p>
+   The <b>get-job-status</b> is a user-initiated (on-demand)
+   request to refresh the status information for a submission. Normal
+   (polled) updates, on the other hand, are the responsibility of the
+   <code>monitor</code>
+   component. The status command nevertheless needs to be implemented in
+   most cases, as it will be called internally just after submission.
+</p>
+<p>
+   The <b>terminate-job</b> command is used to terminate a previously
+   launched batch or interactive job. <b>Note</b>: if the submission type is interactive, the
+   <b>terminate-job</b> command usually does not need to be implemented,
+   as the process termination will be handled internally. However, in some
+   cases (such as PBS -I) which require the interactive job to run as a
+   pseudo-terminal, one may need this command in order to force its
+   termination externally.
+</p>
+<p>
+   The
+   remaining <b>-job</b> commands are operations which can be executed on
+   jobs submitted to job schedulers (batch-systems) and do not apply to
+   resource managers which connect to interactive runtime-systems such as
+   OpenMPI or PE.
+</p>
+<p>
+   The <b>button-action</b> is an arbitrary
+   command associated with a button exposed through the
+   <code>Launch Tab</code>
+   (see further [[#Action Button | below]]).
 </p>
+[[Image:08command.jpeg]]
+There are a number of attributes that can be specified on the <code>command-type</code> element:
+<dl>
+<dt>directory (defaults to the "home" or working directory of the remote control connection, i.e. the <code>control.working.dir</code> property)
+<dd>Specifies the location on the remote systemto where the command will be executed
+<dt>redirectStderr (default false)
+<dd>Merge both the output and error streams onto <code>stdout</code>
+<dt>streamBufferLimit (not currently used)
+<dd>Used to specify the buffer sizes on the stream readers
+<dt>replaceEnvironment (default false)
+<dd>Specifies if the environment set on the command should entirely replace the shell environment (the default behavior is to append the command environment)
+<dt>ignoreExitStatus (default false)
+<dd>Specifies that no error should be thrown in the case of non-zero exit of the command
+<dt>flags
+<dd>An OR'd string of three possible values:
 <ul>
-	<li>NONE (default)</li>
+   <li>NONE (default)</li>
-	<li>ALLOCATE_PTY (allocates a pseudo-terminal)</li>
+   <li>ALLOCATE_PTY (allocates a pseudo-terminal)</li>
-	<li>FORWARD_X11</li>
+   <li>FORWARD_X11</li>
 </ul>
+<dt>waitForId (default false)
+<dd>Indicates to the resource manager that the output stream for the command is being parsed for an id which will appear as a property in the environment during the command execution, and that the execution should not return until it sees this id. Very often the submit commands will have these semantics.
+<dt>keepOpen (default false)
+<dd>Indicates that the command should be held open for potentially repeated redirection of input (see next paragraph). There can only be one such process open inside the resource manager at a time (these semantics will usually pertain to interactive managers using a pseudo-terminal mode to control job submission). NOTE: if keepOpen is false, then the internal command will not complete until the remote process has finished.
+</dl>
+	<hr>
+<blockquote>
+	<b>BEST PRACTICE: WAIT-FOR-ID</b>
+	<p>
+		As [[#Resource Manager Command Types|previously]] described, there are two basic types of commands,
+               <b>(uu)id</b> commands which are related to a specific job, and generic (non-uuid) commands.
+		<code>submit-</code>
+		and
+		<code>-job-</code>
+		commands fall into the former category; those in the latter category
+		are usually simple interactive commands, and their output is not
+		redirected to the console, but uuid-commands have several subclasses:
+	</p>
+        <br><p>
+	<table align="center" rules="all">
+		<tr>
+			<th>COMMAND TYPE</th>
+			<th>WAIT FOR ID</th>
+			<th>TOKENIZER STATES</th>
+		</tr>
+		<tr>
+			<td>batch</td>
+			<td>yes</td>
+			<td>SUBMITTED</td>
+		</tr>
+		<tr>
+			<td>interactive</td>
+			<td>no</td>
+			<td></td>
+		</tr>
+		<tr>
+			<td>interactive</td>
+			<td>yes</td>
+			<td>RUNNING</td>
+		</tr>
+		<tr>
+			<td>interactive, open</td>
+			<td>no</td>
+			<td></td>
+		</tr>
+		<tr>
+			<td>interactive, open</td>
+			<td>yes</td>
+			<td>(SUBMITTED,) RUNNING</td>
+		</tr>
+		<tr>
+			<td>interactive, open, pseudo-terminal</td>
+			<td>no</td>
+			<td></td>
+		</tr>
+		<tr>
+			<td>interactive, open, pseudo-terminal</td>
+			<td>yes</td>
+			<td>(SUBMITTED,) RUNNING</td>
+		</tr>
+	</table>
+        </p><br>
+	<p>
+		This table shows the various combinations that are feasible between
+		command type and waiting states. Simple interactive commands, whether
+		held open for further input or not, can wait or not wait, but in the
+		case of waiting, they should be provided with a [[#Stream Parsers|stream tokenizer]] which recognizes and sets
+		the RUNNING state. Batch jobs, on the other hand, will usually have a
+		tokenizer which recognizes and sets SUBMITTED. In the case of some
+		open-process jobs (e.g.,
+		<code>qsub -I</code>
+		for PBS, which also requires a pseudo-terminal to connect remotely),
+		there will usually be a pause before the terminal job is actually
+		scheduled; in order to allow the user to see that the job has been
+		accepted and is pending, the tokenizer needs to set both SUBMITTED and
+		RUNNING states, the latter when the job has actually started.
+	</p>
+</blockquote>
+	<hr>
+===== The <code>arg</code> Element =====
 <p>
-	Two crucial fields for command control are <i>waitForId</i> and <i>keepOpen</i>.
+   The first argument of the command must be the name/path
-	The former indicates to the resource manager that the output stream for
+   of the executable only. Its own arguments can then be subsequently
-	the command is being parsed for an id which will appear as a property
+   specified by an arbitrary number of
-	in the environment during the command execution, and that the execution
+   <code>arg</code>
-	should not return until it sees this id. Very often the submit commands
+   elements. See [[#The arg Element |above]] for more details.
-	will have these semantics. The latter field indicates that the command
+   More than one argument can be specified using an <code>arg</code> type.
-	should be held open for potentially repeated redirection of input (see
-	next paragraph). There can only be one such process open inside the
-	resource manager at a time (these semantics will usually pertain to
-	interactive managers using a pseudo-terminal mode to control job
-	submission).
 </p>
+===== The <code>input</code> Element =====
 <p>
-	We have discussed the
+   The
-	<code>arg</code>
+   <code>input</code> element is similar to the <code>arg</code> element
-	type [[line and arg types]]. Note that the command arguments, input and
+   but is directed to the input stream of the command. If the <i>keepOpen</i>
-	even environment variables make use of this element. For the
+   attribute is true, a check will be made to see if an open process already exists
-	<code>name-value-pair</code>
+   (which is also alive), which will then be used; otherwise, the
-	type comprising the latter one can as simple alternative set the <i>value</i>
+   arguments are executed, then the input arguments are given to the
-	attribute to a string (which will be resolved first in the current
+   process. With an open command/process, the input arguments can be fed
-	environment); finer-grained control over the resolution of the value,
+   repeatedly to the same process. This allows, for instance, for
-	however, requires the use of the
+   continuous testing of an interactive job in the same interactive
-	<code>arg</code>
+   session.
-	type. When there is
+</p>
-	<code>input</code>
-	present, it is directed to the input stream of the command. If the <i>keepOpen</i>
+===== The <code>environment</code> Type =====
-	attribute is true, a check will be made to see if there already exists
+<p>
-	an open process (which is also alive), and use that; otherwise, the
+The <code>environment</code> element can be used to pass strings containing properties or attributes to the target system as environment variables. The <i>name</i> attribute is used to specify the name of the environment variable that will be set on the target system. The <i>value</i> attribute specifies the string value of the environment variable. This string is resolved against the variable map, so can contain property or attribute references.
-	arguments are executed, then the input arguments are given to the
+</p>
-	process. With an open command/process, the input arguments can be fed
+<p>
-	repeatedly to the same process; this allows, for instance, for
+This element can be used in conjunction with the <i>replaceEnvironment</i> attribute on the command.
-	continuous testing of an interactive job in the same interactive
-	session.
 </p>
@@ Line 663: / Line 892: @@
 <p>
-	One can attach parsers (which we also refer to as tokenizers) to the
+   One can attach parsers (which we also refer to as tokenizers) to the
-	output and error streams of any
+   stdout and stderr streams of any
-	<code>command</code>
+   <code>command</code> element using the <code>stdout-parser</code>
-	in order to capture information and use it to side-effect existing
+   and <code>stderr-parser</code> elements respectively. The <code>redirect-parser</code>
-	properties or attributes, or to generate new ones on the fly. While the
+   element can be used to parse a combined stdout and stderr stream.
-	parser is not completely general, it is capable of a wide range of
+   This allows output to be captured and used to side-effect existing
-	tasks which would typically be required in the handling of output from
+   properties or attributes, or to generate new ones on the fly. While the
-	batch and runtime systems. We here explain the main elements (
+   parser is not completely general, it is capable of a wide range of
-	<code>target, match, test</code>
+   tasks which would typically be required in the handling of output from
-	) used by the tokenizer; one should further consult the [[Tokenizer
+   batch and runtime systems. We here explain the main elements (
-	Examples]] demonstrating various usage scenarios.
+   <code>target, match, test</code>
+   ) used by the tokenizer; one should further consult the [[#Tokenizer Examples | tokenizer examples]] demonstrating various usage scenarios.
 </p>
@@ Line 679: / Line 909: @@
 <p>
-	The
+   The built-in tokenizer can read the stream in two different ways. If <i>delim</i>
-	<code>type</code>
+   is provided, the stream is split using the indicated value. The string
-	element will most commonly not be set, meaning the built-in parser will
+   should only be one character in length (escaped or non-escaped).
-	be used; however, it is possible to implement a custom parser as a
+   Provision is made for the '\r\n' (Windows) two-character delimiter
-	contribution to the
+   internally; in this case the delimiter should be set to "\r" (however,
-	<code>org.eclipse.ptp.rm.jaxb.core.streamParserTokenizer</code>
+   as already mentioned, PTP does not generally guarantee that system
-	extension point, in which case this element should be set to its
+   calls will work on Windows). Setting <i>includeDelim</i> means that the
-	extension id.
+   delimiter will appear as the last char on the returned stream segment.
 </p>
 <p>
-	The built-in tokenizer can read the stream in two different ways. If <i>delim</i>
+   The second way to read from the stream is to provide a <i>maxMatchLen</i>
-	is provided, the stream is split using the indicated value. The string
+   size; what this indicates is that whatever substring needs to be found
-	should only be one character in length (escaped or non-escaped).
+   on the stream will not exceed this length. The stream is then read in
-	Provision is made for the '\r\n' (Windows) two-character delimiter
+   swatches of <i>maxMatchLen</i>, with the internal buffer set to twice
-	internally; in this case the delimiter should be set to "\r" (however,
+   this size, so that each successive read shifts the buffer to the "left"
-	as already mentioned, PTP does not generally guarantee that system
+   by one length. This guarantees that all such substrings will eventually
-	calls will work on Windows). Setting <i>includeDelim</i> means that the
+   be matched.
-	delimiter will appear as the last char on the returned stream segment.
 </p>
 <p>
-	The second way to read from the stream is to provide a <i>maxMatchLen</i>
+   Sometimes a sort of "look-ahead" paradigm is necessary. For instance,
-	size; what this indicates is that whatever substring needs to be found
+   one may need to match a segment or segments whose position is defined
-	on the stream will not exceed this length. The stream is then read in
+   from the end of the output, but you do not know in advance the actual
-	swatches of <i>maxMatchLen</i>, with the internal buffer set to twice
+   stream length. In this case, one can opt to read until the end of the
-	this size, so that each successive read shifts the buffer to the "left"
+   stream (<i>all</i>="true"), retaining only the last <i>N</i> buffer-lengths or
-	by one length. This guarantees that all such substrings will eventually
+   delimited segments, as indicated by the <i>save</i> field. When the
-	be matched.
+   parser reaches the end of the stream, it will then apply the various
+   targets to each saved segment in order.
 </p>
 <p>
-	Sometimes a sort of "look-ahead" paradigm is necessary. For instance,
+   The <i>applyToAll</i> attribute is discussed further under
-	one may need to match a segment or segments whose position is defined
+   [[#Target Type|<code>target</code>]]
-	from the end of the output, but you do not know in advance the actual
+   . The
-	stream length. In this case, one can opt to read until the end of the
+   <code>exit-on</code>
-	stream (<i>all</i>="true"), retaining only the last N buffer-lengths or
+   element indicates that the tokenizer should quit immediately when it
-	delimited segments, as indicated by the <i>save</i> field. When the
+   encounters this pattern;
-	parser reaches the end of the stream, it will then apply the various
+   <code>exit-after</code>
-	targets to each saved segment in order.
+   indicates that the tokenizer should quit when it encounters this
+   pattern, but should first apply the current segment to its targets.
 </p>
-<p>
+====== The <code>type</code> Element ======
-	<i>applyToAll</i> is discussed further under
+	<p>
-	<code>target</code>
+		The
-	. The
+		<code>type</code>
-	<code>exit-on</code>
+		element will most commonly not be set, meaning the built-in parser
-	element indicates that the tokenizer should quit immediately when it
+		will be used; however, it is possible to implement a custom parser as
-	encounters this pattern;
+		a contribution to the
-	<code>exit-after</code>
+		<code>org.eclipse.ptp.rm.jaxb.core.streamParserTokenizer</code>
-	indicates that the tokenizer should quit when it encounters this
+		extension point, in which case this element should be set to its
-	pattern, but should first apply the current segment to its targets.
+		extension id. Note that the extension requires the class to implement
-</p>
+		<code>org.eclipse.ptp.rm.jaxb.control.internal.IStreamParserTokenizer</code>
+		, which is a Runnable interface with an initialize method which passes in
+		any job id plus the current environment map; the details of such a
+		parser's implementation are not, however, configured from the XML
+		document.
+	</p>
-====== Target Type ======
+====== The <code>target</code> Element ======
 <br>
 [[Image:091target.jpeg]]
 <p>
-	A tokenizer may be given any number of
+		A tokenizer may be given any number of
-	<code>target</code>
+		<code>target</code>
-	elements. The target denotes a particular value (object) currently in,
+		elements. The target denotes a particular value (object) currently in,
-	or to be written to, the environment, which will be side-effected on
+		or to be written to, the environment, which will be side-effected on
-	the basis of the result of the tokenization. A target in turn contains
+		the basis of the result of the tokenization. A target in turn contains
-	<code>match</code>
+		<code>match</code>
-	elements and
+		elements and
-	<code>test</code>
+		<code>test</code>
-	elements; the former are run as part of the stream processing; the
+		elements; the former are run as part of the stream processing; the
-	latter are run after the stream processing has been completed.
+		latter are run after the stream processing has been completed. The
-</p>
+		optional
+		<code>else</code>
+		element is applied only if there are no other tests defined or if none
+		of the defined tests succeed).
+	</p>
 <p>
-	The target object is either to be constructed at match time, or it
+   The target object is either to be constructed at match time, or it
-	pre-exists in the environment. If constructed, the <i>type</i> field is
+   pre-exists in the environment. If constructed, the <i>type</i> field is
-	used to indicate whether the object is a
+   used to indicate whether the object is a
-	<code>property</code>
+   <code>property</code>
-	or an
+   or an
-	<code>attribute</code>
+   <code>attribute</code>
-	type; otherwise, <i>ref</i> points to the name of the
+   type; otherwise, <i>ref</i> points to the name of the
-	<code>property</code>
+   <code>property</code>
-	or
+   or
-	<code>attribute</code>
+   <code>attribute</code>
-	in the environment (recall that for the runtime job identifier, <i>@jobId</i>
+   in the environment (recall that for the runtime job identifier, <i>@jobId</i>
-	is used).
+   is used).
 </p>
 <p>
-	<b>Note</b>: when new targets are constructed, there is a merge
-	operation at the end of tokenization which attempts to combine objects
+		<b>Note</b>: when new targets are constructed, there is a merge
-	into a single instance identified by their <i>name</i> attribute. This
+		operation at the end of tokenization which attempts to combine objects
-	assumes that such names will be unique and that any other values to be
+		into a single instance identified by their <i>name</i> attribute. This
-	set on the object which are not explicitly bound in some way to that
+		assumes that such names will be unique and that any other values to be
-	name via the match pattern will appear on the stream before a new name
+		set on the object which are not explicitly bound in some way to that
-	does (see ex. 5 in [[Tokenizer Examples]]).
+		name via the match pattern will appear on the stream before a new name
-</p>
+		does (see ex. 5 in [[#Tokenizer Examples | tokenizer examples]]).
+		<b><i>The default behavior of this merge is that it will fail
+				if two objects with the same name but differing values are generated
+				by the parsing.</i> (This excludes <code>add</code> and <code>put</code>
+			operations which create a list or map; in these cases, the two
+			collections or maps will be combined into one. This does <i>NOT</i>
+			work, however, for <code>append</code> actions.) To allow duplicates,
+			set the <i>allowOverwrites</i> to true; in this case, successive duplicates
+			simply replace the preceding object. </b>
+	</p>
 <p>The default behavior of the tokenizer read-match sequence is as
-	follows:</p>
+   follows:</p>
 <ol>
-	<li>read from the stream either a set number of chars or until the
+   <li>read from the stream either a set number of chars or until the
-		delimiter is found;</li>
+      delimiter is found;</li>
-	<li>for each target:
+   <li>for each target:
-		<ul>
+      <ul>
-			<li>for each match:
+         <li>for each match:
-				<ul>
+            <ul>
-					<li>if the match is positive, process each of the actions, and
+               <li>if the match is positive, process each of the actions, and
-						go to (1).</li>
+                  go to (1).</li>
-				</ul>
+            </ul>
-		</ul></li>
+      </ul></li>
 </ol>
 <p>
-	Only one qualifying target is processed for any given segment read, and
+   Only one qualifying target is processed for any given segment read, and
-	for the given target, the first pattern matched is the one processed
+   for the given target, the first pattern matched is the one processed
-	for its actions. This is basically the "OR" semantics of normal logic
+   for its actions. This is basically the "OR" semantics of normal logic
-	programming; hence <i>the implementer must be careful to arrange
+   programming; hence <i>the implementer must be careful to arrange
-		the matches inside a target in such a way that the more specific match
+      the matches inside a target in such a way that the more specific match
-		patterns precede the more general</i>.
+      patterns precede the more general</i>.
 </p>
 <p>Three boolean fields allow you to modify this behavior.</p>
 <ol>
-	<li>The <i>applyToAll</i> field on the <code>tokenizer-type</code>
+   <li>The <i>applyToAll</i> field on the <code>tokenizer-type</code>
-		element means take the unmatched part of the read stream and pass it
+      element means take the unmatched part of the read stream and pass it
-		to the next target, even if there was a previous match; this allows
+      to the next target, even if there was a previous match; this allows
-		you to capture more than one regex pattern per stream segment (see ex.
+      you to capture more than one regex pattern per stream segment (see ex.
-in [[Tokenizer Examples]]).</li>
+in [[#Tokenizer Examples | tokenizer examples]]).</li>
-	<li>The <i>matchAll</i> field on the <code>target-type</code>
+   <li>The <i>matchAll</i> field on the <code>target-type</code>
-		element means do not try to match an already matched expression until
+      element means do not try to match an already matched expression until
-		all the others are matched (i.e., a logical AND instead of OR governs
+      all the others are matched (i.e., a logical AND instead of OR governs
-		the set of matches at successive calls to the target match operation);
+      the set of matches at successive calls to the target match operation);
-		this allows one to use, for instance, .* repeatedly but set different
+      this allows one to use, for instance, .* repeatedly but set different
-		fields of the object with the resulting match (see ex. 5 in
+      fields of the object with the resulting match (see ex. 5 in
-		[[Tokenizer Examples]]).</li>
+      [[#Tokenizer Examples | tokenizer examples]]).</li>
-	<li>The <i>moveToTop</i> field on the <code>match-type</code>
+   <li>The <i>moveToTop</i> field on the <code>match-type</code>
-		element indicates to the tokenizer that the matched target be promoted
+      element indicates to the tokenizer that the matched target be promoted
-		to first position in the list of targets. This is useful when there is
+      to first position in the list of targets. This is useful when there is
-		an ordering which expects types of attributes or properties to be
+      an ordering which expects types of attributes or properties to be
-		grouped in sequence on the stream (see ex. 4 in [[Tokenizer
+      grouped in sequence on the stream (see ex. 4 in [[#Tokenizer Examples | tokenizer examples]]).</li>
-		Examples]]).</li>
 </ol>
 <p>When a match is found, the set of action types it contains are
-	all applied.</p>
+   all applied.</p>
-====== Match Type ======
+====== The <code>match</code> Element ======
 <br>
 [[Image:10match.jpeg]]
 <p>Each of these types corresponds to an action to be taken on the
-	indicated field of the target object.</p>
+   indicated field of the target object.</p>
+<br>
 <table align="center" rules="all">
-	<tr>
+   <tr>
-		<td><code>set</code></td>
+      <td><code>set</code></td>
-		<td>sets the value of that field</td>
+      <td>sets the value of that field</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>append</code></td>
+      <td><code>append</code></td>
-		<td>adds to a string buffer, whose string value will be set on
+      <td>adds to a string buffer, whose string value will be set on
-			the field</td>
+         the field</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>add</code></td>
+      <td><code>add</code></td>
-		<td>adds to a trst to which the value of that field will be set</td>
+      <td>adds to a list to which the value of that field will be set</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>put</code></td>
+      <td><code>put</code></td>
-		<td>places a key-value pair in a map to which the value of that
+      <td>places a key-value pair in a map to which the value of that
-			field will be set</td>
+         field will be set</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>throw</code></td>
+      <td><code>throw</code></td>
-		<td>throws an exception and (optionally) also sets the value of
+      <td>throws an exception and (optionally) also sets the value of
-			the field</td>
+         the field</td>
-	</tr>
+   </tr>
 </table>
+<br>
+	<p>
+		The actions listed here all have
+		<code>entry-type</code>
+		children, either single (
+		<code>set, throw</code>
+		) or potentially multiple. All of these except
+		<code>throw</code>
+		also allow you to force the creation of a new object (<i>forceNewObject</i>)
+		each time it is applied; the new object then replaces the current one
+		for successive actions in the match.
+	</p>
-<p>
+====== The <code>entry</code> Element ======
-	The actions listed here all have
-	<code>entry-type</code>
-	children, either single (
-	<code>set, throw</code>
-	) or potentially multiple.
-</p>
-====== Entry Type ======
 <br>
 [[Image:11entry.jpeg]]
 <p>
-	This value-abstraction allows one to set <i>key</i> (for maps) and <i>value</i>
+   This value-abstraction allows one to set <i>key</i> (for maps) and <i>value</i>
-	as literals or references to other properties or attributes to be
+   as literals or references to other properties or attributes to be
-	resolved in the current environment; to reference the matched segment
+   resolved in the current environment; to reference the matched segment
-	parts one sets <i>keyIndex</i> and <i>valueIndex</i> if the regex was
+   parts one sets <i>keyIndex</i> and <i>valueIndex</i> if the regex was
-	used to split the segment; otherwise, <i>keyGroup</i> and <i>valueGroup</i>
+   used to split the segment; otherwise, <i>keyGroup</i> and <i>valueGroup</i>
-	refer to the capture group of the regex pattern, with group 0 referring
+   refer to the capture group of the regex pattern, with group 0 referring
-	to the entire match.
+   to the entire match.
 </p>
-====== Test Type ======
+====== The <code>test </code> Element ======
 <br>
 [[Image:12test.jpeg]]
 <p>
-	As mentioned [[Target Type]], the
+   As mentioned [[#Target Type |above]], the
-	<code>test-type</code>
+   <code>test-type</code>
-	elements are all run after the tokenization has reached the end of the
+   elements are all run after the tokenization has reached the end of the
-	stream. This class of actions is useful for setting values based on
+   stream. This class of actions is useful for setting values based on
-	other values produced during tokenization. A test is one or more
+   other values produced during tokenization. A test is one or more
-	comparison operations plus a set of actions to apply to the target
+   comparison operations plus a set of actions to apply to the target
-	fields in the case of either success or failure (the "else" element);
+   fields in the case of either success or failure (the "else" element);
-	see ex. 3 or the "get-job-status" example in[[Tokenizer Examples]].
+   see ex. 3 or the "get-job-status" example in [[#Tokenizer Examples|tokenizer examples]].
 </p>
 <p>
-	The <i>op</i> attribute can be one of the following comparisons:
+   The <i>op</i> attribute can be one of the following comparisons:
 </p>
 <table align="center">
-	<tr>
+   <tr>
-		<td>EQ</td>
+      <td>EQ</td>
-		<td>:</td>
+      <td>:</td>
-		<td>equals</td>
+      <td>equals</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>LT</td>
+      <td>LT</td>
-		<td>:</td>
+      <td>:</td>
-		<td>less than</td>
+      <td>less than</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>LE</td>
+      <td>LE</td>
-		<td>:</td>
+      <td>:</td>
-		<td>less than or equal to</td>
+      <td>less than or equal to</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>GT</td>
+      <td>GT</td>
-		<td>:</td>
+      <td>:</td>
-		<td>greater than</td>
+      <td>greater than</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>GE</td>
+      <td>GE</td>
-		<td>:</td>
+      <td>:</td>
-		<td>greater than or equal to</td>
+      <td>greater than or equal to</td>
-	</tr>
+   </tr>
 </table>
+<br>
 <p>
-	When the operation is set to one of these, it is expected that the two
+   When the operation is set to one of these, it is expected that the two
-	<code>value</code>
+   <code>value</code>
-	elements will be used. As usual, these elements can be literals or can
+   elements will be used. As usual, these elements can be literals or can
-	contain variables to be resolved into a string type;
+   contain variables to be resolved into a string type;
-	<code>#FIELD</code>
+   <code>#FIELD</code>
-	refers to the value of the given field on the current target; the
+   refers to the value of the given field on the current target; the
-	strings will be converted in conformity with the inferred (primitive)
+   strings will be converted in conformity with the inferred (primitive)
-	type of the comparison. The
+   type of the comparison. The
-	<code>else</code>
+   <code>else</code>
-	element also pertains to comparison tests; the actions listed there
+   element also pertains to comparison tests; the actions listed there
-	will be taken upon failure of the comparison.
+   will be taken upon failure of the comparison.
 </p>
 <p>
-	The <i>op</i> attribute can also be a logical operator [AND, OR, NOT],
+   The <i>op</i> attribute can also be a logical operator [AND, OR, NOT],
-	in which case the embedded
+   in which case the embedded
-	<code>test</code>
+   <code>test</code>
-	object should be used; these can be nested to an arbitrary depth, but
+   object should be used; these can be nested to an arbitrary depth, but
-	of course must bottom out in a comparison operation.
+   of course must bottom out in a comparison operation.
 </p>
-===== Tokenizer Examples
+====== Tokenizer Examples ======
-([[tokenizer-examples.xml|tokenizer-examples.xml]]) =====
+See: [[PTP/designs/resource_manager_xsd#Tokenizer Examples|Tokenizer Example XML]].
-<br>
 <table align="center" rules="all">
-	<tr>
+   <tr>
-		<th>Example</th>
+      <th>Example</th>
-		<th>Description</th>
+      <th>Description</th>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>1</td>
+      <td>1</td>
-		<td>output is a list of line-separated queue names to be assigned
+      <td>output is a list of line-separated queue names to be assigned
-			to the known property "available-queues"</td>
+         to the known property "available-queues"</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>2</td>
+      <td>2</td>
-		<td>output is to be searched for its final line which should
+      <td>output is to be searched for its final line which should
-			contain a job id of the form "[digits].[chars]"</td>
+         contain a job id of the form "[digits].[chars]"</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>3</td>
+      <td>3</td>
-		<td>indeterminate number and order of lines containing parts of
+      <td>indeterminate number and order of lines containing parts of attribute definitions, but each line bearing a distinct id (e.g., openMPI attribute discovery)</td>
-			attribute definitions, but each line bearing a distinct id (e.g.,
+   </tr>
-			openMPI attribute discovery)</td>
+   <tr>
-	</tr>
+      <td>4</td>
-	<tr>
+      <td>indeterminate number of definitions, but grouped by caption; use of <i>moveToTop</i> to promote the target to the top of the list when the caption appears</td>
-		<td>4</td>
+   </tr>
-		<td>indeterminate number of definitions, but grouped by caption;
+   <tr>
-			use of <i>moveToTop</i> to promote the target to the top of the list
+      <td>5</td>
-			when the caption appears</td>
+      <td>similar to 4, but without delimiter (implicit ordering)</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>5</td>
+      <td>6</td>
-		<td>similar to 4, but without delimiter (implicit ordering)</td>
+      <td>similar to 4, but with indeterminate type order and using
-	</tr>
+         buffer + DOTALL | UNIX_LINES</td>
-	<tr>
+   </tr>
-		<td>6</td>
+   <tr>
-		<td>similar to 4, but with indeterminate type order and using
+      <td>7</td>
-			buffer + DOTALL | UNIX_LINES</td>
+      <td>indeterminate number of property definitions, but on single
-	</tr>
+         line</td>
-	<tr>
+   </tr>
-		<td>7</td>
+   <tr>
-		<td>indeterminate number of property definitions, but on single
+      <td>8</td>
-			line</td>
+      <td>looking for values interspersed in the stream but which will
-	</tr>
+         not exceed 32 chars</td>
-	<tr>
+   </tr>
-		<td>8</td>
+   <tr>
-		<td>looking for values interspersed in the stream but which will
+      <td>9</td>
-			not exceed 32 chars</td>
+      <td>successive names/values, in order, but staggered</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>9</td>
+      <td>10</td>
-		<td>successive names/values, in order, but staggered</td>
+      <td>forced merge</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>10</td>
+      <td>11</td>
-		<td>forced merge</td>
+      <td>exit-on</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>11</td>
+      <td>12</td>
-		<td>exit-on</td>
+      <td>exit-after</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td>12</td>
+      <td>13</td>
-		<td>exit-after</td>
+      <td>get-job-status (use of tests)</td>
-	</tr>
+   </tr>
-	<tr>
-		<td>13</td>
-		<td>get-job-status (use of tests)</td>
-	</tr>
 </table>
 <br>
-==== 4. The Launch Tab ====
+===== The <code>pre-launch-cmd</code> and <code>post-launch-cmd</code> Elements =====
 <p>
-	The UI component which needs to be configured in conjunction with the
+It is sometimes necessary to execute commands just prior to and just after submitting the job. This is particularly the case for <code>-debug</code> commands, where the debugger needs cleanup and/or other types of actions to launch correctly.
-	resource manager control is the <b>Launch Tab</b>, which is actually
-	the main area of the <b>Resources Tab</b>, one of the six tabs
-	comprising the <b>Run Configuration</b> wizard or the seven tabs
-	comprising the <b>Debug Configuration</b> wizard. The purpose of this
-	component is to allow the user to set up the job by providing values
-	relating to resource requirements or to the properties of the job in
-	general. In other words, it is in this component that the user can set
-	the values on the
-	<code>attribute</code>
-	elements making up the configurable environment as defined in the XML.
 </p>
 <p>
-	Specifying the composition and layout of this component can be tricky
+The <code>pre-launch-cmd</code> and <code>pre-launch-cmd</code>elements only allow simple command execution. Stream parsers and other <code>command</code> elements are not available.
-	and tedious because it deals with widgets defined by the Eclipse
+</p>
-	<code>JFace</code>
+<p>
-	library and with their underlying implementation in
+Available attributes include:
-	<code>SWT</code>
+<dl>
-	. We intend in the near future to provide some standard "templates" for
+<dt><i>exec</i>
-	typical widget groups that can be cut and pasted into the XML
+<dd>Specifies the command to be launched along with any arguments. Property and attribute substitution is performed on this string.
-	definition. Some of this can already be achieved by importing into the
+<dt><i>directory</i>
-	workspace a provided definition (such as for the PBS resource manager)
+<dd>Specifies the directory in which the command will be executed. If not supplied, the main command directory will be used.
-	and modifying it or borrowing from it. This is, in fact, the approach
+<dt><i>ignoreExitStatus</i>
-	demostrated in the [[JAXBDemo.pdf|tutorial]].
+<dd>Allows the exit status of the command to be ignored. If not set, and the command exits with a non-zero exit status, the entire submission will be aborted.
+<dt><i>wait</i>
+<dd>Specifies that the submission should not proceed until the command has completed. Should only be used if the command completes in a reasonable time.
+</dl>
 </p>
-[[Image:13launch-tab.jpeg]]
+==== The <code>launch-tab</code> Element ====
 <p>
-	As seen from this top-level specification, the Launch Tab can have any
+   The UI component which needs to be configured in conjunction with the
-	number of
+   resource manager control is the <b>Launch Tab</b>, which is actually
-	<code>dynamic</code>
+   the main area of the <b>Resources Tab</b>, one of the six tabs
-	tab-controllers, "dynamic" meaning configurable from the XML. There is
+   comprising the <b>Run Configuration</b> wizard or the seven tabs
-	also a fixed
+   comprising the <b>Debug Configuration</b> wizard. The purpose of this
-	<code>import</code>
+   component is to allow the user to set up the job by providing values
-	controller type; the text of this element is the label/title that will
+   relating to resource requirements or to the properties of the job in
-	appear on the tab. This component allows you to browse the workspace to
+   general. In other words, it is in this component that the user can set
-	set a custom script as the one to run. Its main text area widget is
+   the values on the
-	read-only; any editing of the script must be done through the workspace
+   <code>attribute</code>
-	editor. The browse is local only, so remote scripts should first be
+   elements making up the configurable environment as defined in the XML.
-	imported into a local project in the workspace (using the remote
-	systems import wizard), and then captured by this controller's browse
-	button.
 </p>
-===== Tab Controller =====
+<p>
+   Specifying the composition and layout of this component can be tricky
+   and tedious because it deals with widgets defined by the Eclipse
+   <code>JFace</code>
+   library and with their underlying implementation in
+   <code>SWT</code>
+   . We intend in the near future to provide some standard "templates" for
+   typical widget groups that can be cut and pasted into the XML
+   definition. Some of this can already be achieved by importing into the
+   workspace a provided definition (such as for the PBS resource manager)
+   and modifying it or borrowing from it. This is, in fact, the approach
+   demostrated in the [[Media:JAXBDemo.pdf|tutorial]].
+</p>
 <p>
-	The
+   <b> Throughout the following, it may generally be assumed that
-	<code>dynamic</code>
+      fields such as <i>style</i>, <i>foreground</i>, <i>background</i>,
-	controllers belong to the
+      etc., take string equivalents of the corresponding <code>SWT</code>
-	<code>tab-controller-type</code>
+      constants (e.g., "SWT.LEFT", "SWT.VERTICAL", "SWT.NONE") which can be
-	. It contains an arbitrary number of
+      OR'd (using "|") wherever they would be in Java code. We will <i>not</i>
-	<code>tab-folder</code>
+      exhaustively specify here which constants pertain to which widgets,
-	or
+      because in most cases we have tried to reflect very closely the names
-	<code>composite</code>
+      of the <code>SWT</code> object types and fields, and this information
-	elements; as will be seen shortly, these both have potentially
+      can be found by consulting the Javadocs relative to the widgets in
-	recursive elements, so that one can nest "containers" in the usual
+      question. </b>
-	manner.
 </p>
+[[Image:13launch-tab.jpeg]]
+	<p>
+		As seen from this top-level specification, the Launch Tab can have any
+		number of
+		<code>dynamic</code>
+		tab-controllers, "dynamic" meaning configurable from the XML, which
+		will discuss in more detail in the following paragraphs.
+	</p>
+	<p>
+		There is also a fixed
+		<code>import</code>
+		controller type (implemented as an extension/subclass of the dynamic
+		controller, in order to provide uniform handling of the environment);
+		this component allows you to browse the workspace to set a custom
+		script as the one to run. Its main text area widget is read-only; any
+		editing of the script must be done through the workspace editor. The
+		browse is local only, so remote scripts should first be imported into
+		a local project in the workspace (using the remote systems import
+		wizard), and then captured by this controller's browse button.
+	</p>
+	<p>
+		The configurable part of the import tab is an optional [[#Attribute Viewer|attribute viewer]] for selecting the
+		variables from the environment which you wish to pass to the launch
+		command along with the script. <i>The values set here should
+			correspond to whatever the script contains; they are set for the benefit
+			of the internal consistency of the PTP client, and do not override
+			those hard-coded in script.</i> For an example, see the
+[http://help.eclipse.org/indigo/topic/org.eclipse.ptp.rm.jaxb.pbs.doc.user/html/PBSHelpPage.html#ImportTab   Import PBS Script]
+ tab.
+	</p>
+===== The <code>tab-controller</code> Element =====
+	<p>
+		The
+		<code>dynamic</code>
+		controllers belong to the
+		<code>tab-controller-type</code>
+		, which contains an arbitrary number of
+		<code>tab-folder,</code>
+		<code>composite,</code>
+		<code>widget,</code>
+		<code>browse,</code>
+		<code>button-group,</code>
+		<code>action,</code>
+		or
+		<code>viewer</code>
+		elements; as will be seen shortly, the first two, which are
+		"containers", are potentially recursive elements, so that one can nest
+		widgets in the usual manner. For all intents and purposes, the
+		<code>tab-controller-type</code>
+		is the same as a
+		<code>composite</code>
+		, but without the <i>group</i> attribute and with the extra field for
+		controlling shared widgets.
+	</p>
 [[Image:14tab-controller.jpeg]]
 <p>
-	The <i>includeWidgetValuesFrom</i> field is important. By default, it
+		The <i>includeWidgetValuesFrom</i> field is important. By default, it
-	is set to empty, meaning that the subset of the total resource manager
+		is set to empty, meaning that the subset of the total resource manager
-	environment represented by the controller's widgets is all that will be
+		environment represented by the controller's widgets is all that will
-	passed to the actual job configuration when the job is launched. This
+		be passed to the actual job configuration when the job is launched
-	behavior can be changed by giving a comma-delimited list of the
+		from this tab. This behavior can be changed by giving a
-	<code>title</code>
+		comma-delimited list of the
-	values for other controllers. In this case, the configuration for the
+		<code>title</code>
-	job, when the "Run" button is pressed with this controller being the
+		values for other controllers. In this case, the configuration for the
-	visible one, will also include the values of the widgets on the listed
+		job, when the "Run" button is pressed with this controller being the
-	controllers. <b>NOTE:</b> while this field controls the inclusion or
+		visible one, will also include <i>all the properties or attributes
-	exclusion of attribute and property names in the configuration, their
+			valid for the listed but currently invisible controllers</i>.
-	values are always shared across the controllers (there is only one
+	</p>
-	environment); thus if a variable value is changed on one controller,
-	and this variable is also exposed on another, the value for that
-	variable will also change on the other controller when it becomes
-	visible; similarly, if the user switches off a controller to another
-	having only a subset of the first controller's variables, the variables
-	not represented on the second controller will become undefined or
-	return to their defaults.
-</p>
+<hr>
+<blockquote>
+	<b>Property/Attribute Names vs Values</b>
+       <p>
+	While this field controls the inclusion or exclusion of
+		attribute and property names in the configuration, their values are
+		always shared across the controllers (there is only one environment);
+		thus if a variable value is changed on one controller, and this
+		variable is also exposed on another, the value for that variable will
+		also change on the other controller when it becomes visible;
+		similarly, if the user switches off a controller to another having
+		only a subset of the first controller's variables, the variables not
+		represented on the second controller will become undefined or return
+		to their defaults.</p>
+		<p><b>Note</b> that the union of valid names is not transitively closed;
+		that is, if Tab1 has <i>includeWidgetValuesFrom</i>="Tab2", and Tab2
+		similarly has <i>includeWidgetValuesFrom</i>="Tab3", Tab1 will get
+		only the valid values local to Tab2. <b>Note</b> also, that if there are
+		names which intersect with strictly local widget targets, the local
+		settings (visibility, selection, enablement) still determine whether
+		they belong in the set of valid property/attribute names.</p>
+</blockquote>
+<hr>
 <p>
-	<b> Throughout the following, it may generally be assumed that
+		The <i>showViewConfig</i> and <i>showViewExcluded</i> fields allow
-		fields such as <i>style</i>, <i>foreground</i>, <i>background</i>,
+		control over whether the "View Configuration" and "View Excluded"
-		etc., take string equivalents of the corresponding <code>SWT</code>
+		buttons appear at the bottom of the tab (the "View Script" button appears if
-		constants (e.g., "SWT.LEFT", "SWT.VERTICAL", "SWT.NONE") which can be
+		the configuration contains a
-		OR'd (using "|") wherever they would be in Java code. We will <i>not</i>
+		<code>script</code>
-		exhaustively specify here which constants pertain to which widgets,
+		element). <i>showViewConfig</i> is true by default; this button brings
-		because in most cases we have tried to reflect very closely the names
+		up a scrolling dialog which allows you to inspect the current
-		of the <code>SWT</code> object types and fields, and this information
+		configuration (that is, the one relevant to the visible controller). <i>showViewExcluded</i>
-		can be found by consulting the Javadocs relative to the widgets in
+		is by default false, and should only be set if the resource manager
-		question. </b>
+		processes attributes or properties discovered by a command at run
-</p>
+		time; this dialog allows you to see which ones have been marked
+		invisible; <b>unlike invisible predefined properties or attributes</b>,
+		which are always <i>included</i>, invisible discovered properties or
+		attributes are always <i>excluded</i> from the configuration and thus
+		do not persist across resource manager sessions.
+	</p>
-===== Tab Folders and Tab Items =====
+===== The <code>tab-folders</code> and <code>tab-items</code> Elements =====
 <br>
 [[Image:15tab-folder.jpeg]]
 <p>
-	The
+   The
-	<code>tab-folder</code>
+   <code>tab-folder</code>
-	is a container which looks like a file-folder, with tabs running across
+   is a container which looks like a file-folder, with tabs running across
-	the top or bottom ("SWT.TOP, SWT.BOTTOM"). Each tab label corresponds
+   the top or bottom (<i>style</i> includes SWT.TOP, SWT.BOTTOM). Each tab label corresponds
-	to a
+   to a
-	<code>tab-item</code>
+   <code>tab-item</code>
-	which provides the actual visible content. The
+   which provides the actual visible content. The
-	<code>tooltip</code>
+   <code>tooltip</code>
-	pertains to the entire folder.
+   pertains to the entire folder.
 </p>
+	<p>
+		The
+		<code>tab-item</code>
+		is another all-purpose "container" in which all widget types can be
+		embedded, but distinguished from the [[#Composite/Group | composite]]
+		in that it can only be the child of a
+		<code>tab-folder</code>
+		and that it can have a tooltip (which appears on the tab).
+	</p>
 <p>
-	The
+   As with
-	<code>tab-item</code>
+   <code>composite</code>
-	is an all-purpose "container" in which all widget types can be
+   (see [[#Composite/Group|below]]), the tab folder and tab item take
-	embedded. It is distinguished from the
+   <code>layout</code>
-	<code>composite group</code>
+   and
-	in that it can only be the child of a
+   <code>layout-data</code>
-	<code>tab-folder</code>
+   elements; these closely reflect their
-	, that it can have a tooltip (which appears on the tab).
+   <code>SWT</code>
+   objects and types; for further explanation we refer the reader once
+   again to the relevant Javadocs.
 </p>
-<p>
+===== The <code>layout</code> Element =====
-	As with
-	<code>composite</code>
-	(see [[Composite/Group]]), the tab folder and tab item take
-	<code>layout</code>
-	and
-	<code>layout-data</code>
-	elements; these closely reflect their
-	<code>SWT</code>
-	objects and types; for further explanation we refer the reader once
-	again to the relevant Javadocs.
-</p>
-===== Layout =====
 <br>
 [[Image:17layout.jpeg]]
 <br>
-===== Layout Data =====
+===== The <code>layout-data</code> Element =====
 <br>
 [[Image:18layout-data.jpeg]]
 <br>
-===== Composite/Group =====
+===== The <code>composite</code> Element =====
 <p>
-	The main container, of course, is the
+   The main container, of course, is the
-	<code>composite</code>
+   <code>composite</code>; its sub-type, the group (<i>group</i>="true"), allows for decoration
-	; its sub-type, the group (<i>group</i>="true"), allows for decoration
+   and a title, while the simple composite acts as a widget holder and has
-	and a title, while the simple composite acts as a widget holder and has
+   no visible characteristics, though it does have a layout. As is
-	no visible characteristics, though it does have a layout. As is
+   evident, composites can be contained in composites and tab-items, and
-	evident, composites can be contained in composites and tab-items, and
+   tab-folders can be contained in composites.
-	tab-folders can be contained in composites.
 </p>
@@ Line 1,186: / Line 1,514: @@
 <br>
+===== The <code>control-state</code> Element =====
+	<p>
+		It should be evident that all the XML elements corresponding to UI SWT
+		<code>control</code>
+		types have been given
+		<code>control-state</code>
+		children.
+	</p>
+	[[Image:22control-state.jpeg]]
+	<p>
+		The purpose of these elements is to provide an optional way of setting
+		the <b>enabled</b> and/or <b>visible</b> states of the control on the
+		basis of the state of button (checkbox/radio) selection. One can
+		define two such elements per control, using one of the show/hide or
+		enable/disable pairs. Each of these is associated with a
+		<code>control-state-rule:</code>
+	</p>
+	[[Image:23control-state-rule.jpeg]]
+	<p>The basic rule is defined by its attributes:</p>
+	<ol>
+		<li>the id of the button control to check;</li>
+		<li>whether its selected state should be true or false.</li>
+	</ol>
+	<p>Rules can be nested to an arbitrary depth to form clauses
+		governed by the logical operators.</p>
+	<p>
+		A
+		<code>SelectionListener</code>
+		is created for this target which subscribes to all the controls in the
+		rule; when it receives an event, it evaluates the rule, and if the
+		result is true, takes the associated action on this target.
+	</p>
+	<p>
+		When one wishes to wire two or more such controls in this way, the
+		"trigger" button control must have an id unique to the resource
+		manager configuration. This will be assigned via the <i>buttonId</i>
+		element on the
+		<code>widget</code>
+		corresponding to the button (true also for the buttons nested in a
+		<code>button-group</code>
+		).
+	</p>
+	<p>Here is an example which sets the enabled and visible properties
+		of both a text field and a related checkbox:</p>
+	<pre>
+               &lt;widget type="checkbox" buttonId="002" title="Host list:" attribute="checked002"&gt;
+                  &lt;layout-data&gt;
+                     &lt;grid-data horizontalAlign="SWT.LEFT" verticalAlign="SWT.TOP"/&gt;
+                  &lt;/layout-data&gt;
+                  &lt;control-state&gt;
+                     &lt;show-if button="001" selected="true"/&gt;
+                  &lt;/control-state&gt;
+               &lt;/widget&gt;
+               &lt;widget type="text" style="SWT.V_SCROLL | SWT.BORDER" attribute="host_list"&gt;
+                  &lt;layout-data&gt;
+                     &lt;grid-data horizontalAlign="SWT.FILL" verticalAlign="SWT.FILL" horizontalSpan="2" grabExcessHorizontal="true"
+                        grabExcessVertical="true"/&gt;
+                  &lt;/layout-data&gt;
+                  &lt;control-state&gt;
+                     &lt;show-if button="001" selected="true"/&gt;
+                     &lt;enable-if&gt;
+                        &lt;and&gt;
+                           &lt;rule button="001" selected="true"/&gt;
+                           &lt;rule button="002" selected="true"/&gt;
+                        &lt;/and&gt;
+                     &lt;/enable-if&gt;
+                  &lt;/control-state&gt;
+               &lt;/widget&gt;
+   </pre>
+	<p>Note also that this implicit dependency graph is checked for
+		cycles; the building of the UI tab will be aborted if such a cycle is
+		discovered.</p>
+<hr>
+<blockquote>
+	<b>BEST PRACTICE: CONTROL STATE</b>
+	<p>
+		An invisible property should be assigned to each checkbox which
+		controls the state of other widgets (e.g., <i>checked002</i> in the
+		example above); this will allow you to restore the state of the tab
+		correctly when it is rebuilt or re-initialized.
+	</p>
+</blockquote>
+	<hr>
+===== The <code>widget</code> Element =====
 <p>
-	We now come to the three non-container widget elements with functional
+   We now come to the five non-container widget elements with functional
-	characteristics. The first of these is the
+   characteristics. The first of these is the
-	<code>widget</code>
+   <code>widget</code>
-	, which comprises a number of simple widget types.
+   , which comprises a number of simple widget types.
 </p>
-===== Widget =====
-<br>
 [[Image:19widget.jpeg]]
 <p>
-	The <i>type</i> of the widget can be one of the following:
+   The <i>type</i> of the widget can be one of the following:
 </p>
 <br>
 <table align="center" rules="all">
-	<tr>
+   <tr>
-		<td><code>label</code></td>
+      <td><code>label</code></td>
-		<td>fixed text for display</td>
+      <td>fixed text for display</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>text</code></td>
+      <td><code>text</code></td>
-		<td>area for entering/editing text</td>
+      <td>area for entering/editing text</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>checkbox</code></td>
+      <td><code>checkbox</code></td>
-		<td>corresponds to a boolean choice</td>
+      <td>corresponds to a boolean choice</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>radiobutton</code></td>
+      <td><code>radiobutton</code></td>
-		<td>usually used in connection with <code>button-group</code></td>
+      <td>usually used in connection with <code>button-group</code></td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>spinner</code></td>
+      <td><code>spinner</code></td>
-		<td>provides a continuous range of integer values</td>
+      <td>provides a continuous range of integer values</td>
-	</tr>
+   </tr>
-	<tr>
+   <tr>
-		<td><code>combo</code></td>
+      <td><code>combo</code></td>
-		<td>select from a list of items</td>
+      <td>select from a list of items</td>
-	</tr>
+   </tr>
-	<tr>
-		<td><code>browse</code></td>
-		<td>a text widget followed by a push button allowing the user to
-			browse for directory or file.</td>
-	</tr>
-	<tr>
-		<td><code>action</code></td>
-		<td>a push button associated with a <code>button-action</code>
-			command</td>
-	</tr>
 </table>
 <br>
 <p>
-	Not all attributes and elements of the
+   Not all attributes and elements of the
-	<code>widget</code>
+   <code>widget</code>
-	descriptor are applicable to all types.
+   descriptor are applicable to all types.
 </p>
 <br>
 <table align="center" rules="all">
-	<tr>
+   <tr>
-		<td><i>title</i>
+      <td><i>title</i>
-		</td>
+      </td>
-		<td>used with <code>checkbox</code> and <code>push button</code>
+      <td>used with <code>checkbox</code> and <code>radiobutton</code> widgets for the button label.</td>
-			widgets (<code>browse, action</code>) for the button label</td>
+   </tr>
-	</tr>
+   <tr>
-	<tr>
+      <td><i>style</i>
-		<td><i>style</i>
+      </td>
-		</td>
+      <td>carries the <code>SWT</code> style constants appropriate to the widget type.</td>
-		<td>carries the <code>SWT</code> style constants appropriate to
+   </tr>
-			the widget type</td>
+   <tr>
-	</tr>
+      <td><i>readOnly</i>
-	<tr>
+      </td>
-		<td><i>directory</i>
+      <td>applies to <code>text</code>, <code>spinner</code> and <code>combo</code> widgets; the text area in this case is not editable.</td>
-		</td>
+   </tr>
-		<td>limit the <code>browse</code> widget to directories</td>
+   <tr>
-	</tr>
+      <td><i>attribute</i>
-	<tr>
+      </td>
-		<td><i>uri</i>
+      <td>applies to all widgets except <code>label</code>; the name of the property or attribute whose value should be set on the basis of the widget's "selected" value or text<br> <b>Note:</b> use of this attribute is mutually exclusive with the <code>fixed-</code> and <code>dynamic-text</code> elements.</td>
-		</td>
+   </tr>
-		<td>have the <code>browse</code> widget set a URI as the text
+   <tr>
-			(default is false, in which case the text is an absolute path)</td>
+      <td><code>items-from</code>
-	</tr>
+      </td>
-	<tr>
+      <td>applies to the <code>combo</code> widget; indicates a property or attribute <i>value</i> (of type <code>java.util.Collection</code>) to use to populate the items.<br> <b>Note:</b> if a combo widget saves its value to an attribute and the present element on the widget descriptor is not set, the attribute will be checked for non-empty <code>choice</code> or <code>items-from</code> values, which then will be used to populate the combo items; the <code>items-from</code> field on the widget is for added flexibility (for instance, linking up a target property with another property providing the list of choices).</td>
-		<td><i>localOnly</i>
+   </tr>
-		</td>
+   <tr>
-		<td>limit the <code>browse</code> widget to the local file system
+      <td><code>fixed-text</code>
-			(otherwise, the choice of local or remote is enabled)</td>
+      </td>
-	</tr>
+      <td>applies to the <code>label</code> and (read-only) <code>text</code> widgets; text is resolved once and remains constant thereafter.</td>
-	<tr>
+   </tr>
-		<td><i>readOnly</i>
+   <tr>
-		</td>
+      <td><code>dynamic-text</code>
-		<td>applies to <code>text</code> and <code>combo</code> widgets;
+      </td>
-			the text area in this case is not editable</td>
+      <td>applies only to (read-only) <code>text</code> widgets; text is resolved each time there is an update of a value from the Launch Tab.</td>
-	</tr>
+   </tr>
-	<tr>
-		<td><i>saveValueTo</i>
-		</td>
-		<td>applies to all widgets except <code>label</code> and <code>action</code>;
-			the name of the property or attribute whose value should be set on
-			the basis of the widget's "selected" value or text<br> <b>Note:</b>
-			use of this attribute is mutually exclusive with the <code>fixed-</code>
-			and <code>dynamic-text</code> elements.</td>
-	</tr>
-	<tr>
-		<td><i>translateBooleanAs</i>
-		</td>
-		<td>applies to the <code>checkbox</code> widget; a
-			comma-delimited pair indicating a string equivalent for the boolean
-			to be used as the actual value for the property or attribute set by
-			the widget<br> <i>Example</i>: "YES,NO" would mean true yields
-			the first string and false the second</td>
-	</tr>
-	<tr>
-		<td><code>sub-layout-data</code>
-		</td>
-		<td>applies to the <code>browse</code> widget; applies to the
-			button component, whereas <code>layout-data</code> applies to the
-			text area</td>
-	</tr>
-	<tr>
-		<td><code>items-from</code>
-		</td>
-		<td>applies to the <code>combo</code> widget; indicates a
-			property or attribute <i>value</i> (of type <code>java.util.Collection</code>)
-			to use to populate the items<br> <b>Note:</b> if a combo widget
-			saves its value to an attribute and the present element on the widget
-			descriptor is not set, the attribute will be checked for non-empty <code>choice</code>
-			and <code>items-from</code> values, which then will be used to
-			populate the combo items; the <code>items-from</code> field on the
-			widget is for added flexibility (for instance, linking up a target
-			property with another property providing the list of choices)</td>
-	</tr>
-	<tr>
-		<td><code>fixed-text</code>
-		</td>
-		<td>applies to the <code>label</code> and (read-only)<code>text</code>
-			widgets; text is resolved once and remains constant thereafter</td>
-	</tr>
-	<tr>
-		<td><code>dynamic-text</code>
-		</td>
-		<td>applies only to (read-only)<code>text</code> widgets; text is
-			resolved each time there is an update of a value from the Launch Tab</td>
-	</tr>
-	<tr>
-		<td><code>button-action</code>
-		</td>
-		<td>applies to the <code>action</code> widget; the <code>button-action-type</code>
-			specifies the name of a command (<code>action</code>) defined in the
-			XML; this can be one of the <code>start-up</code>, <code>shut-down</code>,
-			commands, or a special <code>button-action</code> command (<b>note
-		</b>that it cannot, however, be a <code>-job</code> command, as the Launch
-			Tab is not coupled to any submitted jobs); the <code>button-action-type</code>
-			also indicates whether the variable affected by the command should be
-			cleared prior to the command execution, and whether the launch tab
-			should be refreshed (default is true) afterwards</td>
-	</tr>
 </table>
 <br>
 <p>Widget foreground, background (color) and font are in principle
-	configurable, though the font setting may not always work. The tooltip
+   configurable, though the font setting may not always work. The tooltip
-	will appear on the widget whereever it usually does.</p>
+   will appear on the widget whereever it usually does.</p>
-===== Button Group =====
+===== The <code>button-group</code> Element =====
-<br>
 [[Image:20button-group.jpeg]]
 <p>
-	The button group provides an explicit selection of a single value based
+   The button group provides an explicit selection of a single value based
-	on a mutually exclusive set of button labels. Just as with the
+   on a mutually exclusive set of button labels. Just as with the
-	<code>widget</code>
+   <code>widget</code>
-	, one uses <i>saveValueTo</i> to indicate what property or attribute
+   , one uses <i>attribute</i> to indicate what property or attribute
-	value to set. The
+   value to set. The
-	<code>button</code>
+   <code>button</code>
-	elements, which can be arranged horizontally or vertically using the <i>style</i>
+   elements, which can be arranged horizontally or vertically using the <i>style</i>
-	attribute ("SWT.HORIZONTAL","SWT.VERTICAL"), will be
+   attribute (SWT.HORIZONTAL, SWT.VERTICAL), will be
-	<code>widget</code>
+   <code>widget</code>
-	elements whose type is either <i>checkbox</i> or <i>radiobutton</i> (if
+   elements whose type is either <i>checkbox</i> or <i>radiobutton</i> (if
-	a different widget type is given, an error will occur); the <i>saveValueTo</i>
+   a different widget type is given, an error will occur); the <i>attribute</i>
-	attribute on the individual button will be ignored. Radio buttons are
+   attribute on the individual button will be ignored. Radio buttons are
-	"sticky" in the sense that once they are selected, they cannot be
+   "sticky" in the sense that once they are selected, they cannot be
-	"unchecked" until a different button in the group is selected. It is
+   "unchecked" until a different button in the group is selected. It is
-	thus advisable always to provide a default value for the property or
+   thus advisable always to provide a default value for the property or
-	attribute associated with the group value if radio buttons are used. If
+   attribute associated with the group value if radio buttons are used. If
-	check boxes are used, they can be individually checked on and off (one
+   check boxes are used, they can be individually checked on and off (one
-	at a time), thus allowing for the value of the group to be potentially
+   at a time), thus allowing for the value of the group to be potentially
-	empty.
+   empty.
 </p>
 <p>Below is an example of a button group using radio buttons,
-	disposed vertically.</p>
+   disposed vertically.</p>
 <br>
 [[Image:ButtonGroup.jpeg]]
 <br>
-===== Attribute Viewer =====
 <br>
+===== The <code>browse</code> Element =====
+[[Image:190browse.jpeg]]
+	<p>
+		A special widget-pair -- a text widget followed by a push button --
+		allowing the user to browse directories or files from either the
+		local or remote file system. Aside from the fields shared with the
+		<code>widget</code>
+		type, we note the following:
+	</p>
+	<br>
+	<table align="center" rules="all">
+		<tr>
+			<td><i>directory</i>
+			</td>
+			<td>limit the <code>browse</code> widget to directories.</td>
+		</tr>
+		<tr>
+			<td><i>uri</i>
+			</td>
+			<td>have the <code>browse</code> widget set a URI as the text
+				(default is false, in which case the text is an absolute path).</td>
+		</tr>
+		<tr>
+			<td><i>localOnly</i>
+			</td>
+			<td>limit the <code>browse</code> widget to the local file
+				system (otherwise, the choice of local or remote is enabled).</td>
+		</tr>
+	</table>
+	<br>
+	<p>
+		<code>layout</code>
+		and
+		<code>control-state</code>
+		are specified as belonging to either the text or the button. Style applies only to the text widget.
+	</p>
+===== The <code>action-button</code> Element =====
+[[Image:191push-button.jpeg]]
+	<p>
+		A push button which is associated with a
+		<code>button-action</code>
+		command. The
+		<code>button-action</code>
+		specifies the name of a command (
+		<code>action</code>
+		) defined in the XML; this can be one of the
+		<code>start-up</code>
+		or
+		<code>shut-down</code> commands, or a special
+		<code>button-action</code>
+		command (<b>note </b>that it cannot, however, be a [[#BEST PRACTICE: WAIT-FOR-ID|<b>(uu)id</b>]]
+		command, as the Launch Tab is not coupled to any submitted jobs); the
+		<code>button-action-type</code>
+		also indicates whether the variable affected by the command should be
+		cleared prior to the command execution, and whether the launch tab
+		should be refreshed (default is true) afterward.
+	</p>
+===== The <code>attribute-viewer</code> Element =====
 [[Image:21attribute-viewer.jpeg]]
 <p>
-	The last of the widget types, the
+		The last of the widget types, the
-	<code>attribute-viewer</code>
+		<code>attribute-viewer</code>
-	, displays visible properties and attributes in a table or tree viewer,
+		, displays visible properties and attributes in a table or tree
-	with the value columns as editable widgets. The type of cell editor is
+		viewer, with the value columns as editable widgets. The type of cell
-	determined implicitly here by the type of the property or attribute;
+		editor is determined implicitly here by the type of the property or
-	the presence of a combo editor is determined by an attribute whose
+		attribute; the presence of a combo editor is determined by an
-	<code>choice</code>
+		attribute whose
-	or
+		<code>choice</code>
-	<code>items-from</code>
+		or
-	element is defined. (This is the only viable possibility for a combo in
+		<code>items-from</code>
-	the table, as there would also need to be a target to save the selected
+		element is defined. (This is the only viable possibility for a combo
-	item to, viz., the value of this same attribute).
+		in the table, as there would also need to be a target to save the
-</p>
+		selected item to, viz., the value of this same attribute.)
+	</p>
-<p>Each entry in the viewer is provided with a checkbox to indicate
+	<p>Each entry in the viewer is provided with a checkbox to indicate
-	which attributes have been chosen. A checkbox is also placed above the
+		which attributes have been chosen (and thus marked valid).
-	viewer to allow for filtering the unchecked items. The viewer outputs
+                A checkbox is also placed above the
-	its checked values to the environment (and on update removes unselected
+		viewer to allow for filtering the unchecked items. The viewer outputs
-	ones). The name of the viewer is also stored against a string of all
+		its values to the environment. The name of the viewer is also stored against a
-	selected values, constructed from a template ( further discussion at
+		string of all selected values, constructed from a template (further
-	the end of this section).</p>
+		discussion at the end of this section).</p>
-<p>
+	<p>
-	The viewer <i>type</i> can be either "table" or "tree"; in the case of
+		The viewer <i>type</i> can be either "table" or "tree"; in the case of
-	the tree, the type, default, status and description fields of the
+		the tree, the type, default, status and description fields of the
-	attribute or property are displayed as collapsible child rows hanging
+		attribute or property are displayed as collapsible child rows hanging
-	off the main name entry. Either table or tree can have up to seven
+		off the main name entry. Either table or tree can have up to <b>six</b>
-	columns assigned to the various fields of the attribute. These are
+		columns whose names/types are based on the following property and
-	defined by the
+		attribute fields:
-	<code>column-data</code>
+	</p>
-	elements. Tooltips for the column appear when hovering over the column
-	header. (<i>A caveat on the foreground and background settings for
-		this type: they are exposed here but may not work as expected: these
-		are currently applied to the column through the cell label provider;
-		foreground does not change the font color but applies the color to the
-		entire cell.</i>)
-</p>
-<p>
+	<table align="center" rules="all">
-	<i>sort</i> is done on the name field/column. This is a toggle sort,
+		<tr>
-	such that alternate clicks reverse the sort direction. <i>headerVisible</i>
+			<td><b>Name</b>
-	and <i>linesVisible</i> control the look of the table. <i>tooltipEnabled</i>
+			</td>
-	enables column-viewer label provider tooltip support. The tooltip
+		</tr>
-	defined for an attribute will appear on the row of the table or the
+		<tr>
-	parent node of the tree. The attribute description will also appear as
+			<td><b>Value</b>
-	a tooltip on the description columns if they exist.
+			</td>
-</p>
+		</tr>
+		<tr>
+			<td><b>Type</b>
+			</td>
+		</tr>
+		<tr>
+			<td><b>Status</b>
+			</td>
+		</tr>
+		<tr>
+			<td><b>Default</b>
+			</td>
+		</tr>
+		<tr>
+			<td><b>Description</b>
+			</td>
+		</tr>
+	</table>
+	<br>
+	<p>
+		These can appear in any order. There is a slight difference in how
+		these columns are treated, depending upon whether <i>tree</i> or <i>table</i>
+		is chosen as type. For <b>table</b> type, the value of the field
+		simply appears in the corresponding column:
+	</p>
-<p>
+	<br>
-	Control over the actual contents of the viewer is achieved through the
+	[[Image:Table.jpg]]
-	<code>viewer-items-type</code>
+	<br>
-	; one can here explicitly list properties or attributes for inclusion
-	using the
-	<code>include</code>
-	element, or one can set one or both of the <i>all</i> attributes; in
-	the latter case, attributes and properties can be excluded from the set
-	using
-	<code>exclude</code>
-	.
-</p>
-<p>
+	<p>
-	A property bearing the name given to this widget is written to the
+		In the <b>tree</b> type, the
-	environment at each update/refresh; the value of the property is a
+		<code>name</code>
-	string formed from the selected (checked) rows of the table using the
+		field displays all fields except
-	<code>template-type</code>
+		<code>value</code>
-	(the
+		as children;
-	<code>value</code>
+		<code>value</code>
-	element). The <i>pattern</i> can be any literal string embedding <i>@name</i>
+		always appears as a separate column on the same row as the name (it
-	and <i>@value</i> tags as placeholders for the row's name and value;
+		contains the cell editor). If the viewer also has the column
-	for instance, '@name="@value"' would generate a string of name-value
+		corresponding to the child field, that field value appears in that
-	assignments. The default <i>separator</i> between patterned segments is
+		column;
-	a single space.
+	</p>
-</p>
+	<br>
+	[[Image:Tree1.jpg]]
+	<br>
+	<p>
+		Otherwise, the child fields display their value in the value column
+		along with the actual
+		<code>value</code>
+		field:
+	</p>
+	<br>
+	[[Image:Tree2.jpg]]
+	<br>
+	<p>
+		Tooltips for the column appear when hovering over the column header. (<i>A
+			caveat on the foreground and background settings for this type: they
+			are exposed here but may not work as expected: these are currently
+			applied to the column through the cell label provider; foreground
+			does not change the font color but applies the color to the entire
+			cell.</i>)
+	</p>
+	<p>
+		<i>sort</i> is done on the name field/column. This is a toggle sort,
+		such that alternate clicks reverse the sort direction. <i>headerVisible</i>
+		and <i>linesVisible</i> control the look of the table. <i>tooltipEnabled</i>
+		enables column-viewer label provider tooltip support. The tooltip
+		defined for an attribute will appear on the row of the table or the
+		parent node of the tree. The attribute description will also appear as
+		a tooltip on the description columns if they exist.
+	</p>
+	<p>
+		Control over the actual contents of the viewer is achieved through the
+		<code>viewer-items-type</code>
+		; one can here explicitly list properties or attributes for inclusion
+		using the
+		<code>include</code>
+		element, or one can set one or both of the <i>all</i> attributes; in
+		the latter case, attributes and properties can be excluded from the
+		set using
+		<code>exclude</code>
+		. The <i>initialCheckAll</i> field, which defaults to true, indicates
+		that if the attribute or property is new or undefined from a previous
+		session, it will appear as checked; turn this off if you want new
+		attributes or properties to appear at first as unchecked.
+	</p>
+	<p>
+		A property bearing the name given to this widget is written to the
+		environment at each update/refresh; the value of the property is a
+		string formed from the selected (checked) rows of the table using the
+		<code>template-type</code>
+		(the
+		<code>value</code>
+		element). The <i>pattern</i> can be any literal string embedding <i>@name</i>
+		and <i>@value</i> tags as placeholders for the row's name and value;
+		for instance, '@name="@value"' would generate a string of name-value
+		assignments. The default <i>separator</i> between patterned segments
+		is a single space.
+	</p>
-<p>Below is an example of the tree version of the attribute viewer.
+	<p>Below is a fuller example of the tree version of the attribute
-	The text area above it displays the current value of its templated
+		viewer. The text area above it displays the current value of its
-	pattern string. The child rows reporting status, default, etc., cannot
+		templated pattern string. The child rows reporting status, default,
-	be selected or unselected (only the parent row can). Hovering over the
+		etc., cannot be selected or unselected (only the parent row can).
-	name (here walltime) displays the tooltip.</p>
+		Hovering over the name (here walltime) displays the tooltip.</p>
 <br>
 [[Image:AttribViewerTree.jpeg]]
 <br>
-=== The Monitor Data Type ===
+=== The <code>monitor-data</code> Element ===
 <p>At present, the monitor component has less elements to define;
-	this may change in future releases, when certain hard-coded features of
+   this may change in future releases, when certain hard-coded features of
-	the driver may be handed over to the client for configuration.</p>
+   the driver may be handed over to the client for configuration.</p>
 [[Image:22monitor.jpeg]]
 <p>
-	This component can be furnished with its own set of properties
+   This component can be furnished with its own set of properties
-	(currently unused). What is necessary to set at present is only the <i>schedulerType</i>
+   (currently unused). What is necessary to set at present is only the <i>schedulerType</i>
-	attribute (e.g., "PBS"). <i>refreshFrequencyInSeconds</i> defaults to
+   attribute. It specifies the target system's job scheduler. Currently supported batch systems are
-, but this can be changed according to the needs of the user. Be
+   COBALT_BG, GRIDENGINE, LL, LL_BG, LSF, OPENMPI, PBS, PE, SLURM, SLURM_ALPS, TORQUE and TORQUE_ALPS.
-	aware that too low a setting will probably not work, as the command on
+   These names can be used as attribute values for <i>schedulerType</i>.
-	an average sized system will take upwards of five seconds to complete
+   <i>refreshFrequencyInSeconds</i> defaults to
-	(XML is being streamed to the driver and a
+, but this can be changed according to the needs of the user. Be
-	<code>diff</code>
+   aware that too low a setting will probably not work, as the command on
-	is sent back to the client ).
+   an average sized system will take upwards of five seconds to complete
+   (XML is being streamed to the driver and a
+   <code>diff</code>
+   is sent back to the client ).
 </p>
 <p>
-	If the
+   If the
-	<code>driver</code>
+   <code>driver</code>
-	element is configured, then the default behavior, which is to stage
+   element is configured, then the default behavior, which is to stage
-	over the necessary driver scripts to the <i>.eclipsesettings</i>
+   over the necessary driver scripts to the <i>.eclipsesettings</i>
-	directory in the user's home, is overridden by connecting to a
+   directory in the user's home, is overridden by connecting to a
-	pre-existent installation. This can be specified either using the
+   pre-existent installation. This can be specified either using the
-	<code>url</code>
+   <code>url</code>
-	, or a combination of the
+   , or a combination of the
-	<code>name</code>
+   <code>name</code>
-	,
+   ,
-	<code>path</code>
+   <code>path</code>
-	and
+   and
-	<code>args</code>
+   <code>args</code>
-	elements.
+   elements.
 </p>

COMMAND TYPE	WAIT FOR ID	TOKENIZER STATES
batch	yes	SUBMITTED
interactive	no
interactive	yes	RUNNING
interactive, open	no
interactive, open	yes	(SUBMITTED,) RUNNING
interactive, open, pseudo-terminal	no
interactive, open, pseudo-terminal	yes	(SUBMITTED,) RUNNING

Breadcrumbs

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Difference between revisions of "PTP/designs/Resource Manager Configuration"

Latest revision as of 07:50, 5 February 2014

Contents

PTP 6.0.0 Changes

PTP 5.0.1 Changes

Introduction

Configuring/Customizing the Resource Manager

Preferences

The JAXB Configurable Resource Manager XML Schema

The resource-manager-data Element

The control-data Element

Resource Manager Environment

The property and attribute Elements

Property and Attribute Values

The regex and file-match Elements

The managed-file Element

The script Element

The line Element

The arg Element

"Just-in-time" resolution of @jobId and managed-file paths

The command Element

The arg Element

The input Element

The environment Type

Stream Parsers

The type Element

The target Element

The match Element

The entry Element

The test Element

Tokenizer Examples

The pre-launch-cmd and post-launch-cmd Elements

The launch-tab Element

The tab-controller Element

The tab-folders and tab-items Elements

The layout Element

The layout-data Element

The composite Element

The control-state Element

The widget Element

The button-group Element

The browse Element

The action-button Element

The attribute-viewer Element

The monitor-data Element

The `resource-manager-data` Element

The `control-data` Element

The `property` and `attribute` Elements

The `regex` and `file-match` Elements

The `managed-file` Element

The `script` Element

The `line` Element

The `arg` Element

"Just-in-time" resolution of @jobId and `managed-file` paths

The `command` Element

The `arg` Element

The `input` Element

The `environment` Type

The `type` Element

The `target` Element

The `match` Element

The `entry` Element

The `test` Element

The `pre-launch-cmd` and `post-launch-cmd` Elements

The `launch-tab` Element

The `tab-controller` Element

The `tab-folders` and `tab-items` Elements

The `layout` Element

The `layout-data` Element

The `composite` Element

The `control-state` Element

The `widget` Element

The `button-group` Element

The `browse` Element

The `action-button` Element

The `attribute-viewer` Element

The `monitor-data` Element