Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

PTP/designs/resource manager xsd

< PTP‎ | designs
Revision as of 13:40, 5 March 2011 by Arossi.ncsa.uiuc.edu (Talk | contribs) (Resource Manager XSD)

Resource Manager XSD

The following is a draft of the proposed Resource Manager XML Schema (XSD). It has been annotated internally where I felt some explanation necessary.

Note that I have not attempted to figure out yet what the monitoring part will look like or what the control and monitoring sections may need to interact; it may also be the case that some things inside the control node should come up a level.

Revised version 1 (17/01/2011):

Revised version 2 (19/01/2011): I have adjusted the schema to accommodate the points raised by our discussions on ptp-dev. More tomorrow, I'm sure. -alr

Revised version 3 (23/01/2011): A few more changes I discovered needed to be made as I work through the XML instance for PBS. -alr

Revised version 4 (24/01/2011): Numerous tweaks. I have added a few specialized widgets and groups involving job attributes whose construction is standardized; this is for economy of expression more than anything else. -alr

Revised version 5 (25/01/2011): Changed how to specify the management of files as per discussion. -alr

Revised version 6 (27/01/2011): Added browse enumeration types; switched script commands to arglists; other minor corrections. -alr

Revised version 7 (09/02/2011): Modified UI elements to expose more granularity (grid-layout, grid-data); corrected PBS xml. -alr

Revised version 8 (22/02/2011): Added 'set' element to parser to take care of injection on JobAttributes; added value to Property. -alr

Revised version 9 (27/02/2011): Numerous structural changes. -alr

Revised version 10 (27/02/2011): More structural changes. Because the page content was becoming long, I've taken down the XSD; it can be examined in the data directory of the org.eclipse.ptp.rm.jaxb.core plugin, along with the example PBS xml. -alr


A working design for the Configurable Regex Stream Tokenizer (05/03/2011):

     An explanation of the ConfigurableRegexTokenizer.
 
     1. Tokenizer loops through its read commands until the stream is exhausted.
 
     2. There are two read modes:
        a) reads until it matches (forward matching, all="false", default);
        b) reads until the stream is closed (all="true"), saving the most recent
           number of delimited parts (save="#"), then reads through each of these,  
           attempting to match each part (in this case, there will be only one  
           read element for the tokenizer).
 
        Read either looks for a delimiting character, or reads a given
        number of characters; in the latter case, "maxMatchLength" will be
        used to determine the buffer size (2*maxMatchLength).
 
        There are two match modes:
        a) mode="or" [default], read returns when one match is satisfied;
        b) mode="and", read returns when all matches are satisfied (in order).
 
        A read can be without target or actions (and even without a regex), in
        which case it acts like a "skip" command.
 
     3. Regex can be used to split the input (split="true"), or applied to capture 
        groups; all the Java flags are available (using strings that look like 
        the or'd int expressions, e.g. "CASE_INSENSITIVE | UNIX_LINES" ).
 
     4. Target is an action either dereferencing or creating the object:
        a) "ref" indicates it should be retrieved from the RM env map (an exception
           is raised if it does not exist);
        b) if there is no name, the "type" field must be set indicating which
           kind of object to create (property, attribute);
        c) if "idFrom" is used, it indicates the group or split index of the
           match which identifies the instance;
        d) anonymously created properties or attributes can be used, but
           one at a time; the most recently created is the one acted on; 
           in other words, this mode is only possible when the parts of the 
           object to be matched are known to be streamed in order.
 
     5. a) Set assigns the given match group or index to the field indicated;
           alternately, one can assign a string to be evaluated 
           (a boolean or numerical expression);
        b) Add takes the indicated groups or indices and adds them to the 
           list<string> assigned to the field;
        c) Append concatenates the groups or indices and appends the string
           to the current value of the field;
        d) Put takes the indicated groups or indices and puts them as key-value
           pairs into the map<string,string> assigned to the field.

XSD SNIPPET

	<xs:element name="stream-parser">
		<xs:annotation>
			<xs:documentation>
				These are attached to the stdout or stderr streams in order to capture the output of the command and add values
				into the resource manager environment. If displayStdout or displayStderr of the command is true, the stream will be passed on to an
				output stream will also be sent to the terminal.
			</xs:documentation>
		</xs:annotation>
		<xs:complexType>
			<xs:sequence>
				<xs:element ref="tokenizer"/>
			</xs:sequence>
			<xs:attribute name="name" use="required" type="xs:string"/>
			<xs:attribute name="stderr" type="xs:boolean" default="false"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="tokenizer">
		<xs:complexType>
			<xs:sequence>
				<xs:choice>
					<xs:element name="type" type="xs:string">
						<xs:annotation>
							<xs:documentation>
								Place to provide a custom tokenizer; type is the extension id contributing to
								"org.eclipse.ptp.rm.jaxb.core.streamParserTokenizer".
							</xs:documentation>
						</xs:annotation>
					</xs:element>
					<xs:sequence>
						<xs:element ref="read" maxOccurs="unbounded"/>
					</xs:sequence>
				</xs:choice>
			</xs:sequence>
		</xs:complexType>
	</xs:element>
	<xs:element name="read">
		<xs:complexType>
			<xs:sequence>
				<xs:choice>
					<xs:element ref="match" minOccurs="0" maxOccurs="unbounded"/>
				</xs:choice>
			</xs:sequence>
			<xs:attribute name="delim" type="xs:string"/>
			<xs:attribute name="includeDelim" type="xs:boolean" default="false"/>
         <xs:attribute name="maxMatchLen" type="xs:int" default="0"/>
			<xs:attribute name="all" type="xs:boolean" default="false"/>
			<xs:attribute name="save" type="xs:int" default="0"/>
			<xs:attribute name="mode" default="or">
            <xs:simpleType>
               <xs:restriction base="xs:string">
                  <xs:enumeration value="or"/>
                  <xs:enumeration value="and"/>
               </xs:restriction>
            </xs:simpleType>
         </xs:attribute>
		</xs:complexType>
	</xs:element>
	<xs:element name="match">
		<xs:complexType>
			<xs:sequence>
				<xs:element ref="regex" minOccurs="0"/>
				<xs:element ref="target" minOccurs="0"/>
				<xs:element ref="set" minOccurs="0" maxOccurs="unbounded"/>
				<xs:element ref="add" minOccurs="0" maxOccurs="unbounded"/>
				<xs:element ref="put" minOccurs="0" maxOccurs="unbounded"/>
				<xs:element ref="append" minOccurs="0" maxOccurs="unbounded"/>
			</xs:sequence>
			<xs:attribute name="errorOnMiss" type="xs:boolean" default="false"/>
			<xs:attribute name="target" type="xs:string"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="target">
		<xs:complexType>
		   <xs:attribute name="ref" type="xs:string"/>
			<xs:attribute name="type" type="xs:string"/>
			<xs:attribute name="idFrom" type="xs:int"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="regex">
      <xs:complexType mixed="true">
         <xs:attribute name="flags" type="xs:string"/>
         <xs:attribute name="split" type="xs:boolean" default="false"/>
      </xs:complexType>
   </xs:element>
	<xs:element name="set">
		<xs:complexType>
			<xs:attribute name="field" type="xs:string"/>
			<xs:attribute name="group" type="xs:int" default="0"/>
			<xs:attribute name="index" type="xs:int" default="0"/>
			<xs:attribute name="expression" type="xs:string"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="add">
		<xs:complexType>
			<xs:attribute name="field" type="xs:string"/>
			<xs:attribute name="groups" type="xs:string"/>
			<xs:attribute name="indices" type="xs:string"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="append">
		<xs:complexType>
			<xs:attribute name="field" type="xs:string"/>
			<xs:attribute name="groups" type="xs:string"/>
			<xs:attribute name="indices" type="xs:string"/>
			<xs:attribute name="separator" type="xs:string"/>
			<xs:attribute name="startTag" type="xs:string"/>
			<xs:attribute name="endTag" type="xs:string"/>
		</xs:complexType>
	</xs:element>
	<xs:element name="put">
		<xs:complexType>
			<xs:attribute name="field" type="xs:string"/>
			<xs:attribute name="keyGroups" type="xs:string"/>
			<xs:attribute name="keyIndices" type="xs:string"/>
			<xs:attribute name="valueGroups" type="xs:string"/>
			<xs:attribute name="valueIndices" type="xs:string"/>
		</xs:complexType>
	</xs:element>

EXAMPLES

<!-- EXAMPLE 1: output is a list of line-separated queue names to be assigned
	             to the known property "available-queues" -->
<tokenizer>
	<read delim="\n">
		<match>
			<regex>.*</regex>
			<target ref="available_queues"/>
			<append field="value" separator="," groups="0"/>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 2: output is to be searched for its final line which should
	             contain a job id of the form "[digits].[chars]" -->
<tokenizer>
	<read delim="\n" all="true" save="1">
		<match>
			<regex>([\d]*)[.].*</regex>
			<target ref="jobId"/>
			<set field="value" group="1"/>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 3: indeterminate number and order of lines containing parts
	             of attribute definitions, but each line bearing a distinct
	             id (e.g., openMPI attribute discovery) -->
<tokenizer>
	<read delim="\n">
		<match>
			<regex>mca:.*:param:([^:]*)value:(.*)</regex>
			<target type="attribute" idFrom="1"/>
			<set field="name" group="1"/>
			<set field="visible" expression="true"/>
			<set field="value" group="2"/>
		</match>
		<match>
			<regex>mca:.*:param:([^:]*):status:(.*)</regex>
			<target type="attribute" idFrom="1"/>
			<set field="status" group="2"/>
			<set field="read-only" expression="${status} == 'read-only'"/>
		</match>
		<match>
			<regex>mca:.*:param:([^:]*):help:(.*)</regex>
			<target type="attribute" idFrom="1"/>
			<set field="toolTip" group="2"/>
		</match>
		<match>
			<regex>(.*):([^:]*)</regex>
			<target type="attribute" idFrom="1"/>
			<set field="name" group="1"/>
			<set field="value" group="2"/>
			<set field="visible" expression="true"/>
			<set field="read-only" expression="true"/>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 4a: indeterminate number of property definitions, but grouped
	              by delimiter -->
<tokenizer>
	<read delim="\n" mode="and">
		<match>
			<regex>&lt;new-property&gt;</regex>
			<target type="property"/>
		</match>
		<match>
			<regex>.*</regex>
			<set field="name" group="0"/>
		</match>
		<match>
			<regex>.*</regex>
			<set field="value" group="0"/>
		</match>
		<match>
			<regex>&lt;/new-property&gt;</regex>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 4b: similar to 4a, but without delimiter (implicit ordering) -->
<tokenizer>
	<read delim="\n" mode="and">
		<match>
			<regex>.*</regex>
			<target type="attribute"/>
			<set field="name" group="0"/>
		</match>
		<match>
         <regex>.*</regex>
         <set field="type" group="0"/>
      </match>
		<match>
         <regex>.*</regex>
         <set field="description" group="0"/>
      </match>
      <match>
         <regex>.*</regex>
         <set field="tooltip" group="0"/>
      </match>
		<match>
			<regex>.*</regex>
			<set field="value" group="0"/>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 5: indeterminate number of property definitions, but on single line -->
<tokenizer>
	<read delim="\n">
		<match>
			<regex>&lt;name&gt;(.*)&lt;/name&gt;&lt;value&gt;(.*)&lt;/value&gt;</regex>
			<target type="property"/>
			<set field="name" group="1"/>
			<set field="value" group="2"/>
		</match>
	</read>
</tokenizer>
 
<!-- EXAMPLE 6: looking for values interpersed in the stream but which will
                not exceed 1024 chars -->
<tokenizer>
   <read maxMatchLen="1024">
      <match>
         <regex>&lt;job&gt;([\d]*):([\w]*)&lt;/job&gt;</regex>
         <target ref="jobStates"/>
         <put field="value" keyGroups="1" valueGroups="2"/>
      </match>
   </read>
</tokenizer>

Back to the top