Difference between revisions of "SMILA/Documentation/JobManagerConfiguration"

Latest revision as of 08:46, 11 November 2011

Available since SMILA 0.9!

Configuring the JobManager

The JobManager is configured via the ClusterConfig service. With the "simple" ClusterConfig service, it uses one properties in the "taskmanager" section of clusterconfig.json:

{
  ...
  "taskmanager": {
    ...
    "maxRetries": 10,
    ...
  },
  ...
}

• maxRetries: Used to decide how often a task should be retried that has failed with an RECOVERABLE_ERROR, either because the "timeToLive" was exceeded or the worker itself reported such an error. If the retry limit is reached, the task will finally fail with a FATAL_ERROR.

See SMILA/Documentation/TaskManager#Configuration for more details.

Predefined JobManager entities

The configuration directory org.eclipse.smila.jobmanager can contain 5 files:

• dataObjectTypes.json: DataObject types for buckets
• workers.json: Description of available workers
• buckets.json Predefined persistent buckets.
• workflows.json: Predefined workflow definitions.
• jobs.json: Predefined job definitions.

These files contain JobManager entity definitions that are available immediately after system startup. We call them "predefined" elements. DataObject types and workers can currently be only predefined, for the other entities additional definitions can be added and updated via the ReST API.

The structure of these files is similar, e.g. the standard dataObjectTypes.json looks like this:

{
   "dataObjectTypes": 
   [
      {
      	"name": "recordBulks", 
        "persistent": {
           "store": "${store}",
           "object": "${_bucketName}/${_uuid}"
        },
        "transient": {
           "store": "${tempStore}",
           "object": "${_bucketName}/${_uuid}"
        }
      }
   ]
}

Each file contains a single JSON object with a single key identical to the filename without suffix ("dataObjectTypes", "workers", "buckets", "workflows", "jobs"). The value for this key is a sequence of appropriate entity definitions, see the respective page for details:

• SMILA/Documentation/DataObjectTypesAndBuckets
• SMILA/Documentation/WorkerAndWorkflows
• SMILA/Documentation/JobDefinitions

Predefined entities cannot be changed or deleted using the ReST API, but only by changing the configuration files and restarting SMILA. Therefore they are marked with a "readOnly": true property when retrieved from the JobManager. You can of course make copies of them with different names to create variants.

If an entity cannot be parsed or is not valid (e.g. a job definition using an undefined workflow) you will find error messages in the SMILA log file.