Difference between revisions of "PTP/designs/scalability"

Revision as of 17:26, 3 October 2012

Overview

PTP is designed to interact with a variety of computer systems in order to aid the development of parallel and scientific applications. Many of these systems are extremely large, comprising many tens or hundreds of thousands of cores. In addition, the applications that run on these systems can also be very large, not just in terms of the number of lines of code, but also in their memory usage and the number of computational tasks required to deliver adequate performance.

Providing a development environment that allows developers to maximize their productivity in these environments is a challenging task. Not only do the applications need to scale, but the tools that are required to monitor, debug, and analyze these application must also scale.

The following document outlines the design elements of PTP that are key to ensuring that it will continue provide a useful and productive development platform, regardless of the system or application scaling requirements.

Further Reading

Key designs that are references in this document are as follows:

• Resource Manager Configuration - A description of the architecture and XML configuration file for the new JAXB resource manager framework
• LML DA Driver - Architecture of the backend data collection engine.

Resource Manager Framework

The primary mechanism for interacting with a target system is via the resource manager framework. This framework allows developers to launch and monitor applications on a variety of different computer systems, and consequently must support interaction via a range of different job schedulers and runtime system.

Previous implementations of the RM framework have suffered from two primary scaling problems. The first of these was a monitoring system that displayed a representation of the entire computer system layout down to the node level, as well as individual tasks for each job run. While this was adequate when systems and jobs comprised only a few thousand elements, the current generation of machines has significantly exceeded this threshold. The second issue was the non-scalability of the proxy communication protocol used to obtain monitoring information from the target system. While attempts were made to add scaling features to the protocol (including a variety of compression techniques), these added significant complexity, and ultimately reduced the overall reliability of the system.

In order to overcome these issues, a new framework is required that provides features that specifically address the scalability requirements of the platform. The major architectural improvement provided by this framework is to separate the control and monitoring aspects of interaction with target system. This separation ensures that the scalability sensitive monitoring functions do no impact on the less sensitive control functions. Detailed key design elements of this framework are detailed below.

Control Framework

The RM control framework provides a set of functions that allow the user to control application launch on the target system without the need for complex proxy agents or protocols. For simple interactive runs, the control framework will be the equivalent of issuing a shell command that starts the application. For more complex batch systems, the control framework is capable of generating the appropriate batch script and submitting this to the job scheduler.

The control framework employs the Java Architecture for XML Binding (JAXB) to enable support for a target system to be completely specified via an XML configuration file. This includes the commands that are required for job submission, definition of the scripts (if required), and a description of the user interface that will be presented to the user in order to specify job attributes and parameters.

The advantage of employing an XML based configuration approach is simplicity and extensibility. In prior designs, it would often take 6 months or more to add support for a new target system. This is because each implementation needed to be coded from scratch, and the complexity of some resource managers (such as LoadLeveler) required considerable effort in order to be fully supported. The same resource manager took 1-2 weeks to implement using the XML approach. The XML schema is also powerful enough to support virtually any type of resource manager, but can be relatively easily extended if additional functionality is required.

The XML configuration file scheme is available from http://eclipse.org/ptp/schemas/rm.xsd.

Plugin Layout

The control framework is defined in four plugins, with sample configuration files contained in a fifth plugin. These are:

org.eclipse.ptp.rm.jaxb.configs 

Sample configuration files

• org.eclipse.ptp.rm.jaxb.control.core
• org.eclipse.ptp.rm.jaxb.control.ui
• org.eclipse.ptp.rm.jaxb.core
• org.eclipse.ptp.rm.jaxb.ui

Each of these plugins will be described in more detail below.

org.eclipse.ptp.rm.jaxb.configs

This plugin contains the sample configuration files to support a range of generic system types. These include:

• IBM LoadLeveler
• IBM Parallel Environment
• Grid Engine
• MPICH2
• Open MPI
• PBS
• Generic remote launch
• SLURM
• TORQUE

User's are presented with a choice of these target system configurations on the Resource tab of the Parallel Application launch configuration (more detail below.)

Monitoring Framework

The monitoring framework is key to achieving scalability in PTP. This framework is divided into two components, a client UI, and a backend data collection engine. Communication between the client and backend is via an XML-based protocol called LML [ref]. The backend data collection engine is designed to be modular, so that support for new systems can be added easily.

The design of the monitoring user interface needs to pay particular attention to scalability issues as it must avoid excessive memory consumption and processing overhead in the JVM, as well as overloading the user with too much detailed information. These issues are dealt with by employing a hierarchical data representation that reduces the amount of information that is presented to the user at any time. Increasing levels of detail can be obtain by user interaction, "drilling into" the display, but otherwise only a moderate level of detail is presented at each level of the hierarchy. The internal model that is used to represent the monitored system is correspondingly structured, so that only the minimum amount of information is transmitted and stored at any particular time.

Consideration to the load the backend data collection engine places on the target system must also be considered. Currently, the backend is restricted to running every 60 seconds to minimize this load. This appears to be adequate for more situations, however, a large number of users could still place a significant burden on the system, particularly if data collection consumes non-trivial resources. One option to overcome this is to centralize the data collection to a single server process, then make this data available via an HTTP service. While the architecture for this is in place, the implementation of such a service is still ongoing. Additional information on the backend data collection engine can be found in the LML DA Driver document.