Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
PTP/planning/1.1
This document describes the 1.1 release of PTP. This release wil add some new functionality as well as enhancements and bug fixes to existing features.
Contents
Release Deliverables
The release deliverables are:
- Source code release for PTP Project
- PTP Project (downloadable)
- PTP runtime components (downloadable as source)
Release Milestones
Current versions of PTP are built with Eclipse platform 3.2.1 and CDT 3.1.
The milestones are:
Nov 3, 2006 - Release Candidate 1 (1.1 RC1) Nov 10, 2006 - Release Candidate 2 (1.1 RC2)
It is expected that RC2 will become the final release, but additional release candidates can be scheduled if necessary.
Target Operating Environments
Platform support for PTP is complicated by the need to operate closely with a variety of parallel runtime systems and debuggers that may in turn support a wide range of architectures. Because of this complexity, only the Java components of PTP will be distributed pre-built. Runtime and debugging support will have to be compiled from source.
| Operating System | Processor Architecture | Window System | Java 2 Platform | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Linux (SUSE 9.1 or RHE WS4) | i386/x86_64 | GTK | Sun Java 2 Standard Edition, version 1.5 for Linux x86 | Linux (SUSE 9.1 or RHE WS4) | i386 | GTK | IBM 32-bit SDK for Linux on Intel architecture, Java 2 Technology Edition, Version 1.4.2 | Mac OS X 10.4 | PPC/i386 | Carbon | Java 2 Standard Edition 1.5 for Mac OS X |
| MPI Runtime | Processor Architecture | Operating System | Open MPI | i386/x86_64 | Linux (SUSE 9.1 or RHE WS4) | Open MPI | PPC/i386 | Mac OS X 10.4 |
|---|
| Operating System | Processor Architecture | MPI Runtime | Linux (SUSE 9.1 or RHE WS4) | i386/x86_64 | Open MPI | Mac OS X 10.4 | PPC/i386 | Open MPI |
|---|
Component Features
Core/Runtime
- Restructure model manager to support asynchronous operation
- OpenMPI improvements
- Bug fixes
- Better error handling
- Support for v1.2
- Support for MPICH2 (runtime only, no debugging)
- Debugger improvements
- Bug fixes
- Scalability (not UI) to handle 1000 processes
- Reduced load on backend
- A Parallel Development Perspective which is comprised of a Machines View, Jobs View, Process View, Legend, and includes a Preferences Page.
- A Machine's View which displays the status of all the machines the user knows of.
- Dynamically updated status of the nodes of machines as those nodes change state (such as 'up', 'down', 'has a job running on it')
- A Job's View which displays all the jobs that were started during the current session. This includes job state and a listing of processes comprising the job (including process state).
- A Process View which displays the status of a single process (though multiple Process Views may be open concurrently). This includes the stdout of the process and has status and exit code fields that are updated dynamically as the state of the process changes.
- Ability to focus on a machine, node, job, or process and display current status of that entity.
- A Legend dialog that displays the various icons for nodes and processes that represent the states these entities can undertake.
- A Preferences Page which lets the user specify settings for the type of monitoring and control system to use.
- A model (series of instantiated data structures) that represents the known universe (machines, nodes, jobs, processes). The model is organized heirarchically and each entity contains attributes (key/value pairs) that represent additional information about the object (such as process state, node ownership, etc.)
- An interface to external control and monitoring systems (runtime system components).
- Open-MPI control and monitoring systems implementations. These interface to Open-RTE through the Java Native Interface (JNI).
- Simulated control and monitoring systems which exercise the runtime system interface, user interface, and allow demonstration in environments without other control/monitoring systems.
- Ability to start a job on a specified machine on a specified number of processes.
- Ability to terminate a running job.
- Ability to create sets of nodes and processes for ease of viewing. Also the ability to delete entries from these sets, add to them, and focus on a given set.
Parallel Debugger
Debug Core
- A parallel debug model which consists of 2 packages: ptp.debug.core (PTP Debug Core) and ptp.debug.external (PTP Debug External). It is based on the CDT/CDI Debug Model. PTP Debug Core acts as a bridge between the Platform Debug Model and PTP Debug External which is connected to the external debugger interface.
- An extension to CDI called PCDI which is suitable for the parallel debug model is shared between PTP Debug Core and PTP Debug External. With CDI and PCDI, the external component (PTP Debug External) can be further extended or replaced without rewriting the core component (PTP Debug Core).
- A process debug model which is inherited from the PTP Core Process Model. All processes in the debug job are listed in the Parallel Debug view. One or more processes can be registered in the usual Debug view for further inspection.
- A data structure called BitList that is very similar to java.util.BitSet with the option of further space optimization of the data storage.
- A process set debug model to describe a set of processes using BitList as the underlying data structure.
- Parallel support for events, breakpoints, and variables (local variables, argument variables, and global variables).
- Event management system for multiple processes. Events from multiple processes can be aggregated into one event. The event system utilizes BitList to avoid overwhelming the platform with events.
- Breakpoint management system for multiple processes. Unlike in the CDT Debug Model where a breakpoint is associated with a single process, in PTP Debug Model, a breakpoint is associated with the debug session. Thus, a single breakpoint can be applied to a set of processes.
- A support to see the variables in the Variable view for selected processes in the Debug view.
- A well-defined interface (IDebugger) in PTP Debug External to be used by external debuggers to connect to PTP. The interface is similar to the HPDF standard albeit without command-line support (e.g. command history, debug focus).
- A simulated parallel debugger as an example of implementation of IDebugger. The debugger allows users to debug multiple processes (e.g. set breakpoint, resume, terminate) in a simulated environment.
Debug UI
- Buttons and shortcuts to create, delete and modify the set by selecting elements in Debug Views.
- Ability to name a new set in Debug Views.
- Change set button to switch the set for examining the status of the elements in Debug Views.
- Parallel Debug view supports mulitple jobs run and ability to switch to specific job to display its processes.
- Parallel Debug view supports process registering or unregistering to and from the Standard Debug view.
- C/C++ Source Editor in Parallel Debug Perspective allows create, delete, enable and disable parallel breakpoint.
- Tooltip display to represent the multiple breakpoints on the same location with different jobs.
- Breakpoint represent its status in different sets in different colors.
- A set of buttons in the parallel debug view allows to resume, suspend, stop, step into, step over and step return all processes in the current set.
- A set of buttons in the debug view allows to control the selected processes in this view.
- Parallel Breakpoint Set view in Standard Breakpoint view which groups multiple sets of breakpoints.
Scalable Debug Manager
- Debugger launched from UI using normal launch configuration
- Support for local or remote debugging (i.e. debugger frontend can be on local or remote machine)
- Support for multiple proxy protocols between Eclipse and debug frontend (initial implementation using TCP)
- Debugger integration with OpenRTE
- Support for basic debug commands (to be extended):
- Set line/function breakpoint
- Single step
- Continue execution
- Evaluate expression
- List local/global variables
- Stackframe manipulation
- Debug commands can be applied to all or specific sets of processes.
- All debug commands operate asynchronously
- Support for three modes of event handling:
- wait for events from all processes
- wait for events from set of processes
- wait for all events within a specified interval
- Debugger will automatically coalesce events of the same type
Fortran Development Tools
- Modifications to the CDT to provide a user interface that is language independent and less focused on C and C++. These modifications include:
- Modify the CDT C/C++ perspective to a "Make Projects" perspective.
- Create "Managed Make" and "Standard Make" projects types and associated wizards to create them.
- Modify CDT icons to reflect changes to the CDT.
- Additions to the CDT to provide an IDE for Fortran. These additions include:
- A Fortran language model.
- A Fortran editor with syntax highlighting.
- Recognition of Fortran file extensions.
- Mixed language toolchains including C, C++, Fortran compilers.
- Toolchain support for IBM, Intel, and GNU Fortran compilers.
- Error parsers for IBM, Intel, and GNU Fortran compilers.
- Dependency analysis enhancements to the Managed Make System to allow projects with multiple Fortran files to be built in the correct order.
- Capability to launch Fortran applications
MPI Development Tools
This will combine the contribution of OpenMP support from IBM with the MPI Development Tools into a new common feature. The aim of this feature will be to provide a range of tools to aid the development of parallel language applications.
- PLDT (Parallel Language Development tools, formerly known as MPI Development Tools)
- Will support OpenMP in addition to MPI, for C programs
- MPI support for Fortran
- ? OpenMP support for Fortran - probably not, but a possibility by then
- MPI New project wizard:*
- MPI Project type
- MPI and compiler versions: specify which MPI implementation, and which compiler/toolchain, for the project
- *Note: the MPI New project wizard may be deferred until CDT new project model changes are in, probably summer '07, meaning Release 2.0.
- Static analysis of one or more C/C++ source files to identify the location of MPI artifacts
- Preference page for setting up MPI header file location and other configuration information.
- Eclipse toolbar buttons for initiating static analysis for artifact identification, and clearing existing artifact markers.
- Identification of "MPI artifacts" from C/C++ code in an Eclipse view listing API name, source file name and line number, artifact type (function or constant), etc. - linked with markers to source code locations.
- Markers on each artifact to identify in source file, and allow navigation to the source line containing each artifact
- Content Assist (Ctrl-space) shows possible completions of arbitrary partial API names in C source files. Mouse or cursor selection aids entry and reduces typing while improving accuracy.
- Context-sensitive help on each identified API in C source files: mouse hover shows API name, brief description, argument types, and MPI types as exemplary fulfillments of argument types (instead of generic C type. e.g. 'MPI_Comm' instead of 'int')
- Help view via F1 on each identified API in C source files - shows more detailed descriptions.
- Help plug-in includes setup, examples, and walk-thru of usage of all features including screen shots.
Other
- TAU[1] (Tuning and Analysis Utilities) Support
- Automatic generation of TAU-instrumented binaries from existing TAU configurations
- Trace and profile output storage in timestamped directories
- Make target support (Craig will do if UOregon doesn't)
Component Test Plans
Core/Runtime
SETUP:
Start on a bproc machine. Make sure Open-MPI is setup and working (not part of this test, just required to utilize Open-MPI). Start with a fresh Eclipse install (including workspace). Install PTP. Compile the PTP Open-MPI JNI library. Acquire some set of nodes for a long period for testing. Launch Eclipse and then launch a new Eclipse with the PTP plugins running.
TEST PLAN:
Using the Parallel Development Preferences Page select the Simulated runtime system. Using the menuing system, open the Machines View and Jobs View. Confirm that the Jobs View shows no jobs since it is a clean start. Confirm that the Machines View displays the current state of the machines. Use the drop-down menus to observe other machines that are known and confirm they too display the current machine state. Create a set of nodes using the user interface and name the set. Confirm that the user can switch between (focus on) the full set of nodes for the given machine and the newly created set. Add a few more nodes to the new set. Focus on a different machine and confirm that the set is no longer visible (since it pertains to the original machine). Create a new C project. Create a new C-MPI source file in the project. The source file will have each process producing periodic output and run for a few minutes (so that the tests can be performed on a running job). Compile the C-MPI application. Create a new Run Configuration for this project, utilizing the Parallel Development configuration to specify the number of processes for this run and a chosen simulated machine. Run the job (under simulated control). Confirm that the appropriate node's change state in the Machines View to specify they contain a running job and that the job starts on the correct machine. Focus on a node where one of the processes has been assigned. Confirm that the Machines View displays the processes on that node, including which job the process belongs to. Double-click on one of the processes in the Machine View to bring up the Process View. Confirm that the MPI rank, node number, job number, and status are correct. Observe process output in the output section of the Process View. Wait for job to terminate. Observe that the process state and exit code correctly display in the Process View, Machine View (for the appropriate node), and Jobs View (for the appropriate Job). Bring the Jobs View to the foreground. Confirm the Job previously run, as well as the processes contained within it, are listed and is shown as terminated. Re-run the same job. Confirm the Job View displays the job as running and the processes as well. Double-click on a process of the job, opening the Process View. Confirm the running state. Terminate the job by using the terminate icon. Confirm the Job View updates to show the terminated state. Confirm the Process View updates to show the terminated state, including an exit-code. Using the Parallel Development Preferences Page select the Open-MPI runtime system. Using the Open MPI Preferences Page under the Parallel Development Preferences Page set the path and arguments to the ORTE daemon (ORTEd). Run the same job from step #12 (under OMPI control). Repeat steps #14 through #26 for this second runtime system (Open-MPI as opposed to simulation). Switch back to the Machine View. Using another terminal change the state of one of the nodes (reboot it, change ownership, etc) and confirm that the node's status changes in the Machine View (both the icon to match the legend as well as the detailed text information to display the new change(s)).
Parallel Debugger
Debug Core
SETUP:
Make sure that the correct version of CDT and Eclipse are installed. Check out these packages from dev.eclipse.org:/home/technology/org.eclipse.ptp: org.eclipse.ptp.core org.eclipse.ptp.debug.core org.eclipse.ptp.debug.external org.eclipse.ptp.debug.ui org.eclipse.ptp.launch org.eclipse.ptp.ui Run the Eclipse Application Create a simple Managed Make C Project with the name: "TestC" Create the main program file with the name: "main.c" Fill in "main.c" with any code since the Debug Simulator will not interpret the source code (it only gets the line number). Edit the file so that it contains only 20 lines. Create the PTP Launch Configuration and fill in the necessary configuration, including the number of processes.
TEST PLAN:
Switch to the Parallel Debug Perspective. Double click the ruler at any line in the source code window to set the breakpoint, set another breakpoint after the first breakpoint. Verify that the breakpoints have been set by looking at the Breakpoints view. Click the debug button for the created PTP Launch Configuration. Process 0 will be registered automatically in the Debug view, 2 Console views will also be opened (one for process 0, and the other for the debugger output). After printing the output for process 0, which can be verified by opening the Console view for process 0, all processes (process 0 and others) will break at the first breakpoint (since the breakpoint applies for all processes). Verify this by looking at the Debug view and see that process 0 is suspended. Look at the Parallel Debug view, all processes should change to yellow. Look at the Console view for process 0, the output should stop. Look at the Console view for the debugger, there should be text saying that BreakpointEvent has been received. Double click process 1 in the Parallel Debug view to register it to the Debug view. Look at the Debug view and see that process 1 has been registered and in the suspended state. Look at the Console view and verify that Console view for process 1 has been created. Select process 0 and click the Step Over button in the Debug view. Look at the Console view for process 0 to verify that the action has been executed successfully. Repeat and verify this action several time Select process 1 and click the Step Into button in the Debug view. Look at the Debug view and see that the number of stackframes for process 1 increases by one. Click Step Into to increase the stackframe and click Step Finish to decrease the stackframe. Repeat and verify this action several times. The maximum number of stackframes is 5. Select the stackframe of process 0 and open the Variables view. Verify that there are 2 argument variables and 2 local variables. Select process 0 and click the Resume button, it will then hit the second breakpoint. Click the Resume button again to run process 0 until completion (it prints 20 lines and then exits). Look at the Debug view and the Console view and verify that process 0 has exited. Select process 1 and click the Terminate button. Look at the Debug view and the Console view and verify that process 1 has been terminated.
Debug UI
SETUP
Create a C/C++ Project named as "TestC" reate a file called "main.c" Open it and write any coding Switch to Debug Parallel Perspective Add any breakpoint on "main.c" file by double click on ruler. OR double click anywhere on source editor and then double click on ruler again. Click launch button to create parallel configuration and then adding the project name, filename and the number of processes. Click debug button to launch this program.
TEST PLAN:
Starting: Parallel Debug View displays the status of processes after launching a parallel program in debug mode. Debug icon of the job represent that it is in debug mode. The new job will be selected at the start up and the all processes of this job will be displayed. Unselected the job, the process view will be cleared. View: Drag across a rectangular region to select some processes. Hold down the left mouse button, move the mouse, and release the button when the desired process is selected. Click with the left mouse button while holding down the Shift key will extent selection. Click with the left mouse button while holding down the Ctrl key will continue selection. A dialog box will popup when press the Create set button. After enter set name, the view will change from root set to created set. There is a validation to check whether a new set name already exist or not. If exists, the ok button will be disabled. Only set can be created after the job is launched. Press Ctrl-A key can select all the processes. Press Page-up key can go up one page. Press Page-down key can go down one page. Auto scrolling while dragging further up or down the views. Breakpoint: The parallel breakpoint is created when double click on ruler of the C/C++ source editor. If the parallel breakpoint is created in the condition that is no job running or unselected all the jobs, this breakpoint will belong to be global for all the jobs and there is an additional overlayered image on the breakpoint. Tooltips displaying details of breakpoint (job name, set name, file name and line number) will appear when mouse hovers on the breakpoint. Color of a breakpoint will change to green when it is created in the current set's process. Color of a breakpoint will change to yellow when it does not exist in any of the set's process. Color of a breakpoint will change to blue when it exists in one of the process in the current set but is not created in this set. Double on the created breakpoint on ruler, the breakpoint will be removed. The global breakpoint only can be created or removed when there is no job selected. Multiple breakpoints can be created in the same source file and the same line number with different jobs. Breakpoint view: All breakpoints will be displayed ordered by the set name when clicking the drop down menu in the breakpoint view and then the parallel breakpoint set. Unchecked the breapoint in Breakpoint View, the breakpoint will be disabled. Check it again, it will be enabled again. Right click on the breakpoint in Breakpoint View and select Goto Set action, the parallel debug view will change to the job and the set indicated in the selected breakpoint. Register / Unregister: Double click on the process in parallel debug view and the process will be registered into the debug view. A registered process is presented by a black rectangle. Selecting multiple processes in the parallel debug view and then click the register button will register all of them in the debug view. If more than 10 processes have been selected to register, a dialog will popup and ask for confirmation. Double click on the registered process will unregister it from the debug view. The black rectangle representing the registration will be removed. Selecting multiple registered processes in the parallel debug view and then click the unregister button will unregister all of them in the debug view. The debug view will only display registered processes which is according to the current job and current set. Annotation If a breakpoint location contains multiple markers, annotations for both these markers will be overlapped when the breakpoint is hit. The registered process marker should appear on top. Tooltips will display process task id of each marker set in this location, grouped by its marker. The annotation will change to another color(not created) when switch to another set. The process icon appears in yellow process is in suspended mode. The process icon appears in green when in running mode. The process icon appears in red when in terminate mode. The resume button and all step buttons in the parallel debug view will only be enabled when one of the processes is suspended. The suspend button in the parallel debug view will only be enabled when one of the processes is running. The terminate button in the parallel debug view will only be enabled when one othe processes is either running or suspended. Resume button in the parallel debug view resumes all suspended processes in the current set. Suspend button in the parallel debug view pauses all running processes in the current set. Terminate button in the parallel debug view stops all suspended and running processes in the current set. Step buttons in the parallel debug view can step into, step over and step return all suspended processes in the current set. Resume button in the debug view can resume selected suspended process(es). Suspend button in the debug view can pause selected running process(es). Terminate button in the parallel debug view can stop any suspended or running process(es). Step buttons in the debug view can step into, step over and step return any suspended process(es).
Scalable Debug Manager
TBD
Fortran Development Tools
In a new workspace, check out the following CDT components from dev.eclipse.org:/home/tools org.eclipse.cdt-build/ org.eclipse.cdt.make.core org.eclipse.cdt.make.ui org.eclipse.cdt.managedbuilder.core org.eclipse.cdt.managedbuilder.gnu.ui org.eclipse.cdt.managedbuilder.ui org.eclipse.cdt-core/ org.eclipse.cdt.core org.eclipse.cdt.core.ARCH org.eclipse.cdt.ui org.ecipse.cdt-debug/ org.eclipse.cdt.debug.core org.eclipse.cdt.debug.mi.core org.eclipse.cdt.debug.mi.ui org.eclipse.cdt.debug.ui org.eclipse.cdt-launch/ org.eclipse.cdt.launch org.eclipse.cdt-doc/ org.eclipse.cdt.doc.isv org.eclipse.cdt.doc.user Next check out the FDT components from dev.eclipse.org:/home/technology org.eclipse.ptp/lang/ org.eclipse.fdt.core org.eclipse.fdt.managedbuilder.core org.eclipse.fdt.managedbuilder.ui org.eclipse.fdt.managedbuilder.tests org.eclipse.fdt.ui cdt_changes From a shell, cd to Eclipse workspace and execute the shell script cdt_changes/install Select org.eclipse.cdt.* projects one by one and refresh them. Verify that workspace builds correctly without any errors. Create Eclipse Application launch configuration (FDT) and run it. Examine icons (set left of debug icon) and make sure that project, folder, and file wizard icons are "M" based not "C" based. Also check that there is a "Make" perspective and a "Make Projects" window. Examine menus corresponding to these icons and make sure they don't have C/C++ in labels. Right click in "Make Project" window and examine icons and menu labels as in #6 and #7 above. Use menu in project icon to create a new "Managed Make" project, name it HelloWorld (check icon). Select Gnu Fortran for project type and click Finish. Use menu in file icon to create a new source file. Name it hello.f90; make sure the file extension is acceptable and check icon and click Finish. Check that new file is automatically opened. Copy contents of org.eclipse.fdt.managedbuilder.tests/src/HelloWorld/hello.f90 into the new file and save the file. Check console to see that "Build complete for project HelloWorld" Create new "Local Application" Run target. Ensure that label in the configurations window is "Local Application" and icon has an "M". Name the configuration HelloWorld and set the Project and Application fields. Set a debugger and click Run. Check that "Hello from Fortran Main." is printed in the console window. Select the HelloWorld target from the Run menu and check console output again. Redo steps 9-15 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Components. That is, create a Managed Make project named Components and build an application the files. Open Debug/subdir.mk and ensure the dependencies are as follows: Driver.o: MonteCarloIntegrator.o MonteCarloIntegrator.o: LinearFunction.o MonteCarloIntegrator.o: RandNumGenerator.o Make sure the project will build and run. Redo steps 9-15 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Mixed. That is, create a Managed Make project named Mixed and build an application the files. Make sure the mixed language project will build and run
MPI Development Tools
(See help files in the plug-in for more details on these)
Launch eclipse with PTP (incl. CDT) Help -> Help Contents, PTP MPI Tools Follow instructions in "Setup for PTP MPI tools". Basically, create a CDT project, a source file with MPI apis (a sample is included in help), and use Preferences to locate MPI header files. Follow instructions in "Running PTP MPI tools" to set up Open MPI artifacts view (Do we need to explicitly add the actions to the toolbar?) select source vile in Navigator, then select MPI action to run analysis. MPI artifact view should be populated with markers, listing API name, source file name and line number, and artifact type (function or constant) Double-click on any artifact, this should take you to the source file and line locating the artifact. In source file, icon should be in marker bar, and squiggles should be under the artifact (e.g. function name) Click on column headings in artifact view to sort by values in that column. Command completion: In editor, type e.g. MPI_ and hit ctrl-space. Possible completions should appear. Continue to type a few characters; possible completions should diminish to match string. Hitting ENTER or double-clicking on any choice should insert the completed API into the file. Hover help: Hover on any MPI API, see yellow popup info on API and arguments. F1 help: click on any MPI API and hit F1 to see Help view appear; click on API in that view and see detailed info on that API.