Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "PTP/planning/1.1"

< PTP‎ | planning
(Parallel Language Development Tools)
 
(8 intermediate revisions by 2 users not shown)
Line 15: Line 15:
 
The milestones are:
 
The milestones are:
  
* Nov 3, 2006 - Release Candidate 1 (1.1 RC1)
+
* Nov 3, 2006 - Release Candidate 1 (1.1 RC1) - API freeze
* Nov 10, 2006 - Release Candidate 2 (1.1 RC2)
+
* Nov 10, 2006 - Release Candidate 2 (1.1 RC2) - Code freeze
  
 
It is expected that RC2 will become the final release, but additional release candidates can be scheduled if necessary.
 
It is expected that RC2 will become the final release, but additional release candidates can be scheduled if necessary.
Line 22: Line 22:
 
== Target Operating Environments ==
 
== Target Operating Environments ==
  
Platform support for PTP is complicated by the need to operate closely with a variety of parallel runtime systems and debuggers that may in turn support a wide range of architectures. Because of this complexity, only the Java components of PTP will be distributed pre-built. Runtime and debugging support will have to be compiled from source.
+
Platform support for PTP is complicated by the need to operate closely with a variety of parallel runtime systems and debuggers that may in turn support a wide range of architectures. Because of this complexity, only the Java components of PTP will be distributed pre-built. Runtime and debugging support will have to be compiled from source. There are no windowing system specific plugins for PTP.
  
{| border="1"
+
The current plan is to support Java 1.5 run-time environments only.
|+ '''PTP Core Supported Platforms'''
+
! Operating System !! Processor Architecture !! Window System !! Java 2 Platform
+
|-
+
| Linux || i386/x86_64 || GTK || Sun Java 2 Standard Edition, version 1.5 for Linux x86
+
|-
+
| Linux || i386 || GTK || IBM 32-bit SDK for Linux on Intel architecture, Java 2 Technology Edition, Version 1.4.2
+
|-
+
| Mac OS X || PPC/i386 || Carbon || Java 2 Standard Edition 1.5 for Mac OS X
+
|}
+
  
 +
=== PTP Core Supported Platforms ===
  
{| border="1"
+
* Linux - x86, x86_64, ppc
|+ '''PTP Supported Runtime Systems'''
+
* Mac OS X - ppc, x86
! Operating System !! Processor Architecture !! Runtime System
+
* Windows - x86 (PLDT and FDT only: that is, Parallel Language Development tools and Fortran Development Tools both work on Windows, but not the parallel runtime or the parallel debugger, etc.)
|-
+
| Linux || i386/x86_64 || Open MPI
+
|-
+
| Linux || i386/x86_64 || MPICH2
+
|-
+
| Mac OS X || PPC/i386 || Open MPI
+
|-
+
| Mac OS X || PPC/i386 || MPICH2
+
|}
+
  
 +
=== PTP Supported Runtime Systems ===
  
{| border="1"
+
* Open MPI 1.0.2, 1.2
|+ '''PTP Debugger Supported Systems'''
+
* MPICH2 (debug not available)
! Operating System !! Processor Architecture !! MPI Runtime
+
|-
+
| Linux || i386/x86_64 || Open MPI
+
|-
+
| Mac OS X || PPC/i386 || Open MPI
+
|}
+
  
 +
=== PTP Supported Debuggers ===
 +
 +
* gdb 6.3, 6.4, 6.5
  
 
== Component Features ==
 
== Component Features ==
Line 108: Line 89:
 
# Bug fixes
 
# Bug fixes
 
# Support for OpenMP in addition to MPI (C programs only)
 
# Support for OpenMP in addition to MPI (C programs only)
# Fortran support
+
#* OpenMP support, besides the identification of OpenMP artifacts, has additional prototype analysis features including identification of several common concurrency errors, concurrency analysis, and #pragma region identification.
#* MPI using Photran infrastructure
+
# Fortran support (this is tentative and may not make it into PTP 1.1)
 +
#* MPI artifact location using Photran infrastructure
 
#* Possible OpenMP support
 
#* Possible OpenMP support
# MPI New project wizard (Note: the MPI New project wizard may be deferred until CDT new project model changes are in, probably summer '07, meaning Release 2.0):
+
# MPI New project wizard  
#* MPI Project type
+
#* For 1.1 this is an added wizard page to the CDT new managed build C project wizard, allowing specification of MPI include and lib paths, and build commands, with reasonable default settings based on PLDT preferences.  This will probably only work for gcc as the compiler.
#* MPI and compiler versions: specify which MPI implementation, and which compiler/toolchain, for the project
+
#*(Note: the MPI New project wizard may be deferred until CDT new project model changes are in, probably summer '07, meaning CDT Release 4.0 and PTP Release 2.0 - this would include a new "MPI Project" type, and user specification of MPI and compiler versions: specify which MPI implementation, and which compiler/toolchain, for the project.)
  
 
=== Other Tools ===
 
=== Other Tools ===
  
# TAU[http://www.cs.uoregon.edu/research/tau/home.php] (Tuning and Analysis Utilities) Support
+
==== TAU [http://www.cs.uoregon.edu/research/tau/home.php] Support ====
#* Automatic generation of TAU-instrumented binaries from existing TAU configurations
+
#* Trace and profile output storage in timestamped directories
+
#* Make target support (Craig will do if UOregon doesn't)
+
 
+
== Component Test Plans ==
+
 
+
=== Core/Runtime ===
+
 
+
==== SETUP: ====
+
 
+
Start on a bproc machine.
+
Make sure Open-MPI is setup and working (not part of this test, just required to utilize Open-MPI).
+
Start with a fresh Eclipse install (including workspace).
+
Install PTP.
+
Compile the PTP Open-MPI JNI library.
+
Acquire some set of nodes for a long period for testing.
+
Launch Eclipse and then launch a new Eclipse with the PTP plugins running.
+
 
+
TEST PLAN:
+
 
+
Using the Parallel Development Preferences Page select the Simulated runtime system.
+
Using the menuing system, open the Machines View and Jobs View.
+
Confirm that the Jobs View shows no jobs since it is a clean start.
+
Confirm that the Machines View displays the current state of the machines. Use the drop-down menus to observe other machines that are known and confirm they too display the current machine state.
+
Create a set of nodes using the user interface and name the set.
+
Confirm that the user can switch between (focus on) the full set of nodes for the given machine and the newly created set.
+
Add a few more nodes to the new set.
+
Focus on a different machine and confirm that the set is no longer visible (since it pertains to the original machine).
+
Create a new C project.
+
Create a new C-MPI source file in the project. The source file will have each process producing periodic output and run for a few minutes (so that the tests can be performed on a running job).
+
Compile the C-MPI application.
+
Create a new Run Configuration for this project, utilizing the Parallel Development configuration to specify the number of processes for this run and a chosen simulated machine.
+
Run the job (under simulated control).
+
Confirm that the appropriate node's change state in the Machines View to specify they contain a running job and that the job starts on the correct machine.
+
Focus on a node where one of the processes has been assigned. Confirm that the Machines View displays the processes on that node, including which job the process belongs to.
+
Double-click on one of the processes in the Machine View to bring up the Process View. Confirm that the MPI rank, node number, job number, and status are correct.
+
Observe process output in the output section of the Process View.
+
Wait for job to terminate.
+
Observe that the process state and exit code correctly display in the Process View, Machine View (for the appropriate node), and Jobs View (for the appropriate Job).
+
Bring the Jobs View to the foreground.
+
Confirm the Job previously run, as well as the processes contained within it, are listed and is shown as terminated.
+
Re-run the same job. Confirm the Job View displays the job as running and the processes as well.
+
Double-click on a process of the job, opening the Process View. Confirm the running state.
+
Terminate the job by using the terminate icon.
+
Confirm the Job View updates to show the terminated state.
+
Confirm the Process View updates to show the terminated state, including an exit-code.
+
Using the Parallel Development Preferences Page select the Open-MPI runtime system.
+
Using the Open MPI Preferences Page under the Parallel Development Preferences Page set the path and arguments to the ORTE daemon (ORTEd).
+
Run the same job from step #12 (under OMPI control).
+
Repeat steps #14 through #26 for this second runtime system (Open-MPI as opposed to simulation).
+
Switch back to the Machine View.
+
Using another terminal change the state of one of the nodes (reboot it, change ownership, etc) and confirm that the node's status changes in the Machine View (both the icon to match the legend as well as the detailed text information to display the new change(s)).
+
 
+
=== Parallel Debugger ===
+
 
+
==== Debug Core ====
+
 
+
SETUP:
+
 
+
Make sure that the correct version of CDT and Eclipse are installed.
+
Check out these packages from dev.eclipse.org:/home/technology/org.eclipse.ptp:
+
org.eclipse.ptp.core
+
org.eclipse.ptp.debug.core
+
org.eclipse.ptp.debug.external
+
org.eclipse.ptp.debug.ui
+
org.eclipse.ptp.launch
+
org.eclipse.ptp.ui
+
Run the Eclipse Application
+
Create a simple Managed Make C Project with the name: "TestC"
+
Create the main program file with the name: "main.c"
+
Fill in "main.c" with any code since the Debug Simulator will not interpret the source code (it only gets the line number). Edit the file so that it contains only 20 lines.
+
Create the PTP Launch Configuration and fill in the necessary configuration, including the number of processes.
+
 
+
TEST PLAN:
+
 
+
Switch to the Parallel Debug Perspective.
+
Double click the ruler at any line in the source code window to set the breakpoint, set another breakpoint after the first breakpoint.
+
Verify that the breakpoints have been set by looking at the Breakpoints view.
+
Click the debug button for the created PTP Launch Configuration.
+
Process 0 will be registered automatically in the Debug view, 2 Console views will also be opened (one for process 0, and the other for the debugger output).
+
After printing the output for process 0, which can be verified by opening the Console view for process 0, all processes (process 0 and others) will break at the first breakpoint (since the breakpoint applies for all processes).
+
Verify this by looking at the Debug view and see that process 0 is suspended. Look at the Parallel Debug view, all processes should change to yellow. Look at the Console view for process 0, the output should stop. Look at the Console view for the debugger, there should be text saying that BreakpointEvent has been received.
+
Double click process 1 in the Parallel Debug view to register it to the Debug view. Look at the Debug view and see that process 1 has been registered and in the suspended state. Look at the Console view and verify that Console view for process 1 has been created.
+
Select process 0 and click the Step Over button in the Debug view. Look at the Console view for process 0 to verify that the action has been executed successfully. Repeat and verify this action several time
+
Select process 1 and click the Step Into button in the Debug view. Look at the Debug view and see that the number of stackframes for process 1 increases by one. Click Step Into to increase the stackframe and click Step Finish to decrease the stackframe. Repeat and verify this action several times. The maximum number of stackframes is 5.
+
Select the stackframe of process 0 and open the Variables view. Verify that there are 2 argument variables and 2 local variables.
+
Select process 0 and click the Resume button, it will then hit the second breakpoint. Click the Resume button again to run process 0 until completion (it prints 20 lines and then exits). Look at the Debug view and the Console view and verify that process 0 has exited.
+
Select process 1 and click the Terminate button. Look at the Debug view and the Console view and verify that process 1 has been terminated.
+
 
+
==== Debug UI ====
+
 
+
SETUP
+
 
+
Create a C/C++ Project named as "TestC"
+
reate a file called "main.c"
+
Open it and write any coding
+
Switch to Debug Parallel Perspective
+
Add any breakpoint on "main.c" file by double click on ruler. OR double click anywhere on source editor and then double click on ruler again.
+
Click launch button to create parallel configuration and then adding the project name, filename and the number of processes.
+
Click debug button to launch this program.
+
 
+
TEST PLAN:
+
 
+
Starting:
+
Parallel Debug View displays the status of processes after launching a parallel program in debug mode.
+
Debug icon of the job represent that it is in debug mode.
+
The new job will be selected at the start up and the all processes of this job will be displayed.
+
Unselected the job, the process view will be cleared.
+
View:
+
Drag across a rectangular region to select some processes. Hold down the left mouse button, move the mouse, and release the button when the desired process is selected.
+
Click with the left mouse button while holding down the Shift key will extent selection.
+
Click with the left mouse button while holding down the Ctrl key will continue selection.
+
A dialog box will popup when press the Create set button. After enter set name, the view will change from root set to created set.
+
There is a validation to check whether a new set name already exist or not. If exists, the ok button will be disabled.
+
Only set can be created after the job is launched.
+
Press Ctrl-A key can select all the processes.
+
Press Page-up key can go up one page.
+
Press Page-down key can go down one page.
+
Auto scrolling while dragging further up or down the views.
+
Breakpoint:
+
The parallel breakpoint is created when double click on ruler of the C/C++ source editor.
+
If the parallel breakpoint is created in the condition that is no job running or unselected all the jobs, this breakpoint will belong to be global for all the jobs and there is an additional overlayered image on the breakpoint.
+
Tooltips displaying details of breakpoint (job name, set name, file name and line number) will appear when mouse hovers on the breakpoint.
+
Color of a breakpoint will change to green when it is created in the current set's process.
+
Color of a breakpoint will change to yellow when it does not exist in any of the set's process.
+
Color of a breakpoint will change to blue when it exists in one of the process in the current set but is not created in this set.
+
Double on the created breakpoint on ruler, the breakpoint will be removed.
+
The global breakpoint only can be created or removed when there is no job selected.
+
Multiple breakpoints can be created in the same source file and the same line number with different jobs.
+
Breakpoint view:
+
All breakpoints will be displayed ordered by the set name when clicking the drop down menu in the breakpoint view and then the parallel breakpoint set.
+
Unchecked the breapoint in Breakpoint View, the breakpoint will be disabled. Check it again, it will be enabled again.
+
Right click on the breakpoint in Breakpoint View and select Goto Set action, the parallel debug view will change to the job and the set indicated in the selected breakpoint.
+
Register / Unregister:
+
Double click on the process in parallel debug view and the process will be registered into the debug view. A registered process is presented by a black rectangle.
+
Selecting multiple processes in the parallel debug view and then click the register button will register all of them in the debug view.
+
If more than 10 processes have been selected to register, a dialog will popup and ask for confirmation.
+
Double click on the registered process will unregister it from the debug view. The black rectangle representing the registration will be removed.
+
Selecting multiple registered processes in the parallel debug view and then click the unregister button will unregister all of them in the debug view.
+
The debug view will only display registered processes which is according to the current job and current set.
+
Annotation
+
If a breakpoint location contains multiple markers, annotations for both these markers will be overlapped when the breakpoint is hit. The registered process marker should appear on top. Tooltips will display process task id of each marker set in this location, grouped by its marker.
+
The annotation will change to another color(not created) when switch to another set.
+
The process icon appears in yellow process is in suspended mode.
+
The process icon appears in green when in running mode.
+
The process icon appears in red when in terminate mode.
+
The resume button and all step buttons in the parallel debug view will only be enabled when one of the processes is suspended.
+
The suspend button in the parallel debug view will only be enabled when one of the processes is running.
+
The terminate button in the parallel debug view will only be enabled when one othe processes is either running or suspended.
+
Resume button in the parallel debug view resumes all suspended processes in the current set.
+
Suspend button in the parallel debug view pauses all running processes in the current set.
+
Terminate button in the parallel debug view stops all suspended and running processes in the current set.
+
Step buttons in the parallel debug view can step into, step over and step return all suspended processes in the current set.
+
Resume button in the debug view can resume selected suspended process(es).
+
Suspend button in the debug view can pause selected running process(es).
+
Terminate button in the parallel debug view can stop any suspended or running process(es).
+
Step buttons in the debug view can step into, step over and step return any suspended process(es).
+
 
+
==== Scalable Debug Manager ====
+
 
+
TBD
+
 
+
=== Fortran Development Tools ===
+
 
+
In a new workspace, check out the following CDT components from dev.eclipse.org:/home/tools
+
org.eclipse.cdt-build/
+
org.eclipse.cdt.make.core
+
org.eclipse.cdt.make.ui
+
org.eclipse.cdt.managedbuilder.core
+
org.eclipse.cdt.managedbuilder.gnu.ui
+
org.eclipse.cdt.managedbuilder.ui
+
org.eclipse.cdt-core/
+
org.eclipse.cdt.core
+
org.eclipse.cdt.core.ARCH
+
org.eclipse.cdt.ui
+
org.ecipse.cdt-debug/
+
org.eclipse.cdt.debug.core
+
org.eclipse.cdt.debug.mi.core
+
org.eclipse.cdt.debug.mi.ui
+
org.eclipse.cdt.debug.ui
+
org.eclipse.cdt-launch/
+
org.eclipse.cdt.launch
+
org.eclipse.cdt-doc/
+
org.eclipse.cdt.doc.isv
+
org.eclipse.cdt.doc.user
+
Next check out the FDT components from dev.eclipse.org:/home/technology
+
org.eclipse.ptp/lang/
+
org.eclipse.fdt.core
+
org.eclipse.fdt.managedbuilder.core
+
org.eclipse.fdt.managedbuilder.ui
+
org.eclipse.fdt.managedbuilder.tests
+
org.eclipse.fdt.ui
+
cdt_changes
+
From a shell, cd to Eclipse workspace and execute the shell script cdt_changes/install
+
Select org.eclipse.cdt.* projects one by one and refresh them.
+
Verify that workspace builds correctly without any errors.
+
Create Eclipse Application launch configuration (FDT) and run it.
+
Examine icons (set left of debug icon) and make sure that project, folder, and file wizard icons are "M" based not "C" based. Also check that there is a "Make" perspective and a "Make Projects" window.
+
Examine menus corresponding to these icons and make sure they don't have C/C++ in labels.
+
Right click in "Make Project" window and examine icons and menu labels as in #6 and #7 above.
+
Use menu in project icon to create a new "Managed Make" project, name it HelloWorld (check icon). Select Gnu Fortran for project type and click Finish.
+
Use menu in file icon to create a new source file. Name it hello.f90; make sure the file extension is acceptable and check icon and click Finish.
+
Check that new file is automatically opened.
+
Copy contents of org.eclipse.fdt.managedbuilder.tests/src/HelloWorld/hello.f90 into the new file and save the file.
+
Check console to see that "Build complete for project HelloWorld"
+
Create new "Local Application" Run target. Ensure that label in the configurations window is "Local Application" and icon has an "M". Name the configuration HelloWorld and set the Project and Application fields. Set a debugger and click Run.
+
Check that "Hello from Fortran Main." is printed in the console window. Select the HelloWorld target from the Run menu and check console output again.
+
Redo steps 9-15 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Components. That is, create a Managed Make project named Components and build an application the files.
+
Open Debug/subdir.mk and ensure the dependencies are as follows:
+
Driver.o: MonteCarloIntegrator.o
+
MonteCarloIntegrator.o: LinearFunction.o
+
MonteCarloIntegrator.o: RandNumGenerator.o
+
Make sure the project will build and run.
+
Redo steps 9-15 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Mixed. That is, create a Managed Make project named Mixed and build an application the files.
+
Make sure the mixed language project will build and run
+
 
+
=== MPI Development Tools ===
+
 
+
(See help files in the plug-in for more details on these)
+
  
Launch eclipse with PTP (incl. CDT)
+
# Automatic generation of TAU-instrumented binaries from existing TAU configurations
Help -> Help Contents, PTP MPI Tools
+
# Trace and profile output storage in timestamped directories
Follow instructions in "Setup for PTP MPI tools". Basically, create a CDT project, a source file with MPI apis (a sample is included in help), and use Preferences to locate MPI header files.
+
# Make target support (Craig will do if UOregon doesn't)
Follow instructions in "Running PTP MPI tools" to set up Open MPI artifacts view (Do we need to explicitly add the actions to the toolbar?)
+
select source vile in Navigator, then select MPI action to run analysis.
+
MPI artifact view should be populated with markers, listing API name, source file name and line number, and artifact type (function or constant)
+
Double-click on any artifact, this should take you to the source file and line locating the artifact.
+
In source file, icon should be in marker bar, and squiggles should be under the artifact (e.g. function name)
+
Click on column headings in artifact view to sort by values in that column.
+
Command completion: In editor, type e.g. MPI_ and hit ctrl-space. Possible completions should appear. Continue to type a few characters; possible completions should diminish to match string. Hitting ENTER or double-clicking on any choice should insert the completed API into the file.
+
Hover help: Hover on any MPI API, see yellow popup info on API and arguments.
+
F1 help: click on any MPI API and hit F1 to see Help view appear; click on API in that view and see detailed info on that API.
+

Latest revision as of 16:10, 1 November 2006

This document describes the 1.1 release of PTP. This release will add some new functionality as well as enhancements and bug fixes to existing features.

Release Deliverables

The release deliverables are:

  • Source code release for PTP Project
  • PTP core (downloadable)
  • PTP runtime components (downloadable as source)

Release Milestones

Current versions of PTP are built with Eclipse platform 3.2.1 and CDT 3.1.

The milestones are:

  • Nov 3, 2006 - Release Candidate 1 (1.1 RC1) - API freeze
  • Nov 10, 2006 - Release Candidate 2 (1.1 RC2) - Code freeze

It is expected that RC2 will become the final release, but additional release candidates can be scheduled if necessary.

Target Operating Environments

Platform support for PTP is complicated by the need to operate closely with a variety of parallel runtime systems and debuggers that may in turn support a wide range of architectures. Because of this complexity, only the Java components of PTP will be distributed pre-built. Runtime and debugging support will have to be compiled from source. There are no windowing system specific plugins for PTP.

The current plan is to support Java 1.5 run-time environments only.

PTP Core Supported Platforms

  • Linux - x86, x86_64, ppc
  • Mac OS X - ppc, x86
  • Windows - x86 (PLDT and FDT only: that is, Parallel Language Development tools and Fortran Development Tools both work on Windows, but not the parallel runtime or the parallel debugger, etc.)

PTP Supported Runtime Systems

  • Open MPI 1.0.2, 1.2
  • MPICH2 (debug not available)

PTP Supported Debuggers

  • gdb 6.3, 6.4, 6.5

Component Features

Core/Runtime

  1. All 1.0 features
  2. Bug fixes
  3. Fully asynchronous operation
  4. Open MPI improvements
    • Better handling of daemon shutdown and other events
    • Support for Open MPI v1.2
  5. Support for MPICH2 (runtime only, no debugging)

Parallel Debugger

Debug Core

  1. All 1.0 features
  2. Bug fixes
  3. Thread support
  4. Full AIF datatype support

Debug UI

  1. All 1.0 features
  2. Bug fixes
  3. Thread support
  4. Variable display using AIF types

Scalable Debug Manager

  1. All 1.0 features
  2. Bug fixes
  3. Redesigned backend for scalability. This employs a broadcast topology for sending commands to the debug servers, and uses aggregation to manage large volumes of events.
  4. Improvements to polling routines to reduced load on backend machine

Fortran Development Tools

  1. All 1.0 features
  2. Integration with Photran
  3. Bug fixes

Parallel Language Development Tools

This will combine the contribution of OpenMP support from IBM with the MPI Development Tools into a new common feature. The aim of this feature will be to provide a range of tools to aid the development of parallel language applications.

  1. All 1.0 features
  2. Bug fixes
  3. Support for OpenMP in addition to MPI (C programs only)
    • OpenMP support, besides the identification of OpenMP artifacts, has additional prototype analysis features including identification of several common concurrency errors, concurrency analysis, and #pragma region identification.
  4. Fortran support (this is tentative and may not make it into PTP 1.1)
    • MPI artifact location using Photran infrastructure
    • Possible OpenMP support
  5. MPI New project wizard
    • For 1.1 this is an added wizard page to the CDT new managed build C project wizard, allowing specification of MPI include and lib paths, and build commands, with reasonable default settings based on PLDT preferences. This will probably only work for gcc as the compiler.
    • (Note: the MPI New project wizard may be deferred until CDT new project model changes are in, probably summer '07, meaning CDT Release 4.0 and PTP Release 2.0 - this would include a new "MPI Project" type, and user specification of MPI and compiler versions: specify which MPI implementation, and which compiler/toolchain, for the project.)

Other Tools

TAU [1] Support

  1. Automatic generation of TAU-instrumented binaries from existing TAU configurations
  2. Trace and profile output storage in timestamped directories
  3. Make target support (Craig will do if UOregon doesn't)

Back to the top