Difference between revisions of "PTP/testing/1.x"

Latest revision as of 08:41, 19 September 2007

Test Plan for PTP 1.x Release

This plan describes the tests that will be undertaken to verify the 1.x series of PTP releases.

Test Setup

The following steps should be carried out prior to testing PTP. Refer to the release notes for the appropriate PTP version if necessary.

1. Install OpenMPI 1.2. Make sure it is configured with the '--with-devel-headers' option.
2. If you are testing on a single node, edit <openmpi_install>/etc/openmpi-default-hostfile and add lines containing 'node0', 'node1', etc. Edit /etc/hosts and add a localhost entry for each name you added to the openmpi-default-hostfile. So, for 'node0', add '127.0.0.1 node0' to /etc/hosts. This will simulate a multi-node machine.
3. Verify OpenMPI is working by compiling and launching a simple program from the command line.
4. Start with a fresh Eclipse and CDT install (including workspace), as per the release notes.
5. Install PTP using instructions from the release notes.
6. Launch Eclipse on the test machine.
7. Switch to the C/C++ Perspective.
8. Create a new Managed Make C Project....
9. [If PLDT is installed ]In the project setup wizard, accept default settings up until the last page: select "Add MPI settings to the project." If you have not put MPI include path into Preferences, cancel out of the new project setup and do it now; then restart new project wizard.
10. [If PLDT is not installed.] Accept the default settings on the new project wizard. Then right-click on the new project and choose 'Properties'. Select the 'C/C++ Build' item. Change the 'GCC C Compiler' and '... C Linker' settings to 'mpicc'.
11. Create a new C source file in the project.
12. Add the following code:

    #include <stdio.h>
    #include <mpi.h>
    int main(int argc, char *argv[])
    {
        int i, rank;
     
        MPI_Init(&argc, &argv);
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
     
        for (i = 0; i < 10; i++) {
           printf("hello from %d loop %d\n", rank, i);
           sleep(1);
        }
     
        MPI_Finalize();
        return 0;
    }
    

13. Save the file. It should automatically build.
14. Verify there are no errors in the build.

Test Matrix

Testers

Committers: Clement Chu, Greg Watson, Craig Rasmussen, Beth Tibbitts

Version Grid

The following grid indicated the tools' versions used to carry out testing.

• I could upgrade to a more recent OpenMPI but since I might be the only one at 1.0.2 it's good to have it tested. I will probably try CDT 3.1.2 too since that has our AST traversal changes in it.

Platform Grid

The following machines/architectures should be considered when testing.

	Clement	Greg	Craig	Beth
Linux x86
Linux x86_64
Linux x86 w/ bproc
Linux x86_64 w/ bproc
Mac OS X ppc
Mac OS X x86
Windows x86 (PLDT only)

Build Grid

The following builds have been tested.

	Clement	Greg	Craig	Beth
pldt-1.1.0-I200701082248
ptp-1.1.0-I200701091500
ptp-1.1.0-I200701101100
ptp-1.1.0-I200701121100
ptp-1.1.0-I200701142000
ptp-1.1.0-I200701161500
ptp-1.1.0-I200701191300
ptp-1.1.0-I200702222000

Test Grid

This shows the test results based on the latest build tested.

Feature	Clement	Greg	Craig	Beth
Core/Runtime Tests
Runtime Views
Node Sets
Parallel Job Launch
Parallel Job Output
Job View
Debug Tests
Debug Launch & Terminate
Parallel Debug View
Register/Unregister Process				bug
Breakpoint Creation
Breakpoint Deletion
Breakpoint Hit
Breakpoint View
Global Breakpoint	Image hasn't changed to 'G'
Annotation
Step Command
Variable View				? bug?, notes in my office
PLDT Tests
MPI Artifacts
Command Completion
Hover Help
F1 Help
OpenMP Artifacts				*

• * some problems finding artifacts in some OpenMP code I hadn't tested for a while, will investigate

Core/Runtime Tests

Runtime Views

1. Switch to the PTP Runtime perspective.
   • Verify that the runtime system starts.
2. If the Machines View and Jobs View are not open, select them from Window->Show View->Other...
   • Verify that the Jobs View shows no jobs since it is a clean start.
   • Verify that the Machines View displays the correct number of nodes and node state for the current machine.
3. Double-click on a node icon in the Machines view.
   • Verify that the node information is displayed correctly in the Node Info panel.
4. Double-click on a different node.
   • Verify the node information is updated.
5. If on a bproc machine, use another terminal window change the state of one of the nodes (reboot it, change ownership, etc)
   • Verify that the node's status changes in the Machine View (both the icon to match the legend as well as the detailed text information to display the new change(s)).

Node Sets

1. Select some nodes in the Node View.
2. Click on the 'Create Set' button and enter a name for the set.
   • Verify that the user can switch between (focus on) the full set of nodes for the given machine and the newly created set.
3. Switch to the 'Root' set. Select some more nodes and select 'Add to set:' from the 'Create Set' menu.
   • Verify the nodes are added to the set.
4. Switch to the 'Root' set. Select a different set of nodes and create a new set.
   • Verify there are now three sets.
5. Select a node from this set and click the 'Remove Elements' button.
   • Verify the element is removed.
6. Click on the 'Delete Set' button
   • Verify the set is deleted.

Parallel Job Launch

1. Open the 'Run Configuration' dialog.
2. Select 'Parallel Application' and click the 'New' button.
3. In the 'Main' tab, select the 'Parallel Project' and 'Application program' to run.
4. In the 'Parallel' tab, specify the number of processes to run (say 4).
5. In the 'Debugger' tab, select the 'SDM' debugger.
6. Click on 'Run'.
   • Verify that the 4 (or whatever number you chose) nodes change state in the Machines View to specify they contain a running job and that the job starts on the correct machine.
7. Wait for the job to terminate.
   • Verify that the state information is correctly displayed in the Machine View (for the appropriate node).

Parallel Job Output

1. Re-run the same program.
2. Double-click on a node where one of the processes has been assigned.
   • Verify that the 'Process Info' panel displays the processes on that node, including which job the process belongs to.
3. Double-click on one of the processes in the 'Process Info' panel to bring up the Process View.
   • Verify that the MPI rank, node number, job number, and status are correct.
   • Verify there is process output in the output section of the Process View.

Jobs View

1. Bring the Jobs View to the foreground.
   • Verify the job previously run, as well as the processes contained within it, are listed and are shown as terminated.
2. Re-run the same job.
   • Verify the Job View displays the job as running and the processes as well.
3. Double-click on a process in the Job View, opening the Process View.
   • Verify the running state.
4. Terminate the job by using the terminate icon (re-run again if necessary).
   • Verify the Job View updates to show the terminated state.
   • Verify the Process View updates to show the terminated state, including an exit-code.

Debug Tests

Debug Launch and Terminate

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
   • Verify that the Parallel Debug View displays the status of processes as suspended (all should be yellow).
   • Verify that the job icon in the Parallel Debug View is a debug icon.
   • Verify that the new job is selected and all processes of this job will be displayed.
   • Verify that process 0 is registered (has a black box around it) and that the stack frame is visible in the Debug View.
   • Verify that the program stops automatically at the first line of code in main().
   • Verify that there is a register process current line marker and an unregistered process current line marker at line 7 of the code.
4. Select the Breakpoint View.
   • Verify that there are no breakpoints visible.
5. Click on the 'Terminate' button on the Parallel Debug View toolbar.
   • Verify that all processes have the exited status icon.
   • Verify that the job icon is shown as terminated.

Parallel Debug View

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Drag across a rectangular region to select some processes in the Parallel Debug View.
   • Verify that clicking with the left mouse button while holding down the Shift key extends the selection. See Bugzilla #169468
   • Verify that clicking with the left mouse button while holding down the Ctrl (Cmd) key adds new processes to the selection.
   • Verify that Ctrl-A (Cmd-A) selects all the processes.
5. If more than one page of processes is displayed:
   • Verify the Page-up key moves up one page.
   • Verify the Page-down key moves down one page.
   • Verify auto scrolling while dragging further up or down the views.
6. Make sure that only processes 0 and 2 are selected.
7. Click the 'Create Set' button. Enter 'even' for the set name.
   • Verify that the view changes from the 'Root' set to the newly created set.
8. Click the 'Change Set' button.
   • Verify that the view changes to the 'Root' set.
9. Select processes 1 and 3.
10. Click the 'Create Set' button.
11. Enter 'even' for the set name.
    • Verify that the OK button is disabled and a message that the name is already used is displayed.
12. Enter 'odd' for the set name and select OK.
13. Click on the background of the left hand portion of the Parallel Debug View (not on any jobs). You may need to remove terminated jobs to do this.
    • Verify that the processes are removed from the right hand portion of the View, and the registered process removed from the Debug View.
    • Verify that the current line markers are removed from the editor.
14. Click on debug job icon.
    • Verify that the processes are restored to the right hand portion of the View, and the registered process appears in the Debug View.
    • Verify that the current line markers are restored to the editor.
15. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Register / Unregister Process

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Double-click on process 3 in Parallel Debug View.
   • Verify that the process is displayed with a black square.
   • Verify that the process stack frame appears in the Debug View.
5. Select all processes in the Parallel Debug View and then click the 'Register Selected Elements' button.
   • Verify that all processes are registered in the debug view.
   • Note: If more than 10 processes have been selected to register, a dialog will popup and ask for confirmation.
6. Double-click on process 0.
   • Verify that the processes is unregistered from the debug view.
   • Verify that the black square is removed.
7. Select all processes in the Parallel Debug View and then click the 'Unregister Selected Elements' button.
   • Verify that all processes are unregistered in the debug view.
8. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Breakpoint Creation

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test.
5. Make sure the 'Root' set is selected.
6. Double-click on ruler at line 15 in the C/C++ source editor.
   • Verify a green breakpoint icon appears on the ruler at line 15.
7. Change to the 'even' set.
   • Verify that the color of the breakpoint at line 15 changes to blue.
8. Double-click on ruler at line 11 in the C/C++ source editor.
   • Verify a green breakpoint icon appears on the ruler at line 11.
9. Change to the 'odd' set.
   • Verify that the breakpoint on line 11 changes color to yellow and the breakpoint on line 15 remains blue.
10. Hover over the breakpoint on line 11.
    • Verfiy that the tooltips appears displaying details of breakpoint (job name, set name, file name and line number).
11. Select the 'Root' set.
12. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Breakpoint Deletion

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Double-click on ruler at line 15 in the C/C++ source editor.
   • Verify a green breakpoint icon appears on the ruler at line 15.
5. Double-click on ruler at line 15 in the C/C++ source editor.
   • Verify that the breakpoint icon is removed.
6. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Breakpoint Hit

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
5. Select the 'Root' set.
6. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that processes 0 and 2 are suspended on line 11 and processes 1 and 3 are suspended on line 15.
   • Note that it will take 10 seconds for processes 1 and 3 to suspend.
7. Double-click on the breakpoint at line 11 to delete it.
8. Select the 'even' set.
9. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that all processes are now suspended on line 15.
10. Select the 'Root' set.
11. Click on the 'Resume' button on the Parallel Debug View toolbar.
    • Verify that all processes are now exited and the job shows a terminated icon.

Breakpoint View

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
5. Select the Breakpoints View (if not already selected).
   • Verify that the breakpoints appear in the view with the same colors as those on the ruler.
6. From the Breakpoints View menu, choose 'Group By->Parallel Breakpoint Set'.
   • Verify that the breakpoints are ordered by set.
7. Uncheck (disable) the breakpoint on line 11 in Breakpoint View, the breakpoint will be disabled.
   • Verify that the breakpoint marker is dimmed.
8. Select the 'Root' set.
9. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that all processes are now suspended on line 15.
10. Right click on the breakpoint at line 11 in Breakpoint View and select Goto Set action
    • Verify that the 'even' set is selected.
11. Select the 'Root' set.
12. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Global Breakpoint

See Bugzilla #168773

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Unselect the current job by clicking on the background of the left panel in the Parallel Debug View.
5. Double-click on ruler at line 11 in the C/C++ source editor.
   • Verify that a global breakpoint is created (indicated with a 'G' symbol).
6. Click on the debug job icon.
7. Click on the 'Terminate' button on the Parallel Debug View toolbar.
8. Click the debug button to launch the debugger.
   • Verify that the global breakpoint remains set.
9. Click on the 'Resume' button.
   • Verify that all processes are suspended at the breakpoint.
10. Click on the 'Terminate' button on the Parallel Debug View toolbar.
11. Unselect the current job by clicking on the background of the left panel in the Parallel Debug View.
12. Double-click on ruler at line 11 in the C/C++ source editor.
    • Verify that a global breakpoint is deleted.

Annotation

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. By default process 0 is registered and all processes are suspended on line 7 (main method).
4. Two annotations should be marked on ruler of source editor at the line 7.
5. Move mouse hover on these annotations.
   • Verify that tooltip display "Suspended on registered process: 0" and "Suspended on unregistered processes: 1-3"
6. Register process 2 by double clicking on the process icon.
7. Move mouse hover annotation again on the ruler of source editor.
   • Verify that tooltip display "Suspended on registered processes: 0,2" and "Suspended on unregistered processes: 1,3"
8. Unselect job by clicking on job viewer.
   • Verify that no job selected, no icon displayed on process viewer, no debug item on Debug view and no Annotation on ruler.
9. Select job by clicking on job viewer.
   • Verify that job is selected, 4 process icons displayed on process viewer, two registered debug items on Debug view and two annotations at line 7 on ruler and display same tooltip as step 4.
10. Click 'Step over' button. All processes should be suspended at line 8.
    • Verify that two annotations at line 8 on ruler and display same tooltip as step 4.
11. Create a set called 'setA' for process 2 and process 3.
12. Process viewer should contain two process icons (2 and 3).
    • Verify that two annotations locate at line 8 on ruler and display tooltip "Suspended on unregistered processes: 0-1,3" and "Suspended on registered process: 2"
13. Click 'Step over' button. Process 2 and 3 will be stopped at line 10.
14. Three annotations will be shown on ruler.
    • Verify that an annotation locates at line 8 and display tooltip "Suspended on unregistered processes: 0-1"
    • Verify that two annotations locate at line 10 and display tooltip "Suspended on unregistered process: 3" and "Suspended on registered process: 2"
15. Switch to Set 'Root'. 4 process icons will be shown on Process viewer.
16. Four annotations will be shown on ruler.
    • Verify that two annotations locates at line 8 and display tooltip "Suspended on unregistered process: 1" and "Suspended on registered process: 0"
    • Verify that two annotations locate at line 10 and display tooltip "Suspended on unregistered process: 3" and "Suspended on registered process: 2"
17. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Step Command

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
5. Select the 'Root' set.
6. Create a breakpoint at line 10.
7. Click the 'Resume' button. All processes should be suspended on line 10.
8. Select the 'even' set.
9. Click the 'Step Over' button several times on the Parallel Debug View toolbar.
   • Verify that processes 0 and 2 step to line 11 and 12, while processes 1 and 3 remain suspended.
10. Select the 'Root' set.
11. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Variable View

See Bugzilla #169475

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Select the Variable View.
   • Verify that all the local variables are displayed in the view.
5. Click on 'Step Over' on the Parallel Debug View toolbar three times in quick succession.
   • Verify that the variables are displayed correctly in the view.
6. Click on the 'Terminate' button on the Parallel Debug View toolbar.

PLDT Tests

(See help files in the plug-in for more details on these)

MPI Artifacts

1. Launch Eclipse with PLDT (incl. CDT)
2. Help -> Help Contents, PTP Parallel Language Development Tools
3. Follow instructions in "Setup for PTP MPI tools". Basically, create a CDT project, a source file with MPI apis (a sample is included in help), and use Preferences to locate MPI header files.
4. Follow instructions in "Running PTP MPI tools" to set up.
5. Select a single source file in Navigator, then select MPI action from toolbar to run analysis.
6. MPI artifact view should be populated with markers, listing API name, source file name and line number, and artifact type (function or constant)
7. Double-click on any artifact, this should take you to the source file and line locating the artifact.
8. In source file, icon should be in marker bar, and squiggles should be under the artifact (e.g. function name)
9. Click on column headings in artifact view to sort by values in that column.
10. Also test selection of a container (e.g. a project) and see that all files within it are analyzed.
11. Also test multiple selection (e.g. of several source files in a project) and they should all get analyzed.
12. Also test hitting the "Cancel" button on the Progress Monitor during analysis. It should stop at least between the analysis of the files (cancelation is checked between the handling of each individual source file.)

Command Completion

1. In editor, type e.g. MPI_ and hit ctrl-space. Possible completions should appear.
2. Continue to type a few characters; possible completions should diminish to match string.
3. Hitting ENTER or double-clicking on any choice should insert the completed API into the file.

Hover Help

1. Hover on any MPI API, see yellow popup info on API and arguments, as well as some text that describes the API.

F1 Help

1. Click on any MPI API and hit F1 (Ctrl-F1 in Linux) to see Help view appear
2. Click on API in that view and see detailed info on that API.

OpenMP Artifacts

(See PLDT help for OpenMP for more information.) Adapted from http://www.llnl.gov/computing/tutorials/openMP/samples/C/omp_hello.c

#include <omp.h>
#include <stdio.h>
#include <stdlib.h>

int main (int argc, char *argv[]) {

int nthreads, tid;

/* Fork a team of threads giving them their own copies of variables */
#pragma omp parallel private(nthreads, tid)
  {

  /* Obtain thread number */
  tid = omp_get_thread_num();
  printf("Hello World from thread = %d\n", tid);

  /* Only master thread does this */
  if (tid == 0) 
    {
    nthreads = omp_get_num_threads();
    printf("Number of threads = %d\n", nthreads);
    }

  }  /* All threads join master thread and disband */
  return 0;
}
	

If you run this example, via launch configuration with environment variable:
OMP_NUM_THREADS=4

You should see output something like:

Hello World from thread = 0
Number of threads = 4
Hello World from thread = 1 
Hello World from thread = 2
Hello World from thread = 3 

1. Proceed like MPI artifacts above for finding artifacts, Help, Content Assist.
2. Problems view should be populated if problems found: e.g. barrier inside barrier.
3. Concurrency Analysis: Highlight an expression, rightClick context menu, choose "Show concurrency" and other expressions should appear in yellow