Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

PTP/testing/2.x

< PTP‎ | testing
Revision as of 10:32, 11 March 2008 by Tibbitts.us.ibm.com (Talk | contribs) (Test Grid for PTP Core)

Test Plan for PTP 2.x Release

This plan describes the tests that will be undertaken to verify the 2.x series of PTP releases.

Test Setup

The following steps should be carried out prior to testing PTP. Refer to the release notes for the appropriate PTP version if necessary.

  1. Install OpenMPI 1.2. Make sure it is configured with the '--with-devel-headers' option.
  2. If you are testing on a single node, edit <openmpi_install>/etc/openmpi-default-hostfile and add lines containing 'node0', 'node1', etc. Edit /etc/hosts and add a localhost entry for each name you added to the openmpi-default-hostfile. So, for 'node0', add '127.0.0.1 node0' to /etc/hosts. This will simulate a multi-node machine.
  3. Verify OpenMPI is working by compiling and launching a simple program from the command line.
  4. Start with a fresh Eclipse and CDT install (including workspace), as per the release notes.
  5. Install PTP using instructions from the release notes.
  6. Launch Eclipse on the test machine.
  7. Switch to the C/C++ Perspective.
  8. Create a new C Project....
  9. [If PLDT is installed ]In the project setup wizard, create a new MPI Project, either one of the wizard projects (MPI Hello World, MPI Pi, or Empty MPI project.) If you have not put MPI include path into Preferences, cancel out of the new project setup and do it now; then restart new project wizard.
  10. [If PLDT is not installed.] Create a new C project. Then right-click on the new project and choose 'Properties'. Select the 'C/C++ Build' item. Change the 'GCC C Compiler' and '... C Linker' settings to 'mpicc'.
  11. If you did not choose a wizard project above, Create a new C source file in the project, and Add the following code:
    #include <stdio.h>
    #include <mpi.h>
    int main(int argc, char *argv[])
    {
        int i, rank;
     
        MPI_Init(&argc, &argv);
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
     
        for (i = 0; i < 10; i++) {
           printf("hello from %d loop %d\n", rank, i);
           sleep(1);
        }
     
        MPI_Finalize();
        return 0;
    }
    

    Save the file. It should automatically build.</li>

  12. Verify there are no errors in the build.
  13. </ol>

    Test Matrix

    Testers

    Committers: Greg Watson, Beth Tibbitts

    Version Grid

    The following grid indicated the tools' versions used to carry out testing.

      Greg Beth
    Eclipse 3.3.1.1 3.3.2
    CDT 4.0.3 4.0.3
    Open MPI 1.2.5 1.2.4
    GDB 6.3,6.4,6.5,6.6 6.6
    Java JDK 1.5,1.6 JDK 1.5

    Platform Grid

    The following machines/architectures should be considered when testing.

      Greg Beth
    Linux x86 Tick.gif Tick.gif
    Linux x86_64
    Linux x86 w/ bproc
    Linux x86_64 w/ bproc
    Mac OS X ppc
    Mac OS X x86 Tick.gif **
    Windows x86 (PLDT & Remote UI) Tick.gif

    ** Beth: I can test Mac too if necessary, but should probably focus on Linux & Windows since Greg is testing Mac

    Build Grid

    The following builds have been tested.

      Greg Beth
    200801232144 Tick.gif  
    200802250908   Tick.gif
    200803101629   Tick.gif

    Notes: 2/25 build: Beth tested Linux, not remote Windows

    Test Grid for PTP Core

    This shows the test results based on the latest build tested.

    ? Should we specify tasks specific to local vs. remote run/debug?

    Feature Greg Beth
    Core/Runtime Tests
    Runtime Views Tick.gif Bug 222232
    Node Sets
    Parallel Job Launch Tick.gif
    Parallel Job Output Tick.gif
    Job View Tick.gif
    Debug Tests
    Debug Launch & Terminate Tick.gif
    Parallel Debug View Tick.gif
    Register/Unregister Process Tick.gif
    Breakpoint Creation Tick.gif
    Breakpoint Deletion Tick.gif
    Breakpoint Hit Tick.gif
    Breakpoint View Tick.gif
    Global Breakpoint Tick.gif
    Annotation Tick.gif
    Step Command Tick.gif
    Variable View Tick.gif

    Test Grid for PLDT

    This shows the test results based on the latest build tested.

    Feature Greg Beth
    PLDT Tests
    New Project Wizards OK
    Command Completion OK
    Hover and F1 Help hover OK; F1 ?
    MPI Artifact Location OK
    MPI Barrier Analysis OK but filed Bug 222223
    OpenMP Artifact Location
    Other OpenMP Features

    Notes:

    • hard to test F1 help on Parallels VM.
    • 3/10 build: filed Bug 222223 Barrier matching view has no child nodes with dup barrier calls (probably minor)

    Core/Runtime Tests

    Runtime Views

    1. Switch to the PTP Runtime perspective.
      • Verify that the runtime system starts.
    2. If the Machines View and Jobs View are not open, select them from Window->Show View->Other...
      • Verify that the Jobs View shows no jobs since it is a clean start.
      • Verify that the Machines View displays the correct number of nodes and node state for the current machine.
    3. Double-click on a node icon in the Machines view.
      • Verify that the node information is displayed correctly in the Node Info panel.
    4. Double-click on a different node.
      • Verify the node information is updated.
    5. If on a bproc machine, use another terminal window change the state of one of the nodes (reboot it, change ownership, etc)
      • Verify that the node's status changes in the Machine View (both the icon to match the legend as well as the detailed text information to display the new change(s)).

    Node Sets

    1. Select some nodes in the Node View.
    2. Click on the 'Create Set' button and enter a name for the set.
      • Verify that the user can switch between (focus on) the full set of nodes for the given machine and the newly created set.
    3. Switch to the 'Root' set. Select some more nodes and select 'Add to set:' from the 'Create Set' menu.
      • Verify the nodes are added to the set.
    4. Switch to the 'Root' set. Select a different set of nodes and create a new set.
      • Verify there are now three sets.
    5. Select a node from this set and click the 'Remove Elements' button.
      • Verify the element is removed.
    6. Click on the 'Delete Set' button
      • Verify the set is deleted.

    Parallel Job Launch

    1. Open the 'Run Configuration' dialog.
    2. Select 'Parallel Application' and click the 'New' button.
    3. In the 'Main' tab, select the 'Parallel Project' and 'Application program' to run.
    4. In the 'Parallel' tab, specify the number of processes to run (say 4).
    5. In the 'Debugger' tab, select the 'SDM' debugger.
    6. Click on 'Run'.
      • Verify that the 4 (or whatever number you chose) nodes change state in the Machines View to specify they contain a running job and that the job starts on the correct machine.
    7. Wait for the job to terminate.
      • Verify that the state information is correctly displayed in the Machine View (for the appropriate node).

    Parallel Job Output

    1. Re-run the same program.
    2. Double-click on a node where one of the processes has been assigned.
      • Verify that the 'Process Info' panel displays the processes on that node, including which job the process belongs to.
    3. Double-click on one of the processes in the 'Process Info' panel to bring up the Process View.
      • Verify that the MPI rank, node number, job number, and status are correct.
      • Verify there is process output in the output section of the Process View.

    Jobs View

    1. Bring the Jobs View to the foreground.
      • Verify the job previously run, as well as the processes contained within it, are listed and are shown as terminated.
    2. Re-run the same job.
      • Verify the Job View displays the job as running and the processes as well.
    3. Double-click on a process in the Job View, opening the Process View.
      • Verify the running state.
    4. Terminate the job by using the terminate icon (re-run again if necessary).
      • Verify the Job View updates to show the terminated state.
      • Verify the Process View updates to show the terminated state, including an exit-code.

    Debug Tests

    Debug Launch and Terminate

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
      • Verify that the Parallel Debug View displays the status of processes as suspended (all should be yellow).
      • Verify that the job icon in the Parallel Debug View is a debug icon.
      • Verify that the new job is selected and all processes of this job will be displayed.
      • Verify that process 0 is registered (has a black box around it) and that the stack frame is visible in the Debug View.
      • Verify that the program stops automatically at the first line of code in main().
      • Verify that there is a register process current line marker and an unregistered process current line marker at line 7 of the code.
    4. Select the Breakpoint View.
      • Verify that there are no breakpoints visible.
    5. Click on the 'Terminate' button on the Parallel Debug View toolbar.
      • Verify that all processes have the exited status icon.
      • Verify that the job icon is shown as terminated.

    Parallel Debug View

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Drag across a rectangular region to select some processes in the Parallel Debug View.
      • Verify that clicking with the left mouse button while holding down the Shift key extends the selection. See Bugzilla #169468
      • Verify that clicking with the left mouse button while holding down the Ctrl (Cmd) key adds new processes to the selection.
      • Verify that Ctrl-A (Cmd-A) selects all the processes.
    5. If more than one page of processes is displayed:
      • Verify the Page-up key moves up one page.
      • Verify the Page-down key moves down one page.
      • Verify auto scrolling while dragging further up or down the views.
    6. Make sure that only processes 0 and 2 are selected.
    7. Click the 'Create Set' button. Enter 'even' for the set name.
      • Verify that the view changes from the 'Root' set to the newly created set.
    8. Click the 'Change Set' button.
      • Verify that the view changes to the 'Root' set.
    9. Select processes 1 and 3.
    10. Click the 'Create Set' button.
    11. Enter 'even' for the set name.
      • Verify that the OK button is disabled and a message that the name is already used is displayed.
    12. Enter 'odd' for the set name and select OK.
    13. Click on the background of the left hand portion of the Parallel Debug View (not on any jobs). You may need to remove terminated jobs to do this.
      • Verify that the processes are removed from the right hand portion of the View, and the registered process removed from the Debug View.
      • Verify that the current line markers are removed from the editor.
    14. Click on debug job icon.
      • Verify that the processes are restored to the right hand portion of the View, and the registered process appears in the Debug View.
      • Verify that the current line markers are restored to the editor.
    15. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Register / Unregister Process

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Double-click on process 3 in Parallel Debug View.
      • Verify that the process is displayed with a black square.
      • Verify that the process stack frame appears in the Debug View.
    5. Select all processes in the Parallel Debug View and then click the 'Register Selected Elements' button.
      • Verify that all processes are registered in the debug view.
      • Note: If more than 10 processes have been selected to register, a dialog will popup and ask for confirmation.
    6. Double-click on process 0.
      • Verify that the processes is unregistered from the debug view.
      • Verify that the black square is removed.
    7. Select all processes in the Parallel Debug View and then click the 'Unregister Selected Elements' button.
      • Verify that all processes are unregistered in the debug view.
    8. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Breakpoint Creation

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test.
    5. Make sure the 'Root' set is selected.
    6. Double-click on ruler at line 15 in the C/C++ source editor.
      • Verify a green breakpoint icon appears on the ruler at line 15.
    7. Change to the 'even' set.
      • Verify that the color of the breakpoint at line 15 changes to blue.
    8. Double-click on ruler at line 11 in the C/C++ source editor.
      • Verify a green breakpoint icon appears on the ruler at line 11.
    9. Change to the 'odd' set.
      • Verify that the breakpoint on line 11 changes color to yellow and the breakpoint on line 15 remains blue.
    10. Hover over the breakpoint on line 11.
      • Verfiy that the tooltips appears displaying details of breakpoint (job name, set name, file name and line number).
    11. Select the 'Root' set.
    12. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Breakpoint Deletion

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Double-click on ruler at line 15 in the C/C++ source editor.
      • Verify a green breakpoint icon appears on the ruler at line 15.
    5. Double-click on ruler at line 15 in the C/C++ source editor.
      • Verify that the breakpoint icon is removed.
    6. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Breakpoint Hit

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
    5. Select the 'Root' set.
    6. Click on the 'Resume' button on the Parallel Debug View toolbar.
      • Verify that processes 0 and 2 are suspended on line 11 and processes 1 and 3 are suspended on line 15.
      • Note that it will take 10 seconds for processes 1 and 3 to suspend.
    7. Double-click on the breakpoint at line 11 to delete it.
    8. Select the 'even' set.
    9. Click on the 'Resume' button on the Parallel Debug View toolbar.
      • Verify that all processes are now suspended on line 15.
    10. Select the 'Root' set.
    11. Click on the 'Resume' button on the Parallel Debug View toolbar.
      • Verify that all processes are now exited and the job shows a terminated icon.

    Breakpoint View

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
    5. Select the Breakpoints View (if not already selected).
      • Verify that the breakpoints appear in the view with the same colors as those on the ruler.
    6. From the Breakpoints View menu, choose 'Group By->Parallel Breakpoint Set'.
      • Verify that the breakpoints are ordered by set.
    7. Uncheck (disable) the breakpoint on line 11 in Breakpoint View, the breakpoint will be disabled.
      • Verify that the breakpoint marker is dimmed.
    8. Select the 'Root' set.
    9. Click on the 'Resume' button on the Parallel Debug View toolbar.
      • Verify that all processes are now suspended on line 15.
    10. Right click on the breakpoint at line 11 in Breakpoint View and select Goto Set action
      • Verify that the 'even' set is selected.
    11. Select the 'Root' set.
    12. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Global Breakpoint

    See Bugzilla #168773

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Unselect the current job by clicking on the background of the left panel in the Parallel Debug View.
    5. Double-click on ruler at line 11 in the C/C++ source editor.
      • Verify that a global breakpoint is created (indicated with a 'G' symbol).
    6. Click on the debug job icon.
    7. Click on the 'Terminate' button on the Parallel Debug View toolbar.
    8. Click the debug button to launch the debugger.
      • Verify that the global breakpoint remains set.
    9. Click on the 'Resume' button.
      • Verify that all processes are suspended at the breakpoint.
    10. Click on the 'Terminate' button on the Parallel Debug View toolbar.
    11. Unselect the current job by clicking on the background of the left panel in the Parallel Debug View.
    12. Double-click on ruler at line 11 in the C/C++ source editor.
      • Verify that a global breakpoint is deleted.

    Annotation

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. By default process 0 is registered and all processes are suspended on line 7 (main method).
    4. Two annotations should be marked on ruler of source editor at the line 7.
    5. Move mouse hover on these annotations.
      • Verify that tooltip display "Suspended on registered process: 0" and "Suspended on unregistered processes: 1-3"
    6. Register process 2 by double clicking on the process icon.
    7. Move mouse hover annotation again on the ruler of source editor.
      • Verify that tooltip display "Suspended on registered processes: 0,2" and "Suspended on unregistered processes: 1,3"
    8. Unselect job by clicking on job viewer.
      • Verify that no job selected, no icon displayed on process viewer, no debug item on Debug view and no Annotation on ruler.
    9. Select job by clicking on job viewer.
      • Verify that job is selected, 4 process icons displayed on process viewer, two registered debug items on Debug view and two annotations at line 7 on ruler and display same tooltip as step 4.
    10. Click 'Step over' button. All processes should be suspended at line 8.
      • Verify that two annotations at line 8 on ruler and display same tooltip as step 4.
    11. Create a set called 'setA' for process 2 and process 3.
    12. Process viewer should contain two process icons (2 and 3).
      • Verify that two annotations locate at line 8 on ruler and display tooltip "Suspended on unregistered processes: 0-1,3" and "Suspended on registered process: 2"
    13. Click 'Step over' button. Process 2 and 3 will be stopped at line 10.
    14. Three annotations will be shown on ruler.
      • Verify that an annotation locates at line 8 and display tooltip "Suspended on unregistered processes: 0-1"
      • Verify that two annotations locate at line 10 and display tooltip "Suspended on unregistered process: 3" and "Suspended on registered process: 2"
    15. Switch to Set 'Root'. 4 process icons will be shown on Process viewer.
    16. Four annotations will be shown on ruler.
      • Verify that two annotations locates at line 8 and display tooltip "Suspended on unregistered process: 1" and "Suspended on registered process: 0"
      • Verify that two annotations locate at line 10 and display tooltip "Suspended on unregistered process: 3" and "Suspended on registered process: 2"
    17. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Step Command

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
    5. Select the 'Root' set.
    6. Create a breakpoint at line 10.
    7. Click the 'Resume' button. All processes should be suspended on line 10.
    8. Select the 'even' set.
    9. Click the 'Step Over' button several times on the Parallel Debug View toolbar.
      • Verify that processes 0 and 2 step to line 11 and 12, while processes 1 and 3 remain suspended.
    10. Select the 'Root' set.
    11. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    Variable View

    See Bugzilla #169475

    1. Switch to the Parallel Debug Perspective.
    2. Ensure that the default debug launch configuration will launch a 4 process job.
    3. Click the debug button to launch the debugger.
    4. Select the Variable View.
      • Verify that all the local variables are displayed in the view.
    5. Click on 'Step Over' on the Parallel Debug View toolbar three times in quick succession.
      • Verify that the variables are displayed correctly in the view.
    6. Click on the 'Terminate' button on the Parallel Debug View toolbar.

    PLDT Tests

    (See help files in the plug-in for more details on these)

    New Project Wizards

    1. Launch Eclipse with PLDT (incl. CDT)
    2. Help -> Help Contents, PTP Parallel Language Development Tools
    3. Preferences Dialog, under Parallel Language Development Tools, MPI, set MPI include path
    4. New C Project, choose MPI hello World project
    5. Fill in dialogs and hit Finish.
    6. Project should build.
    7. Do the same for OpenMP project, and Empty projects

    Command Completion

    1. In editor, type e.g. MPI_ and hit ctrl-space. Possible completions should appear.
    2. Continue to type a few characters; possible completions should diminish to match string.
    3. Hitting ENTER or double-clicking on any choice should insert the completed API into the file.

    Hover and F1 Help

    1. Hover on any MPI API, see yellow popup info on API and arguments, as well as some text that describes the API.
    2. Click on any MPI API and hit F1 (Ctrl-F1 in Linux) to see Help view appear
    3. Click on API in that view and see detailed info on that API.
    4. Similar testing for OpenMP

    MPI Artifact Location

    1. Select a single source file in C/C++ projects view, then select Find MPI Artifacts action from PLDT toolbar icon menu, to run analysis.
    2. MPI artifact view should be populated with markers, listing API name, source file name and line number, and artifact type (function or constant)
    3. Double-click on any artifact, this should take you to the source file and line locating the artifact.
    4. In source file, icon should be in marker bar, and squiggles should be under the artifact (e.g. function name)
    5. Click on column headings in artifact view to sort by values in that column.
    6. Also test selection of a container (e.g. a project) and see that all files within it are analyzed.
    7. Also test multiple selection (e.g. of several source files in a project) and they should all get analyzed.
    8. Also test hitting the "Cancel" button on the Progress Monitor during analysis. It should stop at least between the analysis of the files (cancelation is checked between the handling of each individual source file.)

    MPI Barrier Analysis

    (Need to fine-tune this, and provide sample code)

    1. Select MPI source file, or container of files, in C/C++ Projects view
    2. PLDT menu, MPI Barrier Analysis
    3. Bring up Barrier View (Window, Show View, MPI, MPI Barrier Sets)
    4. See sets of matching barriers, if any
    5. Cause a barrier mismatch and do the above.
    6. Do the same on a large project with Barriers found (and matched) in multiple files.

    OpenMP Artifact Location

    (See PLDT help for OpenMP for more information.) The following source Adapted from http://www.llnl.gov/computing/tutorials/openMP/samples/C/omp_hello.c - can also use sample code in the wizard.

    #include <omp.h>
    #include <stdio.h>
    #include <stdlib.h>
    
    int main (int argc, char *argv[]) {
    
    int nthreads, tid;
    
    /* Fork a team of threads giving them their own copies of variables */
    #pragma omp parallel private(nthreads, tid)
      {
    
      /* Obtain thread number */
      tid = omp_get_thread_num();
      printf("Hello World from thread = %d\n", tid);
    
      /* Only master thread does this */
      if (tid == 0) 
        {
        nthreads = omp_get_num_threads();
        printf("Number of threads = %d\n", nthreads);
        }
    
      }  /* All threads join master thread and disband */
      return 0;
    }
    	
    
    

    If you run this example, via launch configuration with environment variable:
    OMP_NUM_THREADS=4

    You should see output something like:

    Hello World from thread = 0
    Number of threads = 4
    Hello World from thread = 1 
    Hello World from thread = 2
    Hello World from thread = 3 
    
    1. Proceed like MPI artifacts above for finding artifacts, Help, and Content Assist.
    2. Problems view should be populated if problems found: e.g. barrier inside barrier.

    Other OpenMP Features

    1. Concurrency Analysis: Highlight an expression, rightClick context menu, choose "Show concurrency" and other expressions should appear in yellow
      • need to attach sample code for this here.

Back to the top