Difference between revisions of "PTP/testing/1.x"

Revision as of 20:47, 19 December 2006

Test Plan for PTP 1.x Release

This plan describes the tests that will be undertaken to verify the 1.x series of PTP releases.

Test Setup

The following steps should be carried out prior to testing PTP. Refer to the release notes for the appropriate PTP version if necessary.

1. Install OpenMPI 1.2. Make sure it is configure with the '--with-devel-headers' option.
2. If you are testing on a single node, edit <openmpi_install>/etc/openmpi-default-hostfile and add lines containing 'node0', 'node1', etc. Edit /etc/hosts and add a localhost entry for each name you added to the openmpi-default-hostfile. So, for 'node0', add '127.0.0.1 node0' to /etc/hosts. This will simulate a multi-node machine.
3. Verify OpenMPI is working by compiling and launching a simple program from the command line.
4. Start with a fresh Eclipse and CDT install (including workspace), as per the release notes.
5. Install PTP using instructions from the release notes.
6. Launch Eclipse on the test machine.
7. Switch to the C/C++ Perspective.
8. Create a new Managed Make C Project. Accept the default settings.
9. [This step only necessary if PLDT is not installed.] Right-click on the new project and choose 'Properties'. Select the 'C/C++ Build' item. Change the 'GCC C Compiler' and '... C Linker' settings to 'mpicc'.
10. Create a new C source file in the project.
11. Add the following code:

    #include <stdio.h>
    #include <mpi.h>
    int main(int argc, char *argv[])
    {
        int i, rank;
     
        MPI_Init(&argc, &argv);
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
     
        for (i = 0; i < 10; i++) {
           printf("hello from %d loop %d\n", rank, i);
           sleep(1);
        }
     
        MPI_Finalize();
        return 0;
    }
    

12. Save the file. It should automatically build.
13. Verify there are no errors in the build.

Testing should be carried out on the following machines/architectures:

• Linux x86
• Linux x86_64
• Linux x86 w/ bproc
• Linux x86_64 w/ bproc
• MacOS X ppc
• MacOS X x86

Tests must be carried out in the order shown below.

Core/Runtime Tests

Runtime Views

1. Switch to the PTP Runtime perspective.
   • Verify that the runtime system starts.
2. If the Machines View and Jobs View are not open, select them from Window->Show View->Other...
   • Verify that the Jobs View shows no jobs since it is a clean start.
   • Verify that the Machines View displays the correct number of nodes and node state for the current machine.
3. Double-click on a node.
   • Verify that the node information is displayed correctly in the Node Info panel.
4. Double-click on a different node.
   • Verify the node information is updated.
5. Switch back to the Machine View.
6. If on a bproc machine, use another terminal window change the state of one of the nodes (reboot it, change ownership, etc)
   • Verify that the node's status changes in the Machine View (both the icon to match the legend as well as the detailed text information to display the new change(s)).

Node Sets

1. Select some nodes in the Node View.
2. Click on the 'Create Set' button and enter a name for the set.
   • Verify that the user can switch between (focus on) the full set of nodes for the given machine and the newly created set.
3. Switch to the 'Root' set. Select some more nodes and select 'Add to set:' from the 'Create Set' menu.
   • Verify the nodes are added to the set.
4. Switch to the 'Root' set. Select a different set of nodes and create a new set.
   • Verify there are now three sets.
5. Select a node from this set and click the 'Remove Elements' button.
   • Verify the element is removed.
6. Click on the 'Delete Set' button
   • Verify the set is deleted.

Parallel Job Launch

1. Open the 'Run Configuration' dialog.
2. Select 'Parallel Application' and click the 'New' button.
3. In the 'Main' tab, select the 'Parallel Project' and 'Application program' to run.
4. In the 'Parallel' tab, specify the number of processes to run (say 4).
5. In the 'Debugger' tab, select the 'SDM' debugger.
6. Click on 'Run'.
   • Verify that the 4 (or whatever number you chose) nodes change state in the Machines View to specify they contain a running job and that the job starts on the correct machine.
7. Wait for the job to terminate.
   • Verify that the state information is correctly displayed in the Machine View (for the appropriate node).

Parallel Job Output

1. Re-run the same program.
2. Double-click on a node where one of the processes has been assigned.
   • Verify that the 'Process Info' panel displays the processes on that node, including which job the process belongs to.
3. Double-click on one of the processes in the 'Process Info' panel to bring up the Process View.
   • Verify that the MPI rank, node number, job number, and status are correct.
   • Verify there is process output in the output section of the Process View.

Jobs View

1. Bring the Jobs View to the foreground.
   • Verify the job previously run, as well as the processes contained within it, are listed and are shown as terminated.
2. Re-run the same job.
   • Verify the Job View displays the job as running and the processes as well.
3. Double-click on a process in the Job View, opening the Process View.
   • Verify the running state.
4. Terminate the job by using the terminate icon (re-run again if necessary).
   • Verify the Job View updates to show the terminated state.
   • Verify the Process View updates to show the terminated state, including an exit-code.

Debug Tests

Debug Launch and Terminate

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
   • Verify that the Parallel Debug View displays the status of processes as suspended (all should be yellow).
   • Verify that the job icon in the Parallel Debug View is a debug icon.
   • Verify that the new job is selected and all processes of this job will be displayed.
   • Verify that process 0 is registered (has a black box around it) and that the stack frame is visible in the Debug View.
   • Verify that the program stops automatically at the first line of code in main().
   • Verify that there is a register process current line marker and an unregistered process current line marker at line 7 of the code.
4. Select the Breakpoint View.
   • Verify that there are no breakpoints visible.
5. Click on the 'Terminate' button on the Parallel Debug View toolbar.
   • Verify that all processes have the exited status icon.
   • Verify that the job icon is shown as terminated.

Parallel Debug View

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Drag across a rectangular region to select some processes in the Parallel Debug View.
   • Verify that clicking with the left mouse button while holding down the Shift key extends the selection.
   • Verify that clicking with the left mouse button while holding down the Ctrl (Cmd) key adds new processes to the selection.
   • Verify that Ctrl-A (Cmd-A) selects all the processes.
5. If more than one page of processes is displayed:
   • Verify the Page-up key moves up one page.
   • Verify the Page-down key moves down one page.
   • Verify auto scrolling while dragging further up or down the views.
6. Make sure that only processes 0 and 2 are selected.
7. Click the 'Create Set' button. Enter 'even' for the set name.
   • Verify that the view changes from the 'Root' set to the newly created set.
8. Click the 'Change Set' button.
   • Verify that the view changes to the 'Root' set.
9. Select processes 1 and 3.
10. Click the 'Create Set' button.
11. Enter 'even' for the set name.
    • Verify that the OK button is disabled and a message that the name is already used is displayed.
12. Enter 'odd' for the set name and select OK.
13. Click on the background of the left hand portion of the Parallel Debug View (not on any jobs).
    • Verify that the processes are removed from the right hand portion of the View, and the registered process removed from the Debug View.
    • Verify that the current line markers are removed from the editor.
14. Click on debug job icon.
    • Verify that the processes are restored to the right hand portion of the View, and the registered process appears in the Debug View.
    • Verify that the current line markers are restored to the editor.
15. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Breakpoint Creation

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test.
5. Make sure the 'Root' set is selected.
6. Double-click on ruler at line 15 in the C/C++ source editor.
   • Verify a green breakpoint icon appears on the ruler at line 15.
7. Change to the 'even' set.
   • Verify that the color of the breakpoint at line 15 changes to blue.
8. Double-click on ruler at line 11 in the C/C++ source editor.
   • Verify a green breakpoint icon appears on the ruler at line 11.
9. Change to the 'odd' set.
   • Verify that the breakpoint on line 11 changes color to yellow and the breakpoint on line 15 remains blue.
10. Hover over the breakpoint on line 11.
    • Verfiy that the tooltips appears displaying details of breakpoint (job name, set name, file name and line number).
11. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Test DB_3: Breakpoint hit

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
5. Select the 'Root' set.
6. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that processes 0 and 2 are suspended on line 11 and processes 1 and 3 are suspended on line 15.
7. Select the 'even' set.
8. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that all processes are now suspended on line 15.
9. Select the 'Root' set.
10. Click on the 'Resume' button on the Parallel Debug View toolbar.
    • Verify that all processes are now exited and the job shows a terminated icon.

Test DB_5: Breakpoint View

1. Switch to the Parallel Debug Perspective.
2. Ensure that the default debug launch configuration will launch a 4 process job.
3. Click the debug button to launch the debugger.
4. Create the 'even' and 'odd' sets used in the 'Parallel Debug View' test and set the breakpoints described in the 'Breakpoint Creation' test.
5. Select the Breakpoints View (if not already selected).
   • Verify that the breakpoints appear in the view with the same colors as those on the ruler.
6. From the Breakpoints View menu, choose 'Group By->Parallel Breakpoint Set'.
   • Verify that the breakpoints are orded by set.
7. Uncheck (disable) the breapoint on line 11 in Breakpoint View, the breakpoint will be disabled.
   • Verify that the breakpoint marker is dimmed.
8. Select the 'Root' set.
9. Click on the 'Resume' button on the Parallel Debug View toolbar.
   • Verify that all processes are now suspended on line 15.
10. Right click on the breakpoint at line 11 in Breakpoint View and select Goto Set action
    • Verify that the 'even' set is selected.
11. Click on the 'Terminate' button on the Parallel Debug View toolbar.

Test DB_4: Step command

1. Select the 'even' set.
2. Click the 'Step Over' button on the Parallel Debug View toolbar.
   • Verify that processes 0 and 2 step to line 12, processes 1 and 3 remain suspended.
3. Click on the 'Terminate' button on the Parallel Debug View toolbar.

1. Double on the created breakpoint on ruler, the breakpoint will be removed.
2. The global breakpoint only can be created or removed when there is no job selected.
3. Multiple breakpoints can be created in the same source file and the same line number with different jobs.

1. If the parallel breakpoint is created in the condition that is no job running or unselected all the jobs, this breakpoint will belong to be global for all the jobs and there is an additional overlayered image on the breakpoint.

Test DB_2: Basic Debug Commands

1. Double click the ruler at any line in the source code window to set a breakpoint, set another breakpoint after the first breakpoint.

1. Process 0 will be registered automatically in the Debug view, 2 Console views will also be opened (one for process 0, and the other for the debugger output).
2. After printing the output for process 0, which can be verified by opening the Console view for process 0, all processes (process 0 and others) will break at the first breakpoint (since the breakpoint applies for all processes).
3. Verify this by looking at the Debug view and see that process 0 is suspended. Look at the Parallel Debug view, all processes should change to yellow. Look at the # Console view for process 0, the output should stop. Look at the Console view for the debugger, there should be text saying that BreakpointEvent has been received.
4. Double click process 1 in the Parallel Debug view to register it to the Debug view. Look at the Debug view and see that process 1 has been registered and in the suspended state. Look at the Console view and verify that Console view for process 1 has been created.
5. Select process 0 and click the Step Over button in the Debug view. Look at the Console view for process 0 to verify that the action has been executed successfully. # Repeat and verify this action several time
6. Select process 1 and click the Step Into button in the Debug view. Look at the Debug view and see that the number of stackframes for process 1 increases by one. # Click Step Into to increase the stackframe and click Step Finish to decrease the stackframe. Repeat and verify this action several times. The maximum number of stackframes is 5.
7. Select the stackframe of process 0 and open the Variables view. Verify that there are 2 argument variables and 2 local variables.
8. Select process 0 and click the Resume button, it will then hit the second breakpoint. Click the Resume button again to run process 0 until completion (it prints 20 lines and then exits). Look at the Debug view and the Console view and verify that process 0 has exited.
9. Select process 1 and click the Terminate button. Look at the Debug view and the Console view and verify that process 1 has been terminated.

Test DB_6: Register / Unregister Process

1. Double click on the process in parallel debug view and the process will be registered into the debug view. A registered process is presented by a black rectangle.
2. Selecting multiple processes in the parallel debug view and then click the register button will register all of them in the debug view.
3. If more than 10 processes have been selected to register, a dialog will popup and ask for confirmation.
4. Double click on the registered process will unregister it from the debug view. The black rectangle representing the registration will be removed.
5. Selecting multiple registered processes in the parallel debug view and then click the unregister button will unregister all of them in the debug view.
6. The debug view will only display registered processes which is according to the current job and current set.

Test DB_7: Annotation

1. If a breakpoint location contains multiple markers, annotations for both these markers will be overlapped when the breakpoint is hit. The registered process marker should appear on top. Tooltips will display process task id of each marker set in this location, grouped by its marker.
2. The annotation will change to another color(not created) when switch to another set.
3. The process icon appears in yellow process is in suspended mode.
4. The process icon appears in green when in running mode.
5. The process icon appears in red when in terminate mode.
6. The resume button and all step buttons in the parallel debug view will only be enabled when one of the processes is suspended.
7. The suspend button in the parallel debug view will only be enabled when one of the processes is running.
8. The terminate button in the parallel debug view will only be enabled when one othe processes is either running or suspended.
9. Resume button in the parallel debug view resumes all suspended processes in the current set.
10. Suspend button in the parallel debug view pauses all running processes in the current set.
11. Terminate button in the parallel debug view stops all suspended and running processes in the current set.
12. Step buttons in the parallel debug view can step into, step over and step return all suspended processes in the current set.
13. Resume button in the debug view can resume selected suspended process(es).
14. Suspend button in the debug view can pause selected running process(es).
15. Terminate button in the parallel debug view can stop any suspended or running process(es).
16. Step buttons in the debug view can step into, step over and step return any suspended process(es).

Parallel Language Development Tools

(See help files in the plug-in for more details on these)

Test PLDT_1: MPI Artifacts

1. Launch eclipse with PTP (incl. CDT)
2. Help -> Help Contents, PTP MPI Tools
3. Follow instructions in "Setup for PTP MPI tools". Basically, create a CDT project, a source file with MPI apis (a sample is included in help), and use Preferences to locate MPI header files.
4. Follow instructions in "Running PTP MPI tools" to set up Open MPI artifacts view (Do we need to explicitly add the actions to the toolbar?)

select source vile in Navigator, then select MPI action to run analysis.

1. MPI artifact view should be populated with markers, listing API name, source file name and line number, and artifact type (function or constant)
2. Double-click on any artifact, this should take you to the source file and line locating the artifact.
3. In source file, icon should be in marker bar, and squiggles should be under the artifact (e.g. function name)
4. Click on column headings in artifact view to sort by values in that column.

Test PLDT_2: Command Completion

1. In editor, type e.g. MPI_ and hit ctrl-space. Possible completions should appear.
2. Continue to type a few characters; possible completions should diminish to match string.
3. Hitting ENTER or double-clicking on any choice should insert the completed API into the file.

Test PLDT_3: Hover Help

1. Hover on any MPI API, see yellow popup info on API and arguments.

Test PLDT_4: F1 Help

1. Click on any MPI API and hit F1 to see Help view appear
2. C\lick on API in that view and see detailed info on that API.

Fortran Development Tools

SETUP

1. In a new workspace, check out the following CDT components from dev.eclipse.org:/cvsroot/tools
   • org.eclipse.cdt-build/
   • org.eclipse.cdt.make.core
   • org.eclipse.cdt.make.ui
   • org.eclipse.cdt.managedbuilder.core
   • org.eclipse.cdt.managedbuilder.gnu.ui
   • org.eclipse.cdt.managedbuilder.ui
   • org.eclipse.cdt-core/
   • org.eclipse.cdt.core
   • org.eclipse.cdt.core.ARCH
   • org.eclipse.cdt.ui
   • org.ecipse.cdt-debug/
   • org.eclipse.cdt.debug.core
   • org.eclipse.cdt.debug.mi.core
   • org.eclipse.cdt.debug.mi.ui
   • org.eclipse.cdt.debug.ui
   • org.eclipse.cdt-launch/
   • org.eclipse.cdt.launch
   • org.eclipse.cdt-doc/
   • org.eclipse.cdt.doc.isv
   • org.eclipse.cdt.doc.user
2. Next check out the FDT components from dev.eclipse.org:/cvsroot/technology
   • org.eclipse.ptp/lang/
   • org.eclipse.fdt.core
   • org.eclipse.fdt.managedbuilder.core
   • org.eclipse.fdt.managedbuilder.ui
   • org.eclipse.fdt.managedbuilder.tests
   • org.eclipse.fdt.ui
   • cdt_changes
3. From a shell, cd to Eclipse workspace and execute the shell script cdt_changes/install
4. Select org.eclipse.cdt.* projects one by one and refresh them.
5. Verify that workspace builds correctly without any errors.

Test: FT_1: User Interface

1. Create Eclipse Application launch configuration (FDT) and run it.
2. Examine icons (set left of debug icon) and make sure that project, folder, and file wizard icons are "M" based not "C" based. Also check that there is a "Make" perspective and a "Make Projects" window.
3. Examine menus corresponding to these icons and make sure they don't have C/C++ in labels.
4. Right click in "Make Project" window and examine icons and menu labels as in #6 and #7 above.

Test: FT_2: Managed Make 1

1. Use menu in project icon to create a new "Managed Make" project, name it HelloWorld (check icon).
2. Select Gnu Fortran for project type and click Finish.
3. Use menu in file icon to create a new source file. Name it hello.f90; make sure the file extension is acceptable and check icon and click Finish.
4. Check that new file is automatically opened.
5. Copy contents of org.eclipse.fdt.managedbuilder.tests/src/HelloWorld/hello.f90 into the new file and save the file.
6. Check console to see that "Build complete for project HelloWorld"
7. Create new "Local Application"
   • Run target. Ensure that label in the configurations window is "Local Application" and icon has an "M". Name the configuration HelloWorld and set the Project and #* Application fields. Set a debugger and click Run.
8. Check that "Hello from Fortran Main." is printed in the console window. Select the HelloWorld target from the Run menu and check console output again.

Test FT_3: Managed Make 2

1. Redo FT_2 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Components. That is, create a Managed Make project named Components and build an application the files.
2. Open Debug/subdir.mk and ensure the dependencies are as follows:
   • Driver.o: MonteCarloIntegrator.o
   • MonteCarloIntegrator.o: LinearFunction.o
   • MonteCarloIntegrator.o: RandNumGenerator.o
3. Make sure the project will build and run.

Test FT_4: Managed Make 3

1. Redo FT_2 for the directory, org.eclipse.fdt.managedbuilder.tests/src/Mixed. That is, create a Managed Make project named Mixed and build an application the files.
2. Make sure the mixed language project will build and run