Difference between revisions of "PTP/release notes"
(→Install or Update Java)
|Line 103:||Line 103:|
=== Install or Update Java ===
=== Install or Update Java ===
Ensure that a Java Runtime Environment (JRE) version 1.
Ensure that a Java Runtime Environment (JRE) version 1.or higher is installed. This can be done with the command:
Revision as of 16:13, 22 June 2016
- 1 Latest Version
- 2 Overview
- 3 Components and Pre-Requisites
- 4 Installation
- 5 Updating from a previous release
- 6 Test your installation
- 7 Troubleshooting
- 8 Known Issues
- 9 Feature Descriptions
The latest version of PTP is 9.1.0 and Photran is 9.1.0 (as of June 22, 2016)
A list of bugs fixed in 9.1.0 is available here.
For bugs fixes in later releases, see the bottom of New and Noteworthy for this release
This document describes how to install PTP and its pre-requisites. Documentation for using PTP is available from the Parallel Development User Guide.
The installation process for PTP depends on a number of factors, including the versions of software you have installed on your system, your operating system, and the types of target systems you want to use. The basic installation steps are:
- Install or Update Java
- Install Eclipse and PTP by doing one of the following:
- Customizing the PTP Debugger
- Install optional RDT server component
Additional installation information is available on:
Note! The XSEDE 2013 tutorial contains detailed information about PTP 7.0.x and installing PTP on your workstation or laptop, including screenshots. You may wish to use the slides as a guide for installing Eclipse and PTP on your workstation or laptop. Then see below for information about installing components on your remote cluster as needed.
Components and Pre-Requisites
PTP is a set of plugins that extend the functionality of Eclipse. The following are the minimum requirements for PTP 9.0:
- Java 1.8 or later
- Eclipse 4.6 or later
- C/C++ Developer Tools (CDT) 9.0 or later
The client side of PTP (the system running Eclipse) is supported on the following platforms:
- Mac OS X
The server-side of PTP (the remote system used to launch/debug and host remote projects) is supported on the following platforms:
- Mac OS X
- Unix (e.g. AIX)
For the parallel debugger to work, PTP must support the remote system for debug in the Target System Configuration / TSC. Currently PTP parallel debugger is implemented for:
- Generic TSCs:
- Generic OpenMPI Interactive
- Generic MPICH2 Interactive
- Generic Remote Interactive
- IBM Parallel Environment
- IBM Platform LSF Interactive
- IBM Platform MPI
- Specific system TSCs:
- Trestles Torque OpenMPI Interactive
- git (if Synchronized projects are used) - version 1.7 or later - see https://bugs.eclipse.org/bugs/show_bug.cgi?id=410695
- Java (if RDT remote projects are used)
- Perl (for System Monitoring) - TBD: More Perl requirements for MPICH parallel debugging
- gdb (for Parallel Debugger)
The PTP plugins are grouped into the following three main components:
- Parallel Tools Platform (PTP) End-User Runtime
- This feature contains the core PTP components, including all supported target system configurations, the Parallel Language Development Tools (PLDT), and the External Tools Framework (ETFw).
- Fortran Development Tools (Photran)
- This feature adds support for editing, compiling, and debugging programs written using the Fortran language.
- Remote Services
- This feature adds support for accessing remote systems. The remote services framework can be used independently of the other PTP components, and adds features such as remote file access, remote shell access, and a remote console.
The PTP optional features are a set of features that provide additional functionality not included in the core PTP components. Most of these features require the installation of third party software or specialized components of CDT. The PTP Optional Features are summarized in the table below:
|PTP Fortran Support||Adds support for Fortran projects to PTP. This component includes Photran, as well as a number of features that add Fortran functionality to PTP.|
|PTP External Tools Framework TAU Extension||Adds support for performance analysis using TAU. Requires installation of TAU on the target system.|
|PTP Parallel Language Development Tools UPC Support||Adds PLDT support for UPC. Requires CDT UPC support.|
|PTP Scalable Communication Infrastructure (SCI)||Installs source code for the Scalable Communication Infrastructure (SCI).|
There are two main steps to installing PTP. First you need to make sure that you have the correct version of Java on your client system. The second step is to download and install the PTP plugins. These steps are outlined in more detail below.
Install or Update Java
Ensure that a Java Runtime Environment (JRE) version 1.8 or higher is installed. This can be done with the command:
In addition an IBM, Sun, or equivalent JRE is required; The gcj-based Java shipped with most Linux distributions does not work with Eclipse. If in doubt, run the above command and check if there is a reference to gcj. If you are using the Linux operating system, download and install either the Sun Java Runtime Environment or the IBM Java Runtime Environment. See also Installing Java for use with Eclipse and PTP.
There are two options for installing PTP: Full install or install into an existing Eclipse. A full install downloads and installs a copy of Eclipse that contains pre-installed PTP plugins. This is the simplest option. Install into an existing Eclipse is useful if you already have Eclipse installed and you just want to add the PTP functionality. Both methods are described in more detail below.
This is the simplest, easiest, and most reliable way to install PTP, and it is recommended for most users.
- Download the "Eclipse for Parallel Application Developers" package from http://eclipse.org/downloads/
- After downloading and extracting Eclipse (zip or tar file), there's nothing else to the installation process from the Eclipse side. In the extracted folder, simply run the eclipse executable file. It will ask for a workspace location; the default is fine.
- See also Updating from the Parallel Package to a more recent release below
- Multi-user installs: Note that the default install assumes that the user has write access to the location where Eclipse in installed (unzipped or un-tarred). See multi-user install information for other scenarios.
Install into an existing Eclipse
This option assumes that you have already installed Eclipse and are familiar with the Eclipse installation process.
We recommend starting with either:
- "Eclipse Standard" - (previously known as Eclipse Classic) if you also plan to do Java and/or plugin development, or
- "Eclipse IDE for C/C++ Developers" - only includes support for C/C++, which is simpler and smaller
Using the table above, choose the components you wish to install, and note the pre-requisites required.
- If you are unsure, just choose "Parallel Tools Platform (PTP) End-User Runtime"
Make sure Eclipse is running, then follow the steps below:
- Open "Help > Install New Software..."
- From the "Work with:" drop down, choose "Mars - http://download.eclipse.org/releases/mars"
- See below for possible more recent updates to PTP
- Open the "General Purpose Tools" category
- Check the PTP components you require
- If you are unsure, just choose "Parallel Tools Platform (PTP) End-User Runtime"
- If any of your selections require CDT as a pre-requisite:
- Open the "Programming Languages" category
- Check "C/C++ Development Tools"
- To add Fortran support:
- Open the "Programming Languages" category
- Check "Fortran Development Tools (Photran)" (See the Photran 9.1 Online documentation for more information on installing and using Photran)
- Update above URL for Mars Photran docs
- Select the "Finish" button and follow the prompts
- You should restart Eclipse when prompted.
Customizing the PTP Debugger
Prior to launching the parallel debugger for the first time on a target system, an executable called the SDM will be automatically copied to the target.
In certain situations you may wish to use a different version of the SDM, or install the SDM in a different location (e.g. a shared directory).
The SDM source code and pre-built binaries are available from the download site. You can use these to customize your installation in any way you wish:
- Download ptp-sdm-9.0.x.zip from the PTP downloads page
- Copy the sdm zip file to the target machine
- Unzip the sdm zip file into a temporary location
- Change to the
At this point you can either use one of the pre-built binaries, located in the
org.eclipse.ptp.<os>_9.0.x.yyyymmddhhmm directories or build your own sdm from source (
<os> corresponds to an operating system type, and
9.0.x.yyyymmddhhmm corresponds to the numeric version and build date of the PTP release.)
To build from source:
- Change to the
- Run the command:
This should build the sdm executable.
PTP assumes a default location of the sdm is
/usr/local/bin. If desired, however, you can copy the sdm executable to any convenient directory. When you configure a debug launch configuration, you will need to select the
sdm executable in this directory.
Installing Open MPI
If you do want to build OpenMPI yourself, here are instructions based on building OpenMPI 1.4.2. See the Open MPI FAQ for more information on building and installing Open MPI.
- Download the openmpi-1.4.2.tar.gz file
- Untar it using the command
tar -xzvf openmpi-1.4.2.tar.gzand cd to the directory it creates
- The default installation location is /usr/local
- To install into a different location, e.g. /usr/local/openmpi1.4.2, use the command:
- To add the Fortran bindings (test): add
- Then don't forget to modify PATH, etc:
- If you have problems with Fortran bindings (e.g. Snow Leopard), you can disable this by adding
--disable-mpi-f77to the configure script
- Build and install with
sudo make install
Then to check your install:
ompi_infoshould return version info including version that matches what you installed above.
Build and run a small MPI test case, for example: testMPI.c
mpicc -o testMPI testMPI.c
mpirun -np 4 testMPI
Updating from a previous release
If you are updating to PTP 9.x from a previous major release (e.g. 8.0, 7.0, etc.) it is recommended that you start with a completely new installation of Eclipse and a new workspace. Existing projects can be imported into the workspace using one of the available import wizards (File > Import).
Convert workspaces created under PTP 7.0
If you have synchronized projects created under 7.0, please use the following procedure to update them in a 9.x workspace. Since synchronized projects can exist for other types of Eclipse projects besides C/C++/Fortran, the synchronized information is kept entirely differently in PTP 9.0.
- Assure the project is synchronized and all local content is on the server as well
- if you have local content that you have filtered and not sync'd to the remote system, back it up elsewhere.
- Delete the project from eclipse (right mouse on project, Delete, check the box for Delete project contents on disk)
- Create a new synchronized project (File > New > Other > Remote > Synchronized C/C++ project or Synchronized Fortran project) and point to the directory on the remote system where the original is located
- If you have any local content not sync'd to the remote system, paste it back into the project now and update the Sync filters to prevent it from syncing again.
Updating from the Parallel Package to a more recent release
The normal way to update PTP is to use Help > Check for Updates, however there are two reasons this may not report any updates are available, even when a PTP update exists.
- If you installed the Eclipse for Parallel Application Developers or some other Eclipse build
- Because, unless you installed a subsequent update from the PTP-specific update site, "Check for Updates" only looks at the site(s) you installed initially from
- If the PTP update site is not enabled in the update manager
If "Check for Updates" does not report any updates, follow the steps below. Note that these steps only need to be done once, and "Check for Updates" should work for subsequent releases.
Installing a new PTP build or release
- Open Help > Install New Software...
- Click on "Available Software Sites"
- Locate the PTP-specific update site and make sure it is checked. If it doesn't exist, create a new one.
- The PTP update site URL is http://download.eclipse.org/tools/ptp/updates/mars
- Click OK to return to the main dialog
- If you are using the Eclipse for Parallel Application Developers:
- Select the PTP update site from the "Work with:" dropdown
- Decide how much you want to update/install:
- Simplest: check the box next to each category to update everything
- Or, to update only the currently-installed PTP features:
- Open each category and check the box next to the items that are already installed (the icon shows two arrows). If the icons are grey, the update is already installed for that item.
gray plug icon - feature already installed and up to date;
two arrows - feature needs to be updated;
color plug - feature is not installed
- Click Next and follow the instructions (Next, Next, Finish, accept license).
- If you are NOT using the version of PTP from your original Eclipse for Parallel Application Developers, that is, if you installed from http://download.eclipse.org/tools/ptp/updates/mars directly since then, then updating should work:
- Click Cancel to quit the update manager
- Select Help > Check for Updates
- Follow the instructions
Once you install from http://download.eclipse.org/tools/ptp/updates/mars, either initially, or on top of Eclipse for Parallel Application Developers, the Help > Check for Updates will work after that to update from this site as well.
Test your installation
The easiest way to test your installation is to create and run an MPI application.
- Start Eclipse. (You may need to use the -clean parameter to force it to recognize new features and plug-ins).
- Switch to the C/C++ perspective (Window > Open Perspective > Other...)
- Create a small MPI C project.
- Create a launch configuration (Run > Run Configurations...) and create a run configuration for a Parallel Application.
- Fill in the information for the particular type of target system configuration
- For example, for an Open MPI application this includes (but may not be limited to):
- On Resources tab, select Number of processes
- On Application tab, select your Project for Parallel Project, and the executable for Application Program.
- On Debugger tab, find the sdm executable that you build from the command line during the installation process as the "Path to debugger executable"
- Launch the application.
More details on using PTP can be found in Help > Help Contents. Click on "Parallel Tools Platform User Guide".
The PTP help is also available online.
- I have an OTP token that's required for use with the my remote system. Can I use this when creating synchronized C/C++ projects and/or the resource manager?
- I created a resource manager but I entered the wrong username/password. How do I change these?
- The parallel debugger does not start (may be stuck in "Upload file" or "Waiting for connection...".)
Solution: Use the OTP password but avoid saving the password with any of the dialogs. If you skip the password in dialogs that prompt for both username and password, you'll get a separate password prompt. Use your OTP password there.
Solution: Open Connection preferences by selecting "Preferences > Remote Development > Connections". In the view click on the name of the connection you used in the resource manager configuration and then "Edit". Note that a connection must be stopped in order to edit it. Of course, if the username/password is wrong, it probably isn't running.
Solution: There are many reasons that this could be happening:
- Check that you can run the MPI program successfully (non-debug), both from the command line, and from within Eclipse
- Check that you have compiled the application for debugging with -g
- Check that the debugger session address on the "Debugger" tab of the launch configuration is set to "localhost"
- If you have installed your own sdm on the remote machine, check that the path is correct on the "Debugger" tab in the launch configuration
- See Known Issues - Debugger below
If still have a problem, ask on ptp-dev mailing list, and be sure to list your platform, PTP version, version of Open MPI (or other system), and what the debugger session address is set to.
Solution: Make sure you have built sdm for your platform (either local or remote). See the above section on Customizing the PTP Debugger. In the debug launch configuration dialog, make sure you have selected the sdm executable that you built.
Solution: If the resource manager is local, see troubleshooting local resource managers below. If the resource manager is remote, then it is probably an issue with your PATH or LD_LIBRARY_PATH (or both) on the remote machine. Try running the command
ssh remote_host ompi_info -a --parseable. If the
ompi_info command is not found, then you need to set your PATH correctly on the remote machine. If you see a message like "ompi_info: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory" then you also need to set the LD_LIBRARY_PATH on the remote machine.
Solution: This is usually because your PATH is not being picked up by Eclipse. Most window systems do not get their paths from the shell, so when you launch Eclipse it will have the wrong PATH set. Consult the documentation on your window system for details on how to set the path, or alternatively, launch Eclipse directly from a shell. See also this PTP FAQ item.
Solution: Open the Eclipse preferences. Go to "Run/Debug" and click on the "Launching" preference. Uncheck "Build (if required) before launching".
/usr/local/etc/openmpi-default-hostfile(or wherever you installed Open MPI) and add lines containing
/etc/hosts and add a localhost entry for each name you added to the
127.0.0.1 node0 127.0.0.1 node1 ...
/usr/local/etc/openmpi-mca-params.conf and add the following to the end:
orte_default_hostfile = /usr/local/etc/openmpi-default-hostfile
Solution: Create a file, say
'hostfile' in your home directory containing the hostnames of the nodes. Then set the
'OMPI_MCA_orte_default_hostfile' environment variable to
'$HOME/hostfile'. You'll need to do this in your .bashrc or .tcshrc (the one which is executed when a new shell is created.) Using .bash_profile or .login won't work.
Solution: Normally just run the eclipse executable - however, you may want to try this from a command line instead of from a "shortcut" - some OS's or windowing systems do not properly send the environment information. And, some OS's send the wrong info from the command line. Your mileage may vary.
Solution: XML tags should be gone in OpenMPI version 1.3.4 and later
Solution: The Remote C/C++ Editor does not currently support CDT breakpoints. Open the C/C++ Editor on the file instead by right-clicking on the file and selecting "Open With > C/C++ Editor. This problem should be addressed in 4.0.1. Update your version of PTP to 4.0.1 or later.
- My launch seems to get stuck at "Opening connection" when starting.
- A "Secure Storage" dialog is seen when starting Eclipse on Linux.
Connecting to a remote system from Eclipse on some client operating systems (in particular Fedora Core 11) generate a Kerberos authentication request. Check the command line where you started Eclipse for a Kerberos prompt. Enter your login name and password when requested.
Your operating system documentation may provide information on how to disable this request from being generated.
Eclipse stores usernames/passwords in secure storage. On some systems, a password (usually your login password) must be entered to unlock this storage before Eclipse can access it. Check your operating system documentation for ways to disable this prompt.
- The debugger gets stuck at "Upload file"
- The debugger gets stuck at "Waiting for connection"
- Accelerator keys (F5, F6) operate on the current process selected in Debug view.
- Debugger won't connect; breakpoints don't work; error dialogs
When the debugger launches onto a remote system, it attempts to make a connection back to your local machine running Eclipse. If you have the "SSH Tunneling" option enabled on the resource manager, then this connection will be made over the main ssh connection using an ssh tunnel. For this to work, your remote target sshd configuration needs to enable the "GatewayPorts" option. See the sshd_config(5) for more information on this option.
Workaround: If you are not able to change the sshd configuration on the remote target, the only option is to not use tunneling. This requires direct access from the remote target to your local Eclipse client machine (i.e. you from the remote target system, you should be able to use "ping <local_client_host>".) Also, you'll need to disable the firewall on the local client machine so that the incoming connection is permitted.
The debugger requires a filesystem that is shared across all nodes in the cluster in order to distribute routing information. If this routing file doesn't exist, then one or more of the debugger server processes will appear to hang.
Workaround: In this situation, the routing file must be manually distributed in order for the server processes to proceed.
This limitation will be addressed in a future version of the debugger.
There is currently no way to use these keys for the Parallel Debug view.
Make sure you've built sdm for your target system. See the section on troubleshooting above - items about debugger.
Remote Development Tools (RDT)
- The Remote C/C++ Editor is used by default for local files
- Unable to set CDT breakpoints in the Remote C/C++ Editor
- Remote projects are always closed when Eclipse restarts
- Remote editors are not properly restored on Eclipse restart
- Quick Type Hierarchy does nothing
RDT currently provides a framework for developing C/C++ programs on remote machines. Although it is our intention that RDT work seamlessly with CDT and other Eclipse plug-ins, there is a problem with the Eclipse platform which hinders interoperabilty between base CDT and RDT, namely that the Remote C/C++ Editor provided by RDT is always used by default for editing C/C++ files.
Workaround: Although this issue can be worked around by manually selecting the CDT editor (Context Menu >Open With >C/C++ Editor) when you are working with a local CDT project, it can be tedious having to do this for every single file you open. As such, we recommend that you install RDT into an Eclipse installation which you do not intend to do local C/C++ development with. Future versions of Eclipse and RDT should better support local development.
This issue is similar to the previous problem. Right-click on the file and select Open With > C/C++ Editor. Note also that debugging remote projects with the CDT debugger has not been tested. It is likely that problems will occur if the debugger assumes files reside locally, but are actually located on a remote machine. Use at your own risk.
When you exit and restart Eclipse, all your remote projects are closed. This happens because the .project file is on the remote server and when eclipse starts there is no connection yet, so eclipse thinks the .project file is missing and closes the project.
Workaround: To work around this issue, simply reopen your projects after the connection to the remote machine has been re-established.
If you leave remote editors open upon closing down the workbench, when you restart, the editors will not be properly initialized, showing a "resource not found" error. Closing and reopening the editor will not alleviate the problem. This problem occurs because on startup of the workbench, the remote connection is not yet established, and hence opening the editors fails.
Workaround: To work around this problem, close all editors opened on remote resources before shutting down the workbench. If the problem occurs, you can alleviate it by closing the affected editors and restarting the workbench.
Invoking Quick Type Hierarchy from the editor context menu does not currently do anything.
Workaround: Use "Open Type Hierarchy" instead to open the Type Hierarchy View for an alternate method of viewing type hierarchy information.
Parallel Language Development Tools (PLDT)
- Analysis of parallel applications located on a remote server doesn't seem to work
Note that not all PLDT features are tested and verified for running with RDT at this time.
Since 4.0.3, finding artifacts (MPI, OpenMP, UPC) does work on remote projects.
Workaround: To use other PLDT analysis features, use a local CDT project, instead of a remote project.
C/C++ Build Issues
- I get the error "The Open MPI wrapper compiler was unable to find the specified compiler gcc in your PATH." when I build an MPI program (e.g. created with the Hello MPI World new project wizard).
- The PATH variable may not be getting communicated to CDT correctly. Copy the contents of your PATH variable in the clipboard, and open the Project Properties, c/c++ build, Environment, add var: PATH and paste in the PATH from cmd line
- The ptp_ibmpe_proxy and ptp_ibmll_proxy proxies for the IBM PE and IBM LoadLeveler resource managers do not work with LoadLeveler. Messages may be displayed stating that libllapi.so could not be loaded on Linux systems.
- Depending on the version of LoadLeveler installed, libllapi.so may be a 32 bit shared library or a 64 bit shared library. This can be determined by locating libllapi.so in /opt/ibmll/LoadL. For 32 bit versions, it will be in /opt/ibmll/LoadL/full/lib, /opt/ibmll/LoadL/scheduler/lib, or /opt/ibmll/LoadL/so/lib. For 64 bit versions, libllapi.so will be in the corresponding lib64 directories. You should set the CFLAGS environment variable to contain either the -m32 or -m64 flags before invoking the BUILD script, for instance CFLAGS=" -m64" sh BUILD configure
|Fortran Development Tools (Photran)||N||Adds Fortran support to Eclipse (PTP not required)|
|IBM XL Fortran Compiler Support||N||Adds support for the IBM XL Fortran compiler to Photran|
|Linux Intel(R) Fortran Compiler Support||N||Adds support for the Intel(R) Linux Fortran compiler to Photran|
|Parallel Tools Platform End-User Runtime||Y||Main PTP feature that combines PTP Core, resource managers, and other components|
|Parallel Tools Platform SDK||N||Adds documentation and APIs for extending PTP functionality|
|PTP Common External Components||N||Components that must be installed for certain target system and the debugger|
|PTP Common Utilities||Y||Common utility functions. Can be installed as a stand-alone component.|
|PTP Contributed Target System Configurations||N||Target configurations to support various systems, including XSEDE systems|
|PTP Core Components||Y||Core components of PTP|
|PTP External Tools Framework||N||Adds support for the integration of external tools|
|PTP External Tools Framework TAU Extension||N||Adds support for the integration of the Tuning and Analysis Utilities (TAU)|
|PTP External Tools Framework TAU Extension for Fortran||N||Adds support for the integration of the TAU with Photran|
|PTP Fortran Project Synchronization Support||N||Adds support for Fortran synchronized projects|
|PTP Fortran Support||N||Adds support for Fortran to PTP. Includes Photran plus a number of other Fortran features|
|PTP JAXB Resource Manager Support||Y||Adds the JAXB resource manager framework|
|PTP LML JAXB Resource Manager||Y||Adds support for monitoring of JAXB target configurations|
|PTP LML Support||Y||Adds target system monitoring services|
|PTP Parallel Language Development Tools||N (recommended)||Adds MPI, OpenMP, LAPI, and UPC analysis and assistance tools|
|PTP Parallel Language Development Tools Fortran Support||N||Adds PLDT support to Fortran projects|
|PTP Parallel Language Development Tools UPC Support||N||Adds PLDT support for UPC projects|
|PTP Remote Project Synchronization Support||Y||Adds synchronized project support to PTP|
|PTP Resource Manager for IBM LoadLeveler||N||Legacy component for supporting LoadLeveler. Superseded by JAXB target configurations.|
|PTP Resource Manager for IBM Parallel Environment||N||Legacy component for supporting Parallel Environment. Superseded by JAXB target configurations.|
|PTP Resource Manager for MPICH2||N||Legacy component for supporting MPICH2. Superseded by JAXB target configurations.|
|PTP Resource Manager for Open MPI||N||Legacy component for supporting Open MPI. Superseded by JAXB target configurations.|
|PTP Resource Manager for SLURM||N||Legacy component for supporting SLURM. Superseded by JAXB target configurations.|
|PTP Scalable Communication Infrastructure (SCI)||N||C++ based multicast/reduction network for scalable communication|
|PTP Scalable Debug Manager (SDM)||N (recommended)||External component to allow debugging of parallel applications. Can be installed as a stand-alone component.|