PTP/ETFw/PTP External Tools Framework
- What is PTFw?
- The PTP Performance Tools Framework aims to decrease the amount of "Eclipse plumbing" that must be done to integrate existing Performance Tools in Eclipse, and provide pieces and parts to provide new tools within Eclipse.
- The PTFw is aimed at primarily C, C++, and Fortran applications, since these are the targets of PTP users.
- The PTFw can be used for Parallel launches, or for regular (e.g. CDT) launches.
- The PTFw provides easy integration of launching existing (e.g. command-line) tools from Eclipse. With just an XML file, an existing app can be launched, with Eclipse UI generated to solicit arguments from the user, launch the tool, and launch any analysis of the results.
- Workflow specified in the tool xml file specifies the steps to take. Each tool can have three distinct sections:
- compile - the compile section specifies commands to use to build the executable (e.g. with instrumentation) prior to running the tool. These are commands to prepend the existing compiler. See xml file details and examples.
- execute - the execute section includes utilities which prepend the actual executable, and run to gather information.
- analyze - the analyze section contains tools which can be launched on data produced after program execution is complete. They run in a predetermined sequence. Analysis tools can be existing (non-Eclipse) apps or command line tools or other visulatization tools. Analysis tools can also be created as eclipse plug-ins and are launched to display output within the eclipse workbench.
- The PTFw can work for any tool that should "wrapper" an executable with an arbitrary number of arguments that can be specified by the user.
See also original design page - Performance Tools Framework Design : Preliminary design of Performance Tools Framework
A Performance Analysis Framework For C/C++ and Fortran Wyatt Spear's talk at EclipseCon2008
Screenshots and brief examples
The following brief example shows how information to be extracted from the user prior to launch is displayed in the eclipse launch dialog, and how the items appear in the tool.xml file
- TAU - Wyatt Spear, U.Oregon
- TAU was the original application for which the PTFw tools were developed by Wyatt Spear (U. Oregon). Much of the original functionality has been generalized in the PTFw, but extensions are available (and utilized for TAU) for any features not currently available. That is, write your own eclipse plug-in code for any additional features.
- TaskFinder - Beth Tibbitts, IBM Research
- A tool for instrumenting a binary executable with the Intel PIN tool, and sampling memory references, was developed by IBM. The PTFw was utilized to provide a convenient user interface for the launching of TF with its arguments, and display an eclipse view (analyze section) that displays the results found, and maps the results back to source code lines. Developed by Beth Tibbitts (IBM Research) and should be available shortly (targeted for PTP 2.1 in August).
PTFw integration can be done one of two ways.
- Use xml file only to integrate a tool - specify the file via Eclipse Preferences
- Use Eclipse extensions to extend one or more of three extension points for more custom integration
PTFw XML file only via Eclipse Preferences
Input an xml file via the Eclipse preference pages. The end-user can do this without any eclipse plug-in programming.
PTFw extension points
Implement eclipse extensions for a more custom integration of your tool
- Handles specification of the steps to be performed in the perf/profile launch
- Three sections: compile, execute, and analyze
- Allows integration of external or command line tools, or eclipse plug-in tools
- Any of the three steps are optional
- allows the workflow XML document to be specified in the plug-in
- without this, a workflow XML document can be specified by the end user in the Preferences, without any Eclipse code needed
- workflow XML doc can specify one or more steps for each:
- compile, execute, and analyze
- Handles specification of the steps to be performed in the perf/profile launch
- Handles additional tabs and UI features in launch dialog
- Handles arbitrary post-processing
- Currently available in PTP 2.0 (Eclipse 3.3/CDT 4.0)
- Works with Eclipse 3.3/CDT 4.0 or Eclipse 3.4/ CDT 5.0 (there is currently no pre-built version for 3.4)
- Also available from the Eclipse CVS Repository
- Repository location: dev.eclipse.org
- Project: org.eclipse.ptp
- In folder: tools/perf
- Other project include those specific for TAU: org.eclipse.ptp.perf.* projects
Annotated Tool XML file
Annotated tool.xml file
This example contains 5 tools, each of which can contain one or more sections of compile, execute, and/or analyze.
<?xml version="1.0" encoding="UTF-8"?> <!-- Use the unqualified 'toolset' only for the old xml format. The current version is toolset.01 --> <toolset.01> <!-- There may be multiple tools per tool file. Each tool should have a unique name tag, which will be used to select it in Eclipse--> <tool name="VampirTrace"> <!-- There are three distinct sections for each tool; compile, execute and analyze. Note that each of these steps is optional --> <compile> <!-- By default the compiler commands set here prepend whatever compiler is already in use in Eclipse. If you set the tag replace="true" for the compile element the compilers will be replaced entirely with the command specified here. Each compiler type, c, c++ and fortran, is defined as shown below. --> <!-- Every command referencing a file on the system should include a group tag. The group tag indicates that the relevant binary files or scripts are located in the same place for each command sharing that tag --> <CC command="vtcc" group="vampirtrace"> <!-- Arguments to be passed to a command may be specified with the argument tag as shown here. --> <argument value="-vt:cc"/> </CC> <CXX command="vtcxx" group="vampirtrace"> <argument value="-vt:cxx"/> </CXX> <F90 command="vtf90" group="vampirtrace"> <argument value="-vt:f90"/> </F90> </compile> <execute> <!-- The execute section is comprised of utilities which prepend the actual executable. --> <utility command="mpirun" group="mpi"> <argument value="-np 4"/> </utility> </execute> <analyze> <!-- The analyze section contains tools which are launched on data produced after program execution is complete. The tools listed here will be run in sequence, allowing the definition of a multi-step processing procedure. The type tag can be set to view for tools that show data or process for tools that process data and (generally) output another file. Use caution when specifying arguments. Some tools produce dynamically named output which is not yet supported by this plugin. --> <utility type="view" command="vampir" group="vampir"> <argument value="a.otf"/> </utility> </analyze> </tool> <tool name="Kojak"> <compile> <!-- The allcompilers compiler definition allows the same command to be used for c, c++ and fortran. If used in conjunction with the specific compiler definitions their commands will override the allcompilers command, but arguments specified for allcompilers will be added to each individual compiler --> <allcompilers command="kinst-pomp" group="kojak"/> </compile> <analyze> <utility command="expert" group="kojak"> <argument value="a.elg"/> </utility> <utility command="paraprof" group="tau"> <argument value="a.cube"/> </utility> </analyze> </tool> <tool name="Perfsuite"> <execute> <utility command="mpirun" group="mpi"> <argument value="-np 4"/> </utility> <utility command="psrun" group="perfsuite"> </utility> </execute> <analyze> <utility command="psprocess" type="process" group="perfsuite"> <argument value="-o xmlprocess.txt"/> </utility> <utility command="paraprof" type="view" group="tau"> <argument value="-f psrun *.xml"/> </utility> </analyze> </tool> <tool name="Valgrind"> <execute> <!-- Use this line only if valgrind is a shell script. The plugin can not execute scripts natively --> <utility command="bash" group="inbin"/> <utility command="valgrind" group="valgrind"/> </execute> </tool> <tool name="CAT"> <execute> <!-- Use this line only if valgrind is a shell script. The plugin can not execute scripts natively --> <utility command="cat" group="cat"/> </execute> </tool> </toolset.01>
The following example shows the integration of an existing command-line dynamic tool, TaskFinder, into Eclipse via the PTFw.
TaskFinder does dynamic instrumentation of binaries and samples the execution to provide suggestions of region that could be parallelized.
The following image shows the launch UI dialog in which the user enters arguments used by the launching of the TaskFinder tool. This is specified in the execute workflow stage of the tf.xml. The use begins the profile launch from the profile icon in the eclipse toolbar, and is presented with the launch dialog. Create a "Performance Analysis" launch configuration and fill in as follows:
1. User indicates executable to launch on the Main tab
2. User indicates application args, if any, on the Arguments tab
3. User indicates perf tool, on the Performance Analysis tab, on the Tool Selection tab within that
4. User enters args, on the Performance Analysis tab, on the TF Options tab within that. Note that the "TF Options" tab is entirely configured from the tf.xml. There can be multiple pages here if desired. Note the arguments field is automatically generated (large Text area detailing how args will be passed to the tool). Need to document how to show/hide this.
The following image shows
- the console log (view at the bottom with red text) as the executation takes place, and
- the eclipse view that is generated from the parsed output of the TaskFinder (command-line) tool after launch and execution (just above the console view). This is specified in the analyze workflow stage of the tf.xml. This shows how the PTFw can integrate with standard eclipse plug-in code to display results. Double-clicking on an item in the TaskFinder view navigates to the associated source code line in the eclipse editor.
The plugin.xml (if used to imbed the tool xml; it can also be specified by the end user in a preference page) used by TaskFinder includes the following two extensions:
- workflows - this specifies the location of tf.xml (shown below) which specifies how the tool is launched and how to present the UI to the user.
- dataManagers - this specifies the post-processing to be performed: in this case, a java class within the TF eclipse plugin which parses the output file, determining its location from the PTFw, and showing the results in an eclipse view.
Plugin.xml snippet for PTFw:
<extension point="org.eclipse.ptp.perf.workflows"> <workflowDefinitionXML XMLFile="data/toolxml/tf.xml"> </workflowDefinitionXML> </extension> <extension point="org.eclipse.ptp.perf.dataManagers"> <!-- Post-processing --> <PerformanceDataManager class="org.eclipse.ptp.pldt.an.taskfinder.TaskfinderPostProcessor"> </PerformanceDataManager> </extension>
The PTFw tool xml (tf.xml) that generates this UI is shown here:
<?xml version="1.0" encoding="UTF-8"?> <toolset.01> <tool name="TaskFinder"> <execute> <utility command="pin" group="pinpath"> <optionpane title="TF options" separatewith=" " enclosevalues="" separatevalues=" "> <togoption label="Multithreaded" optname="-mt" tooltip="Allow multithreaded programs" defstate="true" /> <togoption label="Tool name" optname="-t " tooltip="PIN tool name (pintlsprof)" defstate="true" required="true"> <optvalue type="file"></optvalue> </togoption> <togoption label="Analyze?" optname="-analyze" tooltip="Analyze execution for task recommendation" defstate="true" visible="false" required="true"/> <togoption label="Analysis algorithm (simple)" optname="-anaalgo" tooltip="algorithm" defstate="true" visible="false"> <optvalue type="text" default="simple"></optvalue> </togoption> <togoption label="Sample size (2500)" optname="-samplesize" tooltip="Sample size for parallelism finder in 1000s of basic blocks (default=1000)" defstate="true"> <optvalue type="text" default="2500"></optvalue> </togoption> <togoption label="Rate (30)" optname="-rate" tooltip="Sampling rate in % of execution" defstate="true"> <optvalue type="text" default="30"></optvalue> </togoption> <togoption label="Ignore stack?" optname="-ignorestack" tooltip="Ignore stack references" defstate="true"> </togoption> <togoption label="Report file" optname="-repfile" tooltip="Location of output report file (tlsprof.out)" defstate="true"> <optvalue type="file"></optvalue> </togoption> <togoption label="Executable appended at end" optname="--" visible="false" tooltip="Executable target for analysis is obtained from main tab" defstate="true" required="true"/> </optionpane> </utility> </execute> <analyze> <!--The org.eclipse.ptp.pldt.an.taskfinder.TaskfinderPostProcessor class, specified in the plugin.xml extensions, will be called for the analyze phase --> <utility command="tfview" group="internal"/> </analyze> </tool> </toolset.01>
Widgets from XML
The PTFw generates SWT UI widgets from the tool xml.
- text area
- file selection dialog
- directory selection dialog?
<optvalue type="dir" >- need to test
- arguments field - document how to make it appear
See the TaskFinder examples above for examples of these
- No widget - an argument is added to the launch arguments, but nothing appears in the UI due to visible="false"
<togoption optname="-anaalgo" visible="false">
- Checkbox - this is the default widget. In this example, the label is "Multithreaded", the option added to the executable arg is "-mt", and the default state="true" indicates the checkbox is initially checked. If the state is false, the argument is not included.
<togoption label="Multithreaded" optname="-mt" tooltip="Allow multithreaded programs" defstate="true" />
- Text area
<optvalue type="text" >
<togoption label="Sample size (2500)" optname="-samplesize" tooltip="Sample size for parallelism finder in 1000s of basic blocks (default=1000)" defstate="true"> <optvalue type="text" default="2500"></optvalue></togoption>
- File Selection Dialog
<optvalue type="file" >
<togoption label="Report file" optname="-repfile" tooltip="Location of output report file (tlsprof.out)" defstate="true"> <optvalue type="file"></optvalue> </togoption>
- Will more SWT widgets be added to those currently available?
- Currently the UI information from the tool.xml file only handles checkboxes, text entry boxes, and File specifications via file dialogs.
- We agree more widgets would be useful.
- Wyatt has done some more extensive UI work by writing Java code via an eclipse extension to the org.eclipse.ptp.configurationTabs extension point.
- So if you need more widgets immediately, you can use this extension point to implement them for your applicaition
- Will the profile launcher handle remote launchers?
- We have not tested this. PTP parallel launch can launch remotely, but we have not tried it.
- We agree this is a useful feature.
- The framework can handle two types of launch: CDT launch and Parallel Launch.
- Parallel Launch (via PTP) is now available. Again, we have note tested this with the Perf Framework
- CDT remote launch is not yet available but should be as part of RDT (Remote Development Tools) being developed by IBM Toronto, which should be available in the near future. At that time we can evaluate adding a remote CDT launch to Perf Framework.
- The framework can handle two types of launch: CDT launch and Parallel Launch.
- File upload/download feature is needed
- The PTP Remote Tools, used by PTP remote launch/monitor/debug have the tools to be able to do this already, but this is not built in to PTP launch currently. PTP 2.1 will have this capability.
- Perf FW does not do this currently.
- If you just need to access remote information from a class that is called in your plug-in extension (like in my Taskfinder Post-processing class) then that should be do-able now by using RemoteTools directly.
- Are instrumentation tools available?
- TAU utilizes several eclipse features for instrumentation - need to describe here.
- Instrumentation of source code may be added to PTFw in the future. Beth has a rough framework for instrumenting java code and hopes to extend that to PTP-compatible sources (C, etc) in the future.
- What does PTFw require?
- PTFw does not require PTP core or PLDT, unless you want to use the Parallel application launch. It can be installed stand-alone
- PTFw requires Eclipse and CDT (Eclipse Europa 3.3/CDT4.0, or Eclipse Ganymede 3.4/CDT 5.0)
- If you require a Parallel Application launch, you need PTP Core