Revision as of 16:42, 24 August 2010

Remote-Enabling the Rest of PTP

Conference Call - August 24, 2010

Host: Chris Recoskie - Notes: Jeff Overbey

1. History/motivation
   1. RDT's current remote model is based on other remote work and requirements developed at IBM
   2. Remote support for CDT
      1. Needed access to system headers, libraries to preprocess code/parse correctly
         1. These headers are often under license agreements that prevent them from being transferred to another machine
         2. Experimented with accessing via SMB or NFS; 4x slower than parsing locally
         3. Parsing/indexing is slow for large projects even when done locally; 4x slowdown was unacceptable
         4. Solution: perform all work involving the index or AST on the remote machine, and return the results in a lightweight form (e.g., as ICElements - see Call Hierarchy example below)
      2. CDT needed better separation between core and UI to support remote indexing
         1. Some things were easy; e.g., there was already an extension point for adding a new content assist provider
         2. Call hierarchy, type hierarchy, search talked directly to the parser and indexer APIs.  These had to be subclassed to override behavior; occasionally code had to be copied and pasted.
      3. Interaction between client and server -- example: Call Hierarchy view
         1. Client wants to find all callers of a particular function
         2. Client UI determines selection region: filename, offset, length; sends this to the server
         3. Server invokes the parser, maps this to an ASTName.
         4. Server queries the index for that ASTName (translates it to a binding, finds references)
         5. Reference bindings are resolved to ICElements
         6. Server returns a list of ICElements
            1. Outline, call hierarchy, etc. are essentially views of the CModel.  The elements of the CModel are ICElements.
            2. The ICElement hierarchy was duplicated to model remote elements
            3. Not all methods work on remote elements since they do not correspond to things on the local filesystem
         7. View is populated from this list of ICElements
      4. Users may have different projects on different machines.
         1. Service provider: e.g., remote builder, remote indexer
            1. Each project has a service configuration, which is a list of the services it's using
         2. Service model: Description of services that are applicable to a given project; they may or may not actually be configured
      5. RDT has separate managed build toolchains definitions for remote toolchains (e.g., Remote XLC)
2. Converting other PTP tools for remote support
   1. Generalities
      1. C/C++ ASTs obtained locally will probably be incorrect since headers not available; anything that uses C/C++ ASTs should be done remotely
      2. Services, subsystems, and miners
         1. RSE Terminology: A miner runs on a remote server; it accepts commands and returns results (e.g., accept search keywords, return search results); in implementation terms, it is a Java class on the server that implements an RSE API
         2. Miners are always tied to a subsystem on the local machine. The subsystem knows how to send a command to a miner.
            
      3. A CDT miner was created; it accepts commands to reindex a project, get content assist for a string, etc.
      4. Other miners (e.g., for PLDT) could talk to the CDT miner
      5. Service providers are not necessarily tied to miners/RSE, but often it's useful to use RSE subsystems/miners to implement a service
      6. Example: Open Declaration
         1. RDT asks for a service provider; this provider provides an Open Declaration API (e.g., which accepts a filename and offset indicating the cursor location and returns the filename, offset, and length of the declaration)
         2. Service provider talks to the remote indexing subsystem on the local machine
         3. Subsystem uses an RSE API to talk to the CDT miner on the remote machine
         4. Miner returns the filename, offset, length of the declaration
   2. Specifics
      1. PLDT - Gets AST, constructs control flow graph, runs analysis
         1. Biggest problem is probably the preprocessor since headers aren't necessarily available; this means the ASTs constructed locally probably aren't complete
         2. Would need to convert analysis to run on the server, then create markers based on its results
         3. This means creating a PLDT miner, subsystem
      2. ETFw
      3. Photran
         1. Builder
            1. RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains
         2. Remote command launcher is responsible for running the build command on the remote machine
            1. Standard make should be reusable directly from CDT
            2. Create remote GNU toolchain
            3. Scanner discovery caused complications for C/C++; ignoring this issue in Photran right now
         3. Indexer
            1. Need to implement a service provider, subsystem, miner
            2. Greg implemented a service that can deploy a payload
               1. Copies server to the remote system, executes it, and tunnels that traffic through SSH
               2. We would need two implementations of our service: one talks to RSE directly, one knows how to tunnel the traffic over SSH