PTP/designs/remote/using

Remote-Enabling the Rest of PTP August 24, 2010 Host: Chris Recoskie Notes: Jeff Overbey

• History/motivation
  • Remote model is based on other remote work and requirements developed at IBM
  • Remote support for C/C++
    • Needed access to system headers, libraries to preprocess code
      • Often under license agreements that prevents them from being transferred to another machine
      • Experimented with accessing via SMB or NFS; 4x slower than parsing locally
      • Parsing/indexing is slow for large projects even when done locally; 4x slowdown unacceptable
      • Solution: perform all work involving the index or AST on the remote machine, and return the results as ICElements (see Call Hierarchy example below)
    • Needed better separation between core and UI to support remote indexing
      • Some things easy; e.g., already an extension point for adding a new content assist provider
      • Call hierarchy, type hierarchy, search talked directly to parser, indexer APIs; had to subclass and override behavior, occasionally copy and paste code
    • Example: Call Hierarchy view
      • Client wants to find all callers of a particular function
      • Client UI determines selection region: filename, offset, length; sends this to the server.
      • Server invokes the parser, maps this to an ASTName.
      • Server queries the index for that ASTName (translates it to a binding, finds references).
      • Reference bindings are resolved to ICElements.
      • Server returns a list of ICElements.
        • Outline, call hierarchy, etc. are essentially views of the CModel. The elements of this model are ICElements.
        • ICElement hierarchy was duplicated to model remote elements
          • Not all methods work on these, since they do not correspond to things on the local filesystem
      • View is populated from this list of ICElements.
    • Users may have different projects on different machines.
      • Service provider: e.g., remote builder, remote indexer
        • Each project has a service configuration, which is a list of the services it's using
      • Service model: Description of services that are applicable to a given project; they may or may not actually be configured
    • RDT has separate managed build toolchains definitions for remote toolchains (e.g., Remote XLC)
• Converting tools for remote support
  • Generalities
    • C/C++ ASTs obtained locally will probably be incorrect since headers not available; anything that uses ASTs should be done remotely
    • Services, subsystems, and miners
      • RSE Terminology: A miner runs on a remote server; it accepts commands and returns results (e.g., accept search keywords, return search results); in implementation terms, it is a Java class on the server that implements an RSE API
      • Miners are always tied to a subsystem on the local machine. The subsystem knows how to send a command to a miner.
    • The CDT miner accepts commands to reindex a project, get content assist for a string, etc.
    • Other miners (e.g., for PLDT) could talk to the CDT miner
    • Service providers are not necessarily tied to miners/RSE
    • Example: Open Declaration
      • Ask for service provider; provides an Open Declaration API (filename, offset, length)
      • Service provider talks to the remote indexing subsystem (on the local machine)
      • Subsystem uses an RSE API to talk to the CDT miner
      • Miner returns the filename, offset, length of the declaration
  • Specifics
    • PLDT - Gets AST, constructs CFG, runs analysis
      • Biggest problem is probably the preprocessor since headers aren't necessarily available
      • Would need to convert analysis to run on the server, then create markers based on its results
      • This means creating a PLDT miner, subsystem
    • ETFw
    • Photran
      • Builder
        • RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains
      • Remote command launcher is responsible for running the build command on the remote machine
        • Standard make should be reusable directly from CDT
        • Create remote GNU toolchain
        • Scanner discovery caused complications for C/C++; ignoring this issue in Photran right now
      • Indexer
        • Need to implement a service provider, subsystem, miner
        • Greg implemented a service that can deploy a payload
          • Copies server to the remote system, executes it, and tunnels that traffic through SSH
          • We would need two implementations of our service: one talks to RSE directly, one knows how to tunnel the traffic over SSH