Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "PTP/designs/remote/using"
m |
m |
||
Line 1: | Line 1: | ||
− | = Remote-Enabling the Rest of PTP = | + | = Remote-Enabling the Rest of PTP = |
− | ''Conference Call - August 24, 2010'' | + | ''Conference Call - August 24, 2010'' |
− | ''Host: Chris Recoskie - Notes: Jeff Overbey'' | + | ''Host: Chris Recoskie - Notes: Jeff Overbey'' |
#History/motivation | #History/motivation | ||
##RDT's current remote model is based on other remote work and requirements developed at IBM | ##RDT's current remote model is based on other remote work and requirements developed at IBM | ||
− | ##Remote support for CDT | + | ##Remote support for CDT |
− | ###Needed access to system headers, libraries to preprocess code/parse correctly | + | ###Needed access to system headers, libraries to preprocess code/parse correctly |
####These headers are often under license agreements that prevent them from being transferred to another machine | ####These headers are often under license agreements that prevent them from being transferred to another machine | ||
####Experimented with accessing via SMB or NFS; 4x slower than parsing locally | ####Experimented with accessing via SMB or NFS; 4x slower than parsing locally | ||
####Parsing/indexing is slow for large projects even when done locally; 4x slowdown was unacceptable | ####Parsing/indexing is slow for large projects even when done locally; 4x slowdown was unacceptable | ||
− | ####Solution: perform all work involving the index or AST on the remote machine, and return the results in a lightweight form (e.g., as ICElements - see Call Hierarchy example below) | + | ####Solution: perform all work involving the index or AST on the remote machine, and return the results in a lightweight form (e.g., as ICElements - see Call Hierarchy example below) |
###CDT needed better separation between core and UI to support remote indexing | ###CDT needed better separation between core and UI to support remote indexing | ||
####Some things were easy; e.g., there was already an extension point for adding a new content assist provider | ####Some things were easy; e.g., there was already an extension point for adding a new content assist provider | ||
− | ####Call hierarchy, type hierarchy, search talked directly to the parser and indexer APIs. These had to be subclassed to override behavior; occasionally code had to be copied and pasted. | + | ####Call hierarchy, type hierarchy, search talked directly to the parser and indexer APIs. These had to be subclassed to override behavior; occasionally code had to be copied and pasted. |
###Interaction between client and server -- example: Call Hierarchy view | ###Interaction between client and server -- example: Call Hierarchy view | ||
− | ####Client wants to find all callers of a particular function | + | ####Client wants to find all callers of a particular function |
− | ####Client UI determines selection region: filename, offset, length; sends this to the server | + | ####Client UI determines selection region: filename, offset, length; sends this to the server |
####Server invokes the parser, maps this to an ASTName. | ####Server invokes the parser, maps this to an ASTName. | ||
− | ####Server queries the index for that ASTName (translates it to a binding, finds references) | + | ####Server queries the index for that ASTName (translates it to a binding, finds references) |
− | ####Reference bindings are resolved to ICElements | + | ####Reference bindings are resolved to ICElements |
− | ####Server returns a list of ICElements | + | ####Server returns a list of ICElements |
− | #####Outline, call hierarchy, etc. are essentially views of the CModel. The elements of the CModel are ICElements. | + | #####Outline, call hierarchy, etc. are essentially views of the CModel. The elements of the CModel are ICElements. |
− | #####The ICElement hierarchy was duplicated to model remote elements | + | #####The ICElement hierarchy was duplicated to model remote elements |
− | #####Not all methods work on remote elements since they do not correspond to things on the local filesystem | + | #####Not all methods work on remote elements since they do not correspond to things on the local filesystem |
− | ####View is populated from this list of ICElements | + | ####View is populated from this list of ICElements |
###Users may have different projects on different machines. | ###Users may have different projects on different machines. | ||
####''Service provider:'' e.g., remote builder, remote indexer | ####''Service provider:'' e.g., remote builder, remote indexer | ||
− | #####Each project has a ''service configuration,'' which is a list of the services it's using | + | #####Each project has a ''service configuration,'' which is a list of the services it's using |
− | ####''Service model:'' Description of services that are applicable to a given project; they may or may not actually be configured | + | ####''Service model:'' Description of services that are applicable to a given project; they may or may not actually be configured |
− | ###RDT has separate managed build toolchains definitions for remote toolchains (e.g., Remote XLC) | + | ###RDT has separate managed build toolchains definitions for remote toolchains (e.g., Remote XLC) |
#Converting other PTP tools for remote support | #Converting other PTP tools for remote support | ||
##Generalities | ##Generalities | ||
Line 37: | Line 37: | ||
###Services, subsystems, and miners | ###Services, subsystems, and miners | ||
####RSE Terminology: A ''miner'' runs on a remote server; it accepts commands and returns results (e.g., accept search keywords, return search results); in implementation terms, it is a Java class on the server that implements an RSE API | ####RSE Terminology: A ''miner'' runs on a remote server; it accepts commands and returns results (e.g., accept search keywords, return search results); in implementation terms, it is a Java class on the server that implements an RSE API | ||
− | ####Miners are always tied to a ''subsystem'' on the local machine. The subsystem knows how to send a command to a miner.<br> | + | ####Miners are always tied to a ''subsystem'' on the local machine. The subsystem knows how to send a command to a miner.<br> |
###A CDT miner was created; it accepts commands to reindex a project, get content assist for a string, etc. | ###A CDT miner was created; it accepts commands to reindex a project, get content assist for a string, etc. | ||
###Other miners (e.g., for PLDT) could talk to the CDT miner | ###Other miners (e.g., for PLDT) could talk to the CDT miner | ||
− | ###Service providers are not necessarily tied to miners/RSE, but often it's useful to use RSE subsystems/miners to implement a service | + | ###Service providers are not necessarily tied to miners/RSE, but often it's useful to use RSE subsystems/miners to implement a service |
###Example: Open Declaration | ###Example: Open Declaration | ||
####RDT asks for a service provider; this provider provides an Open Declaration API (e.g., which accepts a filename and offset indicating the cursor location and returns the filename, offset, and length of the declaration) | ####RDT asks for a service provider; this provider provides an Open Declaration API (e.g., which accepts a filename and offset indicating the cursor location and returns the filename, offset, and length of the declaration) | ||
− | ####Service provider talks to the remote indexing subsystem on the local machine | + | ####Service provider talks to the remote indexing subsystem on the local machine |
− | ####Subsystem uses an RSE API to talk to the CDT miner on the remote machine | + | ####Subsystem uses an RSE API to talk to the CDT miner on the remote machine |
− | ####Miner returns the filename, offset, length of the declaration | + | ####Miner returns the filename, offset, length of the declaration |
##Specifics | ##Specifics | ||
###PLDT - Gets AST, constructs control flow graph, runs analysis | ###PLDT - Gets AST, constructs control flow graph, runs analysis | ||
− | ####Biggest problem is probably the preprocessor since headers aren't necessarily available; this means the ASTs constructed locally probably aren't complete | + | ####Biggest problem is probably the preprocessor since headers aren't necessarily available; this means the ASTs constructed locally probably aren't complete |
####Would need to convert analysis to run on the server, then create markers based on its results | ####Would need to convert analysis to run on the server, then create markers based on its results | ||
− | ####This means creating a PLDT miner, subsystem | + | ####This means creating a PLDT miner, subsystem |
− | ###ETFw | + | ###ETFw |
###Photran | ###Photran | ||
####Builder | ####Builder | ||
− | #####RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains | + | #####RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains |
− | ####''Remote command launcher'' is responsible for running the build command on the remote machine | + | ######''Remote command launcher'' is responsible for running the build command on the remote machine |
#####Standard make should be reusable directly from CDT | #####Standard make should be reusable directly from CDT | ||
#####Create remote GNU toolchain | #####Create remote GNU toolchain | ||
− | #####Scanner discovery caused complications for C/C++; ignoring this issue in Photran right now | + | #####Scanner discovery caused complications for C/C++; ignoring this issue in Photran right now |
####Indexer | ####Indexer | ||
#####Need to implement a service provider, subsystem, miner | #####Need to implement a service provider, subsystem, miner |
Revision as of 16:42, 24 August 2010
Remote-Enabling the Rest of PTP
Conference Call - August 24, 2010
Host: Chris Recoskie - Notes: Jeff Overbey
- History/motivation
- RDT's current remote model is based on other remote work and requirements developed at IBM
- Remote support for CDT
- Needed access to system headers, libraries to preprocess code/parse correctly
- These headers are often under license agreements that prevent them from being transferred to another machine
- Experimented with accessing via SMB or NFS; 4x slower than parsing locally
- Parsing/indexing is slow for large projects even when done locally; 4x slowdown was unacceptable
- Solution: perform all work involving the index or AST on the remote machine, and return the results in a lightweight form (e.g., as ICElements - see Call Hierarchy example below)
- CDT needed better separation between core and UI to support remote indexing
- Some things were easy; e.g., there was already an extension point for adding a new content assist provider
- Call hierarchy, type hierarchy, search talked directly to the parser and indexer APIs. These had to be subclassed to override behavior; occasionally code had to be copied and pasted.
- Interaction between client and server -- example: Call Hierarchy view
- Client wants to find all callers of a particular function
- Client UI determines selection region: filename, offset, length; sends this to the server
- Server invokes the parser, maps this to an ASTName.
- Server queries the index for that ASTName (translates it to a binding, finds references)
- Reference bindings are resolved to ICElements
- Server returns a list of ICElements
- Outline, call hierarchy, etc. are essentially views of the CModel. The elements of the CModel are ICElements.
- The ICElement hierarchy was duplicated to model remote elements
- Not all methods work on remote elements since they do not correspond to things on the local filesystem
- View is populated from this list of ICElements
- Users may have different projects on different machines.
- Service provider: e.g., remote builder, remote indexer
- Each project has a service configuration, which is a list of the services it's using
- Service model: Description of services that are applicable to a given project; they may or may not actually be configured
- Service provider: e.g., remote builder, remote indexer
- RDT has separate managed build toolchains definitions for remote toolchains (e.g., Remote XLC)
- Needed access to system headers, libraries to preprocess code/parse correctly
- Converting other PTP tools for remote support
- Generalities
- C/C++ ASTs obtained locally will probably be incorrect since headers not available; anything that uses C/C++ ASTs should be done remotely
- Services, subsystems, and miners
- RSE Terminology: A miner runs on a remote server; it accepts commands and returns results (e.g., accept search keywords, return search results); in implementation terms, it is a Java class on the server that implements an RSE API
- Miners are always tied to a subsystem on the local machine. The subsystem knows how to send a command to a miner.
- A CDT miner was created; it accepts commands to reindex a project, get content assist for a string, etc.
- Other miners (e.g., for PLDT) could talk to the CDT miner
- Service providers are not necessarily tied to miners/RSE, but often it's useful to use RSE subsystems/miners to implement a service
- Example: Open Declaration
- RDT asks for a service provider; this provider provides an Open Declaration API (e.g., which accepts a filename and offset indicating the cursor location and returns the filename, offset, and length of the declaration)
- Service provider talks to the remote indexing subsystem on the local machine
- Subsystem uses an RSE API to talk to the CDT miner on the remote machine
- Miner returns the filename, offset, length of the declaration
- Specifics
- PLDT - Gets AST, constructs control flow graph, runs analysis
- Biggest problem is probably the preprocessor since headers aren't necessarily available; this means the ASTs constructed locally probably aren't complete
- Would need to convert analysis to run on the server, then create markers based on its results
- This means creating a PLDT miner, subsystem
- ETFw
- Photran
- Builder
- RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains
- Remote command launcher is responsible for running the build command on the remote machine
- Standard make should be reusable directly from CDT
- Create remote GNU toolchain
- Scanner discovery caused complications for C/C++; ignoring this issue in Photran right now
- RDT has a remote make builder (i.e., the one for standard make projects) and a generated makefile builder which uses remote toolchains
- Indexer
- Need to implement a service provider, subsystem, miner
- Greg implemented a service that can deploy a payload
- Copies server to the remote system, executes it, and tunnels that traffic through SSH
- We would need two implementations of our service: one talks to RSE directly, one knows how to tunnel the traffic over SSH
- Builder
- PLDT - Gets AST, constructs control flow graph, runs analysis
- Generalities