Difference between revisions of "TM and RSE FAQ"
(→What is the difference between RSE IFileService and EFS?)
|Line 158:||Line 158:|
== Working with TM / RSE as a User ==
== Working with TM / RSE as a User ==
==== Why is the Outline View empty when editing a remote PHP or C file? ====
==== Why is the Outline View empty when editing a remote PHP or C file? ====
Revision as of 06:58, 2 April 2008
- 1 Introduction
- 2 General
- 3 Download, Installation and Bug Reports
- 3.1 Which TM build should I download?
- 3.2 Where can I find TM / RSE using the Install/Update Manager from the Europa Discovery Site?
- 3.3 Which operating systems does TM / RSE support?
- 3.4 How do I ask questions?
- 3.5 How do I report a bug or request a feature?
- 3.6 Where is this .log file that I hear so much about?
- 4 Working with TM / RSE as a User
- 5 Working on TM / RSE
- 5.1 How do I debug a problem with TM / RSE?
- 5.2 How do I use eclipse to develop eclipse?
- 5.3 How do I build RSE from CVS if I want a more recent build than is on the downloads page?
- 5.4 How do I modify the code
- 5.5 How do I run or debug with my changes?
- 5.6 How do I export it so that it can be used with an external Eclipse installation?
- 5.7 How do I submit a fix for a bug?
- 5.8 How do I submit a contribution beyond a simple bug fix?
- 5.9 How do I distribute my changes to my customers?
- 6 TM and RSE Architecture
How do I contribute to this FAQ?
Simply edit this page and add content. You can now use your bugzilla username and password to gain access.
How can I get notified of additions to that FAQ?
Log in to the Wiki. On your personal Preferences page, enable E-Mail notification. Then, click the watch tab of this page.
Is it really that easy?
What is the Target Management Project?
It's all about remote computer systems (targets, hosts, boards, grids, farms, nodes...) and developing software on them: remote build, connect, get status, download, run, debug, ...
Since there are many different vendors and solutions around for remote access and especially in the device software space, the main charter of target management is to provide data models and frameworks that are flexible and open enough for vendor-specific extensions. But TM is not limited to device software at all - it can also be used for remote grid or mainframe operations. In fact, the things to be done are often similar, just the terminology is different. Sample implementations are being provided for TCP/IP connections, FTP data transfer and gdb remote launching in the CDT environment. The basis for our implementations is the Remote System Explorer (RSE).
Apart from the Extender's perspective, TM and RSE is also a toolkit that can be used out of the box: a framework, widgets and tools to manage remote systems under a single, consistent user interface.
- For a more official answer, see the
- For an excellent document we prepared when the project was started (but is still fully correct!) on the terminology we choose and the whole list of use cases we want to address in the long term, see the TM Use Cases Document.
- The EclipseCon 2007 Short Talk slides give a good quick overview of the project. PPT slides include comments with additional explanations.
What is RSE?
The Remote System Explorer (RSE) is a consistent framework and UI for accessing remote compute resources from Eclipse.
So what is the difference between TM and RSE?
The Target Management Project works in two repositories:
- org.eclipse.tm.core - Core Components for remote access that can be re-used without other dependencies. Currently, these include Jakarta Commons/Net, Zeroconf Discovery and a Terminal view.
- org.eclipse.tm.rse - The Remote System Explorer (RSE).
We are delivering RSE for download, which integrates some (but currently not all) core components into a consistent framework and UI. TM is the "project", RSE is the "product".
How is TM / RSE licensed?
RSE consists of software produced by the TM / RSE team combined with third party software developed from other Open Source Projects. The software produced by the TM / RSE team is licensed under the Eclipse Public License (EPL). The software designed by third parties is made available under their respective licenses. Refer to the about.html file in the root directory of every RSE plugin for specific licensing information.
How do I find out about future releases of TM / RSE?
If you wish to contribute to the development of RSE, we welcome the opportunity to work with you. The plans will be updated to reflect the commitments made by contributors to this projects. See #Working on RSE for information on how to get started.
Is the TM Project also working on a lightweight Open Source Agent framework in C?
- Yes! An initial contribution of the Target Communication Framework (TCF) has been made by Wind River and is currently pending Eclipse Legal IP Review via bug 210751. While dstore will remain the Java-based agent framework for large, powerful targets, TCF is a small footprint, very lightweight agent framework for less powerful embedded systems.
- A PPT presentation on TCF Concepts and Requirements is now available (PPT, 500K)
- For more details, see the DSDP/TM/TCF FAQ.
Download, Installation and Bug Reports
Which TM build should I download?
The latest stable release version is available from the TM downloads page.
Where can I find TM / RSE using the Install/Update Manager from the Europa Discovery Site?
All components of TM are available for download under Remote Access and Device Development in the Install/Update manager. The specific downloads from TM under this category are:
- Remote Systems Explorer End-User Runtime
- Remote Systems Explorer Extender SDK
- Remote Systems Explorer C/C++ Remote Debug Launcher
- Target Management Terminal
- Target Management Service Discovery
Remote Systems Explorer C/C++ Remote Debug Launcher and TM Service Discovery require additional plug-ins. When downloading these components make sure to click "Select Required" to get the additional plug-ins.
Which operating systems does TM / RSE support?
The RSE Framework is platform independent. It will run where Eclipse will run. According to the TM Project Plan, we are testing on a set of Reference Platforms including Windows, Linux, MacOS X and Solaris.
For the remote side, we are testing the following combinations:
- The remote dstore agent is being tested on Windows, Linux and Solaris but should also run fine on any OS that supports a Java 1.4 VM or later. There are known implementations on AIX, OS/390, and z/OS.
- FTP is being tested against many public available servers including Windows, Linux, FileZilla, ProFTPD, and VxWorks. VMS based servers are known to have problems bug 161238.
- Ssh works against any ssh2 compliant server.
See also the test matrix on the TM 2.0 Testing pages and below.
If you are using on a platform not mentioned here, please make yourself heard on the dsdp-tm-dev developer mailing list. You can contribute test results there.
If you have a favorite platform we highly encourage you to get involved and volunteer to own a feature that does not currently have an implementation that works on your platform of choice. See #Working on RSE for more information.
How do I ask questions?
Target Management and RSE related questions that are not answered in this FAQ or the documentation should be posted to the TM newsgroup. You will need a newsreader and a password. You can also use this simple web interface or this more advanced web interface to browse the newsgroup. General Questions about the Eclipse SDK which includes the Eclipse Platform, JDT (Java Development Tools), or PDE (Plugin Development Environment) should be posted to the Eclipse newsgroup.
Keep in mind that these newsgroups are public, so do not include any confidential information in your questions. You should also read "How to ask questions the smart way" by Eric Raymond before participating in the newsgroups. NOTE: Please submit bugs to bugzilla, not to the newsgroups. #How do I report a bug or request a feature? section of this document.
People will still come into a newsgroup asking questions that have been answered before and often will not provide any information about what versions they have installed, and what the problem is. You will be much more likely to get help if you provide enough information to reproduce the problem. The section on #How do I report a bug or request a feature? gives a list of some information which could be useful.
How do I report a bug or request a feature?
The Target Management Project and RSE (like the Eclipse Project) uses bugzilla as its bug and feature tracking system. Around, this, we have developed some queries and best practices on the TM Bug Process Page.
Entering a bug/feature report is as simple as filling in a web form on the eclipse bugzilla page. The first time you enter a bug you will need to create a new bugzilla account for yourself by providing an email address and choosing a password.
Entering a bug report or enhancement request is really simple, and we encourage users to just go ahead and do it so the request gets tracked. If, on the other hand, you prefer to first search if a similar bug has been reported before, just use the bugzilla search form. The TM Bug Process Page also has some pre-defined queries that you can modify (just press the "Edit this search" link after running the query). If you find a bug report that outlines the problem you are seeing, you can simply annotate it with your comments to let the developers know that you have also hit the bug. Also you can add yourself to the CC list of the bug so that you will notified when the status of the bug changes or someone adds comments.
Once you have searched bugzilla and not found anything, you can go ahead and enter a new bug report. Don't let yourself be constrained too much by the bug writing guidelines. Following these is helpful for us but not an absolute must. Fill in what's easily accessible for you from the following:
- 1. The build level of Eclipse that you are using. For example, "Eclipse 3.2.1"
- 2. The build level of TM that you are using. For example, "TM 1.0 build I20061104"
- 3. Your computer's specifications (OS version + patch level, memory, other pertinent info)
- 4. The contents of your .log file (or lack thereof). This is especially important if you get a dialog that reports an internal error. See #Where is this .log file I hear so much about? for information on finding your .log file.
- 5. The Java runtime or development kit you are using to run eclipse (use java -version or java -fullversion)
- 1. A description of what you were doing,
- 2. A description of what behavior that you observed, and
- 3. An explanation of how the observed behavior differs from the expected behavior
Note that once you have filled in your environmental settings in one bug report, you can "Remember values as bookmarkable template" in bugzilla to easily file your next bug report on the same computer the next time.
Where is this .log file that I hear so much about?
The .log file is located in the workspace/.metadata directory.
The .log file is used by the Eclipse Platform to log runtime errors. It is useful to include it in bug reports because it contains stack traces that occur in plug-ins.
You can also see the .log file in Eclipse from Help > About > Configuration Details > View Error Log or, if you have installed eclipse-SDK (with JDT and PDE) from Window > Show View > Other > PDE Runtime > Error Log.
When you report a bug, including backtraces or error info from your .log is tremendously helpful! If you see multiple backtraces that seem to be related to your problem, it's best to just go and attach your entire .log file to the bug.
Working with TM / RSE as a User
How can I use a remote workspace over SSH?
There are multiple possibilities:
- 1. Use rsync to copy your remote workspace to the local machine, work on it, and rsync it back to the remote when done. RSync comes pre-installed on most Linux machines, is available in Cygwin for Windows or you can use the DeltaCopy installable wrapper on Windows.
- Advantages of this approach: fast local access to all files, static analysis with parsers works fine
- Disadvantages of this approach:
- tools which are only available on the remote cannot be used
- updates can only be previewed after syncing back with rsync, which is problematic when e.g. working on a remote website that should be previewed as you edit
- approach is problematic when the workspace is huge and only few files need to be looked at
- problematic when multiple users work on the remote workspace at the same time (merge issues)
- 2. Use RSE to transparently edit remote files
- Advantages of this approach: directly editing on the remote, minimal data transfer (works fine on slow connections), provides immediate preview, no merge issues and remote tools are available remotely
- Disadvantages of this approach: since local Eclipse tooling like CDT, PDT, Web Tools etc are not aware of the remote files being edited in RSE, static analysis / parsers do not work (see also the next FAQ entry)
- 3. Use the RSE EFS Provider to make the remote resources transparently available to Eclipse as Workspace Resources. Not all Eclipse tooling is properly aware of EFS yet; gives some background. Also, bug 181460 is open against RSE because purely remote projects (i.e. where the .project file is on the remote, and should be shared through EFS) fail to open automatically on Eclipse startup.
- Advantages of this approach: directly editing on the remote, provides immediate preview, no merge issues and remote tools are available remotely
- Disadvantages of this approach:
- Very slow refresh when the remote file tree is huge on a slow connection. Since EFS maps the entire remote file tree into the workspace, and the Eclipse Workspace wants to do a "deep refresh" on all remote resources on startup, refresh can easily take in the range of hours on a slow connection. bug 219164 is open to track this situation, but it's unlikely this can be improved unless some APIs are broken in the way the Resources system works.
- Not all Eclipse tools support EFS yet (see this newsgroup posting for some background). Currently, Eclipse Platform, PDT and WST should work fine while CDT is still struggling, see bug 177994.
- On fully remote projects (when the .project file is remote), the RSE EFS provider currently cannot automatically re-open the project on Eclipse startup. bug 181460 is open against RSE for this problem, but it's hard to fix; alternative EFS providers could be used to overcome this problem. The PTP/planning/remote Wiki summarizes the PTP project's efforts in this area.
Summing up, I would recommend rsync when the remote workspace is not too huge, no merge issues are expected and a sufficient number of tooling is available locally. The EFS approach is the right one for very fast, almost-local connections and when the tooling supports it. The RSE] approach is preferred on slow connections or huge remote workspaces, but you'll need to sacrifice some static analysis tooling support and you'll need to run your compilers etc. in a remote shell.
Why is the Outline View empty when editing a remote PHP or C file?
PHPEclipse and CDT need a project context in order to fill the outline view. Remote Files accessed through RSE are not associated with any particular project, so they are treated as "external files" and don't have a PHP or CDT personality attached.
If you want full tooling support for remote files, you should try setting up the remote files through EFS. For PDT, which is based on the WST editor framework, this should work properly since the Europa Fall Maintenance release (28 Sep 2007). It would work as follows:
- Create a new local PDT project (initially empty)
- New > Folder > Advanced > Link to Folder in File System > Deselect default, select RSE File System > Browse to Remote Folder
As of Eclipse Europa, only few tooling projects support EFS, but support is growing. For more details, see the EFS Wiki page. CDT, particularly, does not yet support remote EFS resources properly, see bug 177994. We suggest filing similar bug reports against other Eclipse based tools to make them more aware of EFS and external files without project context.
An update on the EFS situation has been posted on the eclipse.dsdp.tm newsgroup lately.
How can I do Remote Debugging with CDT?
There is two very different kinds of setup for remote debugging:
- 1. Have the Debugger run on the remote system, and
- 2. Have the Debugger run on the local system, as a cross debugger, with some kind of thin agent on the remote.
The first, "remote debugging" case, can be addressed with CDT alone: typically, compilation will happen on the remote host, and all tools need to be available remotely including the debugger. An NFS or Samba mount can be used to make the remote sources visible locally for static analysis and editing. The make command is changed into "ssh remotehost make -C remoteDir" and the debug command is changed into "ssh remotehost gdb". See bug 30094 comment 13 for more info.
For the second, "cross debugging" case, there are multiple options but they all have one prerequisite in common: you need a cross toolchain installed locally. That is, your compiler runs locally (e.g. on Windows) and produces output that the remote (e.g. Linux) system can execute. Similarly, your debugger runs locally (e.g. gdbppc) and is capable of debugging the remote system. Such a cross toolchain is available from commercial vendors, or you can build one yourself. See Wikipedia:Cross-compilation for a starting point.
When you have your cross toolchain set up (and tested on the commandline), you can start integrating it into Eclipse. There's multiple options for this:
- 1. Use the Target Management / RSE Remote CDT Launch. It uses any RSE file subsystem and shell subsystem, so it works with SSH, Telnet, FTP or whatever communication mechanism you have plugged into RSE. It requires a gdbserver executable installed on the remote, and is capable of uploading your executable to debug to the remote. Here is a mailing list entry that should help you get started.
- 2. Use the Device Debugging / DSF gdb/mi integration's remote launch capabilities. Here is a mailing list entry. The DSDP/DD/GDB Wiki gives more information about this component.
- 3. Use the Target Communication Framework (TCF) component of Target Management. TCF provides a thin plain-C agent that runs out of the box on Linux and uses ptrace for debugging. There are two separate debugger integrations for Platform/Debug and DD-DSF available. TCF is still incubating, so from a User's perspective the debugging capabilities are still very limited. But if you want to integrate your own debugger or cannot use gdbserver for any reason, you should be looking at TCF.
For a general summary about CDT Remote Development, see this mailing list entry which has some more pointers into bugzilla, as well as the PTP/planning/remote Wiki. There was also a CDT Remote / RDT Presentation at EclipseCon 2008, which suggests even forming a new Eclipse project for remote development in general.
Working on TM / RSE
How do I debug a problem with TM / RSE?
When you installed RSE into Eclipse SDK, you can easily self-host:
- Switch to JDT or PDE perspective
- File > Import > Plug-in Development : Plug-ins and Fragments
- From the target platform
- Select from All Plugins
- Binary projects with linked content
- --> Next
- Filter "*rse*", select and add the plugins you need, "Add", Finish
- Press Ctrl+Shift+T to navigate to an interesting RSE class, place breakpoint
- or do Run > Add Java Exception Breakpoint if you want to trace a particular exception
- With any of your imported projects selected, do Run > Debug As > Eclipse Application
How do I use eclipse to develop eclipse?
The self-hosting instructions explain how to use eclipse to develop eclipse.
- If you want to work with a more recent version of any Eclipse plugin than you downloaded (this is only rarely needed), you will need to connect to the Eclipse Project CVS repository. To connect to the Eclipse Project CVS repository, open the CVS repositories view (Perspective->Show View->Other...->CVS->CVS Repositories) and create a new CVS repository location (right click->New->CVS Repository Location. Paste the following information into the "Add CVS Repository" dialog, "Host" field:
- For connecting to the TM and RSE CVS repository, see instructions on th TM CVS Setup page.
NOTE: When you are connected as anonymous you will have read rights to the repository, but you will not be able to commit any code. See #How do I submit a fix for a bug? below for how to contribute back your changes.
How do I build RSE from CVS if I want a more recent build than is on the downloads page?
Start Eclipse SDK, and import a CVS Team Project Set as explained on the TM CVS Setup page.
How do I modify the code
Change any file you want. When you save it, it will be built.
How do I run or debug with my changes?
After successfully building TM / RSE inside Eclipse, one typically wants to run an instance of Eclipse with the freshly built plugins (perhaps after making some changes to the source code). This is very easy to do in the PDE. Here are the steps:
- 1. Open the "Plug-in Development" perspective (you may have to go to "Others" to find it).
- 2. Set breakpoints in the code where you need them
- 3. Ensure that the Project you changed is selected in the Project Explorer
- 4. Select the menu action Run -> Debug As... -> Eclipse Application. (
- 4a. You may also -> Run As..., or use the toplevel run/debug icon, which also allows you to edit or re-launch the previous launch.
How do I export it so that it can be used with an external Eclipse installation?
- You can either:
- a) Export an RSE feature via File->Export->Plugin Development->Deployable Features. This will automatically export all the required plugins.
- b) Export all the plugins etc. individually or all at once via File->Export->Plugin Development->Deployable Plugins and Fragments. However, this is more error prone and you're better off doing a).
- c) Use the ANT stuff in org.eclipse.tm.rse/releng to build RSE the way the nightly build does.
How do I submit a fix for a bug?
While using the Eclipse SDK to develop your plug-in, you found a bug in RSE. You submitted a bug report, but need the fix now. You've debugged the problem, and there is a simple fix. So you figured out how to use eclipse to develop eclipse and have written a fix for the bug you found. Now you want to release the fix to the eclipse community. How do you do this?
First, create a patch. You can create a patch by using the Team patch creation facility.
- 1. Select the project you have patched. It must be connected to the TM CVS repository, so it's best if you got the project from our team project set.
- 2. Right click->Team->Create Patch...
- 3. The Create Patch wizard will prompt you for a file name. You should name your patch with the bugzilla bug report id.
Now you can submit the patch to the appropriate component developer mailing list. Be sure to include information about what your patch fixes. The committers for the component will evaluate your patch to see if it fixes the bug and is acceptable. If your patch is accepted, it will be released by the component team into the repository.
When you submit your patch to bugzilla, it helps streamlining the process when you add a few simple statements verifying that you authored the patch yourself and you are authorized (by your employer) to contribute it under the EPL. See the TM committer HOWTO for a template statement you can copy and paste.
How do I submit a contribution beyond a simple bug fix?
We are always keen to get contributions beyond simple bug fixes as well. But since we need to remain within our charter, and make sure that all code coming from Eclipse.org is really clean in terms of IP and legal / copyright issues, we need to be a little bit more careful with substantial contributions (exceeding 250 lines of code and documentation).
- 1. Make yourself known to the TM team, either by entering an enhancement request for the stuff you can contribute, or through the developer mailing list. Describe what you want to contribute, and verify that the TM team can receive it. Our charter and process requires that we verify with the DSDP PMC (Project Management Council) that the actual contribution is welcome. This is mostly in order to avoid babylonic disgression and unnecessary duplication of features which are also available in other Eclipse projects you might not know about.
- 2. Make sure that your employer is OK with you contributing the code. We're most happy if you can contribute it under the Eclipse Public License (EPL). If you employer asks questions, there is a good EPL FAQ available online, as well as other guide to legal documents. If you can not contribute under the EPL but want some other license, please contact us directly.
- 4. Add your legal statement verifying that you are actually authorized to contribute. By doing that, it's official already and you publish the code under your license (hopefully EPL). Anybody can look at your code now, download it and try it out! But, before we can accept it into the CVS Repository we'll need to have it reviewed by the Eclipse.org legal team (we call that "due diligence"). So be prepared to wait (hopefully not more than 1 month for EPL contributions) before your code shows up in an official Eclipse / Target Management release - the benefit of this review is that even large, cautious companies will adopt your code into their products because they are pretty sure that everything is OK from a legal standpoint.
That's it! - The process may sound scary at first, but all the steps are important and it's really not that bad once you've done it. Thanks and Kudos - we'll happily give you credit on our website for your generous contribution!
How do I distribute my changes to my customers?
Anyway that you see fit! Actually, if anybody has suggestions for this answer, please fill them in here.
TM and RSE Architecture
How can I understand the RSE architecture and learn programming on it?
- Surely the best resource right now is the EclipseCon 2008 Tutorial on RSE material - click on the "Presentation Material" link on top of that page, or download the slides and sample code separately from the TM homepage. The tutorial an architectural overview, sample code, and concrete steps on how to get started.
- Your next best bet is the RSE Developer Documentation, which also includes a tutorial and is available online or as part of your RSE SDK installation from the Help > Help Contents menu. There is a special section on RSE Architecture.
- If you have other questions, feel free to ask on the newsgroup or developer mailing list.
What is the difference between RSE IFileService and EFS?
- RSE API allows filters for directory contents retrieval in the API:These filters can be evaluated at the remote side for performance, if there is an agent remotely.
IFileService.getFiles(progress, parent, filter)
- RSE API supports multi-list and multi-file queries to reduce the number of roundtrips for high-latency connections
- RSE API has some hi-level getters like
- RSE is geared towards persistent caching: for downloads, IFileService gets a File instance where the remote file should be stored; EFS deals with Streams only.
- Since TM 2.0M6a, RSE also includes a well-working EFS provider that builds a bridge between services registered as RSE IFileService and EFS. The benefit of having RSE the provider is that it handles all credential storage and connection properties. The little drawback is that these services require a UI and the Core Resources plugin, so remote projects shown through RSE are not automatically re-opened on startup. For more details, see bug 185921 comment 3, the EFS Wiki page and the Blog about TM 2.0M6a.
- See also Eclipse File Service APIs Compared, as well as the EFS Wiki