Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "CDT/Archive/Flexible Project Structure"
| Line 1: | Line 1: | ||
=== Overview === | === Overview === | ||
| − | The fact that Eclipse projects need to map so closely to the file system is very limiting to many CDT users. | + | The fact that Eclipse projects need to map so closely to the file system is very limiting to many CDT users. They often have existing directory structures that contain their code which don't map easily to Eclipse projects. As such we'd like to provide the flexibility to add files and directories from anywhere into their projects and have all existing Eclipse functionality work properly with them (and that's the hard part). Another problem is that often times there is too much content under the project itself. This causes unnecessary overhead in things like refreshing the workspace. It also requires CDT to allow users to set source folders, or folders that CDT should be interested for source files. A main reason behind this is so we don't spend unnecessary time indexing files that the user isn't interested in. So to summarize, we just need to make project content more flexible. |
| − | |||
| − | + | === Issues === | |
| + | This section will (attempt) to describe how Eclipse projects currently work (currently meaning 3.3 and the little I've seen of 3.4 to this point), and the issues that C/C++ developers might have. | ||
| − | + | The platform creates a .project file wherever the project creator tells it to (new project wizard for example). We'll refer to this location as the root project directory. The content of the project is determined by the file system content under the root project directory. Note that this can be anywhere in the file system, not just in the workspace. So if you have an existing code base somewhere, you'd want to make the project root directory include your code base so the files you're interesting show up in views like the Project Explorer. If any files you're interested don't fall under the root directory, you'll have a much more difficult time working with those files. They won't be displayed in the project views, won't be indexed, won't be tied to your project settings (include paths, symbols, etc.). | |
| − | + | There are several problems with this approach. Because the project content mirrors the file system, you'd have to find the shortest common path in the file system that contains all of the files you're interested in. Typically this isn't that hard to do, but in severe cases it could be the root of a drive! Here are some examples: | |
| − | + | C:\MyProjects\CommonCode\ | |
| + | C:\MyProjects\ProjectA\ | ||
| + | C:\MyProjects\ProjectB\ | ||
| + | C:\MyProjects\ProjectC\ | ||
| + | C:\MyProjects\ProjectD\ | ||
| − | + | Assuming ProjectA uses some of the common code shared across the different projects, you'd want the root project directory to be C:\MyProjects\. But now you have ProjectB, C, D, etc. all be under ProjectA's root. | |
| − | + | The project location issue gets more problematic when you need to work on multiple projects at the same time. When you create ProjectB, the "natural" project root is C:\MyProjects\ as well. But you can't create it there because the .project file (and .cproject file, etc) from ProjectA are already created in that physical directory. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438 for more info. Your only choices are: | |
| − | + | 1) Make the root C:\ - I think you can understand why this is not a good choice :) | |
| + | 2) Make the root C:\MyProjects\ProjectB - now you're missing all of the common code under Project B | ||
| + | |||
| + | |||
| + | One might ask at this point why you don't just have one Eclipse project for all of these projects rooted at C:\MyProjects\. There are several reasons for this. Each of these projects A-D were divided in the first place for a reason. They each have their own build artifacts and project properties like include paths and symbols. What would Eclipse's builder actions like build and clean do? C/C++ projects just don't fit into that model. | ||
| + | |||
| + | Another issue with .project files is that some users will want to share these in version control along with their code base, while others will not want them to "contaminate" their code base. Many teams have developers using different IDE's, and most other IDE's allow their project files to be created anywhere in the file system. This allows the user to decide if they're needed in version control or not. With Eclipse, it must reside in the code base. It should be noted that most version control systems have "ignore" capabilities by file extension - so this would seem to not be a big issue. But we've run into at least one Eclipse plugin from a version control provider that simply deletes files it's not aware of. Things don't work really well when the .project file is deleted out from under you! :) | ||
| + | |||
| + | |||
| + | === Background === | ||
| + | The Eclipse platform tried to address some of the issues with project content flexibility by introducing linked resources. In many cases this was just what the doctor ordered. But there are several things about linked resources that aren't so good. | ||
| + | |||
| + | 1) This is by far the most serious of the issues, and the main reason we feel that linked resources cannot be used at all. The team support simply doesn't work with linked resources. Right now it flat out ignores them. We could potentially fix this in the CVS team support, but what about all of the third party plugins out there? It's hard to ship and IDE without team support, and users just use a wide variety of version control systems. I don't think linked resources can be taken seriously as an option for project flexibility until at least all of the major version control suppliers support them in their plugins. | ||
| + | |||
| + | 2) You can't make file system changes with links. For example, if you want to rename a file or folder, or move a file around, you can't do this with linked resources. It only changes the link itself, not the underlying resource. | ||
| + | |||
| + | 3) Creating new resources in a project with links is confusing at best. Let's say you have a project with a linked folder and file at the root. If you create a new file or folder at the root, it is created in the workspace, not where the other folder/file are in the file system. But if you create a new file under the linked folder, it gets created where you'd expect. | ||
| + | |||
| + | |||
| + | === Possible Solutions === | ||
| + | |||
| + | Two possible solutions that have been discussed: | ||
| + | |||
| + | 1) Linked resources | ||
| + | |||
| + | It's not clear if this can work or not - mainly due to the team support problems mentioned above. Even if we could, it doesn't do anything for the .project file location problem. It does for those that don't want the project files checked into version control. But for those that do, they'll have the same problems they do now. | ||
| + | |||
| + | 2) EFS | ||
| + | |||
| + | We could potentially implement a EFS file system that allows you to add and exclude files and directories from anywhere using URLs. This file system then presents a logical hierarchy to the IResource tree and handles the parent/child mapping. The issue we run into immediately is that there is a lot of code in the CDT (and elsewhere?) that assumes that the parent/child mapping in the IResource tree matches the physical file system and cheats by not asking EFS for that information but going directly to java.io.File. | ||
| + | |||
| + | There also may be a lot of code that uses IResource#getLocation rather than IResource#getLocationURI, which would at least appear to be a problem. Files are actually first class citizen resources in this model at least (not linked that is). So in theory version control would work. We could potentially used the new "hidden" resource attribute for hiding resources even when using EFS. This is available in Eclipse 3.4 and would appear to be all we need to be able to hide files/folders under a project that we're not interested in. | ||
| + | |||
| + | Preliminary investigation leads me to believe that EFS is not a workaround for the .project file location issue. In theory it sounds possible, but there's no way to determine if the file being requested in FileStore#getChild is a temp "project" file, or some file being created by the user. A crude filename test could be done I suppose, but there are many more of the temp project type files created. For example, CDT creates a .cproject file, and any other plugins could be creating their content in the .project file location. We'd want all of these types of files to be created in some arbitrary location, and then the content of the project to be fetched from natural project root. | ||
| + | |||
| + | |||
| + | === Other Considerations === | ||
| + | Assuming we can find a way to make projects more flexible, this section lists some possible things to think about from a CDT perspective. | ||
| + | |||
| + | Should we do away with source paths? If everything under the project is indeed of interest, then do we still need source paths? Perhaps you may still only want to index a subset of the project. Or for managed builder, you only want to build the content of certain folders. Are these the same set, or should these be controlled separately? | ||
| + | |||
| + | What type of UI would we need to hide/unhide resources? | ||
Revision as of 18:29, 14 February 2008
Overview
The fact that Eclipse projects need to map so closely to the file system is very limiting to many CDT users. They often have existing directory structures that contain their code which don't map easily to Eclipse projects. As such we'd like to provide the flexibility to add files and directories from anywhere into their projects and have all existing Eclipse functionality work properly with them (and that's the hard part). Another problem is that often times there is too much content under the project itself. This causes unnecessary overhead in things like refreshing the workspace. It also requires CDT to allow users to set source folders, or folders that CDT should be interested for source files. A main reason behind this is so we don't spend unnecessary time indexing files that the user isn't interested in. So to summarize, we just need to make project content more flexible.
Issues
This section will (attempt) to describe how Eclipse projects currently work (currently meaning 3.3 and the little I've seen of 3.4 to this point), and the issues that C/C++ developers might have.
The platform creates a .project file wherever the project creator tells it to (new project wizard for example). We'll refer to this location as the root project directory. The content of the project is determined by the file system content under the root project directory. Note that this can be anywhere in the file system, not just in the workspace. So if you have an existing code base somewhere, you'd want to make the project root directory include your code base so the files you're interesting show up in views like the Project Explorer. If any files you're interested don't fall under the root directory, you'll have a much more difficult time working with those files. They won't be displayed in the project views, won't be indexed, won't be tied to your project settings (include paths, symbols, etc.).
There are several problems with this approach. Because the project content mirrors the file system, you'd have to find the shortest common path in the file system that contains all of the files you're interested in. Typically this isn't that hard to do, but in severe cases it could be the root of a drive! Here are some examples:
C:\MyProjects\CommonCode\ C:\MyProjects\ProjectA\ C:\MyProjects\ProjectB\ C:\MyProjects\ProjectC\ C:\MyProjects\ProjectD\
Assuming ProjectA uses some of the common code shared across the different projects, you'd want the root project directory to be C:\MyProjects\. But now you have ProjectB, C, D, etc. all be under ProjectA's root.
The project location issue gets more problematic when you need to work on multiple projects at the same time. When you create ProjectB, the "natural" project root is C:\MyProjects\ as well. But you can't create it there because the .project file (and .cproject file, etc) from ProjectA are already created in that physical directory. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438 for more info. Your only choices are:
1) Make the root C:\ - I think you can understand why this is not a good choice :) 2) Make the root C:\MyProjects\ProjectB - now you're missing all of the common code under Project B
One might ask at this point why you don't just have one Eclipse project for all of these projects rooted at C:\MyProjects\. There are several reasons for this. Each of these projects A-D were divided in the first place for a reason. They each have their own build artifacts and project properties like include paths and symbols. What would Eclipse's builder actions like build and clean do? C/C++ projects just don't fit into that model.
Another issue with .project files is that some users will want to share these in version control along with their code base, while others will not want them to "contaminate" their code base. Many teams have developers using different IDE's, and most other IDE's allow their project files to be created anywhere in the file system. This allows the user to decide if they're needed in version control or not. With Eclipse, it must reside in the code base. It should be noted that most version control systems have "ignore" capabilities by file extension - so this would seem to not be a big issue. But we've run into at least one Eclipse plugin from a version control provider that simply deletes files it's not aware of. Things don't work really well when the .project file is deleted out from under you! :)
Background
The Eclipse platform tried to address some of the issues with project content flexibility by introducing linked resources. In many cases this was just what the doctor ordered. But there are several things about linked resources that aren't so good.
1) This is by far the most serious of the issues, and the main reason we feel that linked resources cannot be used at all. The team support simply doesn't work with linked resources. Right now it flat out ignores them. We could potentially fix this in the CVS team support, but what about all of the third party plugins out there? It's hard to ship and IDE without team support, and users just use a wide variety of version control systems. I don't think linked resources can be taken seriously as an option for project flexibility until at least all of the major version control suppliers support them in their plugins.
2) You can't make file system changes with links. For example, if you want to rename a file or folder, or move a file around, you can't do this with linked resources. It only changes the link itself, not the underlying resource.
3) Creating new resources in a project with links is confusing at best. Let's say you have a project with a linked folder and file at the root. If you create a new file or folder at the root, it is created in the workspace, not where the other folder/file are in the file system. But if you create a new file under the linked folder, it gets created where you'd expect.
Possible Solutions
Two possible solutions that have been discussed:
1) Linked resources
It's not clear if this can work or not - mainly due to the team support problems mentioned above. Even if we could, it doesn't do anything for the .project file location problem. It does for those that don't want the project files checked into version control. But for those that do, they'll have the same problems they do now.
2) EFS
We could potentially implement a EFS file system that allows you to add and exclude files and directories from anywhere using URLs. This file system then presents a logical hierarchy to the IResource tree and handles the parent/child mapping. The issue we run into immediately is that there is a lot of code in the CDT (and elsewhere?) that assumes that the parent/child mapping in the IResource tree matches the physical file system and cheats by not asking EFS for that information but going directly to java.io.File.
There also may be a lot of code that uses IResource#getLocation rather than IResource#getLocationURI, which would at least appear to be a problem. Files are actually first class citizen resources in this model at least (not linked that is). So in theory version control would work. We could potentially used the new "hidden" resource attribute for hiding resources even when using EFS. This is available in Eclipse 3.4 and would appear to be all we need to be able to hide files/folders under a project that we're not interested in.
Preliminary investigation leads me to believe that EFS is not a workaround for the .project file location issue. In theory it sounds possible, but there's no way to determine if the file being requested in FileStore#getChild is a temp "project" file, or some file being created by the user. A crude filename test could be done I suppose, but there are many more of the temp project type files created. For example, CDT creates a .cproject file, and any other plugins could be creating their content in the .project file location. We'd want all of these types of files to be created in some arbitrary location, and then the content of the project to be fetched from natural project root.
Other Considerations
Assuming we can find a way to make projects more flexible, this section lists some possible things to think about from a CDT perspective.
Should we do away with source paths? If everything under the project is indeed of interest, then do we still need source paths? Perhaps you may still only want to index a subset of the project. Or for managed builder, you only want to build the content of certain folders. Are these the same set, or should these be controlled separately?
What type of UI would we need to hide/unhide resources?