WTP CVS Restructuring
In an effort to standardized groups on dev.eclipse.org especially in light of our recent WTP Projects Refactoring, the page is to propose and document how to restructure our CVS directories, so their access can be managed by associating directories with Unix Groups. In principle, we could leave all the directories the way they are now, and just carefully assign the Unix groups on a directory by directory bases ... but, this would be very complicated, error prone, and hard to scale in the future.
The document will start by focusing on the old wst and jst directories, as they need the most work, and other subprojects addressed later, towards the end of this proposal.
I have opened bug 207212 to track implementation and resolution of this issue. Please comment there if there are questions or suggestions for this document.
First, I'll describe my best interpretation of what our standardized groups will be.
The following are the Unix groups that will be associated with "code", in the cvs repository.
webtools.common-dev webtools.datatools-dev webtools.ejbtools-dev webtools.jeetools-dev webtools.servertools-dev webtools.sourceediting-dev webtools.webservices-dev webtools.maps-dev webtools.releng-dev webtools.incubator-dev webtools.jsf-dev * webtools.dali-dev * webtools.atf-dev * * these are very similar in name to what they already are.
Per our WTP policy, all committers to any WTP project get full access to all of the WTP websites, so, we just need one group for that:
webtools.all (currently, our "webtools" group is similar, but think webtools-home is used for this)
For reference, the current (cvs) website locations are
org.eclipse/www/webtools (main one) org.eclipse/www/wtp (incubator) org.eclipse/www/jsf (used? do we still need this? need separate?) org.eclipse/www/dali Used for redirect purposes; would like to keep the re-direct in place, but we should remove all of the content here org.eclipse/www/atf
Do we need new, separate ones for each of the new subprojects? (No, I don't think so).
Download site write access
This is currently limited to the "build team". The group is currently "webtoolsadmin" and I think the proposal for consistent naming is to use "webtools.downloads".
Map File Directories
Notice I'm proposing all the map file directories be moved to under one higher level "webtools.maps" directory. The purpose of that is, I'd like the map files to be writable by all, hence associated with webtools-all. Some of the divisions of the map files themselves can come after the main refactoring.
The reason is that occasionally someone from another project may have to releaes, or revert a particular map file from another project.
And, I checked, and there's no particular legal reason to "regulate" the map files ... they are similar to the web pages, there's no particular content to protect with IP rules, and I think there's little danger of someone accidently touching someone elses, unless there'd been prior agreement.
WST and JST to Refactored Projects
These initial WTP projects were created with the following sort of hierarchy -- abbreviated and idealized here to simplify explaining the principles of the restructuring. See the Detailed Listings section below the literal contents of the proposed new structure.
wst components assembly features plugins server features plugins tests xml features plugins org.eclipse.wst.xml.core org.eclipse.wst.xml.ui tests examples development thirdparty xsd plugins org.eclipse.wst.xsd.core org.eclipse.wst.xsd.ui tests
The new proposed structure would look as follows
webtools.maps releng.common releng.sourceediting releng.servertools releng.webservices releng.datatools releng.jeetools releng.ejbtools releng.dali releng.jsf webtools.releng features plugins org.eclipse.wtp.releng org.eclipse.wtp.releng.tests org.eclipse.wtp.releng.webupdatesite releng.builder releng.control releng.wtpbuilder releng.wtptools working sourceediting features plugins org.eclipse.wst.xml.core org.eclipse.wst.xml.ui org.eclipse.wst.xsd.core org.eclipse.wst.xsd.ui tests examples development thirdparty servertools features plugins tests
The above examples show a few points related to this restructuring.
- The best top-level directory name should be similar to what the access group name will be: toplevel.subproject.component-subcomponent. We do not currently have components controlled by access restrictions, so I think we in webtools need only define the toplevel.subproject groups for now, hence the top level directories can be simply the subproject name, since 'webtools' is in its own cvs repository, so 'webtools' does not need to be part of the directory name. Also note that component and subcomponents can be added in the future, but best to start off with simplest, and add detail later, when needed.
- Thus, everyone in a particular Eclipse subproject can be associated with a top level webtools directory, and automatically get access to everything in that directory, without the need for more complicated administration.
- Note the previous "assembly" features and plugins will be become part of releng subproject. These are the Eclipse product plugins, and the features that define our builds. Each project will be responsible for their own features delivered to WTP (currently called the build-component features).
- Note the example in source editing where what used to be directories, xml and xsd, are no longer distinguished by cvs directories, but all the plugins flattened under sourceediting/plugins.
- Conceptually and architecturally, xml and xsd could still be called 'components' but they do not have separate access control, so it's a bit overkill (and ultimately restrictive) to define them in separate cvs directories.
- Note that the "access control" for components such as xml and xsd is (still) social in nature. Legally, and in principle, anyone in the source editing project could change the code there, but in practice, there are individuals or small teams that normally do. Others, even in the same project, usually would attach a patch to a bugzilla if they wanted one of those experts to review and commit the code.
- The map files will not have access restrictions per se (all committers will have access to 'webtools.maps' but the maps will be refactored in directories that contain map files only for that subproject.
The following are the detailed listing of directories in what would be their new structure. Hopefully the directory names, and groups names are obvious.
To Do Items and Schedule
- 10/25 Proof on Concept
- try using CVS snapshots, on local, test machines with some move scripts to test to be sure we've accounted for everything.
- Note: I've included some scripts in one of our cvs projects that work against a snapshot to do the reorganization. I'll see if those can be evolved to be the real things, to use when the real thing is done. See
- Have Web Master sanity check the scripts and whole approach.
- 11/ 1 By the 11/1 Status meeting,
- Project leads to have reviewed the "detailed listings" for accuracy. Post requested revisions to wtp-releng, so revised listings can be created, preferably before the meeting.
- Releng (dw) to make sure all modules in all map files (for WTP 2.0 and 3.0) are listed in the modules file of CVS, so all (important) plugins show up in the flat listing.
- JSF, ATF, Dali should make a firm decision if they want to participate in re-org, to stay consistent, or, leave their directory structure as-is.
- 11/ 8 By the 11/8 Status meeting,
- spell out committer ID's for each Unix group
- make SED scripts to fix up map files, module files ... maybe project team sets?
- 11/ 13 By end-of-day
- Committers create patch files of any "unfinished" work that's not checked in, just for their personal backup.
- Committers check "unfinished" work into a temporary branch, if it's not ready for HEAD, so it will be moved during the mass move.
- Committers should create "Team Project Sets" of their favorite "in workspace" projects (assuming a SED script, or other editing can convert them).
- 11/ 16 1:00 PM - 5 PM Move day
- Begin the mass move
- Update modules file
- Update map files
- Do Test Build
- Do some quick Workspace tests
- Open the flood gates
JSF, JPA, and ATF
The directory-to-Unix-group associations will be pretty easy for these three subprojects and they can be left the way they are. Or, if any of those projects want to flatten their directory structure, this might be the time to do it. Each of those projects can provide their preference and will remain open for discussion, for now.
Reference and Notes
Committers can see our current CVS directories
Snapshots of our CVS can be obtained at
There is a SED Script to "edit" old path/module names to new path/module names that can be found in one of our utility releng projects in cvs: org.eclipse.wtp.releng.utilities. It is named sedscript.sed