ICE Development Process and Guidelines
The ICE development process has evolved over the years to better fit the needs of the developers and stakeholders. The process was created using two flavors of the Rational Unified Process framework (Model-Driven Systems Development and Agile Model Driven Development) with techniques from Extreme Programming and simple common sense added to keep the team flexible, lean, and responsive.
The team tries to follow the guidelines laid down in the Agile Manifesto and the Twelve Principles of Agile in so far as to make technical sense for the project. As it turns out, they almost always do, with the one exception of planning. (Working for the U.S. DOE means that we sometimes have to develop a plan and stick to it, although we hate deadlines fiercely.)
This page is under construction, just like everything else on the project.
ICE is developed iteratively with each month-long iteration starting and ending on the 10th of the calendar month. Each iteration focuses on several key activities and goals. "Review, retrospect, and plan" (RRP) meetings are held as needed, but never more than once per iteration, to track progress and define the direction of the next iteration.
The ICE team used to meet every morning between 9:00 and 9:30 in ICE HQ to plan activities for the day and review progress from the previous day. With the addition new team members working remotely on the west coast, the daily meeting has been shifted to 13:30.
- 1 The Most Important Thing: Deployment and Testing
- 2 Development Phases
- 3 Change Tracking and Version Control
- 4 Task Planning and Project Management
- 5 UML Modeling
- 6 Code Development Guidelines
- 7 What Happens if the Build Breaks?
- 8 Code Text Style
The Most Important Thing: Deployment and Testing
The ICE Development Team very firmly believes that working software is the primary measure of progress. Working software is to software engineering as an experiment is to science. Just as Richard Feynman said of scientific experimentation, "The principle of science, the definition, almost, is the following: The test of all knowledge is experiment. Experiment is the sole judge of scientific ‘truth’." The team believes successful deployment and happy customers are the sole judge of software development success.
The ICE Development Team practices continuous integration and test-driven development to stay on track and continuously deliver a working product. The team also requires that each developer commit a working build and that each developer test their pieces of ICE, including launching the binary, before new code or updated code is committed.
The development of new or existing pieces of ICE happens in phases, with activities from each phase happening within each development iteration. Activities from the different phases are not performed in a certain order in a given iteration and, for a single task, multiple activities from different phases may be performed.
- Business modeling - High-level discussions with stakeholders about a new feature or modification that would be valuable.
- Requirements gathering - Interviews, discussions, literature reviews and code prototyping to determine the requirements necessary to add new capabilities to, or extend the current capabilities of ICE.
- Analysis and design - In-depth design and analysis activities for a particular capability, possibly including but not limited to: modeling, in-depth team design meetings, definition of tests and task development, and prioritization.
- Code authoring - Production code is developed based on the requirements, the analysis of those requirements, and the new designs. Code authoring includes authoring tests.
- Further testing and deployment - ICE is tested continuously and new binary builds are released.
Change Tracking and Version Control
ICE is managed in a Git repository. See Getting ICE for more information.
The GitHub project page also displays recent updates on the front page, as well detailed statistics about commits and traffic using the tabs located on the right-hand side. Information about the success or failure of the continuous build cycle is emailed to the
ice-dev <at> eclipse.org list and archived for later review.
Task Planning and Project Management
Jay Jay Billings is the principal investigator (PI), lead architect and developer, and general handyman of the ICE team. One of his roles is as the ICE "product owner"; he is responsible for communicating the needs of sponsors and customers to the ICE team. (Note that he is not a "product owner" in the SCRUM sense because he actively participates in development.)
Since Jay returned to graduate school to complete a PhD, Alex McCaskey has stepped in to manage day-to-day operations of the ICE project, while Jay still remains as PI on the ICE project.
Deliverables and milestones for the NEAMS program are managed in the PICS:NE system and approved by the NEAMS program managers. Milestones for non-NEAMS work are managed by Jay and/or Alex, and the respective contacts on those projects.
The high-level milestones and deliverables for the different projects are used to scope the work and development activities of the ICE team. Tasks are developed and assigned at Jay's discretion, but based on the input of the ICE Development Team. Most of the tasks are created and assigned after a RRP meeting, but some tasks are created over the course of the iteration.
Bugs and regressions are reported to the appropriate section on the ICE Bugzilla tracker. Tickets are assigned to the appropriate ICE team member or members depending on the task. Feature requests are reported to the same system and handled in the same way.
The Unified Modeling Language (UML) is a fantastic way to share ideas about software and document those ideas for the team.
Using UML provides a common, graphical language in which ideas and techniques can be discussed, refined and settled upon before authoring code. This helps manage the complexity of the system by maintaining scope and traceability of the requirements. It also minimizes bloating of the source code because only the things that are modeled are implemented. In a way, this combats a problem that is the exact opposite of analysis paralysis: it keeps developers focused and on task when they are developing code and prevents authoring unnecessary functionality. Models in UML can be very detailed and may even be extended and customized to represent individual domains. A UML model also helps with test-driven development.
ICE has a very large UML model that is used by the team for exactly the reasons described above. The UML model is translated directly to source code using model-to-source transformations. The team has traditionally required that every part of ICE be modeled in UML, however, this has become difficult practice to sustain as the project grows rapidly.
It's important to note that it's not necessary to model every piece of every bundle. Sometimes it does not make sense to model a small class that is only used in one place, an attribute that has a very specific type, or one that is extremely private. For that matter, it also doesn't make sense to model many private operations and attributes because they expose implementation details in the model. (Protected and public operations and attributes should always be modeled because they are used by others.) The choice of what private things are modeled and how far the model goes is left up to the developer with the approval of the Lead Architect.
In some cases, entire packages may not be explicitly modeled in the UML. The battery package in data structures is a good example, because it is over 25,000 lines of code that are automatically generated from an XML schema called "BatML" from the energy.ornl.gov folks. The item.nuclear package in the Item bundle is another good example, because the SHARP classes there do not require any special pieces, are located there mostly to be consistent with other nuclear energy packages, and are identical in design to most external ICE plugins. Again, decisions on whether or not packages like this should be modeled are left up to the Lead Architect.
Plugins that simply subclass pieces of these bundles or implement service interfaces do not require UML modeling. The Eclipse UI bundles in ICE have been extensively modeled, but UML modeling is no longer required for those bundles because of the difficulty of reverse engineering Eclipse into a UML model.
Code Development Guidelines
Developing code is only part of the whole development process in ICE and it accounts for less than half of the development activities. There are really only a few rules to remember for writing code in ICE:
- You will not break the build.
- You WILL NOT break the build.
- If someone says there is not enough documentation, you have to write more.
- Every class must be tested.
- One commit to the repository at a time.
- No cursing in source code, no slang in source code.
- Code development will go on as long as it has to.
- If it's your first time contributing code to ICE, you have to write the code.
Each of these rules have a purpose:
- A working build is critical to successfully delivering a working product often and continuously. This is the one cardinal sin on the ICE project. Each developer must commit a working build and each developer must test their pieces of ICE, including launching the binary, before new code or updated code is committed.
- Documentation is critical to the success of any source project and we all have to do our part to make sure that ICE is fully documented.
- All classes in ICE are tested in some way, even though some classes are tested more thoroughly than others.
- All of the code for a new capability or extension should be committed to the repository at the same time to prevent bugs and to keep our build from breaking.
- The source code for ICE is developed for an audience of professionals and it should reflect that. Curse words and slang in the source code is embarrassing in that context. Slang and funny themes are allowed in tests, though, because they are not distributed with ICE and they make writing the huge tests more enjoyable.
- We avoid deadlines. Writing good code requires time and testing it requires more time.
- Every new member of the ICE development team goes through a training exercise to teach them the skills necessary to contribute code to ICE. The first piece of code that they contribute to ICE must be written solely by them to prove that they are ready to contribute to a very active project.
That all being said, these are not exact hard and fast rules. They're more like guidelines actually, and examples likely exist where all of them have been violated. Following them as much as possible, though, keeps us all working smoothly.
What Happens if the Build Breaks?
A broken build is considered the worst possible thing that can happen to our team aside from losing funding to continue doing our work, because it is one of the only few other things that can stop us from developing our product. We are a little atypical among DOE teams because of our focus on continuous integration and testing, so we get asked this a lot and the conversation goes something like this:
- What happens if the ICE build breaks?
- The project is headed for a disaster of biblical proportions.
- What do you mean, "biblical"?
- What he means is Old Testament, my friend, real wrath-of-God type stuff.
- Fire and brimstone coming down from the skies! Rivers and seas boiling!
- Forty years of darkness! Earthquakes, volcanoes...
- The dead rising from the grave!
- Human sacrifice, dogs and cats living together... mass hysteria!
- All right, all right! I get the point!
Most of the time the build breaks, one or more developers will show up in the office of the offending committer or send emails to complain about the situation. Normally one or more of these developers will also join the developer who broke the build in a team programming session to fix the problem if it is not a simple fix. If the error cannot be easily fixed, the changes are reverted in the master until the build passes again and, if needed, put into a branch. The decision to make a branch is based on the size of the commit that broke the build and how much work it will take to fix it.
Reverting is a rare occurrence on the project because most of the team members can fix even very large problems in thirty minutes or less. It has only happened a few times in the history of the project. If the master undergoes a revert, it is the responsibility of the person who broke the build to fix the problem and verify it with other developers before committing to the trunk again.
Code Text Style
K&R is the default style in Eclipse and the preferred style of development on ICE. Your code can be auto-formatted into K&R by hitting Ctrl+Shift+F in the Eclipse IDE.
There are a few naming conventions in ICE. One deals with naming subclasses or interfaces. They are normally named in a way that reflects the class they are subclassing or the interface being implemented. For example, the class DataComponent reflects that it is a realization of the Component interface and the class EntryComposite reflects that it is a subclass of Entry.
This is just a convention and it is okay to name a class something else, as long as the name is descriptive.
The other set of guidelines is a much more fluid, structural approach to general naming conventions for the attributes and methods of a class. A class (or a class that implements an interface) contains a set of actions to invoke (methods) and properties. Usually, these actions modify the properties set on these classes. In essence, many of the attributes and methods in ICE are set accordingly to this metaphoric grammatical relationship: an attribute's name should be a noun, and a method a verb. In addition, these entities should also start lower case and only utilize upper case letters to symbolize spaces for phrases (i.e. camel case). Developers should avoid using underscores or any other special characters unless absolutely necessary.
Restricted Language Features
Some features of Java and C++ have restricted use in ICE because they encourage bad programming or are obscure.
- The use of Java's
instanceofoperator is strictly forbidden in ICE, unless it is used in a routine related to I/O, or a class related to the Eclipse UI. The
instanceofoperator is relatively efficient and makes coding easy, but that comes a cost of violating the abstraction established by ICE's class hierarchy, and the proliferation of switch statements across the code. That said, the
instanceofoperator is required when working with the HDF5 libraries and is the de facto standard for type discovery in Eclipse widgets. It is also used in ICE in the database routines to quickly determine whether or not a component should be skipped because it is, for example, some heavyweight thing like an IReactorComponent. If a developer feels that
instanceofis required in a class, they have to bring it to Jay for approval (which was the case in all three of the above cases).