Virgo/Community/F2F/Journal November 2010/Day2
Florian arrived! Apparently he has had a woeful time stranded in Frankfurt and had to take a 'bus from Heathrow to Southampton as flights to So'ton were cancelled.
Glyn asked us all to identify ourselves (barring memory loss from last night), and this helped us all to attach names again.
Glyn then read minutes for yesterday for our perusal. (I hope this journal doesn't get reviewed too severely :-D) The summary was masterly (even recognisable) — well done Glyn.
Since the minutes are such a good summary, I'm going to limit these journal entries to extra-topic observations (unless I get carried away). Hope it isn't too ephemeral.
09:30 start agenda - compare Virgo to other OSGi server solutions
This was really a nice session — we were able to pat ourselves on the back a bit. However, Dmitry did remind us that the others were 'catching up' and that we need to get new compelling function in, or at least make the app developers job significantly smoother to reamin near the front of the pack.
Glyn had some observations around the use of Aries deployment on Virgo. This highlighted for us how hard it is for someone to place other components on our platform. Although most of the problems turn out to be dependency issues not of our making, this is still sobering — others will struggle to get these things working.
Out of this session (it was not a dissing exercise, honest) some requirements arose — these were nearly all simple things (not all) so there is no shortage of work for contributors/committers in the near future. The roadmap discussion later today will be very interesting. One of these was enhancements to administration — the use of Admin Console to dynamically manage logging, for example — and there were requests to make persistence issues much easier to handle.
Although this session wandered a bit, it has to be said that the comparisons were not at all bad for us. Other topics raised here were EJB support (?), JavaEE web profiles, and JPA solutions which cut down management/update burden for that sort of application ("98% of all apps use datasources" - D).
10:23 development vs production modes
It is clear that the idea of having special restrictions placed on servers in production would be quite a useful feature for admins. Coupled with a notion of 'captured resolution' for artifacts, this would go some way to allowing a production system to be robust, and potentially faster in deployment. The security of production systems (Krasimir talks about security later) could also be different, though it would be nice to switch this on in development for testing. In general we agreed that the 'mode' of a system should be visible on every (or nearly every) system admin panel displayed. Chris can surely manage this :-)
Chris gave an outline of the Jetty work — which involves having a Jetty layer instead of the web layer (including Gemini.web and tomcat) on top of the kernel to produce a new server config. Chris has successfully gotten Jetty running as a prototype, and, apart from a few refactorings and tweaks, splash and admin console should be able to run on top of this platform. It has to be said that some features are lost here — and import bundle is one of them — but this is not supposed to be functionally equivalent to the tomcat solution.
Chris again, and he told us about the adventures he has had building, testing and basically keeping working the Snaps technology. (In case you've never heard of it, it is a fork of the Slices facility, which allows front end web apps to be built of independent components, which can be dynamically installed and uninstalled — by doing the same to the underlying bundles that implement them.)
Snaps is exciting stuff, and only requires a little more push to become a reliable repository which is published and supported as part of Virgo. We enjoyed hearing about this, and many ideas were put forward as to how this might be exploited by other Virgo facilities, for example the Admin Console, a configuration management facility and possibly a Spring Batch platform extension to the kernel.
Quip of the day, by Dmitry: "We're not pure RESTafarians."
Violeta spoke about the Web container, including Tomcat and Servlet implications. We expect to need to move to Tomcat 7 soon, partly because that might mean we don't have to run with a (SpringSource) modified version of Tomcat, to enable embedding. This also implies a move to Servlet 3.x. Certain restrictions can be removed thereby and extra features of Tomcat supported. I have to admit to feeling out of my depth here.
12:23 Dmitry's list
Dmitry spoke about JPA and JNDI (for Gemini Naming) and some experience with Tomcat changes. The main point here is that we need to include all those web developers out there (98% of them, according to D) who need to use datasources and JPA. These people should have a smooth and easy path in. We all agreed, of course. D was able to highlight the management problem from personal experience with several customers: it is not good to have to manage datasource definitions that are duplicated (up to three times) in one application system. D has a lot of good requirements (and use cases) that could be fixed with a (small) matter of programming. (The problem this week appears to be lots of ideas for things to do, but not enough resources to do them all!) I did catch, though, that D offered to do something (with snaps?) — to get the DNS definitions in one place, perhaps? It is not clear exactly what this is, but if anyone can prototype this, D can.
In this period mention was made of a Spring Batch extension on the kernel platform that enabled simple management of jobs, each of which can be dynamically deployed and then executed in an OSGi framework. Probably with a Web front end. It was questioned whether this was really necessary — by which I took the questioner to mean does this leverage the OSGi container in any real sense? D seemed to think so. Glyn asked if there was a customer using Spring Batch who could benefit. There are interesting possibilities; how to begin?
At this point the Agenda was bent to include things that were planned for this afternoon.
Krasi told us of all the things he would like to see to support what he termed 'security'. This turns out to mean not only permissions and access control, but also system self-protection against faulty (or even malicious) applications. We are talking 'isolation', API restrictions, OS access denial, including file system hiding (perhaps).
This led to a notion of a sandbox region (or do I mean subsystem?), where 'risky' apps could be run. Can we implement this using the region concept in the kernel (with perhaps some LTW technology on top for API protection)? Under the old nested frameworks api, yes, K reckons we can. Under the new framework resolution hooks, it is not so clear. This led to a discussion of the new region implementation — Glyn seemed to think this was an interesting test of the new function in Equinox 3.7. I had the temerity to point out that it might even prove the Equinox spec proposals (implemented in 3.7) infeasible. Of course, this was time for lunch.
Here we started to outline what was on the near (and not so near) horizon. Glyn took us through the plans (tentative though they are) that we (s2) see coming.
There is Spring 3.0.5 upgrade (trivial?); user region re-implementation (on Equinox 3.7) (not so trivial now, eh Glyn?); Bundlor and tooling donation to Eclipse; extension to plan schema (allowing URI as well as artifact TxNxV); Equinox services updates (Config Admin — done, Log — easy, Event Admin — not so easy); p2 repository in chain (not actually in our plan); Spring DM to Blueprint; Jetty (2/3 sprints left).
Of course Krasi had as long a list as this (and went through it). Tomcat 7 (snap); p2 repos (nearly snap, but more); deployer refactoring (extensibility) PoCs; declarative services; OSGi Tools alignment; (and on and on — see the minutes for the full story). I cannot do justice to the plans K presented here.
We talked and talked, and in the end Glyn was pleased that we had so much to do, and that we were really 'getting going' now. I think I agree.
Dmitry wasn't to be outdone and waded in with config sets; container speedup; tracing and event logging enhancements; configuration persistence and dynamic management; central datasource management; Config Admin in subsystems. Wow. Florian mentioned the httpService for RAP users and before we got too carried away, Glyn called for a bring forward of the committer part of the Agenda. (It might slow some of these whizz kids down.)
15:10 New Committers
What makes a good committer? What responsibilities does a committer have? These were the things we discussed here, along with the sort of design and code guidelines our development team expects of its people. Glyn asked for a show of hands of those who might want to be committers. I said that they should be asked again after Glyn outlined the heavy responsibilities of such a person :-)
Florian here showed us a few pictures (amusing, pithy and relevant) of advice given to developer/designers starting out. The two I noted down were: "Test first" and "One task at a time". They were all good priming for the next session.
Glyn outlined the committer röle. Bringing others on, and gatekeeper for the design and code base, were ones that I remember. The emphasis on quality is perhaps the highest I have seen in my 33 years in the industry (and 39 years programming). Let's keep it that way. There's lots to help us, and the base is quite clean, really.
The test coverage comments provoked lots of suggestions about generating coverage reports in the CI builds, nightly full builds, and findbugs runs — all of which are feasible now. That's the attitude guys (and gal, sorry Violeta).
Chris again — he's very productive here — talking about the shell (as was) and Admin Console. Here we had a brief outline of the design and the technology (including what we had to give up in the shell). This shows what can be done. The use of the JMX interface means that anything we can do in these apps can be done remotely using the model/manipulation mechanisms exposed thereby. This was a deliberate design point. The Admin Console is an app; it has no privileged access (neither did the shell although the commands all ran in the kernel).
Hristo then talked about the shell features to come — recovering the shell facilities we had before, to some extent. Based on Equinox 3.6.1 we should be able to get tab completion, history command line and ssh access (not telnet — too porous) including the vsh command and the class loading commands SAP have put in. (Glyn did a quick public test of the clhas command, and it works! Of course. We immediately suggested improvements — this is what happens to user-facing interfaces like this, everyone has an idea for 'improvement'. Chris had the same trouble with the admin console GUI.)
So Hristo went on to talk about the Equinox shell. Here we talked about the problems with Telnet; the trouble with tab completion (not on parms); and the platform independence issues. (There is nothing new under the sun.) We are blessed with class loading query commands, nested frameworks commands, declarative services commands and more to come, I'm sure. H went on to talk about the implementation in Equinox 3.7, but I cannot bring myself to type all this in here. Suffice to say that a fragment is required at present, so we need to fix that when we have the right Equinox support.
16:40 pause for breath
Since we were eating away at Thursday's topics we decided to stop here; and anyway I was exhausted (again!). If the level of enthusiasm is anything to go by we are going to get a lot done in the next quarter, and the Virgo code base is going to be in good hands.
And so to bed. (Actually, I reckon that most of the team — dare I begin to use that term? — went to the pub. I went home. See you all tomorrow.)