Today I and some hundred others on LinuxTag in Berlin attended a keynote by Mark Shuttleworth, the “head dreamer” of the ubuntu Linux distribution.
He had pretty few slides with hardly any words on it. The headlines were “Cadence”, “Quality” and “Design”. As I have been working towards openSUSE quality, this interested me most – so this is what this text is going to be about. His thoughts were mostly what I was thinking anyway.
Quality works like this: if Factory (equivalent of Debian/sid or whatever it is called in ubuntu) breaks horribly every other week, few people will feel inclined to keep running on Factory or even on factory-snapshot. Mark spoke about a difference in number of testers of “one or two orders of magnitude” (which would mean a factor of 10-100 in mathematical terms). And having fewer testers during development can only mean lower quality for the final release.
This is why efforts should be taken to ensure a working Factory. Automated testsuites and reviews were specifically mentioned, among others. Some packages like gcc include own test-suites, but most do not. When asked how that was handled in ubuntu, he described an approach with scripts using computer vision, recognizing/clicking buttons which most likely referred to sikuli which has many similarities to what I did, just with a different focus (more user-friendliness) and quite some sophistication. Also for text-mode there is still good old “expect”.
Another thing that was not mentioned there, but which I was always thinking of as an odd shortcoming of openSUSE is that Debian has a “testing” distribution which is not like factory-snapshot. Debian/testing contains only packages migrated from sid after a certain time with no bugs above a certain criticality threshold. Of course, this needs a way to track automatically which bug applies to which package (something that ubuntu does in launchpad).
Launchpad is also nicely integrated with bazaar, so that adding “(LP: #12345)” into a changelog will cause the bug to have a link to the proposed fix and later be automatically closed, once the branch was merged into the main trunk.
This is not only cool, but plain useful. It can save a lot of work and frustration. I think Adrian Schröter will be working into that direction with better integration of buildservice into bugzilla and other openSUSE tools.
There is also a big human component. Especially with testers which are non-technicians. It is not so much about motivating them, but about not demotivating them (which can happen surprisingly easy in some cases). I have seen crash messages along the lines of “xxx has crashed. This is in no way your fault. You could help us by …” which is a very good thing in this regard.
Yet another related note: there are openSUSE-Promo and openSUSE-Biarch DVD iso variants which are so well hidden, that few people even know about their existence, so that there can not be much public testing. However, those are the variants that are given away to hordes of interested people at openSUSE booths around the world, but those are also the variants that are least tested. I still have bad memories of the 11.1 or 11.0 one which was degraded to coaster after three unsuccessful tries on different machines. If bandwidth or mirror-space is a problem, offering them only via bittorrent could be a solution.
Overall, still some way to go, but IMHO we are moving into the right direction.
