We'll see | Matt Zimmerman

Leading up to the Ubuntu 8.10 release, Ubuntu developers and quality assurance engineers have been very busy sorting bugs, deciding what can and should be fixed for the final release, and what cannot.  They make these decisions by estimating the importance of each bug, identifying whether it is a regression, assessing the risk of potential fixes, and by applying their best judgement.  Developers can then focus their efforts where they are most needed.

On the whole, I think that we do remarkably well at this.  In September, for example, the total number of open bugs in Ubuntu increased by only 70.  This doesn’t sound like much of an achievement, if not for the fact that in the same time period, 7872 new bug reports were filed.  The remaining 7802, or over 99%, were resolved (some duplicates, some invalid, some fixed, etc.).

The news isn’t all good, of course.  There are currently over 46000 open Ubuntu bug reports in Launchpad.  Even at this impressive rate of throughput, and even if we were to freeze all development and stop accepting new bug reports entirely, I estimate it would take over half a year just to sift through the backlog of reports we have already received.  There is a lot of noise in that data.

When 8.10 is released, as with each previous release, some users will be disappointed that it has a bug which affects them.  This is regrettable, and I feel badly for affected users each time that I read about this, but it is unlikely to ever change.  There will never be a release of Ubuntu which is entirely free of bugs, and every non-trivial bug is important to someone.

So, what do we do?  What should be our key goals where quality is concerned?  We don’t currently have a clear statement of this, but here’s a strawman:

Prioritize bug reports effectively.  It’s usually difficult to say whether a bug report is valid or serious until a human has reviewed it, so this means having enough people to review and acknowledge incoming bug reports, and helping them to work as efficiently as possible.  The Ubuntu bug squad is a focal point for this type of work, though a great deal of this is done by developers in their everyday work as well.  Projects like 5-a-day are a good way to get started.

Measure our performance objectively.  By tracking metrics for each part of the quality assurance process, we can understand where we need to improve.  The QA team has been developing tools, such as the package status pages, to collect hard data on bugs.

Improve incrementally.  By minimizing regressions from one release to the next, and making some progress on old bugs, we can hope to make each Ubuntu release better, in terms of overall quality, than the one before it (or at least no worse).  The regression tracker (which will hopefully move to the QA site soon) will help to coordinate this effort.

Ensure that the most serious bugs are identified and fixed early.  It’s important to the success of the project that we continue to produce regular releases, and showstopper bugs risk delaying our release dates and adversely affecting the next release cycle.  The release management weather report is one tool which helps monitor this process, though a great deal of coordination is required in order to provide it with useful data.

Communicate about known bugs.  It is inevitable that there will be known bugs remaining in each release, and we should do our best to advise our users about them, including any known workarounds.  The Ubuntu 8.10 beta release notes are a good example of this.

I think that to do well in all of these areas would be a good goal for quality in Ubuntu.

Credit: The Kitten's Toe

On Friday, 24th October, I attended an Open Rights Group event where Neil Gaiman spoke about the impact of digital “piracy” on authors such as himself.  His thesis was that the free availability of content from books, even when it is contemporaneously for sale in print, is in fact beneficial for the creator.  This is, he said, because their greatest challenge, in a world crowded with writers, is to be “tasted” by potential readers and publishers.

He recounted a few of his own experiences with publishing free content on his website, noting that in each instance, the net effect was an increase in his book sales.  He explained that the free availability of books enables new readers to try them out, and (somewhat counter-intuitively) does not discourage potential buyers from buying them.  Some of the explanations he offered were:

Reading books is more pleasant than reading from a computer screen. This seems intuitively true for most people, as the practical advantages of books (low cost, durability, light weight, ease of use, and so on) outweigh those of digital content.  He quoted Douglas Adams as having said that “books are sharks”, highly evolved to suit their purpose, and that “nothing is better at being a book than a book”.  He acknowledged that this could change in the future, but pointed out that there were other ways that he could uniquely profit from his works, such as live readings.

Book buyers derive pleasure from the tangible experiences of owning books and passing them on to friends.  Digital storage and copying, while extremely efficient, do not evoke the same feelings as these physical acts.  Contrast browsing a bookshelf with scanning filenames, or receiving an email attachment with opening a neatly wrapped book.  They offer very different social and sensory experiences.

Readers buy books explicitly to support (“give back to”) the author.  Gaiman compared this to his experience of tasting many flavors of ice cream at Baskin Robbins and then feeling compelled by guilt to purchase one.  I found this one the least compelling, but he does seem to have many loyal readers.

I think there is some truth in each of these, and would be interested to see research on this subject.  What does motivate buyers, and does this change when they are aware of corresponding content which is free?

There were several good questions and comments, though I didn’t have the opportunity to present one.  This is because I didn’t put my hand up.  I have always been hesitant to ask questions from within an audience.  This, for some reason, makes me even more self-conscious than addressing an audience, perhaps because many of the people are behind me.

I wanted to pose a question about the impact of technological advancement.  The model that Gaiman described for digital books relies on the existence of related works and experiences which cannot be digitally reproduced: physical books, live performances, autographs, and so on.  The technological trend, however, is that digital experiences are becoming richer and richer, and science fiction writers have long extrapolated that they will become indistinguishable from first-hand sensory experiences.

The experience of browsing a bookshop, wandering around an art museum, or attending a musical performance, will eventually be able to be reproduced with such fidelity that it can be exchanged as easily as a photograph.

What will be left to sell when a digital reproduction is virtually as good as the real thing?  How will creativity be rewarded in a world where most of today’s creative works are merely information, and information is truly free?

This situation exists today for the creators of digital works, and the answers are unclear.  The growth and diversity of these works will depend on whether we find ways to sustain their creators.  Tools like the GNU GPL and Creative Commons BY-NC-SA help us to define boundaries around our work, but the system as a whole is not yet well defined or understood, particularly where free software is concerned.

There are many experiments underway in free software economics.  Canonical sells services associated with the software we distribute.  Some developers publish their work in hope of earning a reputation, followed by a job, in much the same way that Gaiman describes.

How about you?  How will your free software works sustain you, so that you can continue to create?

Free software is a remarkable phenomenon.  It is a highly evolved form of collaboration: compared to other creative endeavors, free software developers all over the world are able to work together on a project with surprisingly little friction.  It is a grassroots political movement which has grown from small online communities to span geographical and national boundaries.  It is a multicultural social group with unique and diverse characteristics.  It has spawned a variety of self-governing organizations and successful corporations.

It is also an interesting example of a gift economy.  In general, participants in free software donate their work to the greater good, with no expectation of an exchange of value.  Some contributions are rewarded by social or professional recognition, where the author achieves standing among their peers or receives gratitude from recipients of their work.  Others indirectly evoke rewards in kind, such as where the creator of a program is rewarded by contributions from others which improve it further.  Some are works for hire, where a corporation commissions a contribution through its employees, in pursuit of its own aims.  There are other types of exchanges where I do not personally understand what motivates the contributor.

There are many recognized roles in free software, but they can be broadly classified into three types:

Developers are the heart of this economy.  They are continuously creating and improving free software technology, and publishing their source code for other developers to use and learn from.  Some developers write one program and then vanish from the community, while others contribute to many different projects over the course of decades.  Highly effective developers are celebrities in the free software community.

Users, also known as “people”, are the reason why software is written.  Their needs and wants determine which software is considered valuable.  Many of them also contribute directly to free software in one way or another, by promoting awareness, providing testing and feedback, supporting other users, writing documentation, or building communication links.  Historically, most users of free software were also its developers, but today, this is no longer true, and millions of people use free software who are not developers.

Packagers connect these two groups.  They gather up the source code produced by developers, wrap it in standardized packaging, and bundle it into collections (distributions) designed to meet the needs of users.  Users experience free software almost exclusively through a distribution.  As the free software stack has grown in size and complexity, so have distributions, and the maintenance of a modern distribution is a large-scale development project in itself: selecting appropriate software and versions, getting the lot working smoothly together, and releasing it in the form of a product which is accessible to users.  Packagers create only a small fraction of the software included in their distribution, but they define several key aspects of “what it is like” to use it.  Users most strongly associate their experience of free software with a distribution.

If free software were film, developers would be some combination of writer and crew, creating and expressing characters and a story.  Users would be viewers (including critics and fans), who receive and interpret the work.  Packagers would be the film crew, realizing the production on film so that it can be seen.

If free software were food, developers would be chefs, developing recipes and cooking.  Users would eat and critique the dishes.  Packagers would be restaurateurs, serving customers and creating an environment in which they can experience the food.

Neither of these analogies are very complete.  In particular, these analogies fail to capture the strange loops of free software.  Every developer is also a user, reliant upon of thousands of other programs which they receive in packaged form order to do their development work.  Every team of packagers develops some software in order to make their distribution work, and they use the distribution itself in order to do so.

Distributions, and the integration work that they do, are a critical part of this ecosystem.  Many of us would not be using free software today if not for the efforts of projects like Debian, whose mission is to produce a complete system out of free software created by others.  In my case, Debian provided both the means and the motivation for my contributions to free software, and later made Ubuntu possible.

Strong, productive relationships between these groups are essential to continuing the growth and development of free software.  Whichever groups you’re part of, get to know your counterparts in other groups.  Talk to the people who are packaging your software, writing the software you package, using your software or packages.  Learn about the problems they face and how you can help each other.  Don’t assume that this communication is someone else’s job: reach out and make it happen.