We'll see | Matt Zimmerman

a potpourri of mirth and madness

Posts Tagged ‘Ubuntu

DEX finishes first batch of derivative patches for Debian

It’s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions.

Our first milestone

The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn’t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old.

I’m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them.

One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project.

This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian.

Conclusions

Although we weren’t quite able to clear the whole list, we still consider the project to be a success because:

  • We ensured that all of the patches received due consideration for inclusion in Debian
  • We proved the concept of DEX, with developers from Debian and derivatives cooperating on a common goal and sharing tools
  • Most importantly, we learned from the experience

What’s next

In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX:

  • Our second major project, nicknamed “big-merges”, will begin soon. Our goal is to identify the few packages which are most diverged between Debian and Ubuntu, and work to get them as close to identical as possible. If you have suggestions for packages to focus on, let us know!
  • Allison Randal is beginning a DEX project to implement the Python 2.7 transition across Debian and Ubuntu
  • Nathan Handler is working on a Summer of Code project to develop specialized tools to help with this kind of cross-distribution teamwork
  • Zack is organizing a derivatives BoF at DebConf 11

We’re looking forward to seeing DEX develop further. If you’d like to get involved, come and join us on the debian-derivatives mailing list or IRC (#debian-derivatives on freenodeOFTC).

Matt Zimmerman and Stefano Zacchiroli

Written by Matt Zimmerman

June 8, 2011 at 14:21

Moving on from Canonical (but not from Ubuntu)

This June, soon after my seventh anniversary with Canonical, I will finish my job as Ubuntu CTO at Canonical. You can read my official announcement on the Canonical blog.

It’s sometimes hard to believe that it’s been that long since we first envisioned a Linux desktop “for human beings”. When I look at how far we’ve come, at the difference we’ve made to so many people, it’s easy to see how the time passed so quickly.

It’s been an incredible experience for me to play a part in building Ubuntu and Canonical. Whether building a decentralized company of hundreds of people, to a global community project of tens of thousands of members, I’m grateful for all of the learning opportunities along the way. It has been a privilege to work with so many brilliant, passionate and thoughtful people under such auspicious circumstances. There is much I will miss, and I have many memories to enjoy, from all-night global hacking sessions driving toward a ship date, to casual singing and playing music at our many face-to-face events

Nonetheless, it is time for me to seek out new challenges and stretch myself in new ways. I’ll be moving back to the US, closer to old friends and family, and starting a new job with a different type of company. I am leaving behind a capable and dedicated team at Canonical, who I am confident will achieve even greater things in the years to come.

I will remain active in the free software community as a volunteer. I intend to continue to participate in Ubuntu, and to serve on the Technical Board. I will also be continuing my work with Debian and the DEX project. This year, I’ve accepted advisory positions with the Ada Initiative and the Freedom Box Foundation, and will continue to support those organizations and their missions.

I’ll be in Budapest at the Ubuntu Developer Summit for the next week, and look forward to seeing my friends in Canonical and the Ubuntu community.

I’ll have more to say soon about what I’ll be doing next professionally. Watch this space for updates!

Written by Matt Zimmerman

May 6, 2011 at 07:51

Posted in Uncategorized

Tagged with , ,

DEX: Debian and its derivatives, getting things done together

Since I resumed active status in Debian, I’ve been thinking about how to bridge the gap between Debian and its derivatives*. I’ve spoken at length with Zack, the attendees of the Derivatives BoF at DebConf 10, and the fine folks at the Derivatives Front Desk about the technical and social issues affecting derivative projects, and could probably write a very thorough series of blog posts on the subject.

Instead, Zack and I decided to try doing something about it: we have begun a project to test out a new approach to the problem.

Introducing DEX

DEX is all about action: merging patches, fixing bugs, crunching data, whatever is necessary to get changes from derivatives into Debian proper. DEX doesn’t try to change the way any existing project works, but adds a “fast path” for getting code from one place to another.

DEX is a joint task force where developers from Debian and its derivatives work together on this common goal. As a pilot project, we’ve established an Ubuntu DEX Team focused on merging code from Ubuntu into Debian. With members from both projects, we hope to be able to resolve blockage anywhere in the pipeline. Whatever needs to get done in order to merge an Ubuntu patch, someone in the Ubuntu DEX team will know what to do. If we get good results with Ubuntu, we hope that other derivatives will follow. With thanks to David Paleino, we’re excited that the Utnubu project is merging into DEX as it aligns well with their goals. I’m very grateful to have Colin Watson and James Westby signed up to contribute as well.

Our first project is simple: turn this list green. This is an archive of quite old patches from Ubuntu, most of which have probably been merged already or made obsolete, but they pre-date any kind of tracking system so they need to be verified. Once that’s done, we’ll move on to a new project with a new todo list.

If you want to see Debian benefit from technical work done in derivatives, DEX is a chance for you to act together to make it happen. If you work on a derivative and want to carry a smaller delta, come and join us. I’m sure we’ll learn a lot from this experience.

* There are many instances of great cooperation between Debian and derivative distributions, including joint package maintenance teams, and some derivatives are even part of the Debian project. Nonetheless, there are areas were most people I’ve spoken to agree that we need to do better. This is what I’ve referred to as the “gap”.

Written by Matt Zimmerman

March 16, 2011 at 21:30

A diversity statement for Ubuntu

The Ubuntu website states that “we aim to make Ubuntu a wonderful place to participate”. We developed the Ubuntu Code of Conduct to set a standard for participants to accept each other in the spirit of cooperation, and have improved it over time to state these principles more clearly.

It is implicit in our philosophy that these and other Ubuntu values should hold equally true for everyone. I would like to propose that we upgrade this to an explicit statement on behalf of the project.

I have spoken with many people who were interested in joining a free software project, but were put off because they felt unwelcome. I know various people who participate in Ubuntu today, but sometimes face difficult social obstacles in order to do so. Going forward, I would like for us, as members of the Ubuntu community, to make the extra effort to accept all kinds of people. This may sound simple, but it can be very difficult to put into practice. People often don’t even notice they’ve gotten it wrong, until the offended party points it out to them. We need tools and guidance to make this a reality.

To that end, I would like to propose a diversity statement for Ubuntu. This draft has already received support from a majority of the Community Council, but I’d like to take it a step further. Because I want this to be a commitment that we can all stand behind, I’m also calling for support from the community as a whole. Please give this issue your consideration, and let me know in the comments if you can get on board with an official statement like this. The more support we have, the more real this commitment can be.

Here’s the text. Many thanks to Mary Gardiner, Valerie Aurora and Benjamin Mako Hill for their review and input.

The Ubuntu project welcomes and encourages participation by everyone. We are committed to being a community that everyone feels good about joining. Although we may not be able to satisfy everyone, we will always work to treat everyone well.

Standards for behavior in the Ubuntu community are detailed in the Code of Conduct and Leadership Code of Conduct. We expect participants in our community to meet these standards in all their interactions and to help others to do so as well.

Whenever any participant has made a mistake, we expect them to take responsibility for it. If someone has been harmed or offended, it is our responsibility to listen carefully and respectfully, and do our best to right the wrong.

Although this list cannot be exhaustive, we explicitly honor diversity in age, culture, ethnicity, genotype, gender identity or expression, language, national origin, neurotype, phenotype, political beliefs, profession, race, religion, sexual orientation, socio-economic status, subculture, and technical ability.

Some of the ideas and wording for this statement were based on diversity statements from the Python community and Dreamwidth Studios (CC-BY-SA 3.0).

Written by Matt Zimmerman

February 7, 2011 at 15:55

Ubuntu Brainstorm Top 10 for December 2010

As I mentioned recently, the Ubuntu Technical Board is reviewing the most popular topics in Ubuntu Brainstorm and coordinating official responses on behalf of the project. This means that the most popular topics on Ubuntu Brainstorm receive expert answers from the people working in these areas.

This is the first batch, and we plan to repeat this process each quarter. We’ll use feedback and experiences from this run to improve it for next time, so let us know what you think.

Power management (idea #24782)

Laptops are now outselling desktops globally, and laptop owners want to get the most out of their expensive and heavy batteries. So it’s no surprise that people are wondering about improved power management in Ubuntu. This is a complex topic which spans the Linux software stack, and certainly isn’t an issue which will be “solved” in the foreseeable future, but we see a lot of good work being done in this area.

To tell us about it, Amit Kucheria, Ubuntu kernel developer and leader of the Linaro working group on Power Management, contributed a great writeup on this topic, with technical analysis, tips and recommendations, and a look at what’s coming next.

I am going to attempt to summarize the various use profiles and what Ubuntu does (or can do) to prolong battery life in those profiles. Power management, when done right, should not require the user to make several (difficult) choices. It should just work – providing a good balance of performance and battery life.

IP address conflicts (idea #25648)

IP addressing is a subject that most people should never have to think about. When something isn’t working, and two computers end up with the same IP address, it can be hard to tell what’s wrong. I was personally surprised to find this one near the top of the list on Ubuntu Brainstorm, since it seems unlikely to be a very common problem. Nonetheless, it was voted up, and we’re listening.

There is a tool called ipwatchd which is already available in the package repository, and was created specifically to address this problem. This seems like a further indication that this problem may be more widespread than I might assume.

The idea has already been marked as “implemented” in Brainstorm based on the existence of this package, but that doesn’t help people who have never heard of ipwatchd, much less found and installed it.

What do you think? Have you ever run into this problem? Would it have helped you if your computer had told you what was wrong, or would it have only confused you further? Is it worth considering this for inclusion in the default install? Post your comments in Brainstorm.

Selecting the only available username to login (idea #6974)

Although Linux is designed as a multi-user operating system, most Ubuntu systems are only used by one person. In that light, it seems a bit redundant to ask the user to identify themselves every time they login, by clicking on their username. Why not just preselect it? Indeed, this would be relatively simple to implement, but the real question is whether it is the right choice for users.

Martin Pitt of the Ubuntu Desktop Team notes that consistency is an important factor in ease of use, and asks for further feedback.

So in summary, we favored consistency and predictablility over the extra effort to press Enter once. This hasn’t been a very strong opinion or decision, though, and the desktop team would be happy to revise it.

Icon for .deb packages (idea #25197)

Building on the invaluable efforts of Debian developers, we work hard to make sure that people can get all of the software they need from Ubuntu repositories through Software Center and APT, where they are authenticated and secure. However, in practice, it is occasionally necessary for users to work with .deb files directly.

Brainstorm idea 25197 suggests that the icon used to represent .deb packages in the file manager is not ideal, and can be confusing.

Matthew Paul Thomas of the Canonical Design Team responds with encouragement for deb-thumbnailer, which makes the icon both more distinctive and more informative. He has opened bug 685851 to track progress on getting it packaged and into the main repository.

I have reviewed the proposed solutions with Michael Vogt, our packaging expert. Solution #1 is straightforward, but we particularly like solutions #5 and #10, using a thumbnailer to show the application icon from inside each package.

Keeping the time accurate over the Internet by default (idea #25301)

It’s important for an Internet connected computer to know the correct time of day, which is why Ubuntu has included automatic Internet time synchronization with NTP since the very first release (4.10 “warty”). So some of us were a little surprised to see this as one of the most popular ideas on Ubuntu Brainstorm.

Colin Watson of the Ubuntu Technical Board investigated and discovered a case where this wasn’t working correctly. It’s now fixed for Ubuntu 11.04, and Colin has sent the patches upstream to Debian and GNOME.

My first reaction was “hey, that’s odd – I thought we already did that?”. We install the ntpdate package by default (although it’s deprecated upstream in favour of other tools, but that shouldn’t be important here). ntpdate is run from /etc/network/if-up.d/ntpdate, in other words every time you connect to a network, which should be acceptably frequent for most people, so it really ought to Just Work by default. But this is one of the top ten problems where users have gone to the trouble of proposing solutions on Brainstorm, so it couldn’t be that simple. What was going on?

More detail in GNOME system monitor (idea #25887)

Under System, Preferences, System Monitor, you can find a tool to peek “under the hood” at the Linux processes which power every Ubuntu system. Power users, hungry for more detail on their systems’ inner workings, voted to suggest that more detail be made available through this interface.

Robert Ancell of the Ubuntu Desktop Team answered their call by offering to mentor a volunteer to develop a patch, and someone has already stepped up with a first draft.

Help the user understand when closing a window does not close the app (idea #25801)

When the user clicks the close button, most applications obediently exit. A few, though, will just hide, and continue running, because they assume that’s what the user actually wants, and it can be hard to tell which has happened.

Ivanka Majic, Creative Strategy Lead at Canonical, shares her perspective on this issue, with a pointer to work in progress to resolve it.

This is more than a good idea, it’s an important gap in the usability of most of the desktop operating systems in widespread use today.

Ubuntu Software Centre Removal of Configuration Files (idea #24963)

One feature of the Debian packaging system used in Ubuntu is that it draws a distinction between “removing” a package and “purging” it. Purging should remove all traces of the package, such that installing and then immediately purging a package should return the system to the same state. Removing will leave certain files behind, including system configuration files and sometimes runtime data.

This subtle distinction is useful to system administrators, but only serves to confuse most end users, so it’s not exposed by Software Center: it just defaults to “removing” packages. This proposal in Ubuntu Brainstorm suggests that Software Center should purge packages by default instead.

Michael Vogt of the Ubuntu Foundations Team explains the reasoning behind this default, and offers an alternative suggestion based on his experience with the package management system.

This is not a easy problem and we need to carefully balance the needs to keep the UI simple with the needs to keep the system from accumulating cruft.

Ubuntu One file sync progress (idea #25417)

Ubuntu One file synchronization works behind the scenes, uploading and downloading as needed to replicate your data to multiple computers. It does most of its work silently, and it can be hard to tell what it is doing or when it will be finished.

John Lenton, engineering manager for the Ubuntu One Desktop+ team, posts on the AskUbuntu Q&A site with tools and tips which work today, and their plans to address this issue comprehensively in the future.

Multimedia performance (idea #24878)

With a cornucopia of multimedia content available online today, it’s important that users be able to access it quickly and easily. Poor performance in the audio, video and graphics subsystems can spoil the experience, if resource-hungry multimedia applications can’t keep up with the flow of data.

Allison Randal, Ubuntu Technical Architect, answers with an analysis of the problem and the proposed solutions, an overview of current activity in this area, and pointers for getting involved.

The fundamental concern is a classic one for large systems: changes in one part of the system affect the performance of another part of the system. It’s modestly difficult to measure the performance effects of local changes, but exponentially more difficult to measure the “network effects” of changes across the system.

Written by Matt Zimmerman

December 10, 2010 at 14:04

Three ways for Ubuntu to help developers

Developers are a crucial part of any successful software platform. In the same way that an operating system is “just” a means for people to use applications, a platform is “just” a means for developers to create applications and make them available to people.

There are three primary ways in which Ubuntu can help developers do their work. They are all related, but distinct, and so we should consider them individually:

1. Developing for Ubuntu

Today, Ubuntu bundles thousands of free software applications, for both clients and servers, most of which are packaged by Debian.

Ubuntu also carries certifications for a variety of third-party ISV software, both open source and proprietary, which are coordinated through Canonical’s partner program.

In both of these cases, many of these applications are actually developed on other platforms, and ported to Ubuntu, either by the free software community or by the creators of the software.

2. Developing on Ubuntu

Ubuntu is already quite popular among developers, who mainly run Desktop Edition on their workstations. They might be developing:

  • web applications (with server-side and browser components)
  • portable applications (e.g. using Java, or Adobe AIR)
  • mobile applications (e.g. for Android or iOS)
  • native applications, which might target Ubuntu Desktop Edition itself, or supporting multiple platforms through a framework like Qt

3. Distributing through Ubuntu

Like other modern operating systems, Ubuntu isn’t just a platform where applications run, but also a system for finding and installing applications. Starting with APT, which originated in Debian, we’ve added Software Center, the ISV partner repository, and various other capabilities in this area. They all help to connect developers with users, facilitating distribution of software to users, and feedback to developers.

So, where should we focus?

Some developers might be interested in all three of these, while others might only care about one or two.

However, most of the developer improvements we could make in Ubuntu would only address one of these areas.

For this reason, I think it’s important that we consider the question of the relative importance of these three developer scenarios. Given that we want Ubuntu to flourish as a platform, how would we prioritize them?

I have my own ideas, which I’ll write about in subsequent posts, but here’s your chance to tell me what you think. :-)

Written by Matt Zimmerman

November 26, 2010 at 11:32

Posted in Uncategorized

Tagged with , , ,

Ubuntu: Project, Platform, Products

When most people talk about Ubuntu, they usually mean our flagship product, Ubuntu Desktop Edition. Sometimes, they might mean the Ubuntu project, or the community of people who work on it, or various other things.

Similarly, Debian might mean the Debian operating system, or the package repositories, or the project, and so on.

This gets a little confusing sometimes. When I’m talking about Ubuntu, I’ve started to use more specific terminology to explain what I mean, and this seems to help people understand better the nature of the whole Ubuntu. In particular, I use the three Ps:

  • a portfolio of products, including Desktop Edition, Server Edition, Netbook Edition, Kubuntu and more. These are software bundles which can be downloaded, pre-installed on retail computers, and so on. Each one is designed to meet a certain set of user needs, and to work on a specific form factor of computer.
  • a technology platform, which can be used to build a wide range of products. It is primarily of interest to developers, who build derivative distributions, OS products, applications and infrastructure using Ubuntu packages. This platform is the common foundation of the Ubuntu products above, and includes things like the global package repository. Joel Spolsky does a good job of explaining why platforms are distinctly different from products, and should be treated as such.
  • an open community project, which collectively produces, distributes, promotes and supports the products and the platform. The Ubuntu project has a philosophy, a government, and various tools and processes to help contributors work together. Canonical supports the Ubuntu project by providing resources and infrastructure, and also directly participates in Ubuntu at various levels.

This breakdown may seem a bit obvious to those of us “on the inside”, but it’s confusing to people who are encountering it for the first time. I’m sharing this in the hope that if more people start using the same words, it will get easier for people to understand how these pieces fit together. I’ll also be linking to it a lot, to help put things into context using this framework.

Written by Matt Zimmerman

November 15, 2010 at 18:39