We'll see | Matt Zimmerman

a potpourri of mirth and madness

Rethinking the Ubuntu Developer Summit

This is a repost from the ubuntu-devel mailing list, where there is probably some discussion happening by now.

After each UDS, the organizers evaluate the event and consider how it could be further improved in the future. As a result of this process, the format of UDS has evolved considerably, as it has grown from a smallish informal gathering to a highly structured matrix of hundreds of 45-to-60-minute sessions with sophisticated audiovisual facilities.

If you participated in UDS 10.10 (locally or online), you have hopefully already completed the online survey, which is an important part of this evaluation process.

A survey can’t tell the whole story, though, so I would also like to start a more free-form discussion here among Ubuntu developers as well. I have some thoughts I’d like to share, and I’m interested in your perspectives as well.

Purpose

The core purpose of UDS has always been to help Ubuntu developers to explore, refine and share their plans for the subsequent release. It has expanded over the years to include all kinds of contributors, not only developers, but the principle remains the same.

We arrive at UDS with goals, desires and ideas, and leave with a plan of action which guides our work for the rest of the cycle.

The status quo

UDS looks like this:

This screenshot is only 1600×1200, so there are another 5 columns off the right edge of the screen for a total of 18 rooms. With 7 time slots per day over 5 days, there are over 500 blocks in the schedule grid. 9 tracks are scattered over the grid. We produce hundreds of blueprints representing projects we would like to work on.

It is an impressive achievement to pull this event together every six months, and the organizers work very hard at it. We accomplish a great deal at every UDS, and should feel good about that. We must also constantly evaluate how well it is working, and make adjustments to accommodate growth and change in the project.

How did we get here?

(this is all from memory, but it should be sufficiently accurate to have this discussion)

In the beginning, before it was even called UDS, we worked from a rough agenda, adding items as they came up, and ticking them off as we finished talking about them. Ad hoc methods worked pretty well at this scale.

As the event grew, and we held more and more discussions in parallel, it was hard to keep track of who was where, and we started to run into contention. Ubuntu and Launchpad were planning their upcoming work together at the same time. One group would be discussing topic A, and find that they needed the participation of person X, who was already involved in another discussion on topic B. The A group would either block, or go ahead without the benefit of person X, neither of which was seen to be very effective. By the end of the week, everyone was mentally and physically exhausted, and many were ill.

As a result, we decided to adopt a schedule grid, and ensure that nobody was expected to be in two places at once. Our productivity depended on getting precisely the right people face to face to tackle the technical challenges we faced. This meant deciding in advance who should be present in each session, and laying out the schedule to satisfy these constraints. New sessions were being added all the time, so the UDS organizers would stay up late at night during the event, creating the schedule grid for the next day. In the morning, over breakfast, everyone would tell them about errors, and request revisions to the schedule. Revisions to the schedule were painful, because we had to re-check all of the constraints by hand.

So, in the geek spirit, we developed a program which would read in the constraints and generate an error-free schedule. The UDS organizers ran this at the end of each day during the event, checked it over, and posted it. In the morning, over breakfast, everyone would tell them about constraints they hadn’t been aware of, and request revisions to the schedule. Revisions to the schedule were painful, because a single changed constraint would completely rearrange the schedule. People found themselves running all over the place to different rooms throughout the day, as they were scheduled into many different meetings back-to-back.

At around this point, UDS had become too big, and had too many constraints, to plan on the fly (unconference style). We resolved to plan more in advance, and agree on the scheduling constraints ahead of time. We divided the event into tracks, and placed each track in its own room. Most participants could stay in one place throughout the day, taking part in a series of related meetings except where they were specifically needed in an adjacent track. We created the schedule through a combination of manual and automatic methods, so that scheduling constraints could be checked quickly, but a human could decide how to resolve conflicts. There was time to review the schedule before the start of the event, to identify and fix problems. Revisions to the schedule during the event were fewer and less painful. We added keynote presentations, to provide opportunities to communicate important information to everyone, and ease back into meetings after lunch. Everyone was still exhausted and/or ill, and tiredness took its toll on the quality of discussion, particularly toward the end of the week.

Concerns were raised that people weren’t participating enough, and might stay on in the same room passively when they might be better able to contribute to a different session happening elsewhere. As a result, the schedule was randomly rearranged so that related sessions would not be held in the same room, and everyone would get up and move at the end of each hour.

This brings us roughly to where things stand today.

Problems with the status quo

  1. UDS is big and complex. Creating and maintaining the schedule is a lot of work in itself, and this large format requires a large venue, which in turn requires more planning and logistical work (not to mention cost). This is only worthwhile if we get proportionally more benefit out of the event itself.
  2. UDS produces many more blueprints than we need for a cycle. While some of these represent an explicit decision not to pursue a project, most of them are set aside simply because we can’t fit them in. We have the capacity to implement over 100 blueprints per cycle, but we have *thousands* of blueprints registered today. We finished less than half of the blueprints we registered for 10.04. This means that we’re spending a lot of time at UDS talking about things which can’t get done that cycle (and may never get done).
  3. UDS is (still) exhausting. While we should work hard, and a level of intensity helps to energize us, I think it’s a bit too much. Sessions later in the week are substantially more sluggish than early on, and don’t get the full benefit of the minds we’ve brought together. I believe that such an intense format does not suit the type of work being done at the event, which should be more creative and energetic.
  4. The format of UDS is optimized for short discussions (as many as we can fit into the grid). This is good for many technical decisions, but does not lend itself as well to generating new ideas, deeply exploring a topic, building broad consensus or tackling “big picture” issues facing the project. These deeper problems sometimes require more time. They also benefit tremendously from face-to-face interaction, so UDS is our best opportunity to work on them, and we should take advantage of it.
  5. UDS sessions aim for the minimum level of participation necessary, so that we can carry on many sessions in parallel: we ask, “who do we need in order to discuss this topic?” This is appropriate for many meetings. However, some would benefit greatly from broader participation, especially from multiple teams. We don’t always know in advance where a transformative idea will come from, and having more points of view represented would be valuable for many UDS topics.
  6. UDS only happens once per cycle, but design and planning need to continue throughout the cycle. We can’t design everything up front, and there is a lot of information we don’t have at the beginning. We should aim to use our time at UDS to greatest effect, but also make time to continue this work during the development cycle. “design a little, build a little, test a little, fly a little”

Proposals

  1. Concentrate on the projects we can complete in the upcoming cycle. If we aren’t going to have time to implement something until the next cycle, the blueprint can usually be deferred to the next cycle as well. By producing only moderately more blueprints than we need, we can reduce the complexity of the event, avoid waste, prepare better, and put most of our energy into the blueprints we intend to use in the near future.
  2. Group related sessions into clusters, and work on them together, with a similar group of people. By switching context less often, we can more easily stay engaged, get less fatigued, and make meaningful connections between related topics.
  3. Organize for cross-team participation, rather than dividing teams into tracks. A given session may relate to a Desktop Edition feature, but depends on contributions from more than just the Desktop Team. There is a lot of design going on at UDS outside of the “Design” track. By working together to achieve common goals, we can more easily anticipate problems, benefit from diverse points of view, and help each other more throughout the cycle.
  4. Build in opportunities to work on deeper problems, during longer blocks of time. As a platform, Ubuntu exists within a complex ecosystem, and we need to spend time together understanding where we are and where we are going. As a project, we have grown rapidly, and need to regularly evaluate how we are working and how we can improve. This means considering more than just blueprints, and sometimes taking more than an hour to cover a topic.
Advertisement

Written by Matt Zimmerman

May 27, 2010 at 16:15

Posted in Uncategorized

Tagged with , , , , , , , , , ,

«
»

17 Responses

Subscribe to comments with RSS.

  1. Disclaimer: UDS Brussels was my first UDS ever, so I’m basing my comments on one (wonderful) experience.

    A way of making the UDS easier to schedule could be to allow people not to already tell what sessions they want to attend right away, but instead what topics they would like to attend.
    Topics could be larger things like ‘Ubuntu One (desktop)’, ‘Unity’, ‘App Menu’; but also smaller things like ‘Bug Squad Mentorship’ (there was _one_ session about this) or ‘IRC Council’.

    Time slots could then be distributed based on the list of things that need to be discussed per topic, the priority of the topics and the (number of) people saying they’re interested in a topic.

    Although not everyone needs to attend every session on the topic, registering per topic would make it easier to group sessions around a topic. Each session could be given a category or tag of the kind we know call ‘tracks’. So you’d have a session as part of the ‘App Menu’ topic that is about getting upstreams involved. That would have the tag ‘community’ and maybe the tag ‘desktop’ as well. Combining the topics and the tags/tracks people want to follow they can select what sessions they want to attend.

    It is really needed to allow longer talking about certain subjects. Sometimes I’ve found that we made some hasty decisions and skipped some possibly interesting thought paths because of a lack of time. Of course, some talks would benefited from a better structured agenda, or any agenda at all, but some really need more time. However, that isn’t always easy to say beforehand. Some sessions get very interesting near the end, all of the sudden, but then you’re discussion is cut short by the people of the next session, anxiously waiting to get in.

    Another issue I noticed was that the level of knowledge on the subject discussed varies widely. A few times some of the people that play a very active role in the discussion don’t know a lot about the session. A lot of people don’t read all the information on the blueprint and on the wiki, or are familiar with the outcomes of previous discussions. Of course you can’t and shouldn’t want to force all attendees to read all material available on the subject discussed, but the people that want to make themselves heard in the discussions should read some more. It is a waste of time when the discussion is lagged or goes down the same paths as previous discussions because you have to tell some people over and over again what the thing that’s up for discussion all about again. It’s even more annoying when you see that a discussion goes over exactly the same points that were already handled earlier, but that some of the vocal people present in the session want to mention again.

    I also think that the preregistering of Blueprints on Launchpad is suboptimal. You write the Blueprints together at the UDS session and you register it afterwards. Only then do you know what you will go through with and what not. Only then do you know who is going to work on what.
    That way the number of useless Blueprints could be reduced. Maybe the registration of Blueprints in the ‘Ubuntu’ project for non-track leads could even be disabled to make the Blueprint lists readable and usable again.

    Sense Hofstede

    May 27, 2010 at 17:24

    • @Sense:

      the problem with registering the blueprint afterwards is, that interested people don’t know what will be (hopefully) discussed during UDS.

      As an non-community attendee this time, I was glad that I could select subscribe to the sessions I was interested in.

      One of the negative points of all this was, that some topics were changed right before UDS or during UDS, where the topic of the session was changed from this to that (in my case: from server to cloud server, which is a big difference).

      Regards,

      \sh

      shermann

      May 27, 2010 at 17:47

  2. I’m the Debian packager of Hadoop, HBase and Zookeeper. I’ve not known, that the UDS will take place and that Hadoop, HBase and Zookeeper will be discussed there.
    Since I was on IRC, Torsten Werner was able to ask me two questions during the session. However, the the audio recording of the session revealed, that I could have answered many more questions if I’d only have been asked.
    Ubuntu could even automate the task of notifying Debian Packagers about sessions that might be interesting for them: Just send a mail to all addresses in the uploaders and maintainers fields in debian/control.
    This mail could then also inform about possibilities to follow a session remotely over audio / video / chat.

    Thomas Koch

    May 27, 2010 at 17:31

    • This is an interesting idea; I’ll bring it up during the retrospective with the organizers.

      Matt Zimmerman

      May 28, 2010 at 08:06

  3. How about holding UDS during DebCamp or the week after DebConf? You might be able to include Debian folks into your discussions that way.

    someone

    May 28, 2010 at 01:50

    • There are usually several Debian developers who participate in UDS, and for 10.10 the DPL attended.

      UDS happens near the beginning of each Ubuntu release cycle (approximately May and November) while DebCamp happens in July or August, so I don’t think it would be feasible to align them.

      I am interested in promoting more cross-participation with Debian, though, and I like the idea (in an earlier comment) about reaching out to Debian developers to make them aware of sessions which might be relevant to them.

      Matt Zimmerman

      May 28, 2010 at 08:06

      • I agree that the idea of having more cross-participation at UDS is interesting, especially because it would very much help in planning together, and reduce the likelihood of forks. The same applies to DebConf/DebCamp, obviously.

        Maybe, what could practically help, is having a public/general invitation to DDs to attend, so that it doesn’t seem too “cabalish” when specific DDs do attend. (I’m not claiming it is cabalish, but you know, “form” counts too :-))

        Of course, that doesn’t mean *sponsoring* or even *accepting* all DDs requests for participation at UDS, since as I’ve witnessed myself, you’re quite close to have some scaling problems.

        Nevertheless, a well-advertised contact point to which DDs can send requests like “I’d like to attend in order to coordinate work on foo/bar/baz, can I come?” would be nice.

        Stefano Zacchiroli

        May 29, 2010 at 11:57

        • The idea of a general invitation sounds reasonable.

          One thing to note (in the context of “accepting” DDs) is that UDS is open to anyone; we are not at a point where we need to limit attendance and anyone can register free to attend.

          Matt Zimmerman

          May 29, 2010 at 12:13

          • A, wonderful, my bad that I didn’t understand that participation was already open to everybody.

            Then, even more so, I think announcements/invitations would be generally appreciated, especially if motivated as “discuss significant changes together to see if we can agree on them before implementation”.

            Cheers.

            Stefano Zacchiroli

            May 30, 2010 at 07:50

  4. OT: What software did you use to draw that wonderful schedule? :)

    Kim

    May 29, 2010 at 18:47

  5. http://en.wikipedia.org/wiki/Dunbar%27s_number

    Might be worth taking into account for optimizing group dynamics.

    Bryce Harrington

    May 30, 2010 at 07:28

  6. Regarding proposal A, I’d suggest trying a lightweight “T-shirt sizing estimation” method to triage blueprints. For instance:
    1. The creator entitles a blueprint and sets its goal (1 single phrase each)
    2. The creator estimates the relative size: S, M, L
    3. Management estimates the business value of the blueprint’s goal: S, M, L.

    Business value and relative size could then be used to earmark the blueprints where effort must be focused, so the business value is maximized each cycle.

    Héctor

    May 30, 2010 at 12:34

  7. The potential of this event should be used in a more efficient way. The biggest downside I see are burnt cycles on things that won’t get implemented in the upcoming release and will have to be rediscussed 6 months later (the world keeps spinning;).

    A better approach could be:

    1) come prepared (lot’s of features can be discussed/drafted prior to UDS) – have a 1st draft of your blueprint available X weeks prior to UDS

    2) apply priorities to the filed features – use a voting mechanism of some sort for community driven efforts/interests mangled with a factor for the business case priority. If available track effort estimates.

    3) know your capacities – how much as a whole can you seriously deliver for the next release? Use effort estimates from #2 to come up with your list of sessions for UDS.

    4) don’t use all your gun powder – plan for only 80% of the UDS time for such topics (thus increase 3 accordingly, the remaining 20% can be fitted in #1) and save about 20% for anything that comes up during the week

    5) finally at the event – reach understanding on open questions but not necessarily on every single detail.

    this has to be done in an appropriate amount of time prior to the event so that all operational tasks (travel, venue,…) can be arranged for.

    The benefits are more directed conversations on less topics that have more impact on the next release. The 20% buffer can be used as casual face time or more strategical forward planning. This doesn’t help solve the complexity of your shown planning algorithm and key people will still get a busy schedule, but at least most things discussed will be part of the next release and are contributing to the business relevance in the same token.

    just a few thoughts from an ubunte newbie;)

    Olli

    June 1, 2010 at 19:28

  8. […] we’ll refine these ideas into a plan for the cycle. We’re organizing UDS a little bit differently this time, with the main program divided into the following tracks to reflect the key […]

    Looking forward to UDS for Ubuntu 11.04 (Natty) | We'll see | Matt Zimmerman

    June 29, 2014 at 12:38


Comments are closed.

%d bloggers like this: