Posts Tagged ‘Technology’
Decoding a .mobileconfig file containing a Cisco IPsec VPN configuration
When someone wants to give you access to a Cisco VPN, they might give you a .mobileconfig file. This is apparently used by MacOS and iOS to encapsulate the configuration parameters needed to connect to a VPN. You should be able to connect to it with open source software (such as NetworkManager and vpnc) as long as you have the right configuration. Some helpful soul has tried to give you that configuration, but it’s wrapped up in an Apple-specific container. Here’s how you rip it open and get the goodies.
File format
A .mobileconfig appears to contain:
- Some binary garbage which is safe to ignore
- An XML document containing the good bits, i.e.:
- The “local identifier” (i.e. IPsec group name)
- The “remote address” (i.e. IPsec gateway host)
- The shared secret (base64 encoded)
- Some more binary garbage which is safe to ignore
…and it looks like this:
<plist version="1.0"> <dict> <key>PayloadContent</key> <array> <dict> <key>IPSec</key> <dict> <key>AuthenticationMethod</key> <string>SharedSecret</string> <key>LocalIdentifier</key> <string>LOCAL_IDENTIFIER_HERE</string> <key>LocalIdentifierType</key> <string>KeyID</string> <key>RemoteAddress</key> <string>REMOTE_ADDRESS_HERE</string> <key>SharedSecret</key> <data> BASE64_ENCODED_SHARED_SECRET_HERE </data> </dict> <key>IPv4</key> <dict> <key>OverridePrimary</key> <integer>0</integer> </dict> <key>PayloadDescription</key> <string>...</string> <key>PayloadDisplayName</key> <string>...</string> <key>PayloadIdentifier</key> <string>...</string> <key>PayloadOrganization</key> <string>...</string> <key>PayloadType</key> <string>com.apple.vpn.managed</string> <key>PayloadUUID</key> <string>...</string> <key>PayloadVersion</key> <integer>1</integer> <key>Proxies</key> <dict> <key>HTTPEnable</key> <integer>0</integer> <key>HTTPSEnable</key> <integer>0</integer> <key>ProxyAutoConfigEnable</key> <integer>0</integer> <key>ProxyAutoDiscoveryEnable</key> <integer>0</integer> </dict> <key>UserDefinedName</key> <string>...</string> <key>VPNType</key> <string>IPSec</string> </dict> </array> <key>PayloadDescription</key> <string>...</string> <key>PayloadDisplayName</key> <string>...</string> <key>PayloadIdentifier</key> <string>...</string> <key>PayloadOrganization</key> <string>...</string> <key>PayloadRemovalDisallowed</key> <false/> <key>PayloadType</key> <string>Configuration</string> <key>PayloadUUID</key> <string>...</string> <key>PayloadVersion</key> <integer>1</integer> </dict> </plist>
The shared secret is base64-encoded, so you can decode it with:
$ echo -n 'BASE64_ENCODED_SECRET_HERE' | base64 -d
Network Manager configuration
- Make sure you have network-manager-vpnc installed
- Click the Network Manager icon, select “VPN Connections”, “Configure VPN…”
- Create a “Cisco-compatible (vpnc)” connection
- Configure the connection settings as follows:
- Enter the “remote address” in the “Gateway” field
- Enter the “local identifier” in the “Group name” field
- Enter the shared secret in the “Group password” field
- To connect, click the Network Manager icon, select “VPN Connections”, and select the connection you just configured
Good luck and enjoy!
Building a personal data locker
If you were building a digital container to store your personal data, what would it look like?
Personal data being information associated with you: your contacts, your photos, the web pages you’ve visited, the places you’ve been, the messages you’ve sent and received, and so on. In short, your stuff.
Here’s my personal wish list of technical requirements:
- It has to be made of free software, of course
- It must keep my data secure, while allowing me to share it when and how I want to
- It needs to handle a range of different data types natively, and be extensible to new types, from photos to real-time sensor data
- It should be able to collect my data from many different places where it is being created and stored
- It should have a rich API, so that I can create applications which access my data
- If I want to, I should be able to host it myself, on my own hardware, without compromising my ability to access and share it
Of course, this isn’t merely an academic exercise, as my new day job at Singly is about building exactly this type of system. With a technical team including Jeremie Miller of Jabber and XMPP fame, our goal is to develop a personal data platform which meets these criteria and more.
There’s a lot of work to do, but today, you can check out the code and run a locker of your own, which can sync in data from Facebook, Twitter, Google, Foursquare, Github and dozens of other services. It’s a bit of a bear to set up, particularly if you don’t already have API keys for these services, but that’s fairly normal at this early stage of development.
If you try it, or have thoughts about what we’re doing, please let me know in the comments.
Where’s your data center?
Thanks to the tremendous growth of “social” applications over the past five years, we have our pick of services for collecting, saving and sharing our experiences online. We each have collections of photos, contacts, messages and more, spread across multiple popular services like Twitter, Facebook, LinkedIn, as well as many less popular services which address particular needs or preferences. We’re also producing a wealth of “exhaust data” through our browsing history, mobile sensors, transactions and other activity streams that we rarely if ever examine directly.
This ecosystem is becoming so complex that it’s easy to lose track of what you’ve created, shared or seen. We need new tools to manage this complexity, to make the most of the wealth of information and connections available to us through various services. John Battelle calls these “metaservices”, and points to growth in the number of connections between the services we use.
I expect that this next age of information tools will center around data rather than services. Data is a common denominator for these online experiences, a bridge across disparate services, technologies, social graphs, and life cycles. Personal data, in particular, has this property: the only thing that links together your photos on Flickr, Facebook, Picasa and Twitpic is…you.
So where’s your “data center”? I don’t anticipate the emergence of a single service where you do everything. There will continue to be innovation in the form of new and specialized services which meet a particular need very well. There won’t be a single service which is everything to everybody.
Instead, I foresee us wanting to track, save, use and control all of our “stuff” across the web. That’s why my new colleagues and I are working to make that possible.
There’s open source code available on github, a vibrant IRC channel (#lockerproject on Freeenode), and lots more I’d like to write about it. But it’s time to get back to work for now…
DEX finishes first batch of derivative patches for Debian
It’s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions.
Our first milestone
The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn’t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old.
I’m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them.
One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project.
This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian.
Conclusions
Although we weren’t quite able to clear the whole list, we still consider the project to be a success because:
- We ensured that all of the patches received due consideration for inclusion in Debian
- We proved the concept of DEX, with developers from Debian and derivatives cooperating on a common goal and sharing tools
- Most importantly, we learned from the experience
What’s next
In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX:
- Our second major project, nicknamed “big-merges”, will begin soon. Our goal is to identify the few packages which are most diverged between Debian and Ubuntu, and work to get them as close to identical as possible. If you have suggestions for packages to focus on, let us know!
- Allison Randal is beginning a DEX project to implement the Python 2.7 transition across Debian and Ubuntu
- Nathan Handler is working on a Summer of Code project to develop specialized tools to help with this kind of cross-distribution teamwork
- Zack is organizing a derivatives BoF at DebConf 11
We’re looking forward to seeing DEX develop further. If you’d like to get involved, come and join us on the debian-derivatives mailing list or IRC (#debian-derivatives on freenodeOFTC).
Matt Zimmerman and Stefano Zacchiroli
DEX: Debian and its derivatives, getting things done together
Since I resumed active status in Debian, I’ve been thinking about how to bridge the gap between Debian and its derivatives*. I’ve spoken at length with Zack, the attendees of the Derivatives BoF at DebConf 10, and the fine folks at the Derivatives Front Desk about the technical and social issues affecting derivative projects, and could probably write a very thorough series of blog posts on the subject.
Instead, Zack and I decided to try doing something about it: we have begun a project to test out a new approach to the problem.
Introducing DEX
DEX is all about action: merging patches, fixing bugs, crunching data, whatever is necessary to get changes from derivatives into Debian proper. DEX doesn’t try to change the way any existing project works, but adds a “fast path” for getting code from one place to another.
DEX is a joint task force where developers from Debian and its derivatives work together on this common goal. As a pilot project, we’ve established an Ubuntu DEX Team focused on merging code from Ubuntu into Debian. With members from both projects, we hope to be able to resolve blockage anywhere in the pipeline. Whatever needs to get done in order to merge an Ubuntu patch, someone in the Ubuntu DEX team will know what to do. If we get good results with Ubuntu, we hope that other derivatives will follow. With thanks to David Paleino, we’re excited that the Utnubu project is merging into DEX as it aligns well with their goals. I’m very grateful to have Colin Watson and James Westby signed up to contribute as well.
Our first project is simple: turn this list green. This is an archive of quite old patches from Ubuntu, most of which have probably been merged already or made obsolete, but they pre-date any kind of tracking system so they need to be verified. Once that’s done, we’ll move on to a new project with a new todo list.
If you want to see Debian benefit from technical work done in derivatives, DEX is a chance for you to act together to make it happen. If you work on a derivative and want to carry a smaller delta, come and join us. I’m sure we’ll learn a lot from this experience.
* There are many instances of great cooperation between Debian and derivative distributions, including joint package maintenance teams, and some derivatives are even part of the Debian project. Nonetheless, there are areas were most people I’ve spoken to agree that we need to do better. This is what I’ve referred to as the “gap”.
Listening to users
In the software community, people hold strong opinions on the subject of listening to users. Some feel that users are an essential source of information for making successful products, as evidenced in the customer development methodology, and seek to involve users deeply in product development. Others believe that users don’t know what they want, invoking the quote attributed to Henry Ford, “If I’d asked customers what they wanted, they would have said ‘a faster horse'”. Some say that user needs are unknowable except through the lens of a marketplace, where people choose in aggregate which products suit them best, and customers “vote with their wallets” (anything else is “anecdata”).
Regular readers will not be surprised that I believe they are all right, but only in certain contexts. The right strategy for involving users in product decisions will depend on factors related to the product itself, the market, and the product development method being used.
One of the most important is the life cycle stage of the product: is it a new and rapidly evolving concept, or a mature commodity, or somewhere in between? Simon Wardley explains this well over on his blog, so I won’t rehash his points here, but will add a few of my own.
If what we’re looking for is inspiration for a new product, it’s here that Henry Ford was right: users generally won’t hand you a complete product vision on a silver platter. They’ll frame their input in terms of what they know, and the choices already available to them. However, this doesn’t mean that users don’t have a role to play in this instance: watching users can be a great source of inspiration. It’s the combination of domain knowledge and passionate imagination which triggers the creative spark. Henry Ford applied his engineer’s interests to a problem which was evident all around him.
If our goal is to test whether a new product is a good fit for its users, there is no substitute for user feedback. We can guess at whether there is a fit, and our intuition may be good, but users are the ultimate judges, and we don’t know if we’re right or wrong until users evaluate it. So ask them! By engaging in dialogue with individual users, we can learn unexpected things which will help to refine the idea. If we don’t find what they think until our new product is released, we risk making something that no one wants. Why wait until it’s too late? It can be challenging to extract useful feedback for a product which doesn’t yet exist, but this effort is well worth it to avoid wasting much more effort in software engineering.
When our objective is to incrementally improve an existing product, individual anecdotes can mislead us. A given change may be an improvement for one user, but a disaster for another. What we want to know is whether the new version is better for the population as a whole, and in this case, we do well to rely on data. There are pitfalls here as well, of course. We need to choose our questions carefully, and realize that users will often resist any change: not because they’re stodgy by nature, but because they have to invest effort in adapting to the change. I think of incremental improvement as a joint investment made between product developers and their users, to improve the whole system of people and technology for the better.
By choosing the right tool for the job, we can make better decisions, improve faster, and ultimately solve the right problem for our users.
Ubuntu Brainstorm Top 10 for December 2010
As I mentioned recently, the Ubuntu Technical Board is reviewing the most popular topics in Ubuntu Brainstorm and coordinating official responses on behalf of the project. This means that the most popular topics on Ubuntu Brainstorm receive expert answers from the people working in these areas.
This is the first batch, and we plan to repeat this process each quarter. We’ll use feedback and experiences from this run to improve it for next time, so let us know what you think.
Power management (idea #24782)
Laptops are now outselling desktops globally, and laptop owners want to get the most out of their expensive and heavy batteries. So it’s no surprise that people are wondering about improved power management in Ubuntu. This is a complex topic which spans the Linux software stack, and certainly isn’t an issue which will be “solved” in the foreseeable future, but we see a lot of good work being done in this area.
To tell us about it, Amit Kucheria, Ubuntu kernel developer and leader of the Linaro working group on Power Management, contributed a great writeup on this topic, with technical analysis, tips and recommendations, and a look at what’s coming next.
I am going to attempt to summarize the various use profiles and what Ubuntu does (or can do) to prolong battery life in those profiles. Power management, when done right, should not require the user to make several (difficult) choices. It should just work – providing a good balance of performance and battery life.
IP address conflicts (idea #25648)
IP addressing is a subject that most people should never have to think about. When something isn’t working, and two computers end up with the same IP address, it can be hard to tell what’s wrong. I was personally surprised to find this one near the top of the list on Ubuntu Brainstorm, since it seems unlikely to be a very common problem. Nonetheless, it was voted up, and we’re listening.
There is a tool called ipwatchd which is already available in the package repository, and was created specifically to address this problem. This seems like a further indication that this problem may be more widespread than I might assume.
The idea has already been marked as “implemented” in Brainstorm based on the existence of this package, but that doesn’t help people who have never heard of ipwatchd, much less found and installed it.
What do you think? Have you ever run into this problem? Would it have helped you if your computer had told you what was wrong, or would it have only confused you further? Is it worth considering this for inclusion in the default install? Post your comments in Brainstorm.
Selecting the only available username to login (idea #6974)
Although Linux is designed as a multi-user operating system, most Ubuntu systems are only used by one person. In that light, it seems a bit redundant to ask the user to identify themselves every time they login, by clicking on their username. Why not just preselect it? Indeed, this would be relatively simple to implement, but the real question is whether it is the right choice for users.
Martin Pitt of the Ubuntu Desktop Team notes that consistency is an important factor in ease of use, and asks for further feedback.
So in summary, we favored consistency and predictablility over the extra effort to press Enter once. This hasn’t been a very strong opinion or decision, though, and the desktop team would be happy to revise it.
Icon for .deb packages (idea #25197)
Building on the invaluable efforts of Debian developers, we work hard to make sure that people can get all of the software they need from Ubuntu repositories through Software Center and APT, where they are authenticated and secure. However, in practice, it is occasionally necessary for users to work with .deb files directly.
Brainstorm idea 25197 suggests that the icon used to represent .deb packages in the file manager is not ideal, and can be confusing.
Matthew Paul Thomas of the Canonical Design Team responds with encouragement for deb-thumbnailer, which makes the icon both more distinctive and more informative. He has opened bug 685851 to track progress on getting it packaged and into the main repository.
I have reviewed the proposed solutions with Michael Vogt, our packaging expert. Solution #1 is straightforward, but we particularly like solutions #5 and #10, using a thumbnailer to show the application icon from inside each package.
Keeping the time accurate over the Internet by default (idea #25301)
It’s important for an Internet connected computer to know the correct time of day, which is why Ubuntu has included automatic Internet time synchronization with NTP since the very first release (4.10 “warty”). So some of us were a little surprised to see this as one of the most popular ideas on Ubuntu Brainstorm.
Colin Watson of the Ubuntu Technical Board investigated and discovered a case where this wasn’t working correctly. It’s now fixed for Ubuntu 11.04, and Colin has sent the patches upstream to Debian and GNOME.
My first reaction was “hey, that’s odd – I thought we already did that?”. We install the ntpdate package by default (although it’s deprecated upstream in favour of other tools, but that shouldn’t be important here). ntpdate is run from /etc/network/if-up.d/ntpdate, in other words every time you connect to a network, which should be acceptably frequent for most people, so it really ought to Just Work by default. But this is one of the top ten problems where users have gone to the trouble of proposing solutions on Brainstorm, so it couldn’t be that simple. What was going on?
More detail in GNOME system monitor (idea #25887)
Under System, Preferences, System Monitor, you can find a tool to peek “under the hood” at the Linux processes which power every Ubuntu system. Power users, hungry for more detail on their systems’ inner workings, voted to suggest that more detail be made available through this interface.
Robert Ancell of the Ubuntu Desktop Team answered their call by offering to mentor a volunteer to develop a patch, and someone has already stepped up with a first draft.
Help the user understand when closing a window does not close the app (idea #25801)
When the user clicks the close button, most applications obediently exit. A few, though, will just hide, and continue running, because they assume that’s what the user actually wants, and it can be hard to tell which has happened.
Ivanka Majic, Creative Strategy Lead at Canonical, shares her perspective on this issue, with a pointer to work in progress to resolve it.
This is more than a good idea, it’s an important gap in the usability of most of the desktop operating systems in widespread use today.
Ubuntu Software Centre Removal of Configuration Files (idea #24963)
One feature of the Debian packaging system used in Ubuntu is that it draws a distinction between “removing” a package and “purging” it. Purging should remove all traces of the package, such that installing and then immediately purging a package should return the system to the same state. Removing will leave certain files behind, including system configuration files and sometimes runtime data.
This subtle distinction is useful to system administrators, but only serves to confuse most end users, so it’s not exposed by Software Center: it just defaults to “removing” packages. This proposal in Ubuntu Brainstorm suggests that Software Center should purge packages by default instead.
Michael Vogt of the Ubuntu Foundations Team explains the reasoning behind this default, and offers an alternative suggestion based on his experience with the package management system.
This is not a easy problem and we need to carefully balance the needs to keep the UI simple with the needs to keep the system from accumulating cruft.
Ubuntu One file sync progress (idea #25417)
Ubuntu One file synchronization works behind the scenes, uploading and downloading as needed to replicate your data to multiple computers. It does most of its work silently, and it can be hard to tell what it is doing or when it will be finished.
John Lenton, engineering manager for the Ubuntu One Desktop+ team, posts on the AskUbuntu Q&A site with tools and tips which work today, and their plans to address this issue comprehensively in the future.
Multimedia performance (idea #24878)
With a cornucopia of multimedia content available online today, it’s important that users be able to access it quickly and easily. Poor performance in the audio, video and graphics subsystems can spoil the experience, if resource-hungry multimedia applications can’t keep up with the flow of data.
Allison Randal, Ubuntu Technical Architect, answers with an analysis of the problem and the proposed solutions, an overview of current activity in this area, and pointers for getting involved.
The fundamental concern is a classic one for large systems: changes in one part of the system affect the performance of another part of the system. It’s modestly difficult to measure the performance effects of local changes, but exponentially more difficult to measure the “network effects” of changes across the system.
Three ways for Ubuntu to help developers
Developers are a crucial part of any successful software platform. In the same way that an operating system is “just” a means for people to use applications, a platform is “just” a means for developers to create applications and make them available to people.
There are three primary ways in which Ubuntu can help developers do their work. They are all related, but distinct, and so we should consider them individually:
1. Developing for Ubuntu
Today, Ubuntu bundles thousands of free software applications, for both clients and servers, most of which are packaged by Debian.
Ubuntu also carries certifications for a variety of third-party ISV software, both open source and proprietary, which are coordinated through Canonical’s partner program.
In both of these cases, many of these applications are actually developed on other platforms, and ported to Ubuntu, either by the free software community or by the creators of the software.
2. Developing on Ubuntu
Ubuntu is already quite popular among developers, who mainly run Desktop Edition on their workstations. They might be developing:
- web applications (with server-side and browser components)
- portable applications (e.g. using Java, or Adobe AIR)
- mobile applications (e.g. for Android or iOS)
- native applications, which might target Ubuntu Desktop Edition itself, or supporting multiple platforms through a framework like Qt
3. Distributing through Ubuntu
Like other modern operating systems, Ubuntu isn’t just a platform where applications run, but also a system for finding and installing applications. Starting with APT, which originated in Debian, we’ve added Software Center, the ISV partner repository, and various other capabilities in this area. They all help to connect developers with users, facilitating distribution of software to users, and feedback to developers.
So, where should we focus?
Some developers might be interested in all three of these, while others might only care about one or two.
However, most of the developer improvements we could make in Ubuntu would only address one of these areas.
For this reason, I think it’s important that we consider the question of the relative importance of these three developer scenarios. Given that we want Ubuntu to flourish as a platform, how would we prioritize them?
I have my own ideas, which I’ll write about in subsequent posts, but here’s your chance to tell me what you think. :-)
Ubuntu: Project, Platform, Products
When most people talk about Ubuntu, they usually mean our flagship product, Ubuntu Desktop Edition. Sometimes, they might mean the Ubuntu project, or the community of people who work on it, or various other things.
Similarly, Debian might mean the Debian operating system, or the package repositories, or the project, and so on.
This gets a little confusing sometimes. When I’m talking about Ubuntu, I’ve started to use more specific terminology to explain what I mean, and this seems to help people understand better the nature of the whole Ubuntu. In particular, I use the three Ps:
- a portfolio of products, including Desktop Edition, Server Edition, Netbook Edition, Kubuntu and more. These are software bundles which can be downloaded, pre-installed on retail computers, and so on. Each one is designed to meet a certain set of user needs, and to work on a specific form factor of computer.
- a technology platform, which can be used to build a wide range of products. It is primarily of interest to developers, who build derivative distributions, OS products, applications and infrastructure using Ubuntu packages. This platform is the common foundation of the Ubuntu products above, and includes things like the global package repository. Joel Spolsky does a good job of explaining why platforms are distinctly different from products, and should be treated as such.
- an open community project, which collectively produces, distributes, promotes and supports the products and the platform. The Ubuntu project has a philosophy, a government, and various tools and processes to help contributors work together. Canonical supports the Ubuntu project by providing resources and infrastructure, and also directly participates in Ubuntu at various levels.
This breakdown may seem a bit obvious to those of us “on the inside”, but it’s confusing to people who are encountering it for the first time. I’m sharing this in the hope that if more people start using the same words, it will get easier for people to understand how these pieces fit together. I’ll also be linking to it a lot, to help put things into context using this framework.