Preparing for Barcelona

What's the issue that deserves the fullest attention of the best minds of the mobile industry representatives who will be gathering next week at the Mobile World Congress event in Barcelona?

That was one of the questions that I got asked in a quick-fire practice session last week, by a journalist who was employed for the morning to take part in a "media training session" for people from the Symbian Foundation launch team. The idea of the session was to bombard participants with potentially awkward questions, so we could test out various ways to respond. The questions ranged from tame to taxing, from straightforward to subtle, and from respectful to riotous.

One possible answer to the question at the top of this posting is that it is the issue of user experience which deserves the fullest attention. If users continue to be confronted by inflexible technology with unfriendly interfaces, they won't get drawn in to make fullest use of mobile devices and services.

Another possible answer is that it is the issue of complexity which deserves the fullest attention. In this line of thinking, overly complex UIs are just one facet of the problem of overly complex mobile technology. Other facets include:

• Overly difficult development cycles (resulting in products coming late to the market, and/or products released with too many defects), and
• Overly exercised CPU cores and overly bloated software (resulting in products with poor battery life and high cost).

However, on reflection, I offer instead the following answer: it is the issue of collaboration which deserves the fullest attention. We need to find better ways for all the good resources of the mobile industry to be productively aligned addressing the same key risks and opportunities, rather than our good intentions ending up pulling in different directions. The problems that we collectively face (including the problems of poor user experience and overly complex software) are surely capable of resolution, if only we can find the way to work together on solutions, rather than our different approaches ending up contradiciting each other and confusing matters.

Open source, whereby people can look at source code and propose changes without having to gain special permission in advance, is part of the solution to improving collaboration. Open discussion and open governance take the solution further. Yet another step comes from collaboration tools that provide first-rate source configuration management and issue tracking.

But collaboration often needs clear leadership to make it a reality: a sufficiently compelling starting point on which further collaboration can take place. Without such a starting point, none of the other items I mentioned can hope to make a lasting difference.

That brings me back to the role of the Symbian Foundation. The Symbian Foundation is offering the entire mobile industry what it claims to be the best possible starting point for further collaboration:

• A tried and tested codebase of 40 million lines of code;
• Processes and disciplines that cope with pressures from multiple divergent stakeholders;
• A visionary roadmap that is informed by the thinking of existing mobile leaders, and which spells out the likely evolution of key mobile technologies.

The Symbian Foundation will be holding a welcome party on Monday evening at Barcelona (8pm-11pm, 16th February). I've been allocated a small number of tickets to this party, to pass to selected bloggers, analysts, and other deep thinkers of the mobile industry. If you'd like to join this party to discuss the points I've made in this posting (or any of the other issues relevant to the formation of the Symbian Foundation), I set you this challenge. Please drop me an email, and provide a link to some of your online writings on applicable topics. (By the way, you don't need to agree with my viewpoint. You just need to demonstrate that you're going to enter into an open-minded, friendly, and constructive debate!)

2009 - end users modifying their mobile phone apps

Here's a scenario I expect to become increasingly common later this year.

(Elements in the following story are made up, of course, but they serve as placeholders for anticipated real people, real phones, and real apps.)

Vijaya is really fond of her new Nokia N225 based on the latest Symbian Platform Release, and is both intrigued and frustrated by features of the Jomo Player app that's built into that phone. The app does some very clever things, but yet, Vijaya thinks it would serve her own needs better if some of the behaviour and functionality were changed. She also has ideas for tweaking the UI.

If this story were set in 2008, that would probably be the end of the story. Vijaya might write about her ideas on Facebook, and her friend Sunil might send them to someone he knows who has a job in the Nokia Devices R&D lab, but the chances are, the original developers of the Jomo Player app would be far too busy to pay attention to what appear to be idiosyncratic, quaint, or overly personalised change suggestions.

Now let's make this story more interesting. Suppose that Vijaya already knows some Symbian C++. Maybe she took a course on it at the local technical university, which is enrolled into the Symbian Academy program. Or maybe she used to work for a phone manufacturer helping to customise their Symbian devices. So, either way, Vijaya starts writing an alternative Jomo Player app, starting from scratch. Her goal is to embody her own ideas on usability and feature set.

But guess what: her alternative Jomo Player falls far short of the performance and power of the built-in app. It's tough to re-create a complex app. Although Symbian in 2008 is an open platform, with rich APIs, it's not at all obvious to Vijaya how to emulate, in her version of the app, many of the features of the original, which she now comes to increasingly recognise as subtle and refined. Some of Vijaya's friends band together to help, but they eventually abandon the project. The original app, they realise, is doing some incredibly complex things under the surface - and their attempted clone comes nowhere close to matching it. So, in 2008, that really is the end of the story.

Now let's re-run this story sometime later on in 2009. The source code for the original Jomo Player app is available for download from the Symbian Foundation Mercurial code repository, under the open source Eclipse Public Licence. What's more, the publicly available SDKs provide enough header files and libraries that Vijaya and her friends can rebuild the entire app. So the starting point is very different. Rather than struggling to create the whole app from scratch, Vijaya can fairly easily locate the parts of the source code she wants to change. As a result, she has a new version of the Jomo Player on her N225 in less than a week. As a result of using this app some more, with its altered features, she and her friends get yet more ideas - and then a major breakthrough flash. The new app quickly evolves into a dramatically better state.

Shortly afterwards, Vijaya makes her new app available via several application stores. It gets rave reviews. These reviews come to the attention of product managers in one or more phone companies. Both the N226 and a new Samsung phone build this version of the app into their ROMs, and reach millions of happy smartphone customers well before Christmas.

Vijaya started this whole process by scratching a personal itch. She wanted to improve a particular app running on her own phone. However, unexpectedly, she now has three different Symbian development houses competing to hire her into their teams.

In parallel, Mika has altered the Voton Reader app so that it's more usable by his mother. (It turns out, afterwards, to be more usable by almost everyone!) Antony has added a whole series of shortcut keys to the Contacts app. And Alexa has produced a stunning new combination of two originally separate apps.

That's the difference between what can be accomplished by an open platform (with published APIs) and by an open source platform (with published, buildable source code).

As 2009 progresses, the mobile phone platforms that publish their source code will increasingly play host to deeper and more interesting forms of innovation, than those mobile platforms which keep their source code closed. The phones from these open source mobile platforms (such as Symbian) will have the best Mojo Player, Voton Readers, and so on - not because the developers inside Symbian are cleverer than those in other mobile phone platform companies, but because these platforms can take greater advantage of the much wider pool of creative and clever people who are outside the company.

Footnote: Credit for key elements of this vision belong to some of my colleagues on the Symbian Foundation launch team, including William Roberts and Antony Edwards.

Disclaimer: The devil's in the detail. Thoughtful readers will realise there are lots of important details missing from the above story. I look forward to returning to these details.

Out with the old, in with the new

The final two hours of 2008 were, for me, the most fraught and tense of any New Years Eve that I remember. A central London car journey that was scheduled to take only 15 minutes became a 2 hour long trauma. We made it to the restaurant 90 minutes late, just one minute before the gongs of Big Ben would ring in 2009.

On paper, the plans for the evening looked clear enough. My mum is staying a few days with us, down from Inverness, so my wife Hyesoon and I planned for the three of us to take in a traditional New Years Eve Viennese Waltz concert at the Barbican, from 7.30-9.15pm, followed by dinner in a classy Belgravia Thai restaurant, the Mango Tree, from 10.30-12.30pm. For some daft reason I decided the trip would be easiest if we drove the whole way. I thought that would allow the greatest flexibility, and that my trusted hi-tech TomTom satnav would guide me through any unfamiliar routes. And in any case, there should be plenty of time for the various parts of the journey: when I checked in advance, both TomTom and Google Maps said that the journey from the Barbican to the Mango Tree would take only 15 minutes. Or so I thought.

The first sign of trouble was in the initial part of the whole journey, from my home in Surbiton (South West London) to the Barbican (East Central London). TomTom predicted 45 minutes. We left home at 6.oo, giving us 90 minutes for that trip. I wasn't sure of the way, but the traffic got thicker and thicker and slower and slower. And slower and slower. After several route recalculations and inspired changes of plan, we finally made it to our seats in the concert hall just as the audience were giving an applause to welcome the leader of the orchestra to the stage. Talk about last minute! Happily, the music was glorious.

Less happily, the concert which I had been told (by Barbican office staff, several days earlier) would finish at 9.15, actually went on till 10pm, by the time the second encore finished. We rushed to the car park to get out ahead of the main crowds, and set off on our anticipated 15 minute journey across central London.

However, the police had erected roadblocks all over the place, to force road traffic away from central London locations such as Trafalgar Square and Parliament Square. Time and again, I had to drive in a different direction to the one I intended. My car wasn't the only one that was frustrated by the re-routings. The few roads that were still open were jammed to a snails pace. We rang the Mango Tree to say we might be, err, 20 minutes, or maybe even 40 minutes late. Come anyway, they said. We're trying, I said. After painfully slow progress along Marylebone Road, Edgware Road and Park Lane, we finally reached the restaurant, just as the DJ was starting the countdown to the chimes signalling 1.1.2009. (Talk about last minute...)

Not unreasonably, the tempo and ambience in the restaurant by that time was a lot noisier than would normally accompany selecting starters from the menu. The staff did a fine job in the circumstances. In the end, I managed at least a grin as the sound system was belting out Mick Jagger's "Honky Tonk Woman" at high volume. And Hyesoon and I got to our feet for some middle-aged boogie to Abba's "Dancing Queen". My mum said, it was quite an experience. Many thanks to both my mum and Hyesoon for being (mainly!) calm and supportive through all this trauma.

Being stuck for so long in slow-moving traffic gives you time for a lot of "if only" thinking. If only I had followed general advice and taken public transport rather than my car. If only I had realised that most routes would be blocked, and had started on a wide berth earlier. If only we had booked a venue closer to home. And, if only my satnav was hooked up with current road and traffic information, rather than relying on hard-wired map information that failed to match the reality of the moment. As a fan of Agile, that's a lesson I ought to have learned already.

I wonder how many other aspects of life will, in 2009, suffer from being similarly misguided by automated or semi-automated responses that are based on out-of-date conceptual maps?

Our collective infrastructure is continuing to change in many ways, probably more than we expect. The market landscape is highly fluid. Items of our economic infrastructure that we thought we could take for granted, are falling away while our attention is focused elsewhere. Woe betide us if we stick on auto-pilot, trusting that our past processes are sufficient to guide us safely through the new terrain.

A few days ago, Kevin Kelleher forecast in GigaOm that 2009 could be "The year of the hacker". In short, our tough new economic climate could result in new creativity from talented people who are struggling in their old jobs (or who lose their jobs completely):

I don’t mean to downplay how hard it is to be unemployed. But with tens of thousands of skilled tech workers being kicked into a hostile job market, the effects could prove to be positive for the Internet and its community over the long term...

I wonder what kind of creativity could be unleashed by workers who, though deprived of a steady paycheck, are freed from tedious tasks. Some could come up with new ideas that help vault the web to a more advanced stage. Others may make micro-contributions that are equally powerful in aggregate. Such creativity could then foster an entirely new generation of startups, which would eventually lure away some of those who had remained at steady jobs all along...

Of course, money will be hard to come by for such labors of love. Some of the best ideas since the last downturn have failed to find a viable business model. A gift economy would be an especially profitless form of innovation. But that notion lies at the heart of the hacking ethic.

Building on Kelleher's ideas, Brad Feld of Mobius Venture Capital plausibly suggests that 2009 could see the "re-rise of open source":

Kevin Kelleher’s article on GigaOm this morning titled 2009: Year of the Hacker made me think back to the rise of open source after the Internet crash of 2001. In the aftermath of the crash, many experienced software developers were out of work for a period of time ranging from weeks to years. Some of them threw themselves into open source projects and, in some cases, created their next job with the expertise they developed around a particular open source project.

We are still in a tense and ambiguous part of the current downturn where, while many developers are getting laid off, some of them are immediately being picked back up by other companies that are in desperate need for them. However, many other developers are not immediately finding work. If the downturn gets worse, the number of out of work developers increases.

If they take a lesson from the 2001 – 2003 time frame, some subset of them will choose to get deeply in an open source related project. Given the range of established open source projects, the opportunity to do this today is much more extensive than it was seven years ago. In addition, most software companies – especially Internet-related ones - now have robust API’s and/or open source libraries that they actively encourage third parties to work with for free. The SaaS-based infrastructure that exists along with maturing source code repositories add to the fun. The ability to hack something interesting together based on an established company’s infrastructure is omnipresent and is one of the best ways to “apply for a job” at an interesting company...

When plans for the Symbian Foundation were announced in June last year, we did not foresee the substantial economic downturn and the fact that many fine software developers would, through no fault of their own, find themselves out of work. This changed landscape, unexpectedly, makes it all the more important for software platforms to be sufficiently open to allow a wide number of developers to engage deeply and easily with the system. If the case was strong last June to open source the Symbian Platform, it has, unexpectedly, become even stronger in the intervening seven months.

If 2009 will be the year of the hacker and/or the year of the re-rise of open source, it changes the priorities of all software systems, to become friendlier to hackers and open source practitioners. The systems that can best leverage this new latent talent pool will be the ones that are the most likely to be flying high in 12 months time when the chimes of Big Ben ring out 2009 and herald in yet another new year. (But on that occasion, I will definitely not be driving anywhere near Central London!)

Order from open source chaos

Various videos and PDFs from the recent Symbian Partner Event are now available online.

One video that amply repays viewing is Jay Sullivan of Mozilla speaking on "Chaos and order: a Mozilla story". You'll find it on the presentations page of the SPE website.

Mozilla's declared mission - "promote choice and innovation on the Internet" - has a lot in common with what Symbian is trying to do. One size does not fit all. Mozilla's declared methods - involving open source, weak copyleft, and an independent foundation - also resonate with those of the Symbian Foundation. Even the sizes of the organisations are broadly comparable (Jay mentioned that Mozilla has around 175 employees).

Mozilla has been travelling along this particular road a lot longer than Symbian. This helps to explain why many Symbian people in the audience were hanging intently on every word in the presentation.

The questions that the presentation sought to answer included:

• How can your organisation harness openness (where more and more things happen in public), rather than fight it?
• How do you get your customers to support each other (peer-to-peer support), rather than always going to the centre for support?
• How can a comparatively small company take advantage of wide public support to compete with huge existing players?
• How can 75 developers inside the company leverage 100s of external daily contributors, 1000s of less frequent contributors, 10s of 1000s of overnight testers, and around one million beta testers?

In part, the answer to these questions is to use appropriate tools. For example, Mozilla relies heavily on the Bugzilla bug-tracking database.

In part, the answer comes down to attitude. Mozilla have adopted widespread openness of information sharing: they use wikis and newsgroups, which are almost all publicly accessible. (The exception is a small amount of personnel information.) Another example: Everyone in the world is able to dial into the company weekly status update meeting. (Jay commented: "We know our competition dials in".)

What I personally found most interesting was Jay's analysis of the potential chaos that ensues from this openness. For example, there can be a great deal of "noise" in the online comments from all sorts of people: it's hard to filter postings that are based on reality, from those based on speculation or fantasy. There's a constant trail of chat, with input from all over the world. Everyone can propose changes to the project. In such an environment, how can real work get done? How can you mediate among 50,000 people who all have ideas to improve a particular dialog box in the UI of an application? How to deal with strongly vocal minorities?

The answers were fascinating (and deeply practical):

• Open doesn't mean democracy
• Decision-making is messy (but that doesn't mean you should step back from openness)
• Be prepared to tolerate some messiness
• Treat disagreements as negotiations
• Managers of the project need to drive towards definite outcomes - focusing on what is the right outcome rather than who has the right ideas
• Organise a chorus (rather than a chaos), around local leaders
• Although anyone can propose changes, you need to earn significant amounts of credibility before you are allowed to implement a change
• Ensure quality through multiple reviews
• Review for performance regressions as well as for functionality
• Educate participants about the vision and the mission of the project, which in turn allows greater micro-level decisions
• Guide participants towards using the appropriate communication channels for particular topics, and to back up their assertions with research and data
• Create small focused teams with responsibility for specific areas of product interest
• Create a common language, to allow discussions to be more productive
• You still need to have clearly identified decision makers, even though you push as much of the discussion out "to the edge" as possible.

These are good thoughts to keep in mind in the midst of the inevitable turmoil as the Symbian Foundation places 40 million lines of code into open source (and makes corresponding changes in processes) over the next 18 months.

The starfish and the spider

In my quest to understand the full potential of open and collaborative methods of working, I recently found myself re-reading "The Starfish and the Spider: The unstoppable power of leaderless organisations" by Ori Brafman and Rod Beckstrom.

I found this book to be utterly engrossing. I expect that its metaphor of the starfish vs. the spider will increasingly enter common parlance - the same way as "Tipping Point" did. In short:

• A starfish has a fully de-centralised nervous system, and can survive and prosper when it undergoes an apparent "head-on" attack;
• A spider has a CEO and a corporate headquarters, without which it cannot function.

The examples in the book show why there's a great deal at stake behind this contrast: issues of commercial revenues, the rise and fall of businesses, the operation of the Internet, and the rise and fall of change movements within society - where the change movements include such humdingers as Slave Emancipation, Female Equality, Animal Liberation, and Al Qaeda.

There are many stories running through the book, chosen both from history and from contemporary events. The stories are frequently picked up again from chapter to chapter, with key additional insights being drawn out. I found some of the stories to be familiar, but others were not. In all cases, the starfish/spider framework cast new light.

The book contains many implications for the question of how best to inspire and guide an open source ecosystem. Each chapter brought an important additional point to the analysis. For example:

• Factors allowing de-centralised organisations to flourish;
• The importance of self-organising "circles";
• The significance of so-called "catalyst" personalities;
• How successful de-centralised organisations often piggy-back pre-existing networks;
• How centralised organisations can go about combatting de-centralised opponents;
• Issues about combining aspects of both approaches.

Regarding hybrid approaches: the book argues that smart de-centralisation moves by both GE and Toyota are responsible for significant commercial successes in these companies. EBay is another example of a hybrid. Managing an open source community surely also falls into this category.

The book spoke personally to me on another level. As it explains, starfish organisations depend upon so-called "catalyst" figures, who may lack formal authority, and who are prepared to move into the background without clinging to power:

• Catalysts enable major reactions to take place, that would otherwise remain dormant;
• They trigger the deployments of huge resources from the environment;
• They make things happen, not by direct power, but by force of influence and inspiration.

There's a big difference between catalysts and CEOs. Think "Mary Poppins" rather than "Maria from Sound of Music". That gave me a handy new way of thinking about my own role in organisations. (I'm like Mary Poppins, rather than Maria! I tend to move on from the departments that I build up, rather than remaining in place.)

Mobile 2.0 keynote

Earlier today, I had the privilege to deliver the opening keynote at the Mobile 2.0 event in San Francisco. This posting consists of a copy of the remarks I prepared.

The view from 2013

My topic is Open Source, as a key catalyst for Mobile Innovation 2.0.

Let’s start by fast forwarding five years into the future. Imagine that we are gathered for the 2013 “Mobile 2.0” conference – though the name may have changed by that time, perhaps to Mobile 3.0 or even Mobile 4.0 – and perhaps the conference will be taking place virtually, with much less physical transportation involved.

Five year into the future, we may look back at the wonder devices of today, 2008: the apparently all-conquering iPhone, the Android G1, the Nokia E71, the latest and greatest phones from RIM, Windows Mobile, and so on: all marvellous devices, in their own ways. From the vantage point of 2013, I expect that our thoughts about these devices will be: “How quaint! How clunky! How slow! How did we put up with all the annoyances and limitations of these devices?”

This has happened before. When the first Symbian OS smartphones reached the market in 2002 – the Nokia 7650, and the Sony Ericsson P800 – they received rave reviews in many parts of the world. These Symbian smartphones were celebrated at the time as breakthrough devices with hitherto unheard of capabilities, providing great user experiences. It is only in retrospect that expectations changed and we came to see these early devices as quaint, clunky, and slow. It will be the same with today’s wonder phones.

Super smart phones

That’s because the devices of five years time will (all being well) be so much more capable, so much slicker, so much more usable, and so much more enchanting than today’s devices. If today’s devices are smart phones, the devices of 2013 will be super smart phones.

These devices will be performing all kinds of intelligent analysis of data they are receiving through their sensors: their location and movement sensors, their eyes – that is, their cameras – and their ears – that is, their always-on recording devices.

They’ll also have enormous amounts of memory – both on-board and on-network. Based on what they’re sensing, and on what they know, and on their AI (artificial intelligence) algorithms, they’ll be guiding us and informing us about all the things that are important to us. They’ll become like our trusted best friends, in effect whispering insight into our ears.

We can think of these devices as like our neo neo-cortex. Just as primates and especially humans have benefited from the development of the neo-cortex, as the newest part of our brains in evolutionary terms, so will users of super smartphones benefit from the enhanced memory, calculation powers, and social networking capabilities of this connected neo neo-cortex.

In simple terms, these devices can be seen as adding (say) 20 points to our IQs – perhaps more. If today’s smartphones can make their users smarter, the super smartphones of 2013 can make their users super smart.

Solving hard problems

That’s the potential. But the reality is that it’s going to be tremendously hard to achieve that vision. It’s going to require an enormously sophisticated, enormously capable, mobile operating system.

Not everyone shares my view that operating systems are that important. I sometimes hear the view expressed that improvements in hardware, or the creation of new managed code environments, somehow reduce the value of operating systems, making them into a commodity.

I disagree. I strongly disagree. It's true that improvements in both hardware and managed code environments have incredibly important roles to play. But there remain many things that need to be addressed at the operating system level.

Here are just some of the hard tasks that a mobile operating system has to solve:

• Seamless switching between different kinds of wireless network – something that Symbian’s FreeWay technology does particularly well;


• Real-time services, not only handling downloads of extremely large quantities of data, but also manipulating that data in real time – decompressing it, decrypting it, displaying it on screen, storing it in the file system, etc;


• All this must happen without any jitter or delay – even though there may be dozens of different applications and services all talking at the same time to several different wireless networks;


• All this rich functionality must be easily available to third party developers;


• However, that openness of developer access must coexist with security of the data on the device and the integrity of the wireless networks;


• And, all this processing must take place without draining the batteries on the device;


• And without bamboozling the user due to the sheer scale and complexity of what’s actually happening;


• And all this must be supported, not just for one device, but in a way that can be customised and altered, supporting numerous different form factors and usage models, without fragmenting the platform.

Finally, please note that all this is getting harder and more complex: every year, the amount of software in a top-range phone approximately nearly doubles. So in five years, there’s roughly up to 32 times as much software in the device. In ten years, there could be 1000 times as much software.

Engaging a huge pool of productive developers

With so many problems that need to be solved, I will say this. The most important three words to determine the long-term success of any mobile operating system are: Developers, Developers, Developers. To repeat: the most important three words for the success of the Symbian platform are: Developers, Developers, Developers.

We need all sorts and shapes and sizes of developers – because, as I said, there are so many deep and complex problems to be solved, as the amount of software in mobile phone platforms grows and grows.

No matter how large and capable any one organisation is, the number of skilled developers inside that organisation is only a small fraction of the number outside. So it comes down to enabling a huge pool of productive and engaged developers, outside the organisation, to work alongside the original developers of the operating system – with speed, creativity, skill, and heartfelt enthusiasm. That’s how we can collectively build the successful super smart phones of the future.

Just two weeks ago, the annual Symbian Smartphone Show put an unprecedented focus on developers. We ran a Mobile DevFest as an integral part of the main event. We announced new developer tools, such as the Symbian Analysis Workbench. We will shortly be releasing new sets of developer APIs (application programming interfaces) in new utility libraries, to simplify interactions with parts of the Symbian programming system that have been found to cause the most difficulty – such as text descriptors, two phase object construction and cleanup, and active objects.

The critical role of open source

But the biggest step to engage and enthuse larger numbers of developers is to move the entire Symbian platform into open source.

This will lower the barriers of entry – in fact, it will remove the barriers of entry. It will allow much easier study of the source code, and, critically, much easier participation in the research and creation of new Symbian platform software. We are expecting a rapid increase in collaboration and innovation. This will happen because there are more developers involved, and more types of developers involved.

That’s why the title of my talk this morning is “Open Source: Catalyst for Mobile Innovation 2.0”. The “2.0” part means greater collaboration and participation than ever before: people not just using the code or looking at the code, but recommending changes to it and even contributing very sizeable chunks of new and improved code. The Open Source part is one of the key enablers for this transformation.

Necessary, but not sufficient

However, Open Source is only one of the necessary enablers. Open Source is a necessary but not sufficient ingredient. There are two others:

1. A stable and mature software base, with reliable processes of integration, which I’ll talk more about in a moment;


2. Mastery of the methods of large-scale Agile development, allowing rapid response to changing market needs.

Fragmentation inside an operating system

Here’s the problem. Fragmentation is easy, but Integration is hard.

Fragmentation means that different teams or different customers pull the software in different directions. You end up, in the heat of development in a fast-moving market, with different branches, that are incompatible with each other, and which can’t be easily joined together. The result is that solutions created by developers for one of these branches, fail to work on the other branches. A great deal of time can be wasted debugging these issues.

Here, I speak from bitter experience. During the rapid growth days of Symbian, we lost control of aspects of compatibility in our own platform – despite our best efforts. For example, we had one version called 7.0s and another called 7.0, but lots of partners reported huge problems moving their solutions between these two versions. Because of resulting project delays, major phones failed to come to the market. It was a very painful period.

Nowadays, in the light of our battle-hardened experience, Symbian OS is a much more mature and stable platform, and it is surrounded and supported by an ecosystem of very capable partners. In my view, we have great disciplines in compatibility management and in codeline management.

The result is that we have much better control over the integration of our platform. That puts us in a better position to handle rapid change and multiple customer input. That means we can take good advantage of the creativity of open source, rather than being pulled apart by the diverse input of open source. Other platforms may find things harder. For them, open source may bring as many problems as it brings solutions.

Fragmentation across operating systems

This addresses the fragmentation inside a single operating system. But the problem remains of fragmentation across different operating systems.

Although competition can be healthy, too many operating systems result in developers spreading their skills too thinly. The mobile industry recognises that it needs to consolidate on a smaller number of mobile operating systems moving forwards. The general view is that there needs to be consolidation on around three or (at the most) four advanced mobile operating systems. Otherwise the whole industry ends up over-stretched.

So, which will be the winning mobile operating systems, over the next five years? In the end, it will come down to which phones are bought in large quantities by end users. In turn, the choices offered to end users are strongly influenced by decisions by phone manufacturers and network operators about which operating systems to prefer. These companies have four kinds of issues in their minds, which they want to see mobile operating systems solve:

• Technical issues, such as battery life, security, and performance, as well as rich functionality;


• Commercial issues, such as cost, and the ability to add value by differentiation;


• Political issues, in which there can be a perception that the future evolution of an operating system might be controlled by a company or organisation with divergent ulterior motivations;


• Reliability issues, such as a proven track record for incrementally delivering new functionality at high quality levels in accordance with a pre-published roadmap.

A time for operating systems to prove themselves

Again, which will be the winning operating systems, over the next five years? My answer is that is slightly too early to say for sure. The next 12-18 months will be a time of mobile operating systems proving themselves. Perhaps three or four operating systems will come through the challenge, and will attract greater and greater support, as customers stop hedging their bets. Others will be de-selected (or merged).

For at least some of the winning smartphone operating systems, there will be an even bigger prize, in the subsequent 2-3 years. Provided these operating systems are sufficiently scalable, they will become used in greater and greater portions of all phones (not just smartphones and high-end feature phones).

Proving time for the Symbian Foundation platform

Here’s how I expect the Symbian platform to prove itself in the next 12-18 months. Our move to open source was announced in June this year, and we said it could take up to two years to complete it. Since then, planning has been continuing, at great speed. Lee Williams, current head of the S60 organisation in Nokia, and formerly of Palm Inc and Be Inc, has been announced as the Executive Director of the Symbian Foundation.

The Foundation will make its first software release midway through the first half of 2009. Up till that point, access to internal Symbian OS source code is governed by our historical CustKit Licence and DevKit Licence. There’s a steep entry price, around 30,000 dollars per year, and a long contract to sign to gain access, so the community of platform developers has been relatively small. From the first Symbian Foundation release, that will change.

The source code will be released under two different licenses. Part will be open source, under the Eclipse Public Licence. This part has no licence fee, and is accessible to everyone. The other part will be community source, under an interim Symbian Foundation Licence. This is also royalty free, but there is a small contract that companies have to sign, and a small annual fee of 1,500 dollars. I expect a large community to take advantage of this.

This interim community source part will diminish, in stages, until it vanishes around the middle of 2010. By then, everything will be open source. We can’t get there quicker because there are 40 million lines of source code altogether, and we need to carry out various checks and cleanups and contract renegotiations first. But we’ll get there as quickly as we can.

There’s one other important difference worth highlighting. It goes back to the theme of reducing fragmentation. Historically, there have been three different UIs for Symbian OS: S60, UIQ, and MOAP(S) used in Japan. But going forwards, there will only be one UI system: S60, which is nowadays flexible enough to support the different kinds of user models for which the other UI systems were initially created.

To be clear, developers don’t have to wait until 2010 before experimenting with this software system. Software written to the current S60 SDK will run fine on these later releases. We’ll continue to make incremental compatible releases throughout this time period.

What you should also see over this period is that the number of independent contributors will increase. It won’t just be Nokia and Symbian employees who are making contributions. It will be like the example of the Eclipse Foundation, in which the bulk of contributions initially came from just one company, IBM, but nowadays there’s a much wider participation. So also for the Symbian Foundation, contributions will be welcome based on merit. And the governance of the Foundation will also be open and transparent.

The view from 2013

I’ll close by returning to the vision for 2013. Inside Symbian, we’ve long had the vision that Symbian OS will be the most widely used software platform on the planet. By adopting the best principles of open source, we expect we will fulfil this vision. We expect there will in due course be, not just 100s of millions, but billions of great devices, all running our software. And we’ll get there because we’ll be at the heart of what will be the most vibrant software ecosystem on the planet – the mobile innovation ecosystem. Thank you very much.

Open Source: necessary but not sufficient

Building the software system for complex smartphone products is one of the toughest engineering challenges on the planet. The amount of software in a high-end phone has been roughly doubling, each year, for the last ten years. In order to reap market success, smartphone software has to meet demanding requirements for performance, reliability, and usability. What’s more, to avoid missing a fast-moving market window, the software needs to pass through its integration process and reach maturity in a matter of months (not years). As I said, it’s a truly tough problem.

[Author's note: a version of this article is appearing in print today, to mark the first day of the Symbian Smartphone Show. I thought that people might like to read it online too.]

In broad terms, there are two ways in which a company can seek to solve this kind of tough problem:

1. Seek to keep careful control of the problem, and rely primarily on resources that are under the tight direction and supervision of the company;
2. Seek to take advantage of resources that are outside the control of the company.

The attraction of the first approach is that it’s easier to manage. The attraction of the second approach is that, in principle, the company can take better advantage of the potential innovation created by users and developers that are outside the company.

Writers and academics who study how innovation works in industry sometimes use the terms “closed innovation” and “open innovation” to describe these two approaches. In his pioneering book “Open innovation, the new imperative for creating and profiting from technology”, Henry Chesbrough lists the following contrasts between open innovation and closed innovation:

The “closed innovation” mindset:

1. The smart people in our field work for us
2. To profit from R&D we must discover it, develop it, and ship it ourselves
3. If we discover it ourselves, we will get to the market first
4. The company that gets an innovation to market first will win
5. If we create the most and the best ideas in the industry, we will win
6. We should control our IP, so that our competitors don’t profit from our ideas.

The “open innovation” mindset:

1. Not all the smart people work for us. We need to work with smart people inside and outside our company
2. External R&D can create significant value; internal R&D is needed to claim some portion of that value
3. We don’t have to originate the research to profit from it
4. Building a better business model is better than getting to market first
5. If we make the best use of internal and external ideas, we will win
6. We should profit from others’ use of our IP, and we should buy others’ IP whenever it advances our own business model.

In the modern world of hyper-complex products, easy communication via the Internet and other network systems, and the “Web 2.0” pro-collaboration zeitgeist, it is easy to understand why the idea of open innovation receives a lot of support. It sounds extremely attractive. However, the challenge is how to put these ideas into practice.

That’s where open source enters the picture. Open source removes both financial and contractual barriers that would otherwise prevent many users and external developers from experimenting with the system. For this reason, open source can boost open innovation.

However, in my view, there’s a lot more to successful open innovation than putting the underlying software platform into open source. We mustn’t fall into the trap of thinking that, because both these expressions start with the same adjective (“open”), the two expressions are essentially equivalent. They’re not.

Indeed, people who have studied open innovation have reached the conclusion that there are three keys to making open innovation work well for a firm (or platform):

• Maximising returns to internal innovation
• Incorporating external innovation in the platform
• Motivating a supply of external innovations.

Let’s dig more deeply into the second and third of these keys.

Incorporating external innovation in the platform

The challenge here isn’t just to stimulate external innovation. It is to be able to incorporate this innovation into the codelines forming the platform. That requires the platform itself to be both sufficiently flexible and sufficiently stable. Otherwise the innovation will fragment the platform, or degrade its ongoing evolution.

It also requires the existence of significant skills in platform integration. Innovations offered by users or external developers may well need to be re-engineered if they are to be incorporated in the platform in ways that meet the needs of the user community as a whole, rather than just the needs of the particular users who came up with the innovation in question.

• This can be summarised by saying that a platform needs skills and readiness for software codeline management, if it is to be able to productively incorporate external innovation.

Codeline management in turn depends on skills in:

• Codeline gate-keeping: not accepting code that fails agreed quality criteria – no matter how much political weight is carried by the people trying to submit that code
• Reliable and prompt code amalgamation: being quick to incorporate code that does meet the agreed criteria – rather than leaving these code submissions languishing too long in an in-queue
• API management, system architecture, and modular design – to avoid any spaghetti-like dependencies between different parts of the software
• Software refactoring – to be able to modify the internal design of a complex system, in the light of emerging new requirements, in order to preserve its modularity and flexibility – but without breaking external compatibility or losing roadmap momentum.

Motivating a supply of external innovations

The challenge here isn’t just to respond to external innovations when they arise. It is to give users and external developers sufficient motivation to work on their ideas for product improvement. These parties need to be encouraged to apply both inspiration and perspiration.

• Just as the answer to the previous issue is skilled software codeline management, the answer to this issue is skilled ecosystem management.

Ecosystem management involves a mix of education and evangelism. It also requires active listening (also known as “being open-minded”), and a willingness by the platform providers to occasionally tweak the underlying platform, in order to facilitate important innovations under consideration by external parties. Finally it requires ensuring that third parties can receive suitable rewards for their breakthroughs – whether moral, social, or financial. This involves the mindset of “growing the pie for mutual benefit” rather than the platform owner seeking to dominate the value for its own sake.

But neither software codeline management nor ecosystem management comes easy. Neither fall out of the sky, ready for action, just by virtue of a platform being open source. Nor can these skills be acquired overnight, by spending lots of money, or hiring lots of intrinsically smart people.

Conclusion: On account of a legacy of more than ten years of trial and error in building and enhancing both a mobile platform and an associated dynamic ecosystem, the Symbian Foundation should come into existence with huge amounts of battle-hardened expertise in both software codeline management and ecosystem management. On that basis, I expect the additional benefits of open source will catalyse a significant surge of additional open innovation around the Symbian Platform. In contrast, other mobile platforms that lack this depth of experience are likely to find that open source brings them grief as much as it brings them potential new innovations. For these platforms, open source may result in divisive fragmentation and a dilution of ecosystem effort.

Footnote: For more insight about open innovation, I recommend the writings of Henry Chesbrough (mentioned above), Wim Vanhaverbeke, and Joel West.

Open source religion

Roger Nolan, my long-time former colleague from both Psion and Symbian, raised some challenging points in his recent piece "Symbian’s open source challenge" on the VisionMobile blog. As Roger sees it, the challenge for Symbian is to get the best out of open source without becoming so fixated by the idea of open source that we fail to address the burning requirement for improved user experience. The worry is that technological or process considerations will get in the way of creating simply delightful products.

In Roger's own words - comparing the possible future evolution of Symbian software with the history of Nokia's Linux-based "maemo" platform for mobile Internet tablets:

Sadly Maemo is ... driven from a technology soapbox. This time, it’s not a features arms race, it’s open-source-or-die. The Maemo team did not sit down and say “Let’s build a great UI for an internet tablet” they sat down and said “What can we do with open source” - open source is the religion, not ease of use and making great devices that are delightful to use.

As Symbian becomes the Symbian foundation and transitions to an open source model, I hope that the open source community will take some of the burden of implementing every last codec and piece of middle-ware and the Symbian foundation can focus on UIs and ease of use. Unfortunately, I fear that they will be overcome following Maemo’s open-source religion.

In other words, is Symbian going on an free software crusade, or are we adopting open source for solidly pragmatic reasons?

My answer is that it's a bit of both, but with a strong emphasis on the pragmatic side of the scale.
The archetypal free software crusader, of course, is Richard Stallman. The 2002 book "Free as in Freedom" by Sam Williams is a sympathetic, interesting and easy-to-read account of Stallman and his very considerable impact on the world of software - but it's no hagiography.

The early chapters in the book take a friendly approach to Stallman's personal idiosyncracies. Reading these chapters, it's easy to develop a strong liking for this pioneering crusader. To my surprise, I found a lot of resonance between Stallman's life experiences and, in smaller scale, my own; for example, we share backgrounds as prodigious mathematicians who were not afraid to be outsiders. (And it seems we're both interested in life extension.)

The last few chapters provide a kind of balance, by highlighting some of the problems caused within the Free and Open Source movements by Stallman's inflexibility, apparent micro-management, and under-developed project management skills.

The narrative in the book jumps around a lot, moving backwards and forwards in time all over the place. Some readers may find that distracting, but I liked it, since it helps to show the remarkable wholeness and integrity to Stallman's conceptions.

The entire text of this book is available online at http://www.faifzilla.org/. Chapter 8 contains a stark example of the clash between the "quasi-religious" approach and the pragmatic one:

Stallman says competitive performance and price, two areas where free software operating systems such as GNU/Linux and FreeBSD already hold a distinct advantage over their proprietary counterparts, are red herrings compared to the large issues of user and developer freedom.

"It's not because we don't have the talent to make better software," says Stallman. "It's because we don't have the right. Somebody has prohibited us from serving the public. So what's going to happen when users encounter these gaps in free software? Well, if they have been persuaded by the open source movement that these freedoms are good because they lead to more-powerful reliable software, they're likely to say, 'You didn't deliver what you promised. This software's not more powerful. It's missing this feature. You lied to me.' But if they have come to agree with the free software movement, that the freedom is important in itself, then they will say, 'How dare those people stop me from having this feature and my freedom too.' ..."

...the underlying logic of Stallman's argument - that open source advocates emphasize the utilitarian advantages of free software over the political advantages - remains uncontested. Rather than stress the political significance of free software programs, open source advocates have chosen to stress the engineering integrity of the hacker development model. Citing the power of peer review, the open source argument paints programs such as GNU/Linux or FreeBSD as better built, better inspected and, by extension, more trustworthy to the average user...

When an audience member asks if, in shunning proprietary software, free software proponents lose the ability to keep up with the latest technological advancements, Stallman answers the question in terms of his own personal beliefs. "I think that freedom is more important than mere technical advance," he says. "I would always choose a less advanced free program rather than a more advanced nonfree program, because I won't give up my freedom for something like that. My rule is, if I can't share it with you, I won't take it."

Such answers, however, reinforce the quasi-religious nature of the Stallman message. Like a Jew keeping kosher or a Mormon refusing to drink alcohol, Stallman paints his decision to use free software in the place of proprietary in the color of tradition and personal belief. As software evangelists go, Stallman avoids forcing those beliefs down listeners' throats. Then again, a listener rarely leaves a Stallman speech not knowing where the true path to software righteousness lies.

Now the nearest thing to a Symbian religion is the published list of our corporate values: Excellence, Innovation, Passion, Integrity, Collaboration, People. We take these values very seriously - as we do our vision - that Symbian OS will be the most widely used software platform on the planet.

Questions such as the extent of our adoption of open source, or usage of proprietary software, are, in the end, weighed up against that list of values. Open source will lead, we believe, to greater collaboration, and to more innovation. That's a good reason to support it. But it's not an end in itself.

Indeed, adopting selected open source principles is only one of the big change initiatives that are taking place in Symbian. As I've mentioned before, we're also adopting selected principles of enterprise agile. What's more, we're looking forward to significantly closer inter-working between the development teams in Symbian and in S60, which will allow faster delivery of important new technology to the market. And last - but definitely not least - there's a whole series of measures to enable improved user experience on Symbian-powered phones. The UI that's on the just-announced Nokia 5800 XpressMusic device is a significant step in this direction.

Open source coexistence with marvellous non-free add-ons

"Has Symbian thought open source through?" That's the question David Meyer of ZDNet asks this morning. David explains the context of his question:

Last week I visited Symbian's labs here in London. The assembled hacks were shown some very interesting stuff, such as what could be done with a quad core mobile chipset... There was also some cool stuff with mobile-based audio EQ, which always pleases me.

We also got shown some of the fruits of Symbian's work with Scalado on the graphics front. The engineer demonstrated very quick loading of and zooming into a 21-megapixel picture, which was very impressive but raised ... unanswered questions: ... what precisely is to happen with this valuable work once Symbian goes open source?

Given that [going open source] will involve stripping out all the third-party, proprietary stuff that can't go open source, why is Symbian still bothering with such partnerships?

Here's my answer. First, I'm sure that there are aspects of going open source which we in Symbian have yet to think through properly. Moving some 400,000 files of source code into open source is bound to pose a whole host of unexpected problems. However, this particular question is one that has received considerable thought.

The highly impressive Scalado mobile imaging software which Symbian licensed earlier this year is only one of a large number of add-on or plug-in solutions which are available, either as part of Symbian OS itself, or as a pre-integrated supplementary solution. For obvious reasons, I won't say anything more about Scalado, but I'll address the general question of an add-on solution A from vendor V, which may be included in a phone created by customer C of Symbian. Suppose that A is currently subject to a license fee F, which is payable:

• Either from C direct to V,
• Or from Symbian to V, with the costs in this case currently being covered as part of the Symbian OS licence fee paid by C to Symbian.

So what happens to this licence fee F once the Symbian platform becomes open source, and there's no longer any licence fee for Symbian platform?

It turns out there are quite a few options available.

For example, the Symbian platform may exclude A, but may instead include a more basic version A0. This will be good enough for many purposes - and will allow customers to build many kinds of successful phones. But customers who want particularly responsive or feature-rich behaviour in the area covered by A will be able to pay fee F directly to V, and will apply A in place of A0 in their phones. So long as the code for A is independent of the Symbian platform code for A0 (in legal terms, so long as A is not a derivative work of A0), there's no obligation on V to licence their code using the EPL applicable to the Symbian platform itself. That is, they won't need to make their source code available.

Is this somehow at variance with the motivation of Symbian in creating an open source platform? It depends what you think the primary motivation is for this move. If you think that motivation is to drive out all cost from phones, you may be surprised by this option. However, once you realise that the main drivers are actually to lower barriers of entry and experimentation, to boost innovation, to deepen collaboration, to raise quality, and to accelerate time-to-market, you won't be so surprised. Open source does not imply low-value! And nor does open source imply that anything which builds on top of it, needs to be zero cost.

Another option is that vendor V will make A available royalty-free as part of the open source platform, but will earn revenues:

• From consultancy work in the area of A
• Or, from making available a chargeable new version A1 that provides even better performance and/or new features.

In short, there will be plenty of ways for creative partner companies to continue to earn handsome income from their add-on and plug-in solutions to Symbian platform software.

I'll close by returning to the last part of the initial question: "...why is Symbian still bothering with such partnerships?" It's because these partnerships collectively generate a huge quantity of impressive add-on and plug-in solutions, which allow our customers to customise and optimise their phones in numerous ways. And that's good for everyone.

The practicalities of producing open source software

I've been reading books and articles about open source software since at least June 1998 - the first time the famous phrase "The cathedral and the bazaar" (*) made an appearance in Symbian's principal internal discussion database (which was, at the time, still called "Psion Software General"). However, I remain keen to keep on testing out my thinking and understanding of open source issues.

After all, there are many different angles to this subject. There's potential huge upside to taking full advantage of the best principles of open source methods - but there's also many risks from applying some of these ideas in a misguided manner.

For that reason, I continue to pick up books on open source software and think hard about what they say. Recently, I accepted the advice of several of my Symbian colleagues, and started reading "Producing open source software" by Karl Fogel.

I'm very pleased that I listened to that advice. In my view, this book is in a class of its own.

It has the great merit of being an intensely practical book. It's clear from numerous examples in the book that the author has extensive real-world experience at the heart of development teams of open source projects that are both significant and successful - including CVS and Subversion.

Some of the chapters contain ideas that have been covered before - like the history of free and open source, advice on how to choose a licence, and aspects of the technical infrastucture of open source projects. However, other chapters delve into material that (to my knowledge) has been covered much less often:

• Social and political infrastructure of successful open source projects;
• "Money": Working in open source projects in which companies are sponsoring parts of the work;
• "Money can't buy you love" - excellent advice on how to avoid corporate sponsorship dampening the enthusiasm of volunteer contributors;
• Communications - including how to communicate with "difficult people";
• Packaging, releasing, and daily development - including dealing with different kinds of codeline, and the special considerations that apply to integration and to making releases;
• Managing volunteers- including the typical roles that probably need to be filled in projects.

Several times while reading a chapter, I found myself thinking: "Yes, this is highly practical material - and interesting too. But I can see there's still another 20+ pages in this chapter. What else is there to say about this subject?" But then as I turned over more pages, I thought, "Oh yes, this is something that really belongs here too - it's what happens in real projects!"

The writing style was pleasant and clear throughout - with an engaging mix of actual examples (both good and bad) and a discussion of the broader lessons to be drawn from these examples.

My recommendation is that project teams should regard this book as a kind of "bible" - it's something that should be regularly dipped into, and the many salient points shared and debated in group discussion. The lessons will make sense on several levels - some apply during early phases of projects, and others as the project becomes more complex.

(Many of the same principles apply in non-open source projects too, by the way! I felt lots of resonance with my own observations over the years about successful software projects inside the "community source" world of Symbian OS development.)

Perhaps the most sobering part of the book is contained in its introduction:

Most free projects fail...

... it's impossible to put a precise number on the failure rate. But anecdotal evidence from over a decade in open source, some casting around on SourceForge.net, and a little Googling all point to the same conclusion: the rate is extremely high, probably on the order of 90–95%. The number climbs higher if you include surviving but dysfunctional projects: those which are producing running code, but which are not pleasant places to be, or are not making progress as quickly or as dependably as they could.

Happily, the introduction continues:

This book is about avoiding failure. It examines not only how to do things right, but how to do them wrong, so you can recognize and correct problems early...

If that whets your appetite, note that you can read the entire book online, by following the links above. Alternatively, check out the reviews on Amazon.com.

(*) Footnote: The June 1998 "Psion Software General" discussion contained a link that is, sadly, now dead. I'd like to belatedly thank Symbian lead Software Engineer Joe Branton for coming to my room, on several occasions, and encouraging me to pay more attention to the ideas in The Cathedral and the Bazaar.

Restrictions on the suitability of open source?

Are there restrictions on the suitability of open source methods? Are there kinds of software for which closed source development methods are inherently preferable and inherently more likely to succeed?

These questions are part of a recent discussion triggered by Nokia's Ari Jaaksi's posting "Different ways and paradigms" that looked for reasons why various open source software development methods might be applicable to some kinds of project, but not to others. As Ari asks,

"Why would somebody choose a specific box [set of methods] for their products?"

One respondent suggested that software with high security and high quality criteria should be developed using closed source methods rather than using open source.

Another stated that,

I firmly believe 'closed' source is best route for targeting consumers and gaining mass appeal/ acceptance.

That brings me back to the question I started with. Are there features of product development - perhaps involving security and robustness, or perhaps involving the kinds of usability that are important to mainstream consumers - to which open source methods aren't suited?

Before answering that, I have a quick aside. I don't believe that open source is ever a kind of magic dust that can transform a failing project into a successful project. Adopting open source, by itself, is never a guarantee of success. As Karl Fogel says in the very first sentence of Chapter 1 in his very fine book "Producing open source software: how to run a successful free software project",

"Most free projects fail."

Instead, you need to have other project fundamentals right, before open source is likely to work for you. (And as an aside to an aside, I believe that several of the current attempts to create mobile phone software systems using open source methods will fail.)

But the situation I'm talking about is when other project fundamentals are right. In that case, my question becomes:

Are there types of software for which an open source approach will be at odds with the other software disciplines and skills (eg security, robustness, usability...) that are required for success in that arena.

In one way, the answer is trivial. The example of Firefox resolves the debate (at least for some parameters). Firefox shows that open source methods can produce software that scores well on security, robustness, and usability.

But might Firefox be a kind of unusual exception - or (as one of the anonymous respondents to Ari Jaaksi's blog put it) "an outlier?" Alternatively - as I myself believe - is Firefox an example of a new trend, rather than an irrelevant outlier to a more persistent trend?

Regarding usability, it's undeniable that open source software methods grew up in environments in which developers didn't put a high priority on ease-of-use by consumers. These developers were generally writing software for techies and other developers. So lots of open source software has indeed scored relatively poorly, historically, on usability.

But history needn't determine the future. I'm impressed by the analysis in the fine paper "Usability and Open Source Software" by David M. Nichols and Michael B. Twidale. Here's the abstract:

Open source communities have successfully developed many pieces of software although most computer users only use proprietary applications. The usability of open source software is often regarded as one reason for this limited distribution. In this paper we review the existing evidence of the usability of open source software and discuss how the characteristics of open-source development influence usability. We describe how existing human-computer interaction techniques can be used to leverage distributed networked communities, of developers and users, to address issues of usability.

Another very interesting paper, in similar vein, is "Why Free Software has poor usability, and how to improve it" by Matthew Paul Thomas. This paper lists no less than 15 features of open source culture which tend to adversely impact the usability of software created by that culture:

1. Weak incentives for usability
2. Few good designers
3. Design suggestions often aren’t invited or welcomed
4. Usability is hard to measure
5. Coding before design
6. Too many cooks
7. Chasing tail-lights
8. Scratching their own itch
9. Leaving little things broken
10. Placating people with options
11. Fifteen pixels of fame
12. Design is high-bandwidth, the Net is low-bandwidth
13. Release early, release often, get stuck
14. Mediocrity through modularity
15. Gated development communities.

As Paul says, "That’s a long list of problems, but I think they’re all solvable". I agree. The solutions Paul gives in his article are good starting points (and are already being adopted in some projects). In any case, many of the same problems impact closed-source development too.

In short, once usability issues are sufficiently understood by a group of developers (whether they are adopting open source or closed source methods), there's no inherent reason why the software they create has to embody poor usability.

So much for usability. How about security? Here the situation may be a little more complex. The online book chapter "Is Open Source Good for Security?" by David Wheeler is one good starting point. Here's the final sentence in that chapter:

...the effect on security of open source software is still a major debate in the security community, though a large number of prominent experts believe that it has great potential to be more secure

The complication is that, if you start out with software that is closed source, and then make it open source, you might get the worst of both worlds. Incidentally, that's one reason why the source code in the Symbian Platform isn't being open-sourced in its entirety, overnight, on the formation (subject to regulatory approval) of the Symbian Foundation. It will take some time (and the exercise of a lot of deep skill), before we can be sure we're going to get the best of both worlds, rather than the worst of both worlds.

Intellectual property and open source

I've just finished reading a third book, within two months, on the topic of open source licensing. The three books are:

1. Heather Meeker's "The Open Source Alternative: Understanding Risks and Leveraging Opportunities" - which I reviewed here;
2. Lawrence Rosen's "Open Source Licensing: software freedom and intellectual property law" - which I reviewed here;
3. Van Lindberg's "Intellectual property and open source: a practical guide to protecting code".

My headline summary is that all three books are well worth reading. They overlap to an extent, but they come at their shared subject from very different viewpoints, so each book has lots of good material that you won't find in the others.

Van Lindberg targets his book at software engineers. He uses many analogies between legal concepts and deeply technical software engineering concepts. For example (to give a flavour of many of the clever pieces of writing in the book):

"One way to think about private goods is to analogize them to locks or mutexes in a multithreaded program. A number of different threads may want to use a protected resource, but control of the lock around the resource is rivalrous..."

Somewhat unexpectedly, the first half of the book hardly mentions open source. There's good reason for this. The first seven chapters of the book cover the basic principles of intellectual property (IP), including patents, copyrights, trademarks, trade secrets, licences, and contracts. I found the very first chapter to be particularly engrossing, as it set out the philosophical foundations for IP. Van Lindberg highlighted the utilitarian justification for IP, in terms of legal measures to counter what would otherwise be two sorts of market failures:

• "The cost of creating knowledge is high, but the cost of consuming it is low.... Therefore there is a societal incentive to not create as much knowledge as we would ideally like to have" (hence the utilitarian rationale for copyright)
• "Secrets are more valuable to you personally, but shared knowledge is more valuable to society.... The resource is valuable to you because you have a key, but it is worthless to everyone else" (hence the utilitarian rationale for patents).

As I said, the very first chapter was particularly engrossing, but I thought the other early chapters dragged a bit. Although all the material was interesting, there were rather too many details for my liking.

Chapter eight ("The economic and legal foundations of open source software") went back to philosophical principles, in an attempt to pinpoint what makes open source different from proprietary software. The difference, according to Van Lindberg, is that:

• Proprietary software is driven by corporate business goals (which inevitably involve profit-maximisation, and therefore - he claimed - a tension between what's best for the customers and what's best for the shareholders)
• Open source software is driven by cooperative goals, in which the goals of the customers have primacy. (Note the difference between the similar-looking words corporate and cooperative.)

This chapter also runs a pretty compelling extended comparison between proprietary software and open source software, on the one hand, and banks and credit unions, on the other hand. Again, the first member of each pair is driven by shareholder goals, whereas the second member of each pair is driven by customer goals (the legal owners are the same people as the customers).

The primary task of open source licences, according to this analysis, is to support cooperation. In more detail, Van Lindberg says that open source licences are intended to solve the "Programmer's Dilemma" version of the famous and well-known "Prisoner's Dilemma" problem from game theory:

"Open source licences serve two functions in a game-theoretic context. First, they allow programmers to signal their cooperative intentions to each other. By placing their code under a licence that allows cooperation, programmers indicate to their peers that they are willing to participate in a cooperative solution. Second... licences are based in copyright law, which allows the original developer to dictate (to some extent) the users and uses of his code. The legal penalties associated with copyright violations change the decision matrix for other programmers, leading to a stable cooperative (and optimal) solution."

This (like everything else in the book) is thought-provoking. But I'm not fully convinced. I think this puts too much importance onto the licence aspect of open source. Yes, picking a good licence is important - but it's insufficient to guarantee the kind of cooperative behaviour that will make an open source project a real success. And as I've argued elsewhere, picking the right licence is no guarantee against the software fragmenting. But despite this quibble, I still think the ideas in this chapter deserve wide readership.

The second half of the book changes gear. With the first eight chapters having carefully outlined the underlying legal framework, the remaining six chapters walk through the kind of real-life IP concerns that will face someone (whether an individual developer, or a company) who wants to become involved in an open source project:

• Issues with standard employment contracts that probably specify that everything you work on - even in your spare time - belongs to your company, and which you therefore are not free to assign to an open source project
• General guidelines on choosing between some of the more popular open source licences
• Legal complications over how to accept patches and other contributions, from outsiders, into your project
• Particular issues with the GPL
• Reverse engineering
• Creating a non-profit organisation or foundation (recommended if your project becomes larger).

There's lots of good advice here. Every chapter of this part of the book has important material - but I was slightly disappointed with some parts. For example, given the careful attention to patents in the first half of the book (where two chapters were devoted to this topic), I was expecting more analysis of how some of the major open source licences differ in their approach to patent licences and patent retaliation clauses. On reflection, that's something that the other two books (ie by Meeker and Rosen) handle better.

The chapter on the issues with the GPL confirmed and extended the opinion about that licence which I'd picked up from my previous reading: the interpretation of the GPL is subject to great uncertainty over ambiguities. The chapter includes a lengthy "Questions and answers" section, to which the answer to nearly every question is "Maybe" or "It depends". (Apart from the last question, which is "Can I depend on the answers in this Q&A to keep me out of trouble?"; the answer to this is "No, this is our best understanding of copyright law as it stands right now, but it could change tomorrow - and nobody really knows...")

Giving more evidence for this view of the ambiguities surrounding the GPL, Van Lindberg mentions an essay by Matt Asay, "A Funny Thing Happened on the Way to the Market". Here's an extract from that essay:

"I asked two prominent representatives of the Free Software Foundation – Eben Moglen, general counsel, and Richard Stallman, founder – to clarify thorny issues of linkage to GPL code, and came up with two divergent opinions on derivative works in specific contexts..."

"...it is telling how widely their responses diverge – there appear to be no definitive answers to the question of what constitutes a derivative work under the GPL, not even from the holders of the licenses in question."

This looks decisive, but it could be argued that this quote from Matt Asay is itself misleading, since Matt's article goes on to state that:

"Fortunately, as I will detail below, this issue has largely gone away, as it has become accepted practice to dynamically link to GPL code [without that code becoming part of the GPL program]. Linus Torvalds helped to build momentum for such a reading of the GPL. While some argue that kernel modules, including device drivers, must be GPL, Torvalds has stated: This [GPL] copyright does *not* cover user programs that use kernel services by normal system calls – this is merely considered normal use of the kernel, and does *not* fall under the heading of 'derived work.'"

However, Van Lindberg seems to be right that the official FAQ about the GPL, maintained by the Free Software Foundation, advocates a stricter interpretation:

"Q: Can I release a non-free program that's designed to load a GPL-covered plug-in?

"A: It depends on how the program invokes its plug-ins. For instance, if the program uses only simple fork and exec to invoke and communicate with plug-ins, then the plug-ins are separate programs, so the license of the plug-in makes no requirements about the main program.

"If the program dynamically links plug-ins, and they make function calls to each other and share data structures, we believe they form a single program, which must be treated as an extension of both the main program and the plug-ins. In order to use the GPL-covered plug-ins, the main program must be released under the GPL or a GPL-compatible free software license, and that the terms of the GPL must be followed when the main program is distributed for use with these plug-ins.

"If the program dynamically links plug-ins, but the communication between them is limited to invoking the ‘main’ function of the plug-in with some options and waiting for it to return, that is a borderline case.

"Using shared memory to communicate with complex data structures is pretty much equivalent to dynamic linking."

Do these ambiguities over the GPL really matter? It's hard to be sure, but I'm personally glad that the Symbian Foundation plans to adopt a licence - the EPL - which avoids these issues.

I'm also glad to have taken the time to read this book - it's helped my understanding grow, in many ways.

Footnote: My thanks go to Moore Nebraska for drawing my attention to the Van Lindberg book.

There’s more to Open Innovation than Open Source

Here’s the challenge: How best to capitalise on the potential innovation that could in theory be created by users and developers who are based outside of the companies that are centrally responsible for a product platform?

This is the question of how best to make Open Innovation work. Recall the following contrasts between Open Innovation and so-called Closed Innovation - taken from the pioneering book by Henry Chesbrough, “Open innovation: the new imperative for creating and profiting from technology”:

The “closed innovation” mindset:

1. The smart people in our field work for us
2. To profit from R&D we must discover it, develop it, and ship it ourselves
3. If we discover it ourselves, we will get to the market first
4. The company that gets an innovation to market first will win
5. If we create the most and the best ideas in the industry, we will win
6. We should control our IP, so that our competitors don't profit from our ideas.

The “open innovation” mindset:

1. Not all the smart people work for us. We need to work with smart people inside and outside our company
2. External R&D can create significant value; internal R&D is needed to claim some portion of that value
3. We don't have to originate the research to profit from it
4. Building a better business model is better than getting to market first
5. If we make the best use of internal and external ideas, we will win
6. We should profit from others' use of our IP, and we should buy others' IP whenever it advances our own business model.

In the modern world of hyper-complex products, easy communication via the Internet and other network systems, and the “Web 2.0” pro-collaboration zeitgeist, it is easy to understand why the idea of Open Innovation receives a lot of support. The challenge, as I said, is how to put these ideas into practice.

It’s tempting to answer that the principal key to successful Open Innovation is Open Source. After all, Open Source removes both financial and contractual barriers that would otherwise prevent many users and external developers from experimenting with the system. (What’s more, “Open Innovation” and “Open Source” share the prefix “Open”!)

However, in my view, there’s a lot more to successful Open Innovation than putting the underlying software platform into Open Source.

To see this, it’s useful to review some ideas from the handy summary presentation by leading Open Innovation researcher Joel West, “Managing Open Innovation through online communities”. Joel makes it clear that there are three keys to making Open Innovation work best for a firm (or platform):

1. Maximising returns to internal innovation
2. Incorporating external innovation in the [platform]
3. Motivating a supply of external innovations.

Let's dig more deeply into the second and third of these keys.

Incorporating external innovation in the platform

The challenge here isn’t just to stimulate external innovation. It is to be able to incorporate this innovation into the platform. That requires the platform itself to be both sufficiently flexible and sufficiently stable. Otherwise the innovation will fragment the platform, or degrade its ongoing evolution.

It also requires the existence of significant skills in platform integration. Innovations offered by users or external developers may well need to be re-engineered if they are to be incorporated in the platform in ways that meet the needs of the user community as a whole, rather than just the needs of the particular users who came up with the innovation in question.

• This can be summarised by saying that a platform needs skills and readiness for software management, if it is to be able to productively incorporate external innovation.

Motivating a supply of external innovations

The challenge here isn’t just to respond to external innovations when they arise. It is to give users and external developers sufficient motivation to work on their ideas for product improvement. These parties need to be encouraged to apply both inspiration and perspiration.

• Just as the answer to the previous issue is software management, the answer to this issue is ecosystem management.

But neither software management nor ecosystem management comes easy. Neither fall out of the sky, ready for action, just by virtue of a platform being Open Source. Nor can these skills be acquired overnight, by spending lots of money, or hiring lots of intrinsically smart people.

Ecosystem management involves a mix of education and evangelism. It also requires active listening, and a willingness by the platform providers to occasionally tweak the underlying platform, in order to facilitate important innovations under consideration by external parties. Finally it requires ensuring that third parties can receive suitable rewards for their breakthroughs – whether moral, social, or financial.

Conclusion: On account of a legacy of more than ten years of trial and error in building and enhancing both a mobile platform and an associated dynamic ecosystem, the Symbian Foundation will come into existence with huge amounts of battle-hardened expertise in both software management and ecosystem management. On that basis, I expect the additional benefits of Open Source will catalyse a dramatic surge of additional Open Innovation around the Symbian Platform. In contrast, other mobile platforms that lack this depth of experience are likely to find that Open Source brings them grief as much as it brings them potential new innovations.

Understanding Open Source Licensing

"What's the best book to read for an introduction to Open Source?"

I've already given one set of answers to this question, in my article, "Clear thinking about open source". One reply to that article - from Joel West, a writer and researcher on Open Innovation and Open Source whose advice I value - urged me to include one more book in my reading list: Lawrence Rosen's "Open Source Licensing: software freedom and intellectual property law". This weekend I've finished reading it. And indeed, I do now endorse it as being clearly written yet also highly insightful.

Initally, I tended to shy away from this book, instead preferring the book by Heather Meeker that I covered in my earlier article. Both books focus on open source licensing issues, but Meeker's was published this year, whereas Rosen's dates from 2004. So Rosen's book makes no mention of GPL v3, or Sun's experience with open-sourcing Java, or even the Eclipse Public License (EPL) which the Symbian Foundation is likely to adopt. That makes Rosen's book appear out of date. However, I realised that one license which the book does cover (comprehensively) is the Common Public License (CPL) which is the precursor of the EPL and which differs from the EPL in very few places. Reassured, I dipped into the book - and then could hardly put it down.

In summary, I now recommend both the Meeker book and the Rosen book for their coverage of open source licensing. They complement each other nicely. There's a bit of overlap, but also lots of good material in each book that you won't find in the other.

Specifically, here are a few of the "aha"s or other learnings I took away from Rosen's book:

1.) The ten principles of the Open Source Definition are actually quite hard to understand in places (this comment came as a relief to me, since I had been thinking the same thing).

2.) Patents and Copyrights should be approached as parallel sets of legal principle - the former applicable to ideas, and the latter to expressions of ideas. That's a far better approach than initially just thinking about Copyrights, and then trying to squeeze in considerations about Patents at the end.

3.) One of the key differences between different open source licenses is in the treatment of patent licenses - and in the different circumstances in which patent licenses (and/or copyright licenses) can be withdrawn in the wake of various kinds of patent infringement suits. There's a tricky balance that has to be drawn between the needs of both licensor and licensee concerning the continuing value of their respective patent portfolios.

4.) One piece of license evolution covered in the book - the difference between v2.0 and v2.1 of the Open Software License (OSL) - closely mirrors the principal difference between the CPL and the EPL: it's a reduction in the circumstances in which a patent license can be withdrawn when a licensee brings a separate patent infringement case against the licensor.

5.) The insistence in GPL v2 about not being compatible with other licenses that introduce additional restrictions (even restrictions that the initial drafters of GPL v2 had not considered), is a real drawback of that license, since it unnecessarily hinders aggregation of code written under similar but different licenses. (Possible restrictions that have emerged more recently include provisions for defence against patent infringement lawsuits or to protect the licensor's trademarks.)

6.) "... sections of the LGPL are an inpenetrable maze of technological babble. They should not be in a general purpose software license." (page 124)

7.) Disclaimers of liability that are generally written into open source licenses may be overridden by general consumer legislation. Recognising this, the CPL (and hence the EPL) introduces a clause that allocates particular responsibility to "commercial contributors" to defend and indemnify all other contributors against losses, damages, or costs.

8.) One possible way for a company to make money from software is via the mechanism Rosen calls "Eventual Source": code is released as open source after some delay period, but recipients can elect to pay an early access license fee to be able to work with the code (under a non-open source license) ahead of its release as open source.

I've still got lots of questions about open source licensing (for example, about the prospects for wider adoption of GPL v3, and about how successful Rosen's own preferred OSL is likely to be in the longer run). I'll be attending the Open Source in Mobile conference in Berlin in September, when I hope to find out more answers! (And no doubt there will be new questions too...)