Why can't we all just get along?

Blogger Tomaž Štolfa asks me, in a comment to one of my previous posts,

I am also wondering why you are not trying to explore a non-os specific scenario?

Developers and service designers do not want to be bound to a single platform when developing a service for the masses. So it would make much more sense to se a bright future with cross-platform standards set by an independent party (W3C?).

If the industry will not agree on standards quickly enough Adobe (or some other company) will provide their own.

It's a good question. I'm actually a huge fan of multi-platform standards. Here's just a few of many examples:

• Symbian included an implementation of Java way back in v4 of Symbian OS (except that the OS was called "EPOC Release 4" at the time);
• Symbian was a founder member of the Open Mobile Alliance - and I personally served twice on the OMA Board of Directors;
• I have high hopes for initiatives such as OMTP's BONDI that is seeking to extend the usefulness of web methods on mobile devices.

Another example of a programming method that can be applied on several different mobile operating systems is Microsoft's .NET compact framework. Take a look at this recent Microsoft TechEd video in which Andy Wigley of Appa Mundi interviews Mike Welham, CTO of Red Five Labs, about the Red Five Labs Net60 solution that allows compact framework applications to run, not only on Windows Mobile, but also on S60 devices.

There's no doubt in my mind that, over time, some of these intermediate platforms will become more and more powerful - and more and more useful. The industry will see increasing benefits from agreeing and championing fit-for-purpose standards for application environments.

But there's a catch. The catch applies, not to the domain of add-on after market solutions, but to the domain of device creation.

Lots of the software involved in device creation cannot be written in these intermediate platforms. Instead, native programming is required - and involves exposure to the underlying operating system. That's when the inconsistencies at the level of native operating systems become more significant:

• Differences between clearly different operating systems (eg Linux vs. Windows Mobile vs. Symbian OS);
• Differences between different headline versions of the same operating system (eg Symbian OS v8 vs. Symbian OS v9);
• Differences between different flavours of the same operating system, evolved by different customers (eg Symbian OS v7.0 vs. Symbian OS v7.0s);
• Differences between different customisations of the same operating system, etc, etc.

(Note: I've used Symbian OS for most of these examples, but it's no secret that the Mobile Linux world has considerably more internal fragmentation than Symbian. The integration delays in that world are at least as bad.)

From my own experience, I've seen many device creation projects very significantly delayed as a result of software developers encountering nasty subtle differences between the native operating systems on different devices. Product quality suffered as a result of these project schedule slips. The first loser was the customer, on encountering defects or a poor user experience. The second loser was the phone manufacturer.

This is a vexed problem that cannot be solved simply by developing better multi-os standard programming environments. Instead, I see the following as needed:

1. Improved software development tools, that alert systems integrators more quickly to the likely causes of unexpected instability or poor performance on phones (including those problems which have their roots in unexpected differences in system behaviour); along this line, Symbian has recently seen improvements in our own projects from uses of the visual tools included in the Symbian Analysis Workbench;
2. A restructuring of the code that runs on the device in order to allow more of that code to be written in standard managed code environments - Symbian's new Freeview architecture for networking IP is one step in this direction;
3. Where possible, APIs used by aspects of the different native operating systems should become more and more similar - for example, I like to imagine that, one day, the same device driver will be able to run on more than one native operating system
4. And, to be frank, we need fewer native operating systems; this is a problem that will be solved over the next couple of years as the industry gains more confidence in the overall goodness of a small number of the many existing mobile operating systems.

The question of technical fragmentation is, of course, only one cause of needless extra effort having to be exerted within the mobile industry. Another big cause is that different players in the value chain are constantly facing temptation to try to grab elements of value from adjacent players. Hence, for example, the constant tension between network operators and phone manufacturers.

Some elements of this tension are healthy. But, just as for the question of technical fragmentation, my judgement is that the balance is considerably too far over to the "compete" side of the spectrum rather than the "cooperate" side.

That's the topic I was discussing a few months back with Adam Shaw, one of the conference producers from Informa, who was seeking ideas for panels for the "MAPOS '08" event that will be taking place 9-10 December in London. Out of this conversation, Adam came up with the provocative panel title, "Can’t We All Just Get Along? Cooperation between operators and suppliers". Here's hoping for a constructive dialog!

New mobile OSes mean development nightmares

Over on TechRadar, Dan Grabham has commented on one of the themes from Monday's Future of Mobile event in the Great Hall in High Street Kensington, London:

The increase in mobile platforms caused by the advent of the Apple iPhone and Google's Android are posing greater challenges for those who develop for mobile. That was one of the main underlying themes of this week's Future of Mobile conference in London.

Tom Hume, Managing Director of developer Future Platforms, picked up on this theme, saying that from a development point of view things were more fragmented. "It's clear that it's an issue for the industry. I think it's actually got worse in the last year or so."

Indeed, many of the panellists representing the major OS vendors said that they expected some kind of consolidation over the coming years as completion in the mobile market becomes ever fiercer.

The theme of collaboration vs. competition was one that I covered in my own opening remarks on this panel. Before the conference, the panel chairman, Simon Rockman of Sony Ericsson, had asked the panellists to prepare a five minute intro. I'll end this posting with a copy of what I prepared.

Before that, however, I have another comment on the event. One thing that struck me was the candid comments from many of the participants about the dreadful user experience that mobile phones deliver. So the mobile industry has no grounds for feeling pleased with itself! This was particularly emphasised during the rapid-fire "bloggers 6x6 panel", which you can read more about from Helen Keegan's posting - provocatively entitled "There is no future of mobile". By the way, Helen was one of the more restrained of that panel!

So, back to my own remarks - where I intended to emphasise that, indeed, we face hard problems within our industry, and need new solutions:

This conference is called the Future of Mobile – not the Present Day of Mobile – so what I want to talk about is developments in mobile operating systems that will allow the mobile devices and mobile services of, say, 5 years time – 2013 – to live up to their full potential.

I believe that the mobile phones of 2013 will make even the most wonderful phones of today look, in comparison, jaded, weak, slow, and clunky. It’s my expectation that the phones used at that time, not just by technology enthusiasts and early adopters, but also by mainstream consumers, will be very considerably more powerful, more functional, more enchanting, more useful, more valuable, and more captivating than today’s smartphones.

To get there is going to require a huge amount of sophisticated and powerful software to be developed. That’s an enormous task. To get there, I offer you three contrasts.

The first contrast is between cooperation and competition.

The press often tries to portray some kind of monster, dramatic battle of mobile operating systems. In this battle, the people sitting around this table are fierce competitors. It’s the kind of thing that might sell newspapers. But rather than competition, I’m more interested in collaboration. The problems that have to be solved, to create the best possible mobile phone experiences of the next few years, will require cooperation between the people in the companies and organisations represented around this table – as well as with people in those companies and organisations that don’t have seats here at this moment, but which also play in our field. Instead of all of us working at odds with each other, spreading our energies thinly, creating incomplete semi-satisfactory solutions that are at odds with each, it would be far better for us to pool more of our energies and ideas.

I’m not saying that all competition should be stopped – far from it. An element of competition is vital, to prevent a market from becoming stale. But we’ve got too much of it just now. We’ve got too many operating systems that are competing with each other, and we’ve got different companies throughout the value chain competing with each other too strongly.

Where the industry needs to reach is around 3 or 4 major mobile operating systems – whereas today the number is somewhere closer to 20 – or closer to 200, if you count all the variants and value-chain complications. It’s a fragmentation nightmare, and a huge waste of effort.

As the industry consolidates over the next few years, I have no doubt that Symbian OS will be one of the small number of winning platforms. That brings me to my second contrast – the contrast between old and new – between past successes and future successes.

Last year, Symbian was the third most profitable software company in the UK. We earned licensing revenues of over 300 million dollars. We’ve been generating substantial cash for our owners. We’re in that situation because of having already shipped one quarter of a billion mobile phones running our software. There are at present some 159 different phone models, from 7 manufacturers, shipping on over 250 major operator networks worldwide. That’s our past success. It grows out of technology that’s been under development for 14 years, with parts of the design dating back 20 years.

But of course, past success is no guarantee of future success. I sometimes hear it said that Symbian OS is old, and therefore unsuited to the future. My reply is that many parts of Symbian OS are new. We keep on substantially improving it and refactoring it.

For example, we introduced a new kernel with enhanced real-time capabilities in version 8.1b. We introduced a substantial new platform security architecture in v9.0. More recently, there’s a new database architecture, a new Bluetooth implementation, and new architectures for IP networking and multi-surface graphics. We’re also on the point of releasing an important new library of so-called “high level” programming interfaces, to simplify developers’ experience with parts of the Symbian OS structure that sometimes pose difficulty – like text descriptors, active objects, and two-phase object construction and cleanup. So there’s plenty of innovation.

The really big news is that the pace of innovation is about to increase markedly – for three reasons, all tied up with the forthcoming creation of the Symbian Foundation:

1. The first reason is a deeper and more effective collaboration between the engineering teams in Symbian and S60. This change is happening because of the acquisition of Symbian by Nokia. By working together more closely, innovations will reach the market more quickly.
2. The second reason is because of a unification of UI systems in the Symbian space. Before, there were three UI systems – MOAP in Japan, UIQ, and S60. Now, given the increased flexibility of the latest S60 versions, the whole Symbian ecosystem will standardise on S60.
3. The third reason is because of the transition of the Symbian platform – consisting of Symbian OS together with the S60 UI framework and applications – into open source. By adopting the best principles of open source, Symbian expects to attract many more developers than before to participate in reviewing and improving and creating new Symbian platform code. So there will be more innovation than before.

This brings me to the third of the three contrasts: openness vs. maturity.

Uniquely, the Symbian platform has a stable, well-tested, battle-hardened software base and software discipline, that copes well with the hard, hard task of large-scale software integration, handling input from many diverse and powerful customers.

Because of that, we’ll be able to cope with the flood of innovation that open source will send our way. That flood will lead to great progress for us, whereas for some other software systems, it will probably lead to chaos and fragmentation.

In summary, I see the Symbian platform as being not just one of several winners in the mobile operating system space, but actually the leading winner – and being the most widely used software platform on the planet, shipping in literally billions of great mobile devices. We’ll get there, because we’ll be at the heart of a huge community of impassioned and creative developers – the most vibrant developer ecosystem on the planet. Although the first ten years of Symbian’s history has seen many successes, the next ten years will be dramatically better.

Footnote: For other coverage of this event, see eg Tom Hume, Andrew Grill, Vero Pepperrell, Jemima Kiss, Dale Zak, and a very interesting Twitter channel (note to self: it's time for me to stop resisting Twitter...)

Mobile 2.0 keynote

Earlier today, I had the privilege to deliver the opening keynote at the Mobile 2.0 event in San Francisco. This posting consists of a copy of the remarks I prepared.

The view from 2013

My topic is Open Source, as a key catalyst for Mobile Innovation 2.0.

Let’s start by fast forwarding five years into the future. Imagine that we are gathered for the 2013 “Mobile 2.0” conference – though the name may have changed by that time, perhaps to Mobile 3.0 or even Mobile 4.0 – and perhaps the conference will be taking place virtually, with much less physical transportation involved.

Five year into the future, we may look back at the wonder devices of today, 2008: the apparently all-conquering iPhone, the Android G1, the Nokia E71, the latest and greatest phones from RIM, Windows Mobile, and so on: all marvellous devices, in their own ways. From the vantage point of 2013, I expect that our thoughts about these devices will be: “How quaint! How clunky! How slow! How did we put up with all the annoyances and limitations of these devices?”

This has happened before. When the first Symbian OS smartphones reached the market in 2002 – the Nokia 7650, and the Sony Ericsson P800 – they received rave reviews in many parts of the world. These Symbian smartphones were celebrated at the time as breakthrough devices with hitherto unheard of capabilities, providing great user experiences. It is only in retrospect that expectations changed and we came to see these early devices as quaint, clunky, and slow. It will be the same with today’s wonder phones.

Super smart phones

That’s because the devices of five years time will (all being well) be so much more capable, so much slicker, so much more usable, and so much more enchanting than today’s devices. If today’s devices are smart phones, the devices of 2013 will be super smart phones.

These devices will be performing all kinds of intelligent analysis of data they are receiving through their sensors: their location and movement sensors, their eyes – that is, their cameras – and their ears – that is, their always-on recording devices.

They’ll also have enormous amounts of memory – both on-board and on-network. Based on what they’re sensing, and on what they know, and on their AI (artificial intelligence) algorithms, they’ll be guiding us and informing us about all the things that are important to us. They’ll become like our trusted best friends, in effect whispering insight into our ears.

We can think of these devices as like our neo neo-cortex. Just as primates and especially humans have benefited from the development of the neo-cortex, as the newest part of our brains in evolutionary terms, so will users of super smartphones benefit from the enhanced memory, calculation powers, and social networking capabilities of this connected neo neo-cortex.

In simple terms, these devices can be seen as adding (say) 20 points to our IQs – perhaps more. If today’s smartphones can make their users smarter, the super smartphones of 2013 can make their users super smart.

Solving hard problems

That’s the potential. But the reality is that it’s going to be tremendously hard to achieve that vision. It’s going to require an enormously sophisticated, enormously capable, mobile operating system.

Not everyone shares my view that operating systems are that important. I sometimes hear the view expressed that improvements in hardware, or the creation of new managed code environments, somehow reduce the value of operating systems, making them into a commodity.

I disagree. I strongly disagree. It's true that improvements in both hardware and managed code environments have incredibly important roles to play. But there remain many things that need to be addressed at the operating system level.

Here are just some of the hard tasks that a mobile operating system has to solve:

• Seamless switching between different kinds of wireless network – something that Symbian’s FreeWay technology does particularly well;


• Real-time services, not only handling downloads of extremely large quantities of data, but also manipulating that data in real time – decompressing it, decrypting it, displaying it on screen, storing it in the file system, etc;


• All this must happen without any jitter or delay – even though there may be dozens of different applications and services all talking at the same time to several different wireless networks;


• All this rich functionality must be easily available to third party developers;


• However, that openness of developer access must coexist with security of the data on the device and the integrity of the wireless networks;


• And, all this processing must take place without draining the batteries on the device;


• And without bamboozling the user due to the sheer scale and complexity of what’s actually happening;


• And all this must be supported, not just for one device, but in a way that can be customised and altered, supporting numerous different form factors and usage models, without fragmenting the platform.

Finally, please note that all this is getting harder and more complex: every year, the amount of software in a top-range phone approximately nearly doubles. So in five years, there’s roughly up to 32 times as much software in the device. In ten years, there could be 1000 times as much software.

Engaging a huge pool of productive developers

With so many problems that need to be solved, I will say this. The most important three words to determine the long-term success of any mobile operating system are: Developers, Developers, Developers. To repeat: the most important three words for the success of the Symbian platform are: Developers, Developers, Developers.

We need all sorts and shapes and sizes of developers – because, as I said, there are so many deep and complex problems to be solved, as the amount of software in mobile phone platforms grows and grows.

No matter how large and capable any one organisation is, the number of skilled developers inside that organisation is only a small fraction of the number outside. So it comes down to enabling a huge pool of productive and engaged developers, outside the organisation, to work alongside the original developers of the operating system – with speed, creativity, skill, and heartfelt enthusiasm. That’s how we can collectively build the successful super smart phones of the future.

Just two weeks ago, the annual Symbian Smartphone Show put an unprecedented focus on developers. We ran a Mobile DevFest as an integral part of the main event. We announced new developer tools, such as the Symbian Analysis Workbench. We will shortly be releasing new sets of developer APIs (application programming interfaces) in new utility libraries, to simplify interactions with parts of the Symbian programming system that have been found to cause the most difficulty – such as text descriptors, two phase object construction and cleanup, and active objects.

The critical role of open source

But the biggest step to engage and enthuse larger numbers of developers is to move the entire Symbian platform into open source.

This will lower the barriers of entry – in fact, it will remove the barriers of entry. It will allow much easier study of the source code, and, critically, much easier participation in the research and creation of new Symbian platform software. We are expecting a rapid increase in collaboration and innovation. This will happen because there are more developers involved, and more types of developers involved.

That’s why the title of my talk this morning is “Open Source: Catalyst for Mobile Innovation 2.0”. The “2.0” part means greater collaboration and participation than ever before: people not just using the code or looking at the code, but recommending changes to it and even contributing very sizeable chunks of new and improved code. The Open Source part is one of the key enablers for this transformation.

Necessary, but not sufficient

However, Open Source is only one of the necessary enablers. Open Source is a necessary but not sufficient ingredient. There are two others:

1. A stable and mature software base, with reliable processes of integration, which I’ll talk more about in a moment;


2. Mastery of the methods of large-scale Agile development, allowing rapid response to changing market needs.

Fragmentation inside an operating system

Here’s the problem. Fragmentation is easy, but Integration is hard.

Fragmentation means that different teams or different customers pull the software in different directions. You end up, in the heat of development in a fast-moving market, with different branches, that are incompatible with each other, and which can’t be easily joined together. The result is that solutions created by developers for one of these branches, fail to work on the other branches. A great deal of time can be wasted debugging these issues.

Here, I speak from bitter experience. During the rapid growth days of Symbian, we lost control of aspects of compatibility in our own platform – despite our best efforts. For example, we had one version called 7.0s and another called 7.0, but lots of partners reported huge problems moving their solutions between these two versions. Because of resulting project delays, major phones failed to come to the market. It was a very painful period.

Nowadays, in the light of our battle-hardened experience, Symbian OS is a much more mature and stable platform, and it is surrounded and supported by an ecosystem of very capable partners. In my view, we have great disciplines in compatibility management and in codeline management.

The result is that we have much better control over the integration of our platform. That puts us in a better position to handle rapid change and multiple customer input. That means we can take good advantage of the creativity of open source, rather than being pulled apart by the diverse input of open source. Other platforms may find things harder. For them, open source may bring as many problems as it brings solutions.

Fragmentation across operating systems

This addresses the fragmentation inside a single operating system. But the problem remains of fragmentation across different operating systems.

Although competition can be healthy, too many operating systems result in developers spreading their skills too thinly. The mobile industry recognises that it needs to consolidate on a smaller number of mobile operating systems moving forwards. The general view is that there needs to be consolidation on around three or (at the most) four advanced mobile operating systems. Otherwise the whole industry ends up over-stretched.

So, which will be the winning mobile operating systems, over the next five years? In the end, it will come down to which phones are bought in large quantities by end users. In turn, the choices offered to end users are strongly influenced by decisions by phone manufacturers and network operators about which operating systems to prefer. These companies have four kinds of issues in their minds, which they want to see mobile operating systems solve:

• Technical issues, such as battery life, security, and performance, as well as rich functionality;


• Commercial issues, such as cost, and the ability to add value by differentiation;


• Political issues, in which there can be a perception that the future evolution of an operating system might be controlled by a company or organisation with divergent ulterior motivations;


• Reliability issues, such as a proven track record for incrementally delivering new functionality at high quality levels in accordance with a pre-published roadmap.

A time for operating systems to prove themselves

Again, which will be the winning operating systems, over the next five years? My answer is that is slightly too early to say for sure. The next 12-18 months will be a time of mobile operating systems proving themselves. Perhaps three or four operating systems will come through the challenge, and will attract greater and greater support, as customers stop hedging their bets. Others will be de-selected (or merged).

For at least some of the winning smartphone operating systems, there will be an even bigger prize, in the subsequent 2-3 years. Provided these operating systems are sufficiently scalable, they will become used in greater and greater portions of all phones (not just smartphones and high-end feature phones).

Proving time for the Symbian Foundation platform

Here’s how I expect the Symbian platform to prove itself in the next 12-18 months. Our move to open source was announced in June this year, and we said it could take up to two years to complete it. Since then, planning has been continuing, at great speed. Lee Williams, current head of the S60 organisation in Nokia, and formerly of Palm Inc and Be Inc, has been announced as the Executive Director of the Symbian Foundation.

The Foundation will make its first software release midway through the first half of 2009. Up till that point, access to internal Symbian OS source code is governed by our historical CustKit Licence and DevKit Licence. There’s a steep entry price, around 30,000 dollars per year, and a long contract to sign to gain access, so the community of platform developers has been relatively small. From the first Symbian Foundation release, that will change.

The source code will be released under two different licenses. Part will be open source, under the Eclipse Public Licence. This part has no licence fee, and is accessible to everyone. The other part will be community source, under an interim Symbian Foundation Licence. This is also royalty free, but there is a small contract that companies have to sign, and a small annual fee of 1,500 dollars. I expect a large community to take advantage of this.

This interim community source part will diminish, in stages, until it vanishes around the middle of 2010. By then, everything will be open source. We can’t get there quicker because there are 40 million lines of source code altogether, and we need to carry out various checks and cleanups and contract renegotiations first. But we’ll get there as quickly as we can.

There’s one other important difference worth highlighting. It goes back to the theme of reducing fragmentation. Historically, there have been three different UIs for Symbian OS: S60, UIQ, and MOAP(S) used in Japan. But going forwards, there will only be one UI system: S60, which is nowadays flexible enough to support the different kinds of user models for which the other UI systems were initially created.

To be clear, developers don’t have to wait until 2010 before experimenting with this software system. Software written to the current S60 SDK will run fine on these later releases. We’ll continue to make incremental compatible releases throughout this time period.

What you should also see over this period is that the number of independent contributors will increase. It won’t just be Nokia and Symbian employees who are making contributions. It will be like the example of the Eclipse Foundation, in which the bulk of contributions initially came from just one company, IBM, but nowadays there’s a much wider participation. So also for the Symbian Foundation, contributions will be welcome based on merit. And the governance of the Foundation will also be open and transparent.

The view from 2013

I’ll close by returning to the vision for 2013. Inside Symbian, we’ve long had the vision that Symbian OS will be the most widely used software platform on the planet. By adopting the best principles of open source, we expect we will fulfil this vision. We expect there will in due course be, not just 100s of millions, but billions of great devices, all running our software. And we’ll get there because we’ll be at the heart of what will be the most vibrant software ecosystem on the planet – the mobile innovation ecosystem. Thank you very much.

Market failure and mobile operating systems

In seeking to talk about economics, and about market failure, I'm moving way outside my depth. There seem to be so many different viewpoints about economics, each appearing reasonably plausible to me (when I read them in isolation), yet all contradicting each other. It's a tough subject! What one writer sees as a market failure, another writer sees instead as a failure of individual actors within that market - and so on.

However, one of my correspondents has made a series of thought-provoking comments about market failure in the particular field of mobile operating systems. I believe these comments are sufficiently unusual and also sufficiently intriguing, to deserve consideration by a wider audience - despite my reticence to broach the subject of economics. The comments arose in the responses to an earlier posting of mine, "De-fragmenting the mobile operating system space". To quote from these responses:

Currently, mobile developers withstand very high development costs due to a very fragmented mobile ecosystem, meanwhile mobile OSes enjoy a much lower developing cost than they would if they had to built compatible OSes between versions: therefore, a de-fragmentation process would move developing costs from mobile developers to mobile OS developers, that is, mobile OS companies/foundations would have to internalize those costs.

But developing an OS with full source or binary compatibility between versions is an order of magnitude more expensive than building one with broken compatibility, and it gets worse with time and versions. Moreover, building a de-fragmented mobile OS requires committing considerable resources (people, money, time) in the present, sunk costs that must have positive expected returns in the future (at least to cover developing costs and money opportunity costs).

Will foundations/consortiums (Symbian, LiMo, OHA/Android), given their non-profit nature, carry these investments in the present to obtain stable software platforms in the future? As Adam Smith wrote, displaying good will and hope is not enough: mobile foundations/consortiums/companies committing resources in the present must charge higher prices in their future and profit handsomely from their risky investments, otherwise the effort will stop...

To paraphrase:

• Economic incentives on individual mobile operating systems will lead these operating systems to diverge from each other;
• This divergence will mean that application developers (and providers of middleware, etc) will suffer greater difficulties, because of having to spread their efforts across larger numbers of diverse mobile operating systems;
• As things stand, no one who is in a position to actually do something to reduce the divergence of mobile operating systems, has a sufficient financial incentive to make that happen;
• So we can actually expect things to get worse for application developers, rather than better.

(I'm over-simplifying what my correspondent actually says; see the original for the full argument.)

Tentatively, I have the following answers:

• I believe that things will actually get better for application developers, rather than worse;
• It's my experience that major players in the mobile phone industry can, on occasion, take actions based on strategy rather than business case;
• This requires strong self-discipline on the part of these companies, but it's not unknown;
• Action on strategic grounds becomes more likely, the more clearly the argument is made that the actions that make sense in the short-term are actually detrimental to longer term interests;
• The other key factor in this decision is whether the various actors can have a high degree of confidence in at least the medium-term viability of the software system they're being asked to collectively support.

So, in line with what I've argued here, what we need to do is to keep on pushing (in creative and diverse ways) the merits of the case in favour of de-fragmenting mobile operating systems - and to keep on highlighting the positive features of the mobile operating systems (such as Symbian OS) that are most likely to enable at least medium-term success for the whole industry.

Incidentally, one big contribution that the shareholders and customers of Symbian are taking, towards that end, is to agree to standardise on S60 as the UI framework for the future. They've taken that decision, even though both UIQ and MOAP(S) have much to commend them as alternative UI frameworks on top of Symbian OS. They've taken that decision for the greater common good. New phones based on the UIQ and MOAP(S) UI frameworks will continue to appear for a while, during a transitional period, but the Symbian Foundation platform software is standardising on the S60 framework. Elements of both UIQ and MOAP(S) will be available inside the Symbian Foundation platform, but the resulting system will be compatible with S60, not UIQ or MOAP(S). That's a decision that will bring some pain, but the shareholders and customers have been able to support it because:

• S60 is now (in contrast to the earlier days) sufficiently flexible and mature, to support the kinds of user experience which previously were available only via UIQ or MOAP(S);
• Indeed, S60 now has flexibility to support new types of UI experiences, whilst maintaining common underlying APIs;
• Distribution of S60 will pass out of the hands of Nokia, into the hands of the independent Symbian Foundation.

I also believe that the disciplines of binary compatibility that have been built up in Symbian, over several years, are significantly reducing the difficulties faced by developers of add-ons and plug-ins for Symbian software. Because of this discipline, it now costs us less to maintain compatibility across different versions of our software. It's true that some practical issues remain here, with surprising incompatibilities still occasionally arising in parts of the software stack on Symbian-powered phones other than Symbian OS itself. But progress is being made - and the leading nature of the Symbian platform will become clearer and clearer.

To finish, I'll give my response to one more comment:

Samsung is gaining market share and producing S60 devices. Let's suppose that Samsung starts snatching portions of market share from Nokia, do you really think anyone believes that Nokia would refrain from intentionally breaking compatibility to derail Samsung, if that would be necessary?

I can see that there might be some temptation towards such a behaviour, inside Nokia. But I don't see that the outcome is inevitable. Nokia would have to weigh up various possible scenarios:

• Symbian Foundation quality assurance tests will notice the compatibility breaks, and will refuse to give Symbian accreditation to this changed software;
• The other licensees could create their own fork of the software (which would have official endorsement from the Symbian Foundation) and build up a lot of momentum behind it.

Instead, Nokia - like all users of Symbian Foundation software - should be inclined to seek differentiation by providing their own unique services on top of or alongside Symbian platform software rather than by illicitly modifying the platform software itself. That's a far better way to deploy skilled software engineers.

Five reasons open source won't work in mobile(?)

John Bruggeman, Chief Marketing Officer of Wind River, gave the stand-out presentation on Day One of OSiM (Open Source in Mobile) conference in Berlin today. It was an extraordinary piece of theatre, that captivated the audience - who were already in the midst of a feast of interesting and thought-provoking presentations from other speakers.

Officially entitled "How open source can drive your next device innovation breakthrough", the majority of the presentation focused on an artful walkthrough of what John called "Five reasons why open source won't work in mobile".

These reasons were presented, with great eloquence and wit, as a wake-up call - "to uncover the ugly truth".

Here's the context. Speakers earlier in the day had outlined enchanting prospects for Open Source Linux to reach an installed base of 10 billion mobile devices within perhaps just five years (that's right - it would involve many people having more than one device - such as separate mobile phones and MIDs). But John said that such prospects were illusory, and he gave the following reasons:

1. Linux phones will never be as good as the iPhone

Here, despite what you might first think, "good" means "good for application developers". Apple iPhone application developers can be confident that, with just one version of their code, they can reach every member of the 15M+ iPhone user community. But in the Linux space, there are already 96 different major and minor variants (based on 10 different mobile Linux platforms). So application developers will have to work much harder, even to begin to reach all Linux phone users.

[Although John was exaggerating some of his rhetoric, for effect, he was quite clear that the figure of 96 was bona fide - and he said that his Wind River colleague Jason Whitmire would show a slide to justify that figure in another presentation later in the day. Unfortunately I missed that later presentation, since I was in another stream at that time.]

2. The mobile industry has been confused and misled by the Symbian Foundation announcement

Previously, open source endeavours in mobile were united (albeit in a highly fragmented sense - see point 1) around the vision of Linux being at the heart of any praiseworthy phones. But with the announcement that the Symbian platform will become open source, everyone has become confused.

[John actually said that "Symbian have polluted the open source message by telling everyone open source is easy" and "Symbian has a gigantic community, but in an evil and dirty way, Symbian is attracting children with candy". As I said, it was an extraordinary piece of theatre - with lots of exaggeration for effect.]

3. Operators insist on hedging their bets

Because there are so many platform choices, without clear winners, operators are fearful of the risk of standardising on the wrong choices. That's a big risk for them. So they bet on multiples - perpetuating the fragmented status quo.

4. Phone manufacturers won't let go of the past

Phone manufacturers have built up huge amounts of legacy code, on numerous operating systems, and they insist on keeping on using these systems. Just like the operators, they fear the risk of giving up what turns out to be a winning system - and in this way, they keep the whole industry confused.

5. Too many people in the mobile value chain just don't get it

Too many people have been confused by the word "free" and have thought that, because Linux itself has zero licensing cost, everything else of value in the emerging new mobile value chain should also be free of cost. They don't realise that the real meaning of "free" is "free access". This distracts people from being able to generate the new kinds of revenues that will strengthen the companies who have key roles to play in the open source mobile value chain.

In summary, "We are all to blame".

After the wake-up call

Was that the end of the talk? (After all, due to previous sessions over-running, it was now well into the time allocated for the lunch session.)

Well, after some dramatic suspense, John went on to offer another set of five points - things that, whilst hard for us to do (he said), would make open source work in mobile after all:

1. Hire new product marketers, who deeply understand the different world of open source;


2. Hire new engineering directors, who (likewise) deeply understand the different world of open source. We need to realise that developing open source software solutions is fundamentally different from the processes in proprietary projects;


3. Stop looking over your shoulder, and learn to innovate;


4. Change your low cost platform into a high value play;


5. Think outside the phone - there are incredible lessons that we can learn from adjacent and complementary markets.

Evaluation

My own view is that the proposed solutions are a lot less convincing than the list of problems. True, open source introduces and demands new thinking models. However, many of the disciplines of large-scale system software development apply in fairly similar ways across both closed-source and open-source projects.

As I see it, the real solution to platform fragmentation is clear platform leadership. Once it becomes clear to operators and handset manufacturers that a given platform is going to allow them strong opportunities for meaningful differentiation, speedy product development, good quality output, and significant revenues, they'll naturally gravitate towards that platform, and other platforms will fade from the scene.

Remember the old adage: the best marketing tool is a winning product. That beats a bunch of pretty powerpoint slides anyday. If you don't have a winning product, then you'll inevitably struggle. Historically, the Symbian platform has been advanced by the visible market successes of a series of breakthrough phone models. (And the very first phone manufacturers who came to us, had all taken a close look at the Psion Series 5 PDA, and liked what they saw.) The longer term success of the Symbian platform will be determined by whether or not there are new breakthrough commercially successful phones in the next 12-24 months. With 92 separate products (*) under development by our customers at the last count, I see plenty of grounds for optimism.

And by the way, if anyone ever does hear me (or anyone else from Symbian) giving the impression that "open source is easy", please pull me up. Building world-beating smartphones is hard, however you go about the business. Recognising that it is a hard task is, paradoxically, part of the beginning of the real solution. Sad to say, clinging to the hope that there's an easy route to smartphone success has been the downfall of many a project.

Footnote: (*) Symbian uses the following principle to count the number of products under development (as opposed to those which are merely "prospects" or "roadmap items"):

Models in development are defined by Symbian as phones prior to launch where licensees

1. have committed a minimum development team; and
2. have a visible plan to launch; and
3. have a minimum expected lifetime shipment for the phone.

De-fragmenting the mobile operating system space

One response to my previous posting, "Addressing the fragmentation of mobile operating systems", gave me a lot to think about. The points made by this anonymous commenter deserve a considered response. Here goes:

>Don’t get me wrong, I would like to have just two operating systems in the mobile space. But I’m quite realistic and tired of hypocrites like Arun Sarin, that call for software homogeneity and then sell every OS available: please, put your money where your mouth is.

I recognise the disconnect that often seems to exist, between the strategy arms of various network operators, that say they are going to reduce the number of mobile operating systems they support, and the local operational arms, that appear to disregard that policy. However, I don't think the disconnect is total. I am aware of several instances where an attractive new smartphone model was rejected by a network operator, on the grounds that the software platform of that smartphone wasn't on their approved short list.

Admittedly, I suspect that if the smartphone model had been knock-out gorgeous, the strategic directive would have been trumped by local sales considerations. But seeing that the model was only "good" (and not "great") the centrally approved policy held sway. So these policies do have some force (albeit not so strong a force as you and I might like).

>So, in your point of view, the losses caused by third parties delays and incurred by mobile manufacturers are so large, that they justify that manufacturers relinquish the short-term gains of incompatible operating systems and join their forces to reduce the number of operating systems and/or accept just one OS.

On reflection, it was unfair of me to pick on delays caused by third party software. Delays caused by malfunctions in second party software (written by the phone manufacturer) and in first party software (provided by the operating system vendor) can be equally bad. Unsuspected incompatibilities in the behaviour or functionality of lower-level software can result in lengthy debugging sessions to eventually understand why higher-level software is inexplicably failing to perform as expected on the latest build of the software platform.

My point remains that it can take a long time for a device creation software team to become truly expert in all the vagaries of the underlying software platform. I wish that weren't the case; I wish that the underlying software platform was more transparent and predictable. Marketing representatives of all the major mobile software platforms might seek to convince people that their own particular platforms are uniquely stable and dependable. And large parts of these software platforms are, no doubt, both stable and dependable. However, because there are many millions of lines of code in the entire platform, and because there are lots of complex fast-changing inter-dependencies between different modules, the unfortunate reality is that the software platforms demand considerable attention from members of the device creation community.

For that reason, manufacturers of advanced mobile phones will put an undue penalty on themselves if they spread the attention of their development teams over several different mobile operating systems. It would be far better for them, over time, to focus on a smaller number of these complex operating systems.

>the natural solution to that problem... [is] to stop outsourcing to third parties those fundamental pieces of code and start producing internally themselves. And maybe, just maybe, that’s what Symbian Foundation is all about: tight control of every software layer

It's true, the Symbian Foundation has the intent of providing a software platform with a greater degree of completeness that has previously been the case. The idea is that mobile phone manufacturers will receive a more self-contained package of software than in the past. They'll have less need to source missing portions.

On the other hand, mobile phone manufacturers will still seek to differentiate their phones, by means of:

• Replacing some of the components provided by the Symbian Foundation
• Optimising other components, with an eye to the unique hardware and network needs of their own products
• Adding in new low-level components, that support new services and user experience.

For this reason, I expect that the device creation marketplace will remain large and vibrant.

>Fragmentation manifests itself as a technical problem (incompatibilities between devices) but it’s not created by technical problems in itself, its real nature is political: the inability of all industry players to reach an agreement to des-fragment the mobile ecosystem and, in absence of that agreement, the power/control of one player to impose a standard

I understand the motivations of the different industry players who each participate in multiple different software foundations. We can call this "irresponsible promiscuity" and/or "a failure of industry alignment", but it can also be seen as a pragmatic hedging of bets. For as long as it remains unclear which of several mobile operating systems will be long-term winners and which will be long-term losers, it's understandable that industry players will loath to constrain themselves to just one or two mobile operating systems.

However, once it starts to become clear that a particular mobile OS is reaching new levels of utility, productivity, and innovation, industry players are likely to focus their minds sharply - and the political landscape will change. Even so, I don't expect there will be just one winner. That would introduce its own problems. But I do expect that the number of mobile operating systems will significantly decrease.

>Short-term, no industry player profits from a des-fragmentation process, since it implies to lower the switching costs

Symbian has thought long and hard about the risks and issues in lowering switching costs. If we provide APIs that are more in line with those used by other platforms (for example, the APIs in our PIPS libraries), the risk is that software that runs on Symbian phones will more easily be ported to run on other devices.

At first sight, that looks like a negative outcome for Symbian. However, I think the actual outcome is positive. Partners and customers dislike being locked into a single operating system. If we make it easier for partners and customers to migrate their software assets to other platforms, it means, paradoxically, they will be happier to create software assets on our platform in the first place.

>Even Symbian, when Nokia S60 programs are not allowed to run on a S60 Samsung due to lack of certificates, is creating artificial fragmentation and switching costs: why waste resources with software compatibility when a certificate breaks it all?

I see this example as an implementation error rather than any deliberate policy. The subject of certificates is tricky, but I sincerely hope that we'll diminish the pains they unintentionally cause. Essential fragmentation is hard enough, without "artificial fragmentation" making things worse.

>Anonymous said...

Final comment: initially I was unsure whether I wanted to reply at length to someone who didn't give their name. I prefer open discussion to shadow discussion. However, I guess that there's good reason for the commenter to seek anonymity here.

Addressing the fragmentation of mobile operating systems

Back in February this year, Vodafone CEO Arun Sarin called on the mobile industry to reduce the number of operating systems in use. As reported in Computer World at the time, "Vodafone CEO calls for mobile OS consolidation":

Mobile phone operating systems are key to [user] experience, he said, but there are too many of them; as many as 30 or 40, Sarin estimated.

"We have to reduce that number. There's no way that developers of cool applications can develop for that many operating systems. If we had three, four, five, that would be better," he said...

One reason for the proliferation of mobile phone operating systems is that, historically, handset suppliers offered their own proprietary code to make the best use of their phones' limited processing and memory resources. Developing applications for such closed systems is difficult.

Today's high-end phones, though, have as much computing power as low-end PCs, spawning a new market for operating systems such as Symbian OS or Windows Mobile that run on phones from multiple manufacturers.

That ought to reduce the number of software platforms in the market, but more are still arriving, including several based on the open-source operating system Linux.

This same topic of there being too many mobile operating systems came up this week at the Symbian Foundation roundtable discussion held alongside the CTIA show in San Francisco. During the Q&A, several of the press and analysts in attendance pressed panellists on this point:

As the number of mobile operating systems increase, will fragmentation become a major issue for customers and developers?

As reported by Brad Smith in Wireless Week,

Christy Wyatt, vice president of Software Platforms and Ecosystem for Motorola, said there is always the danger there could be a complete fragmentation of operating systems, especially with the growing varieties of “open source” operating systems. But she said the reality is that companies involved in software services will work to focus on a few operating systems.

Marin Perez in InformationWeek takes up the story:

Additionally, Wyatt said she expects to see a "sharp contraction" in the number of mobile platforms in the future, including one consistent version of mobile Linux.

At the same time, the roundtable panellists suggested that resourceful and determined developers would find ways to ensure that their applications can run on the platforms with significant market share and revenue potential:

Because of the diversity in hardware, there will never be one unified language, said Oren Levine, product market manager in Nokia's S60 group. But Levine thinks developers will have significant incentive to make their applications work on multiple platforms due to the size of the overall mobile market.

So how much trouble is being posed to developers, in practice, by this diversity of mobile operating systems? Will a multitude of different operating systems continue in existence (contrary to the request of Arun Sarin), each driven and supported by its own factors? Will the mobile ecosystem as a whole find ways to smooth across the differences and fragmentation caused by this variety?

Just one day after the Symbian Foundation roundtable took place, one of the CTIA keynote speakers initially sounded bleak while addressing this same set of questions:

"Developing applications for the fragmented mobile ecosystem is a Herculean effort that often results in developers creating an application that serves the least common denominator of mobile devices with a poor user experience, and an inability to effectively scale," said Marco Boerries, executive vice president, Yahoo! Connected Life.

But then he unveiled a potential solution:

"Yahoo! Blueprint solves the most challenging problem plaguing today's mobile landscape -- now with one click, you can write once and have mobile services run across a critical mass of devices and operating systems, potentially reaching millions of users. We believe Yahoo! Blueprint is simply the best way to create mobile Internet services -- and we expect will change the world of mobile development moving forward."

Cade Metz of The Register recorded the message from Marco Boerries as follows:

Previously, the Blueprint platform was merely a means of developing widgets that run atop Yahoo! Go, an application suite available for various and sundry mobile phones.

"Blueprint is our way to help us and the rest of the industry to develop mobile services," Yahoo! mobile president Marco Boerries said this morning, during his keynote at the CTIA Wireless IT and Entertainment trade show in downtown San Francisco.

"You can write a Blueprint service, and with one click, you can generate a standalone application that runs on hundreds of Java devices, on Windows Mobile devices, on Symbian devices and that you can freely distribute. Never has it been so easy to build mobile applications that are this powerful."

Of course, this sounds very similar to the oft-repeated promise of "Write Once, Run Anywhere (WORA)" (sometimes also expressed as "Write Once, Run Everywhere"). Back in January 1996, Sun's press release covering Java 1.0 stated,

"Java's write-once-run-everywhere capability along with its easy accessibility have propelled the software and Internet communities to embrace it as the de facto standard for writing applications for complex networks," said Alan Baratz, JavaSoft's newly appointed president. "We're delighted to invite developers to download Java 1.0 immediately and start building the next killer application."

Twelve years later, after a history of (at best) mixed success with runtimes that aim to provide WORA, the prospect of a real implementation of this promise remains alluring. And it's not just Yahoo! that are proclaiming new implementations of this ideal. Another product announcement at CTIA, this time by the Frisco, Texas-based software company Recursion Software, used similar language:

Recursion Unveils Universal Android, .NET CF and Symbian Platform

Voyager pervasive platform achieves "Write Once, Run Everywhere" for application messaging and communications

Recursion Software, Inc., an innovative provider of next-generation platforms and intelligent middleware, today at CTIA Wireless and IT 2008 announced a new release of Voyager, a pervasive software platform that blurs the lines between Android, Symbian, Microsoft .NET and Compact Framework (CF), and JEE.

Voyager turns traditional software development on its head by allowing developers to natively write and maintain one code-set in either Java or .NET and publish the code to the device(s) of their choice regardless of the virtual machine they employ, instantly expanding an application’s device and market reach. Combined with Voyager’s existing support for more than 15 embedded operating systems, this release signifies Voyager’s evolution into a pervasive platform for all "Screens of Life" (TV, PC, phone, car, ultra mobile PC).

There are many other products that sound, on first hearing, to be making claims similar to those advanced on behalf of Yahoo! Blueprint or Recursion Voyager. I've not seen any detailed analysis as to the comparative strengths of these different products. (I'll be grateful to any reader of this blog who provides pointers to such analysis.) However, I have no doubt that these products can significantly reduce the effort that developers need to invest, to create certain kinds of mobile software. At least some of the pain and frustration of multiple mobile operating systems can be reduced.

But whilst these intermediate platforms address issues in the market for add-on applications, they provide less help in the arguably even more important market of "device creation software". That's the market of software components that are built into mobile phones:

• Filling gaps in the operating systems themselves
• Providing special optimisations for particular hardware or particular networks
• Enabling low-level differentiation between different devices.

Software at this level tends to need to be written in the native language (usually C or C++) and to use the native APIs of the underlying software platform. Intermediate platforms can't be used for these purposes, as they don't provide sufficiently intimate connections to the device firmware.

My own experience - confirmed by many discussions over the years with people deeply involved in specific phone creation projects - is that device creation projects often suffer back-breaking delays because third party software has difficulty in slotting into the overall device software. These delays can occur, even though the third party providers in question seem to have a good track record with previous versions of their software. The problem is that incompatibilities between different versions of the operating system software result in unexpectedly different behaviour from the third party plug-ins. The changes at the platform level invalidate some of the design, the optimisation parameters, or the testing of the plug-ins.

If this kind of delay results from fairly modest incompatibilities between two different versions of the mobile operating system, how much greater is the delay caused by the incompatibilities between two totally different operating systems?

For me, this is the real reason why the mobile industry needs to evolve towards fewer mobile operating system. The companies that keep working with too many different systems are likely to find their development resources stretched too thinly - and (despite possible short-term gains) they'll lose their effectiveness as a result.

Footnote: The topic of multiple intermediate programming environments will be covered in the keynote panel session "Who will win the runtime race?" on day two (22nd Oct) of the 2008 Symbian Smartphone Show. The panellists will include:

• Pete Barr-Watson, Senior Business Development/Deployment Manager, Microsoft Silverlight
• Andreas Constantinou, Research Director, VisionMobile
• Antony Edwards, VP Developer Product Marketing, Symbian
• Matt Millar, Director of Mobile Devices EMEA, Adobe
• Simon Nicholson, Director Operator Engagement, Client Software Group, Sun Microsystems
• Jürgen Scheible, Author of "Mobile Python".

The five laws of fragmentation

As discussion of the potential for the Symbian Foundation gradually heats up, the topic of potential fragmentation of codelines keeps being raised. To try to advance that discussion, I offer five laws of fragmentation:

1. Fragmentation can have very bad consequences

Fragmentation means there's more than one active version of a software system, and that add-on or plug-in software which works fine on one of these versions fails to work well on other versions. The bad consequences are the extra delays this causes to development projects.

Symbian saw this with the divergence between our v7.0 and v7.0s releases. (The little 's' was sometimes said to stand for "special", sometimes for "strategic", and sometimes for "Series 60".) UIQ phones at the time were based on our v7.0 release. However, the earliest Series 60 devices (such as the Nokia 7650 "Calypso") had involved considerable custom modifications to the lower levels of the previous Symbian OS release, v6.1, and these turned out to be incompatible with our v7.0. As a pragmatic measure, v7.0s was created, that had all of the new technology features introduced for v7.0, but which kept application-level compatibility with v6.1.

On the one hand, v7.0s was a stunning success: it powered the Nokia 6600 "Calimero" which was by far the largest selling Symbian OS phone to that time. On the other hand, the incompatibilities between v7.0 and v7.0s caused no end of difficulties to developers of add-on or plug-in software for the phones based on these two versions:

• The incompatibilities weren't just at the level of UI - UIQ vs. Series 60
• There were also incompatibilities at many lower levels of the software plumbing - including substantial differences in implementation of the "TSY" system for telephony plug-ins
• There were even differences in the development tools that had to be used.

As a result, integration projects for new phones based on each of these releases ran into many delays and difficulties.

Symbian OS v8 was therefore designed as the "unification release", seeking as much compatibility as possible with both of the previous branches of codeline. It made things considerably better - but some incompatibilities still remained.

As another example, I could write about the distress caused to the Symbian partner ecosystem by the big change in APIs moving from v8 to v9 (changes due mainly to the new PlatSec system for platform security). More than one very senior manager inside our customer companies subsequently urged us in very blunt language, "Don't f****** break compatibility like that ever again!"

Looking outside the Symbian world, I note the following similar (but more polite) observation in the recent Wall Street Journal article, "Google's Mobile-Handset Plans Are Slowed":

Others developers cite hassles of creating programs while Android is still being completed [that is, while it is undergoing change]. One is Louis Gump, vice president of mobile for Weather Channel Interactive, which has built an Android-based mobile weather application. Overall, he says, he has been impressed by the Google software, which has enabled his company to build features such as the ability to look up the weather in a particular neighborhood.

But he says Weather Channel has had to "rewrite a few things" so far, and Google's most recent revision of Android "is going to require some significant work," he says.

2. Open Source makes fragmentation easier

If rule 1 was obvious (even though some open source over-enthusiasts seem to be a bit blind to it), rule 2 should be even clearer. Access to the source code for a system (along with the ability to rebuild the system) makes it easier for people to change that software system, in order to smooth their own development purposes. If the platform doesn't meet a particular requirement of a product that is being built from that platform, hey, you can roll up your sleeves and change the platform. So the trunk platform stays on v2.0 (say) while your branch effectively defines a new version v2.0s (say). That's one of the beauties of open source. But it can also be the prelude to fragmentation and all the pain which will ensue.

The interesting question about open source is to figure out the circumstances in which fragmentation (also known as "forking") occurs, and when it doesn't.

3. Fragmentation can't be avoided simply by picking the right contract

Various license contracts for open source software specify circumstances in which changes made by users of an open source platform need to be supplied back into the platform. Different contracts specify different conditions, and this can provoke lengthy discussions. However, for the moment, I want to sidestep these discussions and point out that contractual obligations, by themselves, cannot cure all fragmentation tendencies:

• Even when users of a platform are obligated to return their changes to the platform, and do so, it's no guarantee that the platform maintainers will adopt these changes
• The platform maintainers may dislike the changes made by a particular user, and reject them
• Although a set of changes may make good sense for one set of users, they may involve compromises or optimisations that would be unacceptable to other users of the platform
• Reasons for divergence might include use of different hardware, running on different networks, the need to support specific add-on software, and so on.

4. The best guarantee against platform fragmentation is powerful platform leadership

Platform fragmentation has some similarities with broader examples of fragmentation. What makes some groups of people pull together for productive collaboration, whereas in other groups, people diverge following their own individual agendas? All societies need both cooperation and competition, but when does the balance tilt too far towards competition?

A portion of the answer is the culture of the society - as reflected in part in its legal framework. But another big portion of the answer is in the quality of the leadership shown in a society. Do people in the group believe that the leaders of the group can be relied on, to keep on "doing the right thing"? Or are the leaders seen as potentially misguided or incompetent?

Turning back to software, users of a platform will be likely to stick with the platform (rather than forking it in any significant way) if they have confidence that the people maintaining the trunk of the platform are:

1. well-motivated, for the sake of the ecosystem as a whole
2. competent at quickly and regularly making valuable new high quality releases that (again) meet the needs of the ecosystem as a whole.

Both the "character" (point 1) and the "competence" (point 2) are important here. As Stephen Covey (both father and son) have repeatedly emphasised, you can't get good trust without having both good character and good competence.

5. The less mature the platform the more likely it will be to fragment, especially if there's a diverse customer base

If a platform is undergoing significant change, users can reason that it's unlikely to coalese any time soon into a viable new release, and they'll be more inclined to carry on working with their own side version of the platform, rather than waiting for what could be a long time for the evolving trunk of the platform to meet their own particular needs.

This tendency is increased if there are diverse customers, who each have their own differing expectations and demands for the still-immature software platform.

In contrast, if the core of the platform is rock-solid, and changes are being carefully controlled to well-defined areas within the platform, customers will be more likely to want to align their changes with the platform, rather than working independently. Customers will reason that:

• The platform is likely to issue a series of valuable updates, over the months and years ahead
• If I diverge from the platform, it will probably be hard, later on, to merge the new platform release material into my own fork
• That is, if I diverge from the platform, I may gain short-term benefit, but then I'll likely miss out on all the good innovation that subsequent platform releases will contain
• So I'd better work closely with the developers of the trunk of the platform, rather than allowing my team to diverge from it.

Footnote: Personally I see the Symbian Foundation codeline to be considerably more mature (tried and tested in numerous successful smartphones) than the codeline in any roughly similar mobile phone oriented Linux-based foundation. That's why I expect that the Symbian Foundation codeline will fall under less fragmentation pressure. I also believe that Symbian's well-established software development processes (such as careful roadmap management, compatibility management, system architecture review, modular design, overnight builds, peer reviews, and systematic and extensive regression testing) are set to transfer smoothly into this new and exciting world, maintaining our track record of predictable high-quality releases - further lessening the risks of fragmentation.

Fragmentation is easy, integration is hard

The Wall Street Journal reports today that "Google's Mobile-Handset Plans Are Slowed". The Inquirer picks up the story and adds a few choice morsels of its own: "Depressing news as Google's Android delayed":

However, life’s little crises just kept getting the Android down and now apparently some mobile network operators like Sprint Nextel, have abandoned any attempt to get an Android on the market until 2009. This is purportedly because the majority of Google's attention and resources have been going to Sprint’s competitor T-Mobile USA, who still hope to have an Android mobile out by the end of Q4. We have it on good authority (from un-named sources of course) that Sprint actually asked Google “Do you want me to sit in the corner and rust, or just fall apart where I'm standing?”...

Director of mobile platforms at Google, Andy Rubin, gloomily noted that trying to develop software while the company’s irritating partners kept pushing for new features, was a time-consuming task. "This is where the pain happens", he sighed.

I recognise this pain. It's something that has occurred many times during Symbian's history. That's why I've emphasised a dilemma facing Android: Fragmentation is easy, but integration is hard. Coping with multiple forceful customers at the same time, while your codebase is still immature, is a really tough challenge. Glitzy demos of v2 features don't help matters: they drive up interest that needs to be deflated, as you have to explain to customers that, no, these features aren't ready to include in the software package for their next phones, despite looking brilliant on YouTube. Instead, the focus needs to go on the hard, hard task of integration.

The dangers of fragmentation

My comments on mobile Linux fragmentation at the Handsets World event in Berlin were picked up by David Meyer ("Doubts raised over Android fragmentation") and prompted a response by Andy Rubin, co-founder of Google's Android team. According to the reports,

On a recent comment by Symbian's research chief, David Wood, that Android would eventually end up fragmented, Rubin said it's all part of the open source game.

Raising the example of a carrier traditionally having to wait for a closed platform developer to release the next version of its software to "enable" the carrier to offer new services, Rubin said carriers could just hire a developer internally to speed up that process without waiting any longer.

"If that fragmentation is what [Wood] is talking about, that's great--let's do it," said Rubin.

Assuming these reports are accurate, they fall into the pattern of emphasising the short-term benefits of fragmentation, but de-emphasising the growing downstream compatibility problems of a platform being split into different variants. They make fragmentation sound like fun. But believe me, it's not!

I noticed the same pattern while watching a panel on Open Source in Mobile at one of the Smartphone Summit events that take place the day before CTIA. The panel moderator posed the question, "Is fragmentation a good or bad thing?" The first few panellists were from consultancies and service providers. "Yes", they said, smiling, "Fragmentation gives more opportunity for doing things differently - and gives us more work to do." (I paraphrase, but only slightly.) Then came the turn of a VP from one of the phone manufacturers who have struggled perhaps more than most with the variations and incompatibilities between different mobile Linux software stacks. "Let's be clear", came the steely response, "fragmentation is a BAD thing, and we have to solve that problem".

Luigi Licciardi, EVP of Telecom Italia, made similar remarks at the ‘Open source in Mobile’ conference in Madrid in September 2007. He said that one thing his network operator needs in terms of software platforms which they would consider using in the mobile phones, is ‘backwards compatibility’ - in other words, a certain level of stability. (This sounds simple, but I know from my own experience that backwards compatibility requires deep skill and expertise in the midst of a rapidly changing marketplace.) Moreover, the software platform has to be responsive to the needs of the individual operators: the operator needs to be able to go and talk to a company and say “give us these changes and modifications”. He also said that the platform needs to be open to applications for network connections and end users, but has to be closed to malware. In other words it has got to have a very good security story. (Incidentally, I believe Symbian has uniquely got a very strong security story, with platform security built deep into the operating system.) Finally, he emphasised that “a fragmented Linux is of no interest to operators”.

This topic deserves more attention. Let me share some analysis from a transcript of a talk I gave at the Olswang "Open Source Summit" in London last November:

The point is that there is a great tendency in the mobile phone space for mobile Linux variants to fragment and split. This was first drawn to my attention more than two years ago by Avi Greengart who is a US-based analyst. He said that mobile Linux is the new Unix, meaning that despite the best intentions of all involved, it keeps on going its own separate ways.

So why is that happening? It is happening first of all because fragmentation is easy. This means that you can take the code and do whatever you like with it. But will these changes be brought back inside the main platform? Well I claim that, especially in a fast moving market such as smartphones, integration is hard. The changes tend to be incompatible with each other. Therefore it is my prediction that, on average, mobile Linux will fragment faster than it unifies.

It is true that there are many people who say it is very bad that there are all these different mobile Linux implementations. It is very bad because it has caused great problems for developers: they have to test against so many stacks. These people ask themselves, “Can't we unify things?” And every few months there is a new group that is formed and says, in effect, “Right, we are going to make a better job of unifying mobile Linux than the last lot, they weren’t doing it fast enough, they weren’t doing it seriously enough, so we are going to change that.” But I see the contrary, that there is a greater tendency to fragment in this space than to unify, and here’s why.

It is always easier and quicker to release a device-specific solution than to invest the extra effort to put that functionality into a reusable platform, and then on into a device. In other words, when you are racing to market, when the market leaders are rushing away from you and leaving more and more clear blue water between you and them, it is much more tempting to say, “well I know we are supposed to be thinking in terms of platform, but just for now I am going to serve the needs of my individual product.”

Interestingly we had the same problem in the early days of Symbian. One of the co-founders of Symbian, Bill Batchelor, coined the phrase “the Symbian Paradox”, which is that we found it hard to put functionality into the platform, rather than just serve it out to eager individual customers via consultancy projects. But we gradually learned how to do that, and we gradually put more and more functionality into the platform, suitable for all customers, and therefore more and more customer projects benefited more widely.

So why is mobile Linux fragmenting in a way that perhaps open source in other areas isn’t fragmenting? First, it is an intense, fast moving industry. Symbian as the market leader, together with our customers, is bringing out significant new functionality every three or four months. So there is no time for other people to take things easy and properly invest in their platforms. They are tempted to cut corners – to the detriment of the platform.

Second, if you look at how some of these consortia are meant to work, they are meant to involve various people contributing source code. If you look at some of their architecture diagrams, you might get one company in say Asia, which is contributing one chunk of software, which is meant to be used by other companies the world over. Well guess what happens in a real life project? Another company, let’s say a company trying to ship a Linux based phone in America, and surprise, surprise, the software doesn’t work, it fails to get FCC approval, it doesn’t meet the network operators’ needs or there are bugs that only show up on the network operators in America. So what do they say? They say to the first group (the people out in Asia) “would you mind stopping what you are doing and come and fix this, we are desperate for this fix for our software”. The group in Asia say, “well we are very sorry, we are struggling hard, and we are behind as well, we would rather prioritise our own projects, if you don’t mind, shipping our own software, debugging it on different networks”.

At this point you may raise the question: isn’t open source meant to be somewhat magical in that you can all just look at it and fix it anyway; you don’t need to get the original person to come and fix it? But here we reach the crux of the matter. The problem is there is just too much code, there are such vast systems, it is not just a few hundred lines, or even a few thousand lines of code, there are hundreds of thousands or even millions of lines of code in these components and they are all interfacing together. So somebody looks at the code and they think, “Oh gosh, it is very complicated”, and they look and they look and they look and eventually they think, “Well if I change this line of code, it will probably work”, but then without realising it they have broken something else. And the project takes ages and ages to progress..

Compare this to the following scenario: some swimmers are so good they can actually swim across the English Channel, they go all the way from England to France. Suppose they now say, “Yes I have cracked swimming, what will I do next? Oh I will swim all the way across the Atlantic, after all, it is just an ocean and I have already swum one ocean, so what is different about another ocean?” Well it is the kind of difference between the places where open source has been doing well already and the broader oceans with all the complications of full telephony in smartphones.

So what happens next in this situation? Eventually one company or another may come up with a fix to the defects they faced. But then they try and put it back in the platform, and the first company very typically disagrees, saying “I don’t like what you have done with our code, you have changed it in a very funny way, it isn’t the way we would have done it”. And so the code fragments – one version with the original company, and the other in the new company. That is how it ends up that mobile Linux fragments more than it unifies.

I say this firstly, because I have contacts in the industry who lead me to believe that this is what happens. Secondly, we have the same pressures inside Symbian, but we have learned how to cope with it. We often get enhancements coming back from one customer engagement project and at first it doesn’t quite fit into the main OS platform, but we have built up the highly specialist skills how to do this integration.

As I mentioned, integration is hard. You need a company that is clearly responsible and capable for that. This company needs to be independent and trustworthy, being motivated - not by any kind of ulterior motive, but by having only one place in the value chain, doing one job only, which is that it is creating large scale customer satisfaction by volume sales of the platform.