Open source vs. Open Standards – complementing or competing?

I have been thinking of this idea for some time especially after the launch of Android/OHA – and a recent email from Curro Dominguez of Vodafone Betavine R and D who attended my IMS course at Oxford university prompted this blog.

I seek thoughts on this – and admittedly I may not have got everything right here. Note also that this article/blog is specific to the mobile domain – however similar principles may apply to the Web as well.

So .. Here we go ..

Interoperability and standards

In any consumer based service, we need interoperable standards for the market to blossom

There are three ways to do enable interoperability – the first is by a monopoly i.e.. A single standard controlled by one company. The second is via open standards and standardization bodies and the third(which I find most interesting) is via open source

The open standards process is now well known. For instance, W3C is a standardization body and its process is outlined HERE

As with any standardization body, it comprises submissions, committees, consensus, working groups, draft proposals – so on and so forth.

The result is – all standards from any standardization body have three problems:

a) They are slow i.e. they take a long time to materialise

b) They are ‘standard’ i.e. they leave little room for differentiation

c) Even when there is some effort towards standardization/consensus, there is no guarantee that the market will not fragment(such as in the case of J2ME)

Besides this, there are other problems – for instance -

a) Some companies actually thrive on fragmentation(transcoding companies/testing companies etc) and on the flip side – there are business models built around standardization by standardization entities themselves (for example testing, compliance mark, certification, training etc)

b) IPR management is also an issue

c) Finally, many companies support standards retrospectively. This means they are ‘compliant’ but only later when it makes little difference anyway.

The result is a slow, imperfect process, which does not always work. However, the options are much worse .. Essentially a commercial monopoly

Open source

Can Open source offer a third way forward?

It appears to be a road which companies are increasingly taking – especially with Android/OHA.

Firstly, for the sake of this discussion, let us differentiate between Open standards, Open systems and Open source

We have already discussed Open standards above – i.e. standards which are consensus driven and created by a consortium of companies with some form of community/committee process

Open systems is a term where users are not restricted by any commercial or technological means i.e. no walled gardens. The meaning of ‘open’ in this case is dependent on context. Recently, facebook opened up it’s APIs to third parties. In that sense, facebook is ‘open’. However, within facebook itself, you cannot contact any user without an introduction(unlike in MySpace). So, in that case, facebook is ‘closed’. Hence the meaning that : open = users should not be restricted by any commercial or technological means – is generally valid

Which brings us to ‘Open source’. For once, the Wikipedia article on Open source is not very clear. Hence, I am using a very simple, concise viewpoint to explain this, rather emotive for some people, term

Open source software is managed and promoted through an organization called Open Source Initiative -which maintains an OSI definition and endorses a set of Open source licenses which comply with the OSI definition.

All this is simple enough. However, the term ‘Open source’ was created to distinguish from another term called ‘Free software’ coined by the free software foundation. .

The principal philosophical difference between the free software foundation and the open source movement lies in ‘what to do with derived works’ also called Copyleft . The free software foundation believes that derived works should also be covered by the same terms as the licence(copyleft). In other words, if you modify software under the FSF licence, then the derived work must also be released back into the community.

In contrast, depending on which license you adopt from the open source foundation, derived works need not be released back to the community(there is a secondary question of what constitutes a derived work – but we will leave that aside for the moment). Specifically, the Apache licence v 2.0 covered under Opensource is not copyleft. In other words, any derivatives need not be released back into the community (thereby preserving IP rights of the person modifying it).

There are philosophical debates about this (and one can argue that the copyleft principle is more viral – see wikipedia link on copyleft above)– but the counter argument is that – companies will want to preserve their modifications for commercial reasons and there will be no commercial incentives for people if you force them to also release any derived works.

Of course, none of the open source (and for that matter the free software) licenses mandate ‘commercially free’ i.e. you are free to charge people (or not) for the software and derivatives.

Open source and Interoperability

Open source conjures up images of pony tailed geeks writing Linux code .. but there is another class of contributors enabling the rise of open source.


Why would companies want to contribute code under open source?

To facilitate interoperability and to grow the market.

In addition, unlike standardization which is top down, complex and committee driven : Open source initiatives are grassroots, simple and needs based since they are often solutions to specific problems. This makes the whole process simple and quick with unviable initiatives cut down at the source.

The biggest proponent of Open source in recent times is Google – with Android. The Android user space software is released under Apache version 2.0.This provides a commercial incentive for companies to develop modifications in the application space. See this fantastic post on why Google chose the apache license over GPL v2 for a detailed discussion on this topic.

Of course we have to reiterate that open source is not always better than standardization and ultimately I expect that the two will coexist. But certainly, it is a space to watch.

Others have spotted the same trend.

For instance,

Readwerite web correctly (in my view) identifies open source as the most promising trend of 2008 but wrongly(again in my view) mixes the three terms Open source, Open systems and open standards(for example the Opera lawsuit is not about Open source – it is about open standards since Opera itself is not open source as I understand it)

Vodafone betavine

Finally, we come to the email from Curro Dominguez of Vodafone who works with Vodafone betavine

I have blogged about betavine before and people like Dan Appelquist and Stephen Wolak who work with it are very clued on guys – so I am watching betavine with interest

Hence, I was interested when Curro told me about MobileScript which is an ECMA script that allows developers to access device functionality (calls, messages) in a simple manner.

It’s also one of the first projects to be released as Open Source (GPL v2)

See this YouTube video

As Vodafone are doing (correctly in my view), I see this to be an increasing trend – and it will be beneficial since companies will address specific problems, they will court developers, Android/OHA will provide legitimacy to the whole area and we will balance the objectives of fast development cycles, interoperability and with no monopolies

Also, I wish to acknowledge the thinking from Andreas of visionmobile which I have used to clarify some of my own questions in understanding Open source.

Any comments welcome

Related blogs coming soon:

The Blu Ray debate about openness

The iPhone debate about openness

Tim Berners Lee keynote at 3GSM

For many attendees of 3GSM, Tim Berners Lee’s keynote was an important hightlight of the conference. The full text of the speech is as per the link HERE. I have reproduced the text below and highlighted the bits which I have found insightful.

The Mobile Web

Ladies and gentlemen, it is a pleasure to be here today. I must say I cannot claim to be an expert in the mobile phone technology, so I speak as a guest in this conference. My career for the last 20 years has unrolled in the context of the Internet. What I find exciting about being here now is that we are at an epic point in telecommunications history, when the mobile platforms discussed here, and the Internet platforms which have enabled such a spectacular growth and innovation, are poised, if we manage this well, to merge.

In the few minutes we have together, I would like to explain to you the essence of an open Web platform — the things without which it would not deserve the name. I would like you to understand that there are plenty of ways in which we could fail to pull this off, and leave ourselves incapacitated, with innovation stifled. By ‘we’ here I mean the whole community of manufacturers, service providers, content providers, consumers, and to a limited extent, legislators. I’d like you also to feel with me the excitement about some of the incredible things we can aim for if we succeed.

Let me tell you where I am coming from, to help you understand my point of view. Twenty years ago, or 52 Web years as we used to say, I was a software engineer at CERN, the particle physics lab in Geneva. CERN is a great place, full of people from across the world tackling the greatest current challenges in physics. It is a great place to work, with lots of very creative people who chat over coffee with views of the vineyards and the Alps.

Now, this diversity of talent had brought with it a diversity of technology. This is pre-Web. Take your minds back — perhaps, younger ones, ask your parents. There are documents stored on computers — papers, manuals, help systems, letters, but each system works on one particular computer — minicomputer, mainframe or PC or Mac in those days — and runs on one particular operating system, VMS, VM-CMS, DOS, Mac OS, and many flavors of Unix. So finding a document involved finding out which computer it was on, knowing which program to run, and learning how to use that program. This was driving me crazy, and by 1989 I’d figured out that a networked hypertext system, a kind of Web, could be used to wrap up each system and make its screens, menus and documents be part of a globally interconnected space which could be viewed from any computer.

Also, just at that point, a critical transition was occurring. Each computer had been connected to a different form of network: Decnet, Cernnet, Bitnet, Appletalk, and so on. The attempts to rationalize these using ISO standards was not doing well. However, the Internet was already connecting universities all over the US. Depending on who you talked to, it was just becoming respectable; it was also becoming possible to connect computers to it at CERN.

I wanted to design the World Wide Web, as I decided to call it, to be usable for any data on any system. I had watched the failure of so many sophisticated documentation access systems which constrained their users to use one type of computer, or operating system. If really anything could be on the Web, then the Web technology should demand almost nothing of its users.

The reason that I could just design the Web by myself and set it running on a couple of computers without asking anyone, was that the Internet in turn had been designed to be used for anything, constraining its users as little as possible. So this is one of the qualities of an open platform: it is built to enable, not to control, and it does not try to second guess the things which will be built using it.

The Web is designed, in turn, to be universal: to include anything and anyone. This universality includes an independence of hardware device and operating system, as I mentioned, and clearly this includes the mobile platform. It also has to allow links between data from any form of life, academic, commercial, private or government. It can’t censor: it must allow scribbled ideas and learned journals, and leave it to others to distinguish these. It has to be independent of language and of culture. It has to provide as good an access as it can for people with disabilities.

The Web worked because of a number of technical and social reasons. It worked because there was no central bottleneck for traffic, no central link database to be kept consistent, no central place to go and register a new page or a new Web site.

It worked because it was valuable, in a novel way. The value added by the Web is the unexpected re-use of information. People learned that if they went to the trouble of putting something on the Web for some reason, that others would benefit later in ways they never anticipated. The experience of surfing the Web, which blew some of the early users away for days and nights, was of discovering things you never knew existed.

So the Web worked. How many projects do we start which have a bright start and fizzle out over time? So many that it is worth celebrating that the Web worked. It is worth noting why. A lot of that has to do with the open Internet platform.

Let me mention one important aspect of the platform. The serendipitous re-use of information happens because when I buy an Internet connection, I don’t specify the Web sites I am going to connect to. If you buy an Internet connection, and you run a Web server, then I can connect to your site. I don’t find my ISP saying that it wants to be my supplier of music, and so it will block access to any site I try to load music from.

This is of course different from the model which the cable companies have had. The relatively recent ability of the Internet to carry video promises to really open up the movie delivery options, and provide an exciting new world of anything on demand any time, and not just anything we currently get from a few large companies, but the long tail, the seething mass of individual and independent films which are waiting to entertain someone out there. When a US cable company threatens to attempt to stifle this aspect of the open Internet platform, we have defended it as Net Neutrality. Net Neutrality was so much of an obvious technical and social prerequisite of the Internet world, that it never needed a name until now. It is a tension of convergence, where different business models and cultures may clash. I am confident that Net Neutrality will be preserved, for the good of us all. But I would urge you to support it whenever you get the chance.

So what else does it take to make an open Internet platform?

What does it take?

It takes, mainly, common standards. The innovation of the WWW was possible because the standards for TCP/IP were already implemented in an interoperable way all over the planet, in advance of the innovation. TCP/IP wasn’t designed with networked hypertext in mind. But it wasn’t designed to prohibit it either — it was and is an open platform.

Web 2.0 community Web sites, eBay, and Flickr are possible because the Web standards, in turn, were widely implemented in an interoperable way, before those innovations. The same for the wikis, like Wikipedia, and blogs, and so on. The Web is a huge platform for innovation because of those standards. Any new genre of communication, any new social networking idea, immediately can gain the value of unexpected re-use by people across the world.

There is a very important difference in attitude between a foundation technology and — well — let’s call it a ceiling technology. A foundation technology is designed to enable innovation, to be the base which will support other even more powerful things to come. A ceiling technology is not. It is designed to provide a value, and for its provider to cash in and cash out. Proprietary music download systems are ceiling technologies to the extent that the technologists design to be also being the only store in town, rather than creating an open market. Though putting a lid on further innovation, they are still providing a service, and making sure they profit from it.

Ceiling technologies are the end of the road for innovation.

When you want to make a foundation technology, you need to look ahead. You need to put aside the short term return on investment questions and look at the long term.

A great example of this is the patent question. In 1989 my colleagues in the Internet community would not have dreamed of patenting the ideas in the Internet protocols. We worked together to figure out new ideas, and implement them as common standards. As the Web grew, we realised we needed to establish a structure for developing common Web technical standards. In 1994 we formed the W3C as a meeting place for this process.

In 1998, we had launched a project to help with the issue of user privacy on the Web. It was a protocol (P3P) to allow a Web browser to automatically read and check a machine-readable version of a Web site’s privacy policy. It was not a very exciting new technology, but it was an important innovation as electronic commerce was being held back in some cases by user fears in this area. At the time when we should have been engaged in deployment and testing, a small company announced that everyone who wished to make a P3P implementation would have to pay royalties. They claimed to have a patent on something to do with information being communicated and stored and affecting future communication.

This has a devastating effect. Anyone working for a large company was told by lawyers never to read anything to do with the work. Anyone working as a volunteer in their garage dropped their work on these tools as they didn’t want to work for free for this company. The mid-sized companies who were running a business specifically around the technology ran into serious trouble. It took us 18 months and $150k to get a legal opinion that royalties were not in fact payable, but that was 4 Web years, during the boom, and that was too long. P3P lost its momentum. The world lost an enabling technology.

How does a company think about standards then if following them may involve losing that short-term ceiling technology return?

It is a game. In the mathematical sense. Here is the payoff matrix: You commit to working on a standard, or not. The standard may take off or it may not.

If you don’t commit to the standard, and it doesn’t work, (which of course it won’t if no one else does) then life, and your proprietary ceiling technology, continues. No innovation.

If you do commit to it and it it does work, then a whole new market is enabled: This is the disruptive case. There is some effort involved moving the company to the standard, and often to help build the standard. You might join W3C to help make it happen. A certain amount of effort. There is a major long term return.

One of the most difficult things for some companies to learn is that this is not a zero-sum game. We are so used to battling over a fixed market, or battling over fixed resources, that we tend to assume everything is such that we can only win what our competitors lose. But when we make a whole new market space, like the Web, or like GSM actually, then we are in fact together battling the human condition such as inefficiency, poverty and ignorance.

Now, what about the corner cases? The fear seems to be of going for the standard and it not taking off. Well, the loss in this case is the engineering time to tool up for the standard, which could have been saved. But it is a very finite loss.

On the other hand, what if you decide not to go for the standard and it does take off? Everything happens, the new market appears, and you are not there. The pace of everything ramps up dramatically, and you are left standing still. The costs of retooling to a standard get much bigger as time passes. In this conference we all can see the stresses on phone companies, and we know the dis-empowerment of all travelers from the fact that the GSM standards and frequencies are not quite global — and we know the benefits from the fact that are becoming so. Other cases spring to mind. On the Internet, for example, streaming media are available in many incompatible formats.

Often this is due to companies wanting to profit from ceiling technologies. This involves making a high income from the technology itself rather than letting it take off. This in turn requires patents, and of course that the owned technology dominates. Hence the battles over VHS and Betamax, HD DVD and Blu-Ray, and so on.

So as the Web platform and the mobile phone converge — what do we want the result to be? A foundation or a ceiling technology? Clearly, a foundation. A mobile phone — or whatever device we carry around which uses GSM technology and its successors — is going to be everywhere, and everyone will have one. It has do be designed to be universal. So that everyone can use it. So that you can do anything with it.

The choice is the new platform being a privately owned walled garden, or a competitive open platform. Both models can work in the medium term. But the open model opens up new things which we can only try to imagine.

What are the standards? Basically, the same standards as the current Web uses. That is the most important point. It is one Web. The Web works on phones. There are effective browsers which can give you access to the same information which you could see from any laptop or desktop. Of course, looking ahead, small devices will get smarter and displays will get more and more pixels, so mobile devices are taking the same track which larger computers did a few years before.

That said, there are ways of making a Web site work much better with a mobile device. The W3C’s Mobile Web Initiative (MWI) is a group of mobile technology companies within the World Wide Web Consortium which realize the importance of this convergence and are working hard to make the Mobile Web a reality.

MWI defines best practices for authoring content. It defines what sort of facilities that you should expect to find on a mobile device. It gives best practices for serving data in the most device-independent way. It recommends finding out what device you are talking to if you can, and sending appropriately formatted content. Some phone browsers set out to be able to provide access to virtually any Web page, but technical limitation on other phones make this impossible. To encourage Web sites to become easily browsable by mobile devices, there is a “mobileOK” mark which may be used by content providers adhering to guidelines.

Designing for the “mobileOK” mark, designing Web sites which are browsable by many different sorts of devices also has important spin-offs. Many of the MWI best practices are in fact good Web design principles, so the whole site will be easier to use for anyone. There is a also a lot of overlap with accessibility. Making a “mobileOK” site and making one which is easily used by people with disabilities involves the same sort of work.

From the beginning, The W3C has fought the “best viewed with 800×600 screen” buttons, and any design patterns which disenfranchise different devices. This was difficult when everyone seemed to have the same sort of laptop, but easier as it became obvious that screens vary a lot. The Mobile Web Initiative is the work we have to do now. It is timely, it is part of a historic convergence of technologies. But it is part of a general strategic principle of keeping the information which is such a huge form of capital in the world in as powerful, and reusable from as we can, for the future generations and people who don’t currently have access.

It isn’t just about making the Web you know today work on mobile phones. We are talking about innovation. The innovations which will really count are the things which I can’t imagine now. They may include new applications built using the familiar AJAX technologies used cross-platform now, well known by developers, and increasingly available on mobile devices. These new applications may also operate across multiple devices. This is where we talk of the Ubiquitous Web. Have you noticed the price of LEDs is coming down, and more and more surfaces are covered with them? Not just at rock concerts and Times Square, but coming soon to all kinds of surfaces near you. Your phone could use these displays, and the abstract task you are doing can really rise above individual devices. Imagine that my phone or my wristwatch has details of a flight I am booking, and I walk into a room where it negotiates to project a map on the wall. And so on. Imagine yourself. Innovate on the mobile Web platform.

Among other things, many of us are hoping that a low-cost open platform will have a much greater penetration in what we currently call the developing world. I personally believe that it is important to humanity to connect peoples across the world as widely as possible. I think we must preserve the diversity of cultures and ideas. But also I think we must connect people to give more global harmony. We should not add connectivity to the long list that the richer countries have and the poorer ones do not, a list which of course has clean water, health care and peace pretty near the top.

As part of the Mobile Web Initiative, W3C held a workshop on the Mobile Web in Developing Countries. One of the concerns is that some of the new phones aimed at the lower cost bracket don’t all have Web browsers. The area is very exciting, and the figures for coverage — 80% of the world’s population I have heard (World Bank, according to Wikipedia), and for market growth in developing countries seem very positive.

So when we look at the choices for the mobile devices, it is clear that they must continue on the path to an open Web platform. That is what the Mobile Web Initiative is about. Huge new markets, and huge opportunities for humanity, depend on this. We know in general how to do it. But there is a lot to do. It has been my pleasure to take a tour of these issues with you, you who are the companies and individuals who are making it happen.