What mobile AJAX can learn from the fragmentation of Java Me

By Paddy Byers

paddy_post.JPG

Edited by Ajit Jaokar

ajit.JPG

As we have seen from previous posts in this blog about Mobile Ajax, Mobile AJAX has a lot of potential. However, many questions still remain about the details of how Mobile AJAX could be implemented and exploited.

Specifically, this post discusses the problems that occurred in the evolution of Java ME (formerly called J2ME) and how Mobile Ajax and other browser technologies can learn from them; both in terms of the browser platform itself, and the associated developer ecosystem.

I’d welcome discussion on how to resolve the issues that are raised.

Mobile apps aren’t just about interactivity

Many inherent advantages of browser-based solutions have been articulated clearly in the preceding posts, and AJAX gives a dramatically superior level of interactivity than was possible with conventional XHTML; but the truth is that AJAX has only just improved interactivity to the point that it is good enough to be a credible experience (assuming services are targeting a typically specified feature phone and aiming for modest level of richness in visual experience). From the point of view of the interactivity enabled, AJAX could be likened to an early MIDP 1.0 Canvas-based application on a typical interpreted virtual machine.

The analogy with MIDP 1.0 doesn’t stop there.

The current mobile AJAX environment has many similarities with those early Java ME days: there are maturity issues (e.g. only one or two implementations, no common behavioural specification or precise API specification); ecosystem issues (eg a developer community naively believing that desktop implementation approaches will transfer seamlessly to mobile); and market issues (a broken value chain as it relates to the proliferation of technology enablers). The painful failure of early attempts to use Java ME as a service delivery platform should be a clear warning to the new generation of web 2.0 service providers aspiring to extend to mobile.

The key parallels, however, are in the level of platform integration offered.

Much of the mobile AJAX discussion has focussed on interactivity and connectivity aspects; but the problem is that the services that extend effectively to mobile will be those that successfully harness the mobile platform and its capabilities.

Some of these include SMS/MMS, multimedia capture, interactive multimedia playback, location, Bluetooth, etc. To implement these features, we require access to the platform i.e. device APIs.

Real mobile apps are not simply a thin projection of an interactive interface to a web-based application.

One of the main reasons for the failure of MIDP1.0 was that none of the key platform capabilities was exposed to the java programmer.

Many – nearly all – potential mobile applications, even if 99% implementable in Java ME, turned out not to be viable because of a critical dependency on some platform feature that simply wasn’t accessible in the Java ME environment.

The Java ME community realised this early on and sought to address it by extending the scope of the platform.

Unfortunately, this is where the problems really began.

Unilaterally defined APIs (in the case of MIDP1.0, this was primarily audio and security) were made by manufacturers and also by operators; and the community process later created formally standardised APIs.

Several years on we have platform APIs that have functionality gaps and are inconsistently implemented; further, there is no common policy for the deployment of APIs, either between operators, within a single operator, or even among the devices of individual manufacturers.

The resulting fragmented environment and ecosystem problems are well documented.

The key point is that Sun, to its credit, set out very deliberately to ensure that the java platform did not get fragmented and tried to put in place every technical, legal and commercial measure to implement its strategy – and it still failed.

Implementation interoperability must be addressed proactively

The current desktop browser market contains a small number of players with comparatively well-understood differences. Content authors can ensure that content is interoperable by coding directly to cater for those differences. Basic market dynamics force browser providers to fix problems and keep functionality in step with the market leaders.

The mobile browser market is different.

For one thing, it is still not following the full web browser standards. Also, there are multiple providers and already some fragmentation in respect of functionality; scripting interoperability is especially limited as a result of weak (or nonexistent) DOM implementations underlying many mobile browsers.

There are also few market pressures to correct this – operators are not (yet) mandating full browser functionality (such as AJAX); much of the functionality is not formally specified by W3C or another body; there are no independent conformance or interoperability tests.

Simply leaving this implementation interoperability issue to market forces to address will hinder mobile AJAX substantially. The operators have a position in which they could either improve or inhibit interoperability.

Possibly the worst of all outcomes is in fact the most likely, which is that a requirement to support XMLHttpRequest and certain scripting functionality is made mandatory, without the detailed technical specification and conformance backup needed for manufacturers to do it reliably; the result will be all manufacturers scrambling to incorporate weak but technically compliant implementations, stalling all efforts for genuinely functional and interoperable implementations.

This interoperability challenge should not be underestimated.

Contrast the browser environment with Java ME in which API specifications are developed under an open and formal process, there are tens of thousands of conformance tests, conformance is mandatory under the license terms for the technology, etc – and still interoperability is poor.

The definition of platform and extension APIs must be decentralised

The Java ME strategy is to define all APIs through an open, formal and consensual process which is centralised. When a requirement for a particular API is identified – location-based services, say – the various stakeholders (operator, device manufacturer, content developer, technology provider, etc) form an expert group to develop the API and associated resources and make it available to the industry. The aim, of course, is to avoid the fragmentary effects of there being multiple independent and competing APIs.

However, the centralised nature of this approach is one of its biggest problems.

First, consensus takes a long time and APIs arrive too late to be useful.

APIs become over-engineered because they not only have to meet the immediate need but also any other foreseeable future needs and edge cases – because once defined they can rarely be extended.

The process, in certain cases, also becomes mired in the competing technical and commercial agendas of expert group members; and the business model surrounding the resulting reference implementations and conformance test suites are problematic, which delays uptake, or stops it altogether.

Ironically, these issues then themselves become the cause of much fragmentation.

Due to cost and the risk of delay, manufacturers and operators do not incorporate new APIs unless there is a clear commercial need; and when they do, they all make different choices as to which capabilities to include.

Prior to standard APIs being available, operators and manufacturers still create private APIs, which must then be supported on an ongoing basis and contain a slightly different feature set from the standard so content cannot easily migrate to use a new API.

What’s needed is a decentralised process which allows platform owners and service providers to define and implement their own APIs.

Although at first sight this would appear to create fragmentation, it doesn’t provided that there’s also a way of writing and deploying libraries or wrappers that can implement one API on top of another.

It’s analogous to using WSDL to specify web services – there doesn’t need to be a centralised definition of any given API because the relationship between alternative APIs is explicit and users of those APIs can react by writing wrappers that implement one in terms of the other, or write their own higher level abstractions that can sit on top of either.

The proliferation of widgets in desktop web apps shows the power of this decentralised approach. The potential role of widgets in mobile web apps is something Ajit discusses extensively in his book.

How to make it happen?

None of these issues requires new technology – the industry has the capability to do all of it. But it will only happen if the market incentivises the relevant parties – the browser providers and device manufacturers – to make it happen. The operator, in principle, is sufficiently resourced and has sufficient control to do this – but so far the operator community doesn’t seem to have realised the significance of the opportunity, let alone confronted the issues that have to be resolved.

Mobile web 2.0: AJAX for mobile devices – why mobile AJAX will replace both J2ME and XHTML as the preferred platform for mobile applications development

opera.JPG

Recently, Opera announced the availability of AJAX on mobile devices through their browser. Considering the popularity of Opera in the browser market(especially in the mobile browser market), this announcement is indeed very significant and has been picked up by many A-list bloggers such as Russell Beattie and Om Malik

However, I believe there is more at stake here ..

Having been involved in creating mobile services for a few years now, I believe AJAX will replace both J2ME and XHTML as the platform of choice for developing mobile applications.

In this article, I will outline my reasoning.

Before I do so, a caveat – I believe that mobile web 2.0 is far more than ‘AJAX on mobile’. Essentially mobile web 2.0 involves applying all seven of the web 2.0 principles to mobility. I will be discussing mobile web 2.0 in subsequent blogs. For a more complete discussion see my article on mobile web 2.0.

Here, I am discussing AJAX only i.e. only one facet of web 2.0.

Overview

In this article, we will discuss

1) What is AJAX (an overview)

2) Current Mobile applications development models

3) Problems the industry faces (in other words shortcomings of the current mobile applications models) and finally..

4) Why AJAX will replace J2ME and XHTML as the preferred development platform

What is AJAX

AJAX is an optional addition to web 2.0. It is not a single technology. Rather, it’s a combination of a number of existing technologies acting together namely

• XHTML and CSS for standards based presentation

• Document Object Model for dynamic display and interaction

• XML and XSLT for data interchange and manipulation

• XMLHttpRequest for asynchronous data retrieval and

• Javascript to tie everything together

Until AJAX came along, it was not easy to replicate the rich and responsive interaction design of native applications. AJAX is different from other previous attempts in addressing this problem since it is based on existing, non-proprietary standards which are already familiar to developers.

In traditional web applications, most user action triggers an HTTP request. The server does some processing and returns the result back to the user. While the server is processing, the user waits! The ‘start-stop-start’ nature of web applications is good from a technical standpoint but not from a user interaction standpoint (since almost all user interaction is resulting in trips to the server and the user is waiting while the server is doing the work).

AJAX solves this problem by using the AJAX engine. At the start of the session, the AJAX application loads the AJAX engine. The AJAX engine is written in Javascript as a Javascript library and sits in a hidden frame. The user interacts with the AJAX engine instead of the webserver. If the user interaction does not require a trip to the server, the AJAX engine handles the interaction on it’s own. When the user interaction needs some data from the server, the AJAX engine makes a call asynchronously (via XML/XMLHttpRequest API ) without interrupting the user’s flow.

In this sense, AJAX is ‘asynchronous’ because the AJAX engine is communicating with the server asynchronously to the user interaction. Thus, the user gets a seamless experience(i.e. the user is not waiting)

There is a momentum behind AJAX at the moment. Developers are already familiar with the technologies underlying AJAX. All the technologies making up AJAX are mature and stable. AJAX is the foundation for many new applications on the web like Google suggest , Google Maps , some features of Flickr and Amazon’s A9.com

Mobile applications development models and their shortcomings

From the above discussion and from the articles referenced , we can see that – AJAX clearly solves two problems – namely a superior UI and a standardised form of data retrieval.

These two problems also apply to mobile devices and by extension, AJAX addresses them as well.

However, I believe that it does far more!

Specifically, it solves the following problems in the mobile context.

a) The problem of market fragmentation

b) Porting woes (specific to downloading applications like those built on J2ME)

c) Application distribution without ‘walls’

Besides, it has the developer community behind it – which is a significant plus!

Lets consider existing mobile applications development. There are two principal ways to categorise mobile applications – Browsing applications and Downloading applications. There are others(like Messaging applications, SIM applications and embedded applications) – but a vast majority of the applications we see today fall under downloading or browsing applications.

Browsing applications: Browsing applications are conceptually the same as browsing the web but take into account limitations which are unique to mobility (for example – small device sizes). Similar to the web, the service is accessed through a microbrowser which uses a URL to locate a service on a wireless web server. The client is capable of little or no processing.

Downloading applications (Smart client applications) : In contrast to browsing applications, downloading applications are applications that are first downloaded and installed on the client device. The application then runs locally on the device. Unlike the browsing application, a downloaded(or smart client) application does not need to be connected to the network when it runs. Downloading applications are also called ‘smart client’ applications because the client(i.e. the mobile device) is capable of some processing and / or some persistent storage(caching). Currently, most Java based games are downloaded applications i.e. they are downloaded to the client, require some processing to be performed on the client and need not be always connected to the network. Enterprise mobile applications such as sales force automation are often also examples of smart client applications.

J2ME is the most common mode of developing downloading applications and XHTML is most common way of developing browsing applications.

Let us elaborate on the problems I have outlined before and then discuss how AJAX will solve them – potentially making XHTML and J2ME less relevant.

Problem One – Market fragmentation

Mobile applications are primarily consumer applications. The mobile data industry is an emerging industry. As with many industries in this phase of evolution, it is fragmented.

To be commercially viable (especially considering the need for the network effect ), consumer applications need a large target audience.

This can come about either by a single proprietary standard such as BREW from Qualcomm (which obviously has it’s disadvantages) or through open standards not controlled by any one entity with few industry barriers.

To illustrate how market fragmentation affects commercial viability of a new service, I often recommend the following approach (Most of the figures can be easily obtained from the web).

The idea is to think in terms of ‘concentric circles’ in trying to find out the target audience for your application. A sample set of steps I use is as below

a) What is the population of the country where you are launching your application?

b) What is the percentage of handset penetration amongst this population?

c) Which operators are you targeting within this population? (Most countries have more than one mobile operator)

d) Which handsets are you targeting within this population (not all operators support all handsets)?

e) What is the technology of deployment for example Java, SMS, WAP etc?

f) Does the application have any special technology needs such as location-based services? How many people have handsets equipped with this technology?

g) What does a segmentation analysis of the subset reveal? (Simplest segmentation is male/female. Prepay/postpay etc)

h) What are the channels to market for the segments we are targeting?

i) What proportion of this subset do we expect to hit and convert to customers based on our marketing budget?(i.e. the conversion rate which can be typically 2% )

This will give you your target audience.

Thus, this target audience times number of potential downloads per month should give you an idea of your monthly revenue. This could then be tied against your cost base including your development costs, porting costs etc to arrive at a more tangible picture of success/failure of the new service.

The above methodology illustrates the problem of fragmentation and it implies that very few mobile services are profitable today. Thus, we have a proliferation of ‘broadcast content applications’ – ex ringtones, pictures but very few utility applications at a mass-market level.

Problem two – Porting woes

This problem is specific to downloaded applications (and more commonly J2ME). Write once run anywhere is a joke in the mobile context! – and through no fault of Sun ..

Consider the case of mobile games(a downloaded application) typically developed using J2ME.

First the good news ..

• Carriers such as Sprint and Vodafone report that mobile games and other data services now account for roughly 10 percent of their annual revenues;

• Industry consulting firm Ovum notes that there are now more than 450 million Java-enabled handsets globally, in addition to the 38 million and 15 million BREW- and Symbian-enabled handsets;

• Mobile-game publishers racked up $1.2 billion in global sales in 2004 and expect an even stronger year in 2005 as more and more consumers discover the tiny gaming consoles already in their pockets.

BUT then the pitfalls ..

Game porting generally requires developers to adapt to differences in screen resolution, processor speed, memory thresholds, and sound capabilities, all of which can vary wildly from device to device. For publishers, this can not only exponentially increase game development and asset creation time, but can also cause them to miss critical time-to-market windows in a hyper-competitive industry. As an example, imagine that you are a mid-sized game publisher with 30 games in your portfolio. To make your games available worldwide in five languages and on only 50 devices, you would need to create 7,500 different builds. At $2,500 per build, you would require a budget of nearly $19 million simply to handle porting.

This limits the business model severely and very few mobile games are profitable.

(original source for this section as per my blog Porting – the big barrier to entry with acknowledgements to Sameer Bhatia as per the blog)

Problem three – Application distribution without walls

The predicament of using J2ME as per the preceding example shows that it’s not enough to merely set up a community process as Sun has done (which works fine as far as the technology is concerned). The technology and the applications built upon it must remain homogeneous and interoperable to enable the network effect and gain critical mass. The fewer the ‘choke points’ for a platform – the better it is for the industry as a whole.

We will discuss this more in the next section(where we talk of how AJAX could address this problem).

Why will AJAX replace J2ME and XHTML as the preferred development platform? Can AJAX solve the preceding problems?

In my view .. YES

AJAX is accessed through the browser. There are two ways a customer can get the browser – either the browser can be pre-installed on the phone by the manufacturer or it can be installed as a separate application

Anyone can download a browser for a smartphone as this Opera link shows for series 60 phones

This means, all customers can potentially install their own browser and if enough people do – we have critical mass with few ‘choke points’ – such as specific restrictions created by mobile operators. In other words, a means to bypass the walled garden.

Further, AJAX offers a superior user experience and already has the developer community supporting it. The possibility of attaining critical mass (due to fewer choke points) means more chance of monetising the application – leading to a virtuous circle of better applications.

J2ME as it stands today, is seriously flawed(not the technology but the business model). XHTML will be an ‘also ran’ because AJAX will offer a superior user experience.

Hence, my belief that AJAX will be the preferred platform of choice for mobile applications at the expense of J2ME and XHTL.

Supporting notes

a) I have said ‘preferred’ and not ‘replace’ i.e. I don’t expect AJAX to replace any technology

b) AJAX won’t solve all problems. You still need to create a service which is useful for mobile customers

c) AJAX is not the only attempt to create a better interface. There have been others with limited success but they are not across the industry(or are proprietary). For example mobile SVG from bitflash , superscape’s swerve technology for 3D gaming (which is the implementation of JSR 184 – Mobile 3D Graphics API for J2ME™ ) and macromedia mobile

d) Not a lot of people are actually browsing the mobile internet. Although WAP usage shows phenomenal growth, these figures include the use of WAP as a transport mechanism – typically for downloading content. In other words, every time you download a ringtone, you implicitly create a WAP page impression. I suspect the real figures used by consumers to actually browse the mobile internet are very low

e) Very few mobile operators have tried to engage with the developer community as such. Practically the only example I can think of is source o2

f) The plight of small developers can be illustrated from my discussions with a Korean vendor when I spoke at imobicon in Seoul. The vendor had finally managed to get his game listed on a UK portal. However, that was because – a Korean aggregator managed to get a deal with a UK aggregator. Thus, he now had two aggregators and one operator taking a slice of revenue! Leaving him with very little. A sad state of affairs. Surely, there must be a way to create and distribute applications globally i.e. you write for the browser and anyone who uses that browser can download and run your application

g) Mobile operators often argue that they handle billing and location services etc. That’s fine – but let’s first worry about getting the numbers. Also, billing comes at a cost and there may be better billing mechanisms on the web.

Conclusions

To recap, Mobile applications are primarily consumer focussed. They need critical mass. Currently, the market is fragmented and the current commercial model is broken.

AJAX offers a potentially better solution in comparison to the incumbents (J2ME and XHTML) due to a combination of fewer potential choke points because of its distribution mechanism. The economic models do not favour J2ME and AJAX offers a superior user experience to XHTML. It has the support of the developer community.

Finally, note that I say AJAX will be ‘preferred’ model and not the ‘only’ model. I don’t expect AJAX to replace either J2ME or XHTML.

Comments welcome: Ajit.Jaokar at futuretext.com

Useful links: Opera AJAX announcement

Dion Hinchcliffe’s The Incredible Ongoing Story of Ajax

Image source: www.opera.com

Permanent Link: http://opengardensblog.futuretext.com/archives/2006/01/mobile_web_20_a.html

For part two see HERE

porting: The big barrier to entry ..

ports.bmp

The goal of the OpenGardens blog is to discuss channels to market. And currently the marketplace is very fragmented. I found this article fascinating. It shows how fragmented the marketplace is -and how costly it is to get critical mass.

Sameer Bhatia says:

First the potential

• In China, there are 300 million mobile subscribers; this is expected to grow to 500 million within the next two years;

• Carriers such as Sprint and Vodafone report that mobile games and other data services now account for roughly 10 percent of their annual revenues;

• Industry consulting firm Ovum notes that there are now more than 450 million Java-enabled handsets globally, in addition to the 38 million and 15 million BREW- and Symbian-enabled handsets;

• Mobile-game publishers racked up $1.2 billion in global sales in 2004 and expect an even stronger year in 2005 as more and more consumers discover the tiny gaming consoles already in their pockets.

BUT then the pitfalls ..

When a mobile game is developed, it is made for one or two key handsets. In order to get that game to work on any other cell phone, the game has to be ported – adapted to work – on each and every other handset in the market. As the absolute number and type of handsets in the market grows almost exponentially, one can imagine the large-scale problem that game publishers face. The problem of disparate handset specifications is further compounded by different operating systems and programming languages, the most common of which are J2ME and BREW. The result? An overwhelming problem that renders a mobile game useless until it can work on the vast majority of handsets in the market.

Game porting generally requires developers to adapt to differences in screen resolution, processor speed, memory thresholds, and sound capabilities, all of which can vary wildly from device to device. For publishers, this can not only exponentially increase game development and asset creation time, but can also cause them to miss critical time-to-market windows in a hyper-competitive industry.

As an example, imagine that you are a mid-sized game publisher with 30 games in your portfolio. To make your games available worldwide in five languages and on only 50 devices, you would need to create 7,500 different builds. At $2,500 per build, you would require a budget of nearly $19 million simply to handle porting. Spending the money to get the ports done is normally not an issue for most publishers – a successful game will easily make them money. However, finding the time and resources to develop all of these ports is major problem.

I have heard the same feedback from many developers. Porting increases cost base BIG TIME!. But you need to port to reach a large sector of the market

This, sadly – is why I am not so optimistic about single player games. Granted that some branded games are making money – but most players are not. But very few will admit it!

source: mobenta/Sameer Bhatia

Image source: HERE

template to assess demand of a new service..

assessing demand.jpg

I use this example often when trying to assess the size of the market opportunity. It often comes back with a sobering picture. You would have to modify this methodology to suit your own application but the basic steps are the same – i.e. think in terms of concentric circles.

Start with the largest population base and work inwards. Some of these stats are not easy to obtain(for example – the population of a country may be easy to find but not the percentage of handsets supporting J2ME in that country). In that case, if you are really serious, I recommend you actually buy some of these stats from research companies. On the long run, they may save you more money!

A sample set of steps I use is as below

a) What is the population of the country?

b) What is the percentage of handset penetration amongst this population

c) Which operators are you targeting within this population?

d) Which handsets are you targeting within this population?

e) What is the technology of deployment for example Java, SMS, WAP etc?

f) Does the application have any special technology needs such as location-based services? How many people have handsets equipped with this technology?

g) What does a segmentation analysis of the subset reveal?(simplest segmentation is male/female. Prepay/postpay etc)

h) What are the distribution channels to market for the segments we are targeting?

i) What proportion of this subset do we expect to hit and convert to customers based on our marketing budget?(i.e. our marketing dollars)

This will give you your target audience.

Thus, your target audience subset times number of potential downloads per month should give you an idea of your monthly revenue. This could then be tied against your cost base including your development costs, porting costs etc to arrive at a more tangible picture of success/failure of the new service.

source: www.opengardens.futuretext.com. All other sources as indicated