Dream(sheep++): A developer’s introduction to Google Android

Do androids dream of electric penguins?

In the decade since its founding in a Palo Alto garage, the name "Google" has become practically synonymous with the Internet. Thus it was that the search company's celebrated entry into the mobile market was met with significant enthusiasm from those who believed that Google would be able to use its immense resources and Internet savvy to produce a next-generation mobile product that would deliver "the cloud," in its vast entirety, into the eager hands of consumers. Some of the biggest names in the tech industry flocked to Google's banner and affirmed their support for the Open Handset Alliance, which promised to liberate the mobile masses by building a blooming garden without walls.

After the fanfare faded, we ended up with Android—a platform that launched with some limitations but nonetheless has significant potential. Although the first Android devices leave a lot to be desired when compared to competing products, the platform itself is evolving quickly, and it offers the advantages of openness and collaborative development. In this article we'll take a close look at the underlying technology of Android and what the platform means for developers.

Design philosophy

Although Android is built on top of the Linux kernel, the platform has very little in common with the conventional desktop Linux stack. In fact, during a presentation at the Google IO conference, Google engineer Patrick Brady stated unambiguously that Android is not Linux.

Much of the Android userspace operates within the constraints of Dalvik, Google's own custom Java virtual machine. Dalvik uses its own bytecode format called Dex, and is not compatible with J2ME or other Java runtime environments. Third-party Android applications are written in Java using Android's official APIs and widget toolkit. The Android SDK includes special compilation tools that will translate Java class files into Dex bytecode and generate an installation package that can be deployed on Android devices.

Android's radically different approach to mobile Linux application development offers some unique advantages, but it also creates a lot of challenges for third-party developers. The biggest advantage is that it provides a very high level of uniformity. In theory, the vast majority of Android applications will be able to run seamlessly on virtually any Android-based device without requiring any further modification.

The problem with Google's approach is that it makes Android an island. The highly insular nature of the platform prevents Android users and developers from taking advantage of the rich ecosystem of existing third-party Linux applications. Android doesn't officially support native C programs at all, so it won't be possible to port your favorite GTK+ or Qt applications to Android. It's also not possible to run existing MIDP applications on Android because it uses an incompatible virtual machine.

Other prominent mobile Linux platform initiatives have taken a very different approach and are making extensive use of existing desktop Linux technologies. Maemo, OpenMoko, ALP, MOTOMAGX, Moblin, and Qt Extended all provide some kind of portability glide path between the desktop and other mobile platforms. For example, it's trivially easy to port an existing GTK+ desktop application to Moblin, ALP, and Maemo devices.

The downside of the GTK+ approach compared to Android is that the GTK+ application will have to be modified to accommodate the hardware capabilities, form factor, and toolkit deviations for each individual device. Android attempts to circumvent that problem by providing its own universal runtime and toolkit. It is too early to judge whether Android's approach is successful because the number of Android-based devices is still somewhat limited.

The SDK and API

Android supplies a comprehensive and well-organized assortment of high-level APIs for building applications and leveraging the underlying functionality of the platform. The APIs provide an extremely high level of abstraction, which makes them relatively intuitive and easy to use.

Third-party applications can replicate or interface with virtually every major component of the platform. For example, Android provides methods for retrieving information from the user's contact list and for extending the contact system with new data fields. It's also easy to make a new dialer or implement custom behaviors for system events like incoming SMS messages. This is one area in particular where I really feel like Google exceeded expectations and delivered on its promises: The APIs truly make it possible to build applications that integrate fully with the rest of the platform.

The Android widget toolkit provides a lot of very useful components right out of the box. The individual widgets are designed specifically for finger-friendly interaction, with features like kinetic scrolling already built in. Developers use an XML-based user interface description language to specify the layout and attributes of the widgets, with the XML descriptions being loaded into the program through the Android resource system. The individual widgets described in the XML layout can be referenced by ID in the program. It's also possible to create widgets programmatically and manipulate the user interface at runtime.

The preferred method for creating layouts is to write the XML descriptions by hand. The Android SDK provides a somewhat primitive visual layout tool that integrates with Eclipse, but it doesn't support all of the widgets, and it doesn't always work consistently. There is also a hierarchy viewer included with the SDK that can be used for layout debugging. The lack of a robust visual layout tool is a bit of a weakness for Android compared to other mobile development solutions, particularly Microsoft's Visual Studio. It's likely that the tools will mature and catch up as Android evolves.

Closer look at the platform

Google has invested a lot of resources into slimming down the platform and making it as efficient as possible. Optimizations primarily focus on reducing size, improving speed, conserving battery power, and lowering the overall memory overhead. This work has been done at several different layers of the stack.

The Dalvik virtual machine was designed carefully with the aim of meeting those performance requirements, and it has a few unusual characteristics. Unlike the conventional Java runtime, Dalvik is register-based instead of stack-based and doesn't support JIT compilation. Each individual application runs in a separate Dalvik VM instance, and memory is shared between these instances to reduce overhead. To improve application startup time, Dalvik has a component called "zygote" that pre-initializes VM instances and forks them as needed.

Another core component of Android that was specially crafted for performance is Bionic, the platform's custom implementation of the standard C library. At only 200K, the highly compact Bionic library is roughly half the size of GNU's libc. It is designed to offer good performance on CPU-constrained devices, and it has its own custom, high-performance pthread implementation. It's not directly compatible with glibc, however, and it omits some POSIX features such as support for wide characters. It has built-in support for some platform-level Android services, including system properties and logging.

Dalvik and Harmony

Dalvik leverages many components from Harmony's class library. The Harmony project is an open source Java implementation that is distributed under the Apache license and is maintained under the umbrella of the Apache Foundation. Google developed parts of the stack to use faster native libraries—such as ICU for character encoding and OpenSSL for encryption.

The Dalvik and Harmony code bases diverged significantly during Google's intense push to get the G1 to market. Now that the product has been released and focus is shifting back toward maintenance, Google is working on narrowing the gap and merging in improvements that were made to Harmony after Dalvik's code was forked. Google also plans to help move some of its own improvements back upstream.

The Harmony community's response to Android has been very positive, and the developers are eager to collaborate with Google for their mutual benefit. Prospects for collaboration were discussed extensively on the Harmony mailing list, where various technical and licensing issues were addressed.

"Now that the dust is settling, we are beginning to contemplate how best to get our library code back in sync with the Harmony mainline. We have changes we've made that we are reasonably sure would be useful to Harmony, and we've made others we are pretty sure don't make sense for Harmony's primary target of desktop/server class machines," wrote Google developer Dan Bornstein in an announcement on the Harmony mailing list. "The Android core library folks have been spending a lot of time lately trying to bring the core library sources up to date with respect to the last year and a half of Harmony development."

Although a lot of care clearly went into Dalvik's design, there are currently numerous contradictory claims about its performance. These should generally be taken with a grain of salt. Reliable benchmarks on the relative performance of mobile runtimes are difficult to conduct, but some preliminary experiments by various developers indicate that Dalvik isn't necessarily faster than other comparable technologies.

For example, some tests show that Mono, an open source .NET runtime, consumes less memory and delivers faster runtime performance than Dalvik on the G1. One of the engineers at image analysis company Occipital also reports that Objective-C code on an iPhone significantly outperforms Dalvik code running on a G1. These aren't particularly scientific benchmarks and they shouldn't be taken at face value, but they show that there are still unanswered questions about the performance advantages of Dalvik's design.

Licensing

In many cases, it seems like licensing considerations had a profound influence on technical decisions. The Android developers clearly went to great lengths to avoid using code that is distributed under GNU's General Public License (GPL). Google's decision to build a new libc from scratch and adopt Harmony are both partially attributed to a strong preference for permissive licensing. The vast majority of the Android middleware stack is distributed under the Apache Software License (ASL), which doesn't impose the reciprocal (copyleft) requirements of the GPL.

The pervasive use of ASL throughout the Android middleware layer makes it possible for handset makers and mobile carriers to make proprietary customizations on top of the platform stack. Although it has attracted some criticism, Google's use of the ASL is really very reasonable and makes a lot of sense, as we explained in our detailed look at Android licensing.

The platform's adoption potential is largely predicated on how much flexibility it gives vendors. Distributing the platform under a permissive license instead of a copyleft license gives commercial Android adopters more latitude for differentiating their products from others in the market. ASL is also compatible with GPLv3, which means that developers who want to integrate Android code into GPL-licensed projects and add reciprocal requirements to their own derivatives can do so.

The one major exception to Android's permissive licensing preference is the kernel itself. The Linux kernel is distributed under the GPLv2 license. In looking at the Android stack, it's almost surprising that they used the Linux kernel at all. It seems like a BSD kernel would have been much more closely aligned with their technical and licensing philosophies.

Building blocks

Android applications consist of content providers, services, broadcast receivers, and activities. These components all serve specific purposes in the Android application stack. The manner in which they can interact with each other is what imbues Android with its remarkable modularity.

Content providers

Content providers serve as an abstraction layer for interacting with various data sources and for sharing persistent data between applications. They expose information through a standardized query interface. Queries are described with a URI syntax, but developers typically use higher-level wrapper classes that generate and manage the query strings. When a program sends a query to a content provider, the response is a cursor object that gives the program database-like access to the underlying content. It is also possible to hook monitoring mechanisms into content providers so that your application can be notified when data changes.

The content provider system offers several significant advantages to Android developers. The most noteworthy advantage is that the system facilitates a high level of interoperability by allowing applications to share data in a very uniform way. Several key platform data sources—including the Android contact list and multimedia storage systems—are exposed through content provider interfaces by default so that they can be easily leveraged from third-party applications.

Background processes

A significant feature that differentiates Android from the iPhone platform is that Android officially supports background processes in third-party applications. These are implemented with Android's service component. (Services are headless operations that run in the background for extended durations.)

An interprocess-communication mechanism facilitates communication with services. The Android Interface Definition Language (AIDL) is used to describe the programmatic interface of the methods that the developer wants to expose through the IPC system. During the build process, the AIDL files will be used to automatically generate Java stubs.

Broadcast receivers

Android's broadcast system is roughly analogous to the concept of D-Bus signals on the Linux desktop. Developers implement broadcast receivers which can detect when specific system messages or events are emitted and perform certain behaviors in response. Programs emit their own messages that can be intercepted by other applications.

Many underlying system events can be monitored with the broadcast system, so it can be used to hook into things like battery status changes, incoming SMS message arrival, hardware button presses, GTalk connection changes, storage mounting, and screen blacking.

Intents

Android's "intent" system is the key to understanding how all of these pieces are used together. Intent objects—which contain an action ID and a data URI—are used to invoke activities, services, and receivers. The action ID specifies a desired behavior, and the optional data URI provides the location of the data on which the action should operate.

For example, if an application wants to launch the dialer and punch in a phone number, it would perform ACTION_DIAL and provide the phone number as either a telephone URI (such as tel:8888888888) or an address book provider URI (such as content://contacts/1). The resulting intent object can be executed by passing it to the startActivity method.

For testing purposes, there is a really useful utility called Intent Playground that will let you invoke arbitrary intents. It even has autocompletion for actions, which makes it a great way to explore the intent system.

Learning more

From the perspective of a programmer, these components of the Android platform necessitate an approach that is radically different from conventional desktop development in many ways. Android is built on a different set of paradigms with new nomenclature, and it is endowed with unfamiliar capabilities and limitations.

All of this newness can make the platform seem a bit intimidating, even to veteran Linux and Java developers. The significance of the individual pieces and the relationships between them aren't always clear at first, so the richness of Android's modularity and flexibility aren't immediately apparent from the surface. There is a lot there to learn, and there are still many aspects of the platform that I haven't delved into with sufficient depth myself.

It's very difficult to convey the true nature of the platform through a top-down overview. In order to really comprehend how it all works and cultivate a more complete understanding, I encourage you to dig in for yourself and experiment. Although the initial learning curve is steep, it gets a lot easier once you have developed a solid grasp on the basics. This is because virtually all of the platform's advanced functionality is exposed in a somewhat consistent way through internally standardized mechanisms.

Open platform

One of the distinguishing characteristics of Android is that it's a very open platform. When Android was first announced, Google promised that the Open Handset Alliance would bring unprecedented openness to the mobile industry. At the LugRadio Live event earlier this year, Google developer Robert Love told the audience that "the goal of Android is to be open to developers, open to the industry, and open to users."

Although Google espoused some lofty ideals during the early stages of development, the company's conduct prior to release fell short in many ways of what was promised. The development process took place entirely behind closed doors, and the only players that had input were Google's business partners. The entire undertaking lacked inclusiveness, and many details about the platform itself were kept tightly under wraps. The vast majority of third-party developers were left out in the cold and were denied access to SDK updates for months while Google's programming contest finalists were given the latest versions under non-disclosure agreements.

This situation alienated some members of the third-party developer community. Several of Google's own engineers were also deeply frustrated with the way that things had unfolded, and they attempted without success to lobby internally for more transparency.

As Google originally promised, things improved dramatically after the launch of the G1. The source code for the entire platform is now open, and Google has published extensive documentation that describes how independent developers can contribute to the project. Those changes in the development process make Android a truly open and participatory project. Patches from external contributors have already been accepted, and Google is also working closely with upstream projects like Harmony.

After the source code was opened, there was still one critical weak spot: the T-Mobile G1, Android's flagship handset, is a closed device that uses code signing to restrict changes to the platform. There is no way to flash the G1 with modified images, which means that platform hackers have no practical way to test their changes on physical hardware. This limitation was an immense disappointment, and it undermined a lot of the value of having an open mobile platform. To address this deficiency, Google launched its own unlocked developer model of the G1 handset. The hackable Google handset, which is available to anyone who registers with the Android App Store, is a fully open device that can be flashed and modified.

Despite a very rough start, Google has fulfilled most of its openness promises. It has delivered an open platform and is making available an open device for developers and enthusiasts who want maximum flexibility. In retrospect, it seems like the lack of transparency that plagued the project prior to the launch of the G1 was largely the consequence of a relentlessly intense timeline and pressure from the mobile carriers and other partners. Google probably overestimated the amount of leverage it would have with its launch partners, and was forced to make serious compromises along the way in order to get a product to market in a timely manner. Now that the G1 has landed, Google is doing its part to remedy the mistakes and accommodate the third-party developer community.

This is all more or less acceptable for a first product, but developers can and should expect more transparency in the future. Fortunately, it seems like Google is on track to deliver that. The source code of the next major version of the Android platform is already accessible to third-party developers.

A tasty Cupcake

During the initial Android rollout, when Google was working to make the G1 source code public, most of the active development work was taking place in a separate, private branch. Google began making this development branch, which became known as Cupcake, available to the public a few months later. It was basically a massive code drop of all the work that was done after the 1.0 release. Google has since merged the Cupcake code into the master branch.

The next major version of Android delivers some important features that will help make the platform more appealing to mobile carriers and hardware makers. One of the most important changes is the new on-screen keyboard, which has opened the door for using Android on a whole new class of devices. There are already several products under development that will take advantage of this feature, including an upcoming media tablet from Archos, a WiFi Skype tablet from GiiNii, and the HTC Magic, which is coming to Vodafone.

Another significant addition is support for the x86 architecture, which could make it possible to bring Android to some netbook devices and Atom-based MIDs.

A wide range of other improvements and bugfixes are also in place for the next release, including support for recording video, new Bluetooth profiles (A2DP and AVRCP), the ability to launch applications with the voice dialer, better POP3 and IMAP mail support, and the ability to save MMS attachments.

Performance and stability should be better throughout much of the platform, too. The web browser is one area where users could notice big improvements, since the underlying drawing system used by the web renderer is now much faster. Google has also adopted WebKit's SquirrelFish JavaScript engine, which will help to improve JavaScript performance. It's important to note that Android is using the regular flavor of SquirrelFish and not SquirrelFish Extreme, which is not supported yet on ARM. Google's decision not to use its own V8 engine reflects the fact that V8's performance advantages still haven't really been brought over to ARM, even though V8 technically does have ARM support.

In all, Android development is moving along nicely, and the new features and fixes will do much to increase the robustness and desirability of the platform. It's definitely more competitive today than it was when the G1 launched. The new features are already in pretty good condition, as shown by the growing number of prototype devices that are taking advantage of them.

Multitouch

The availability of multitouch support in Android is a question that seems to come up a lot. The bottom line is that it seems to be possible, but it's not entirely clear yet why it's not being used.

The G1 capacitive touchscreen uses Synaptics' ClearPad, which has also been adopted by RIM for touchscreen Blackberry products. Although the G1 user interface doesn't take advantage of multitouch technology, the device could theoretically have everything it needs to support multitouch interaction. The ClearPad hardware is designed to handle advanced gestures—such as finger pinch zooming—and the drivers for handling this behavior are actually included in Android. Synaptics is a member of Google's Open Handset Alliance and contributed support for implementing "advanced, multi-touch gestures for the Android platform."