whitenoise

technology notes… mobile and embedded.

Archive for the ‘technology’ Category

An Android 101: Hardware and HAL

with 2 comments

Unlike traditional Linux based systems, Android applications communicate with the underlying hardware through Java APIs not by system calls.

image

             Figure 1: Android vs a Tradition Linux system.

Most of the ‘shared libs’ in the above diagram represent the HAL modules.  The Hardware Abstraction Layer or HAL is the glue between any device (part of the kernel) and the corresponding JNI interface.

The details about the HAL is usually not of interest to application developers. However it is an important module for hardware vendors and OEMs involved in  porting android into their own hardware and adding new hardware.

Folder structure:
   The HAL source code reside under the hardware folder . OEMs maintain a top level root directory and internally there are several HAL modules for the corresponding hardware.

On device ,the HALs live as .so’s  either in ‘/system/lib/hw’ or ‘/vendor/lib/hw’. The HAL modules follow a certain nomenclature as defined here . Android relies on this naming convention to load the right HAL module for a given hardware and board configuration.

image

Runtime:
The HALs get loaded by its clients using the  hw_get_module. This method relies on the values set to ro.hardware , ro.product.board and some other ro properties to load the correct .so. The complete list is here.
For eg. The SurfaceFlinger is the system wide compositor and the hardware composer is a HAL that abstracts device specific composition logic. The SurfaceFlinger loads the hardware composer using the hw_get_module method to get a handle of the HAL.

Each HAL implements a set of hooks which get called by their invoking service. The contract between the system service and the HAL is well defined. Device makers should ensure these interfaces are implemented as expected by the Android services. The figure below tries to show some of the HALs on the Android ICS version and key methods that HAL writes should implement.

image

With the newer versions of Android ,the scope of HALs have expanded and it is likely they will be used not just to abstract hardware but to abstract any device specific behavior.

References:

1. Good site about  adding support for a Camera : http://processors.wiki.ti.com/index.php/TI-Android-DevKit-Camera_Porting_Guide
2. Patrick Brady’s Google I/O talk: https://sites.google.com/site/io/anatomy–physiology-of-an-android
3. Figure 1 courtesy: http://www.cmg.org/measureit/issues/mit78/m_78_3.pdf
4, Class diagrams created using Umbrello ,an open source UML Modeler.

Update:  Here is a new post on the same topic with much more detail:  http://www.opersys.com/blog/extending-android-hal

Advertisements

Written by sujai

June 25, 2012 at 7:42 am

An Android 101 : An overview on Binder framework.

with 2 comments

While the underlying platform makes use of Linux concepts, the Android applications are pretty unique. Unlike traditional Java apps, Android apps do not have a an entry point (like a main() function ).  The framework could invoke a particular entry point within an application to achieve the requested functionality.
Each android application is a mixture of several components . The components can be broadly grouped into 4 ‘component types’.  You could read more about components here.

android_component_system
[Image credit : Android binder thesis by  Thorsten Schreiber]

The different components may need to  exchange data and it is realized through inter component/ inter process communication .The communication works with so called intents.Intents are representations for actions to  be performed. Each intent is made of a URI and an ‘action’. The URI identifies the destination and action defines the operation to be performed on the component identified by the URI.  As we learn about components and interactions it becomes increasingly evident that IPC forms a major part of  the Android framework.

The Binder framework :
Unlike traditional Linux machines, Android uses its own mechanism for IPC called the Binder. Binder was originally developed as OpenBinder and was used as the IPC in BeOS . The current Binder implementation in Android is a customized implementation of the OpenBinder. This was mainly to ensure the new implementation uses a license that is compatible with the Android’s user space code.

Each Binder object is an implementation of the IBinder implementation . An important difference between Binder and other IPC mechanisms is the use of  Binder token. A binder token is value that uniquely identifies any  Binder. This capability to uniquely identify a binder across processes allows the binder id to act as a ‘shared token’ across multiple processes.

Communication model:
The Binder framework communication is a client server model . Each client initiates communication and waits for response from the server. Each client would have a proxy  object for the client side communication. The server side constitutes a pool of worker threads.The server shall spawn a new thread for each new Binder request from the client. The bridge between the client and the server process is the binder driver. The Binder driver is a character device that is part of kernel space. This module  ensures the client reaches the appropriate destination  across process boundaries.

binder_commn

                                                                    [Image credit : Android binder thesis by  Thorsten Schreiber]

AIDL : As with any other RPC , there is a need to write a proxy and stub class that would be interacting with the client and server. Developers  write the interface   for their remote services using AIDL .The AIDL parser takes care of generating Java classes for the proxy and stub and logic to convert data into parcels ,that the Binder middleware understands.
If you are in the business of writing AIDLs you should  keep in mind that the Binder protocol is always ‘request gives response’ there is no inbuilt constructs to provide asynchronous functions. Asynchronous mechanisms have to be built on  top  of the Binder framework.

Binder vs SysV IPC: You may wonder why not reuse SysV IPC that comes with Linux. The Binder documentation has good explanation on why Google  has stayed away from using SysV IPC.

One of the design requirements of Binder is to ensure there is no ‘resource leak’ when any service dies improperly . i.e.Ensure that any kernel level resources (like semaphores/binder objects)  are cleared up in case the parent process is killed or crashes prematurely. There is no way to release a SysV semaphore if the process that created it is explicitly killed or it crashes. Killing processes automatically to make room for new ones is very common with the Android framework.

For an in-depth discussion on Binders, you could refer to some of the links below that I used to learn about Binders.

References:
1. eLinux Binder documentation : http://elinux.org/Android_Binder
2. A nice ppt with implementation details : http://blog.kmckk.com/archives/3676340.html
2. NDK reference documentation about SYS V IPC :  <NDK INSTALL ROOT>\docs\system\libc\SYSV-IPC.html .
3. Comparison of IPC mechanisms : http://lwn.net/Articles/466304/
4. A good overview on OpenBinder : http://www.osnews.com/story/13674/Introduction-to-OpenBinder-and-Interview-with-Dianne-Hackborn/

Written by sujai

December 28, 2011 at 11:27 pm

An Android 101 : Boot and platform init.

leave a comment »

Over the past few months I have been dabbling with Android  SDK and a some amount  of programming.  Sharing what I learnt .

53731_android_101_training_course_vancouver_255255255 

A Linux system:

Apart from the application framework, much of Android uses concepts from Linux directly. This includes file systems , partitions and init scripts (like the init.d). A typical Android device would have the following partitions :

image

Besides there may be some device specific partitions (/modem etc) that the OEMs may use for their own (non –android ) purpose.

Of these ,the recovery partition is particularly interesting. Normal android boot would start running the image in the  boot partition. However if you power on device in the ‘recovery mode’ , you could get started with an alternate boot image (the recovery image).

It was originally designed for OEMs to provide OTA upgrades. But custom ROM makes have come up with custom recovery images  that could flash other kernel images (like CynogenMod etc). A popular custom recovery image is the ClockworkMod Recovery.

AndroidBootSequence

Bootup:

When a normal boot happens, the Android platform starts executing code in the /boot partition .The bootloader then starts the kernel. The kernel does the basic initialization (of hardware ,memory subsystems) before mounting the root file system ( / ). Once / is mounted the ‘init’ process is started.

init.rc :
Like normal (desktop ) Linux systems ,Android init parses and runs the various commands as mentioned in the init.rc  script. The bulk of Android platform init happens here. The init.rc script mounts various partitions , It then starts  daemons like the adbd, service manager (responsible for IPC ).,rild, netd and others.  Finally the init.rc also invokes app_process which actually results in the process called zygote.

Zygote: Every new app is hosted on a dedicated VM. However each VM is not started afresh (not a cold start).  Zygote takes care of loading all system libraries beforehand . When a new app is started ,zygote forks itself to provide a new VM that has most components preloaded .
There is more detail on this here : http://elinux.org/Android_Zygote_Startup

SystemServer:   Once Zygote is fully launched, we have the VM initialized . The platform then starts the SystemServer.(frameworks/base/services/java/com/android/server/SystemServer.java)

SystemServer is the first Java component to run on the system . This module starts all ‘Android services’  like the PowerManager, ActivityManager etc. Once the SystemService completes, Android’s bootup is considered complete.

The platform sends a standard broadcast action called ‘ACTION_BOOT_COMPLETED ‘. Clients that want to do some action once boot up is complete could register for this module.

In the next post I will share about other components of Android , like its IPC mechanism (Binder)  and shared memory implementations (ASHMEM)

[Image courtesy: http://www.techvibes.com/ , http://hmtsay.blogspot.com/2010/10/android-startup.html ]

Written by sujai

November 7, 2011 at 7:15 pm

Innovative iPhone and Android apps– 2011 Winter edition

leave a comment »

After my first post this year, there have been many more interesting apps that caught my attention.It is pretty good to see a lot of innovation in the mobile industry by people outside of this domain. So here is my next list for the year. We could say it’s the ‘Winter edition’ of 2011!

The Invisible Universe:   Joshua Peek  is a fellow at the Columbia University studying about galaxies. As an astronomer he sees a lot of things in the space through his telescopes that we couldn’t see.  So he has come up with AR app called Invisible Space. The app is loaded with a database of full sky images of the galaxy.

When someone holds the phone up in the sky,this app would load the image of the galaxy at that orientation.. I liked this app for bringing in lot of informative stuff using simple techniques.

Here is a good intro video: 

 

 

This app is hosted here on the Android Market: https://market.android.com/details?id=com.lbi.iu .

PitPatt : The next app is pretty cool and pretty scary too. This is what happens when Minority Report meets Android.  Researchers at CMU have come up with an advanced Pattern recognition algorithm that allows anyone to know your real identity in under 60 seconds , by just taking a your picture with your smartphone.

Pitpatt has been acquired by Google and soon we could see this tech on Android.The app is not publically available yet.There is more detail on this app here.

Vocre :  I mentioned about translations apps in my  previous review too. This time there is one more vocre-logotranslator app that has won lot of accolades. Vocre from myLanguage can translate what you speak into nearly six different languages (including Mandarin).  Usage is pretty simple .You speak in your language and tilt the phone to translate. There is a nice demo at Vocre’s home page explaining the idea.

Appmates from Disney : My last innovative app is also the best on this list . This app is soo cool that I would buy and iPad just to try this app.  Disney’s new app allows you to drive real cars on virtual space and the virtual terrain reacts to the cars!! 

All you need is an iPad with the free Disney app and get one of those special  Lightning Mcqueen toys. Here is an elaborate demo.Have a look and you’ll be quite impressed.

 

 

The tech around this is a hot topic on Google + and I was able to glean some info on that. The inventor has couple patents on it.Now that is ONE cool idea to be patented !  Here is more info on this concept .

That’s it for now. Like many I would be eagerly following the ‘Lets Talk iPhone’ event today (Oct 4,2011). I am particularly interested to see what they are going to announce about Virtual Personal Assistant using the technology they acquired from Siri.

Written by sujai

October 4, 2011 at 7:04 pm

Kernel.org has been hacked.

leave a comment »

On Aug31 kernel.org broke the news that many of its servers have been compromised by unknown attackers. They broke in using a compromised user account and installed a rootkit that was silently monitoring user activity among other things. The gory details are best described here

Youve_Been_HackedIt should be pretty embarrassing when when this happens around the 20th anniversary  of Linux.

But the hack has made us all know some very important details about the integrity of kernel source
and the resilience of git itself against such attack.

Resilient Git For each file in the git repository a cryptographically secure hash is generated and the hash uniquely identified the content of that file along with its history.  So any modification into an old commit WITHOUT changing the hash is very difficult.
So from a source code perspective we could be fairly sure that no change could be injected onto it. This blog post at the Linux foundation explains it well.

However Aside from Git, kernel.org also hosts the signatures and some key components used to sign the kernel drops.
See the post here to know how the signing works. If the intruder got the private key ,then he could modify source (outside of git) and make tar archives and sign it.

This may be the reason ALL archives and patches hosted in kernel.org has been pulled down .Even the AOSP project  has pulled down its link ((link: http://android.git.kernel.org/) to the source code .So for now it is advisable not to take any of the archives hosted in the server until we hear an update on this.

I expect that the private key is updated and a new public key is put up here .Only then it would be safe to download the new kernels.

Damage control:
  Swift action has been taken on this .The passwords of all the users of the kernel.org (448 of them) have been reset . The attacker accessed the kernel.org using the credentials of one of the user. While all of this is happening the kernel source has temporarily moved to GitHub.

In the end,this entire episode has let many lay users(like me ) to learn more about git’s integrity and some insight on how the entire kernel.org release happens. Also there is no need for us to  worry about the kernel source being compromised now or even in the foreseeable future.

Written by sujai

September 6, 2011 at 12:17 am

Posted in technology, Uncategorized

Tagged with , , , , ,

Mobile fast-food ..thoughts on the new mobile phone making.

leave a comment »

I got chance to attend a recent gathering of several OEMs shipping phones to India.
The Qualcomm Partner Summit  is an annual event by home_09Qualcomm India. Apart from the marketing buzz, it also showcases the wares from many OEMs who plan to launch mobile devices in India and a good indication of what is in store for the next year in the mobile phone landscape.

A clear phenomenon you could see is arrival of several Chinese OEMs and ODMs in that market. Their growth has been phenomenal in past few years and they are already making good inroads into the market share of market leader Nokia . Until 2010 the small time OEMs were only  competing at the feature phone space but with Android the dynamics are going to change dramatically.

Chinese OEMs Indian Distributors: The Chinese OEMs do not have a brand presence in India .Their main customers  are ‘Distributors’ .Companies that make bulk orders of these devices and sell with their own brand name.

Unlike the OEMs ,the distributors do not involve in engineering or the manufacturing , they just give the requirements and book orders and there are mobile phone shops in China that can churn out models based on your need much like ordering breakfast at a Subway joint.

Some of the prominent distributors in India are Micromax and Lawa .And this list is growing rapidly.

Welcome Android : The high end(Tier1) OEMs like Nokia and Samsung still had an edge on the software that had been tested for good  usability, and had passed several rounds of QA.
There was lot of investment on  the Software platforms that was keeping  away the small time OEMs entering the premium segment like the smartphone.

With the advent of Android this is all set to change. Google’s Android platform comes for free and OEMs could churn out  a decent smart phone will all of ‘Android goodies’.  This was pretty evident in the summit I had attended.

HTCDesire_Lookalike There was  HTC-Desire  and few feet away there is an  exact replica by another less known  OEM.  Needless to say each OEM had their own version of an iPhone clone and an iPad clone.

So with Android , the Tier2 OEMs are not just going to be in Feature phones, they are going to come into smart phones  and even some iPad clones. Although there hasn’t been initial success with the integration , they have managed to bring down the Android smart phone price drastically in India.

It may be difficult for the Tier1 OEMs in the long run. For now they are able to maintain a lead by keeping their premium segment intact and also with some exclusive partnership deals with Google and Microsoft.

Written by sujai

May 9, 2011 at 8:34 am

On apps and security .

with 2 comments

While the appstore phenomena has brought is a lot of cool utilities to your smart phone, there are side effects too. Last week saw one of the most dramatic demonstration of what many have been fearing for sometime now. Malware on Smartphone!

And with some intelligent social engineering tricks, it could invade the ecosystem in the matter of days .

evil_android_thumb Several android web pages reported of a very powerful malware that got into the Android market place. Here is how the app installs the malware.

The malware publisher (goes by name Myournet) takes some of the  popular android games and then injects root exploits into the application package and republishes the apps back. All these are variants of the original popular apps, but but they are available FREE. Within days several users download the app and install it. A detailed report is here.

How it works: The malware actually installs a rootkit and ‘steals’ all personal information on your device and sends to a remote location. Besides the malware always opens up a backdoor with your device allowing more worms/malicious code to be run.  This blog post by a mobile security firm has the gory details.

Some points to note:
* The malware’s existence was not known by Google until someone posted about it in public domain .
* While Google was (somewhat) swift to respond , the damage was done. Private data from many phones have been sent to a nefarious user. Would have been very good if the app didn’t enter the ecosystem in the first place.
* Community to the rescue : Google responded by nuking the app with its kill switch. But then the first fixes came from members of XDA.

We can be happy that though the Android market is ‘open’ Google has put in some features in platform to recover back .

How about genuine apps ? Malware isn’t the only security threats to private data. The  Freedom to tinker website had a  blog post recently on the information shared by some of the popular Smartphone apps. The author had used sniffing tools like wire shark to sniff his Android phone.

F5T0XHAF04FMXU0.MEDIUM copy[19]The Android face book app sends stuff out in the open so anyone could read your posts..  It is also possible to make bogus posts, the author says. Some things to be careful of.

Wired’s ‘open’ edition: I’ll close this blog with a note about a recent Wired magazine issue that sums it up all.Some of the wired magazine  subscribers got a ‘very personal’ edition of their copy.   It had lot of information that Wired had gleaned about the subscriber!! The editors used online browsing history, electoral records and ‘social networks’. The magazine just shows how much information we expose unknowingly.

When they do this exercise the next time, they will have yet another ‘gold mine’ of private information -your ‘smart’ phone!

[Image credit : Mobilecrunch , Prohack]

Written by sujai

March 12, 2011 at 9:41 pm

%d bloggers like this: