technology notes… mobile and embedded.

Android HOWTOs : Memory leak debugging with libc

with one comment

Android’s  libc module provides a way to run in ‘debug’ mode.

In this mode it keeps track of allocated memory and also adds extra data to detect memory corruption /overflow. This tools is useful to debug issues related to memory corruptions and memory leaks too.

Before looking at using this tool it would be good to know the symptoms of a memory corruption/leak.


Memory corruptions usually manifest as native crashes in dlfree or *alloc methods with invalid pointers like 0xdeadbaad . Here is sample call stack where you could suspect a heap memory corruption.



In the above scenario the module calling calloc crashed as the heap is already corrupted. The next steps would be to enable libc memory debugging options to identify source of corruption.

Memory leaks usually trigger Out of Memory Java exceptions or will result in OOM killer invoked. The procrank and librank output will show high memory used by the application  (Assumption: The data was collected before the app/process was killed)


In the above scenario, com.android.foo.bar app has a lot of memory allocated. It is likely this app is leaking memory . We could use libc’s memory debug options to identify the source of leak.

Enabling the option:

The next step is to enable the debug options of libc.
On a rooted device , we could enable libc malloc debugging with following steps:


The libc would now run in debug mode and will track memory allocations and corruptions.

Debug level 10 is a good option to setup. It would detect both leaks and corruptions. See here for a complete list of debug levels. 

Now run the original use case that caused the crash/corruption. There would be extra logs from the libc module if there is corruption/leak.  Memory leaks will usually show the call stack of the allocation and the number of bytes leaked.


Memory corruptions will have a slightly different signature. Here is the log when the library detects corrupt heap.


False Positives:

When using this mode we have to be aware of false positives too.

For e.g.: If an application was terminated abruptly (via ‘adb shell kill –9 <pid>’)  then memory that was not properly de allocated would be reported as ‘leaks’ by libc.

Likewise there could be false memory corruption logs if we use  memory optimization modules that manipulate the heap nodes.

However such false positives are very rare. Even if they occur ,they usually show up in generic modules (like Android shell) which could not be suspected to have leaks.


Written by sujai

October 14, 2013 at 6:53 pm

An Android 101: Hardware and HAL

with 2 comments

Unlike traditional Linux based systems, Android applications communicate with the underlying hardware through Java APIs not by system calls.


             Figure 1: Android vs a Tradition Linux system.

Most of the ‘shared libs’ in the above diagram represent the HAL modules.  The Hardware Abstraction Layer or HAL is the glue between any device (part of the kernel) and the corresponding JNI interface.

The details about the HAL is usually not of interest to application developers. However it is an important module for hardware vendors and OEMs involved in  porting android into their own hardware and adding new hardware.

Folder structure:
   The HAL source code reside under the hardware folder . OEMs maintain a top level root directory and internally there are several HAL modules for the corresponding hardware.

On device ,the HALs live as .so’s  either in ‘/system/lib/hw’ or ‘/vendor/lib/hw’. The HAL modules follow a certain nomenclature as defined here . Android relies on this naming convention to load the right HAL module for a given hardware and board configuration.


The HALs get loaded by its clients using the  hw_get_module. This method relies on the values set to ro.hardware , ro.product.board and some other ro properties to load the correct .so. The complete list is here.
For eg. The SurfaceFlinger is the system wide compositor and the hardware composer is a HAL that abstracts device specific composition logic. The SurfaceFlinger loads the hardware composer using the hw_get_module method to get a handle of the HAL.

Each HAL implements a set of hooks which get called by their invoking service. The contract between the system service and the HAL is well defined. Device makers should ensure these interfaces are implemented as expected by the Android services. The figure below tries to show some of the HALs on the Android ICS version and key methods that HAL writes should implement.


With the newer versions of Android ,the scope of HALs have expanded and it is likely they will be used not just to abstract hardware but to abstract any device specific behavior.


1. Good site about  adding support for a Camera : http://processors.wiki.ti.com/index.php/TI-Android-DevKit-Camera_Porting_Guide
2. Patrick Brady’s Google I/O talk: https://sites.google.com/site/io/anatomy–physiology-of-an-android
3. Figure 1 courtesy: http://www.cmg.org/measureit/issues/mit78/m_78_3.pdf
4, Class diagrams created using Umbrello ,an open source UML Modeler.

Update:  Here is a new post on the same topic with much more detail:  http://www.opersys.com/blog/extending-android-hal

Written by sujai

June 25, 2012 at 7:42 am

An Android 101 : An overview on Binder framework.

with 2 comments

While the underlying platform makes use of Linux concepts, the Android applications are pretty unique. Unlike traditional Java apps, Android apps do not have a an entry point (like a main() function ).  The framework could invoke a particular entry point within an application to achieve the requested functionality.
Each android application is a mixture of several components . The components can be broadly grouped into 4 ‘component types’.  You could read more about components here.

[Image credit : Android binder thesis by  Thorsten Schreiber]

The different components may need to  exchange data and it is realized through inter component/ inter process communication .The communication works with so called intents.Intents are representations for actions to  be performed. Each intent is made of a URI and an ‘action’. The URI identifies the destination and action defines the operation to be performed on the component identified by the URI.  As we learn about components and interactions it becomes increasingly evident that IPC forms a major part of  the Android framework.

The Binder framework :
Unlike traditional Linux machines, Android uses its own mechanism for IPC called the Binder. Binder was originally developed as OpenBinder and was used as the IPC in BeOS . The current Binder implementation in Android is a customized implementation of the OpenBinder. This was mainly to ensure the new implementation uses a license that is compatible with the Android’s user space code.

Each Binder object is an implementation of the IBinder implementation . An important difference between Binder and other IPC mechanisms is the use of  Binder token. A binder token is value that uniquely identifies any  Binder. This capability to uniquely identify a binder across processes allows the binder id to act as a ‘shared token’ across multiple processes.

Communication model:
The Binder framework communication is a client server model . Each client initiates communication and waits for response from the server. Each client would have a proxy  object for the client side communication. The server side constitutes a pool of worker threads.The server shall spawn a new thread for each new Binder request from the client. The bridge between the client and the server process is the binder driver. The Binder driver is a character device that is part of kernel space. This module  ensures the client reaches the appropriate destination  across process boundaries.


                                                                    [Image credit : Android binder thesis by  Thorsten Schreiber]

AIDL : As with any other RPC , there is a need to write a proxy and stub class that would be interacting with the client and server. Developers  write the interface   for their remote services using AIDL .The AIDL parser takes care of generating Java classes for the proxy and stub and logic to convert data into parcels ,that the Binder middleware understands.
If you are in the business of writing AIDLs you should  keep in mind that the Binder protocol is always ‘request gives response’ there is no inbuilt constructs to provide asynchronous functions. Asynchronous mechanisms have to be built on  top  of the Binder framework.

Binder vs SysV IPC: You may wonder why not reuse SysV IPC that comes with Linux. The Binder documentation has good explanation on why Google  has stayed away from using SysV IPC.

One of the design requirements of Binder is to ensure there is no ‘resource leak’ when any service dies improperly . i.e.Ensure that any kernel level resources (like semaphores/binder objects)  are cleared up in case the parent process is killed or crashes prematurely. There is no way to release a SysV semaphore if the process that created it is explicitly killed or it crashes. Killing processes automatically to make room for new ones is very common with the Android framework.

For an in-depth discussion on Binders, you could refer to some of the links below that I used to learn about Binders.

1. eLinux Binder documentation : http://elinux.org/Android_Binder
2. A nice ppt with implementation details : http://blog.kmckk.com/archives/3676340.html
2. NDK reference documentation about SYS V IPC :  <NDK INSTALL ROOT>\docs\system\libc\SYSV-IPC.html .
3. Comparison of IPC mechanisms : http://lwn.net/Articles/466304/
4. A good overview on OpenBinder : http://www.osnews.com/story/13674/Introduction-to-OpenBinder-and-Interview-with-Dianne-Hackborn/

Written by sujai

December 28, 2011 at 11:27 pm

An Android 101 : Boot and platform init.

leave a comment »

Over the past few months I have been dabbling with Android  SDK and a some amount  of programming.  Sharing what I learnt .


A Linux system:

Apart from the application framework, much of Android uses concepts from Linux directly. This includes file systems , partitions and init scripts (like the init.d). A typical Android device would have the following partitions :


Besides there may be some device specific partitions (/modem etc) that the OEMs may use for their own (non –android ) purpose.

Of these ,the recovery partition is particularly interesting. Normal android boot would start running the image in the  boot partition. However if you power on device in the ‘recovery mode’ , you could get started with an alternate boot image (the recovery image).

It was originally designed for OEMs to provide OTA upgrades. But custom ROM makes have come up with custom recovery images  that could flash other kernel images (like CynogenMod etc). A popular custom recovery image is the ClockworkMod Recovery.



When a normal boot happens, the Android platform starts executing code in the /boot partition .The bootloader then starts the kernel. The kernel does the basic initialization (of hardware ,memory subsystems) before mounting the root file system ( / ). Once / is mounted the ‘init’ process is started.

init.rc :
Like normal (desktop ) Linux systems ,Android init parses and runs the various commands as mentioned in the init.rc  script. The bulk of Android platform init happens here. The init.rc script mounts various partitions , It then starts  daemons like the adbd, service manager (responsible for IPC ).,rild, netd and others.  Finally the init.rc also invokes app_process which actually results in the process called zygote.

Zygote: Every new app is hosted on a dedicated VM. However each VM is not started afresh (not a cold start).  Zygote takes care of loading all system libraries beforehand . When a new app is started ,zygote forks itself to provide a new VM that has most components preloaded .
There is more detail on this here : http://elinux.org/Android_Zygote_Startup

SystemServer:   Once Zygote is fully launched, we have the VM initialized . The platform then starts the SystemServer.(frameworks/base/services/java/com/android/server/SystemServer.java)

SystemServer is the first Java component to run on the system . This module starts all ‘Android services’  like the PowerManager, ActivityManager etc. Once the SystemService completes, Android’s bootup is considered complete.

The platform sends a standard broadcast action called ‘ACTION_BOOT_COMPLETED ‘. Clients that want to do some action once boot up is complete could register for this module.

In the next post I will share about other components of Android , like its IPC mechanism (Binder)  and shared memory implementations (ASHMEM)

[Image courtesy: http://www.techvibes.com/ , http://hmtsay.blogspot.com/2010/10/android-startup.html ]

Written by sujai

November 7, 2011 at 7:15 pm

RIP–Steve Jobs

leave a comment »


11 - 1

We will miss you ..



Written by sujai

October 5, 2011 at 7:21 am

Posted in life

Innovative iPhone and Android apps– 2011 Winter edition

leave a comment »

After my first post this year, there have been many more interesting apps that caught my attention.It is pretty good to see a lot of innovation in the mobile industry by people outside of this domain. So here is my next list for the year. We could say it’s the ‘Winter edition’ of 2011!

The Invisible Universe:   Joshua Peek  is a fellow at the Columbia University studying about galaxies. As an astronomer he sees a lot of things in the space through his telescopes that we couldn’t see.  So he has come up with AR app called Invisible Space. The app is loaded with a database of full sky images of the galaxy.

When someone holds the phone up in the sky,this app would load the image of the galaxy at that orientation.. I liked this app for bringing in lot of informative stuff using simple techniques.

Here is a good intro video: 



This app is hosted here on the Android Market: https://market.android.com/details?id=com.lbi.iu .

PitPatt : The next app is pretty cool and pretty scary too. This is what happens when Minority Report meets Android.  Researchers at CMU have come up with an advanced Pattern recognition algorithm that allows anyone to know your real identity in under 60 seconds , by just taking a your picture with your smartphone.

Pitpatt has been acquired by Google and soon we could see this tech on Android.The app is not publically available yet.There is more detail on this app here.

Vocre :  I mentioned about translations apps in my  previous review too. This time there is one more vocre-logotranslator app that has won lot of accolades. Vocre from myLanguage can translate what you speak into nearly six different languages (including Mandarin).  Usage is pretty simple .You speak in your language and tilt the phone to translate. There is a nice demo at Vocre’s home page explaining the idea.

Appmates from Disney : My last innovative app is also the best on this list . This app is soo cool that I would buy and iPad just to try this app.  Disney’s new app allows you to drive real cars on virtual space and the virtual terrain reacts to the cars!! 

All you need is an iPad with the free Disney app and get one of those special  Lightning Mcqueen toys. Here is an elaborate demo.Have a look and you’ll be quite impressed.



The tech around this is a hot topic on Google + and I was able to glean some info on that. The inventor has couple patents on it.Now that is ONE cool idea to be patented !  Here is more info on this concept .

That’s it for now. Like many I would be eagerly following the ‘Lets Talk iPhone’ event today (Oct 4,2011). I am particularly interested to see what they are going to announce about Virtual Personal Assistant using the technology they acquired from Siri.

Written by sujai

October 4, 2011 at 7:04 pm

Kernel.org has been hacked.

leave a comment »

On Aug31 kernel.org broke the news that many of its servers have been compromised by unknown attackers. They broke in using a compromised user account and installed a rootkit that was silently monitoring user activity among other things. The gory details are best described here

Youve_Been_HackedIt should be pretty embarrassing when when this happens around the 20th anniversary  of Linux.

But the hack has made us all know some very important details about the integrity of kernel source
and the resilience of git itself against such attack.

Resilient Git For each file in the git repository a cryptographically secure hash is generated and the hash uniquely identified the content of that file along with its history.  So any modification into an old commit WITHOUT changing the hash is very difficult.
So from a source code perspective we could be fairly sure that no change could be injected onto it. This blog post at the Linux foundation explains it well.

However Aside from Git, kernel.org also hosts the signatures and some key components used to sign the kernel drops.
See the post here to know how the signing works. If the intruder got the private key ,then he could modify source (outside of git) and make tar archives and sign it.

This may be the reason ALL archives and patches hosted in kernel.org has been pulled down .Even the AOSP project  has pulled down its link ((link: http://android.git.kernel.org/) to the source code .So for now it is advisable not to take any of the archives hosted in the server until we hear an update on this.

I expect that the private key is updated and a new public key is put up here .Only then it would be safe to download the new kernels.

Damage control:
  Swift action has been taken on this .The passwords of all the users of the kernel.org (448 of them) have been reset . The attacker accessed the kernel.org using the credentials of one of the user. While all of this is happening the kernel source has temporarily moved to GitHub.

In the end,this entire episode has let many lay users(like me ) to learn more about git’s integrity and some insight on how the entire kernel.org release happens. Also there is no need for us to  worry about the kernel source being compromised now or even in the foreseeable future.

Written by sujai

September 6, 2011 at 12:17 am

Posted in technology, Uncategorized

Tagged with , , , , ,

%d bloggers like this: