whitenoise

technology notes… mobile and embedded.

Archive for the ‘android’ Category

Android HOWTOs : Memory leak debugging with libc

with one comment

Android’s  libc module provides a way to run in ‘debug’ mode.

In this mode it keeps track of allocated memory and also adds extra data to detect memory corruption /overflow. This tools is useful to debug issues related to memory corruptions and memory leaks too.

Before looking at using this tool it would be good to know the symptoms of a memory corruption/leak.

Symptoms:

Memory corruptions usually manifest as native crashes in dlfree or *alloc methods with invalid pointers like 0xdeadbaad . Here is sample call stack where you could suspect a heap memory corruption.

heap_corruption

 

In the above scenario the module calling calloc crashed as the heap is already corrupted. The next steps would be to enable libc memory debugging options to identify source of corruption.

Memory leaks usually trigger Out of Memory Java exceptions or will result in OOM killer invoked. The procrank and librank output will show high memory used by the application  (Assumption: The data was collected before the app/process was killed)

procrank_highmem

In the above scenario, com.android.foo.bar app has a lot of memory allocated. It is likely this app is leaking memory . We could use libc’s memory debug options to identify the source of leak.

Enabling the option:

The next step is to enable the debug options of libc.
On a rooted device , we could enable libc malloc debugging with following steps:

enable_libc

The libc would now run in debug mode and will track memory allocations and corruptions.

Debug level 10 is a good option to setup. It would detect both leaks and corruptions. See here for a complete list of debug levels. 

Now run the original use case that caused the crash/corruption. There would be extra logs from the libc module if there is corruption/leak.  Memory leaks will usually show the call stack of the allocation and the number of bytes leaked.

leak_signature

Memory corruptions will have a slightly different signature. Here is the log when the library detects corrupt heap.

corruption_signature

False Positives:

When using this mode we have to be aware of false positives too.

For e.g.: If an application was terminated abruptly (via ‘adb shell kill –9 <pid>’)  then memory that was not properly de allocated would be reported as ‘leaks’ by libc.

Likewise there could be false memory corruption logs if we use  memory optimization modules that manipulate the heap nodes.

However such false positives are very rare. Even if they occur ,they usually show up in generic modules (like Android shell) which could not be suspected to have leaks.

Advertisements

Written by sujai

October 14, 2013 at 6:53 pm

An Android 101: Hardware and HAL

with 2 comments

Unlike traditional Linux based systems, Android applications communicate with the underlying hardware through Java APIs not by system calls.

image

             Figure 1: Android vs a Tradition Linux system.

Most of the ‘shared libs’ in the above diagram represent the HAL modules.  The Hardware Abstraction Layer or HAL is the glue between any device (part of the kernel) and the corresponding JNI interface.

The details about the HAL is usually not of interest to application developers. However it is an important module for hardware vendors and OEMs involved in  porting android into their own hardware and adding new hardware.

Folder structure:
   The HAL source code reside under the hardware folder . OEMs maintain a top level root directory and internally there are several HAL modules for the corresponding hardware.

On device ,the HALs live as .so’s  either in ‘/system/lib/hw’ or ‘/vendor/lib/hw’. The HAL modules follow a certain nomenclature as defined here . Android relies on this naming convention to load the right HAL module for a given hardware and board configuration.

image

Runtime:
The HALs get loaded by its clients using the  hw_get_module. This method relies on the values set to ro.hardware , ro.product.board and some other ro properties to load the correct .so. The complete list is here.
For eg. The SurfaceFlinger is the system wide compositor and the hardware composer is a HAL that abstracts device specific composition logic. The SurfaceFlinger loads the hardware composer using the hw_get_module method to get a handle of the HAL.

Each HAL implements a set of hooks which get called by their invoking service. The contract between the system service and the HAL is well defined. Device makers should ensure these interfaces are implemented as expected by the Android services. The figure below tries to show some of the HALs on the Android ICS version and key methods that HAL writes should implement.

image

With the newer versions of Android ,the scope of HALs have expanded and it is likely they will be used not just to abstract hardware but to abstract any device specific behavior.

References:

1. Good site about  adding support for a Camera : http://processors.wiki.ti.com/index.php/TI-Android-DevKit-Camera_Porting_Guide
2. Patrick Brady’s Google I/O talk: https://sites.google.com/site/io/anatomy–physiology-of-an-android
3. Figure 1 courtesy: http://www.cmg.org/measureit/issues/mit78/m_78_3.pdf
4, Class diagrams created using Umbrello ,an open source UML Modeler.

Update:  Here is a new post on the same topic with much more detail:  http://www.opersys.com/blog/extending-android-hal

Written by sujai

June 25, 2012 at 7:42 am

An Android 101 : An overview on Binder framework.

with 2 comments

While the underlying platform makes use of Linux concepts, the Android applications are pretty unique. Unlike traditional Java apps, Android apps do not have a an entry point (like a main() function ).  The framework could invoke a particular entry point within an application to achieve the requested functionality.
Each android application is a mixture of several components . The components can be broadly grouped into 4 ‘component types’.  You could read more about components here.

android_component_system
[Image credit : Android binder thesis by  Thorsten Schreiber]

The different components may need to  exchange data and it is realized through inter component/ inter process communication .The communication works with so called intents.Intents are representations for actions to  be performed. Each intent is made of a URI and an ‘action’. The URI identifies the destination and action defines the operation to be performed on the component identified by the URI.  As we learn about components and interactions it becomes increasingly evident that IPC forms a major part of  the Android framework.

The Binder framework :
Unlike traditional Linux machines, Android uses its own mechanism for IPC called the Binder. Binder was originally developed as OpenBinder and was used as the IPC in BeOS . The current Binder implementation in Android is a customized implementation of the OpenBinder. This was mainly to ensure the new implementation uses a license that is compatible with the Android’s user space code.

Each Binder object is an implementation of the IBinder implementation . An important difference between Binder and other IPC mechanisms is the use of  Binder token. A binder token is value that uniquely identifies any  Binder. This capability to uniquely identify a binder across processes allows the binder id to act as a ‘shared token’ across multiple processes.

Communication model:
The Binder framework communication is a client server model . Each client initiates communication and waits for response from the server. Each client would have a proxy  object for the client side communication. The server side constitutes a pool of worker threads.The server shall spawn a new thread for each new Binder request from the client. The bridge between the client and the server process is the binder driver. The Binder driver is a character device that is part of kernel space. This module  ensures the client reaches the appropriate destination  across process boundaries.

binder_commn

                                                                    [Image credit : Android binder thesis by  Thorsten Schreiber]

AIDL : As with any other RPC , there is a need to write a proxy and stub class that would be interacting with the client and server. Developers  write the interface   for their remote services using AIDL .The AIDL parser takes care of generating Java classes for the proxy and stub and logic to convert data into parcels ,that the Binder middleware understands.
If you are in the business of writing AIDLs you should  keep in mind that the Binder protocol is always ‘request gives response’ there is no inbuilt constructs to provide asynchronous functions. Asynchronous mechanisms have to be built on  top  of the Binder framework.

Binder vs SysV IPC: You may wonder why not reuse SysV IPC that comes with Linux. The Binder documentation has good explanation on why Google  has stayed away from using SysV IPC.

One of the design requirements of Binder is to ensure there is no ‘resource leak’ when any service dies improperly . i.e.Ensure that any kernel level resources (like semaphores/binder objects)  are cleared up in case the parent process is killed or crashes prematurely. There is no way to release a SysV semaphore if the process that created it is explicitly killed or it crashes. Killing processes automatically to make room for new ones is very common with the Android framework.

For an in-depth discussion on Binders, you could refer to some of the links below that I used to learn about Binders.

References:
1. eLinux Binder documentation : http://elinux.org/Android_Binder
2. A nice ppt with implementation details : http://blog.kmckk.com/archives/3676340.html
2. NDK reference documentation about SYS V IPC :  <NDK INSTALL ROOT>\docs\system\libc\SYSV-IPC.html .
3. Comparison of IPC mechanisms : http://lwn.net/Articles/466304/
4. A good overview on OpenBinder : http://www.osnews.com/story/13674/Introduction-to-OpenBinder-and-Interview-with-Dianne-Hackborn/

Written by sujai

December 28, 2011 at 11:27 pm

An Android 101 : Boot and platform init.

leave a comment »

Over the past few months I have been dabbling with Android  SDK and a some amount  of programming.  Sharing what I learnt .

53731_android_101_training_course_vancouver_255255255 

A Linux system:

Apart from the application framework, much of Android uses concepts from Linux directly. This includes file systems , partitions and init scripts (like the init.d). A typical Android device would have the following partitions :

image

Besides there may be some device specific partitions (/modem etc) that the OEMs may use for their own (non –android ) purpose.

Of these ,the recovery partition is particularly interesting. Normal android boot would start running the image in the  boot partition. However if you power on device in the ‘recovery mode’ , you could get started with an alternate boot image (the recovery image).

It was originally designed for OEMs to provide OTA upgrades. But custom ROM makes have come up with custom recovery images  that could flash other kernel images (like CynogenMod etc). A popular custom recovery image is the ClockworkMod Recovery.

AndroidBootSequence

Bootup:

When a normal boot happens, the Android platform starts executing code in the /boot partition .The bootloader then starts the kernel. The kernel does the basic initialization (of hardware ,memory subsystems) before mounting the root file system ( / ). Once / is mounted the ‘init’ process is started.

init.rc :
Like normal (desktop ) Linux systems ,Android init parses and runs the various commands as mentioned in the init.rc  script. The bulk of Android platform init happens here. The init.rc script mounts various partitions , It then starts  daemons like the adbd, service manager (responsible for IPC ).,rild, netd and others.  Finally the init.rc also invokes app_process which actually results in the process called zygote.

Zygote: Every new app is hosted on a dedicated VM. However each VM is not started afresh (not a cold start).  Zygote takes care of loading all system libraries beforehand . When a new app is started ,zygote forks itself to provide a new VM that has most components preloaded .
There is more detail on this here : http://elinux.org/Android_Zygote_Startup

SystemServer:   Once Zygote is fully launched, we have the VM initialized . The platform then starts the SystemServer.(frameworks/base/services/java/com/android/server/SystemServer.java)

SystemServer is the first Java component to run on the system . This module starts all ‘Android services’  like the PowerManager, ActivityManager etc. Once the SystemService completes, Android’s bootup is considered complete.

The platform sends a standard broadcast action called ‘ACTION_BOOT_COMPLETED ‘. Clients that want to do some action once boot up is complete could register for this module.

In the next post I will share about other components of Android , like its IPC mechanism (Binder)  and shared memory implementations (ASHMEM)

[Image courtesy: http://www.techvibes.com/ , http://hmtsay.blogspot.com/2010/10/android-startup.html ]

Written by sujai

November 7, 2011 at 7:15 pm

%d bloggers like this: