Android Advanced debugging, Get the machine code and tracing instructions

Android is a system that allows developers to write code for the virtual machine without knowing the actual machine code. The compiler can detect many issues during compiling and optimize the code for a polished final product. However, code changes can sometimes cause problems, resulting in unexpected behavior when the app runs. In this post, we’ll explore compiling an Android app and reverse engineering to understand why some code crashes at runtime.

This article is a Tech Pulse published by my former colleague Ray. With his permission, I have excerpted some quotes and codes from it. Ray is a senior Android engineer at Yahoo and is passionate about Android development. Here are his GitHub and LinkedIn links for reference.

Author: Ray Yuan Liu
https://github.com/YuanLiou
https://www.linkedin.com/in/rayyuanliu/

What issue did we meet?

We have a feature that lets users create a shortcut. Unfortunately, this feature only supports Android 7 and above. If this option has been triggered in the old version, it will crash. To prevent a user still on the old machine from clicking this button and crashing the app, we have to check its version and return the execution. However, According to our implementation, The checking statement is not working and still causing the app to crash.

Here is what the code looks like:

// This will cause a crash on old devices
private val shortcutManager: ShortcutManager?
  // this is because ShortcutManager added in API 25
  @TargetApi(Build.VERSION_CODES.N_MR1)
  get() = if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    null
  } else getSystemService(MainApplication.context, ShortcutManager::class.java)

The caller will get a null object if the current app runs on the Android version below the N_MR1. Otherwise, we’ll call the getSystemService method to get ShortcutManager. However, This code will cause a crash on the old devices.

Let’s look at the crash log, the crashing log said we’re still accessing the unexisting ShortcutManager:

java.lang.NoClassDefFoundError: Failed resolution of: 
Landroid/content/pm/ShortcutManager; 
  at ShortCutCompat.getShortcutManager(ShortCutCompat.kt:41) 
  at ShortCutCompat.isShortCutPinned(ShortCutCompat.kt:171) 
  at ShortCutCompat.isShortCutPinned(ShortCutCompat.kt:188)

Syntactic sugar

Kotlin can quickly create a getter to provide value. What if we write our own getter without the syntactic sugar? Surprisingly everything works after all.

// This works on old devices
@TargetApi(Build.VERSION_CODES.N_MR1)
private fun provideShortcutManager(): ShortcutManager? {
  if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    return null
  } else {
    return getSystemService(MainApplication.context, ShortcutManager::class.java)
  }
}

What is different under the hood? When we trace code in Kotlin, they should be the same. However, we get two other runtime behaviors.

How about we uplift the return statement and make the code shorter? When we ran the codes below, it crashed. Kotlin is doing something wrong when we use syntactic sugar.

// This will cause a crash on old devices
@TargetApi(Build.VERSION_CODES.N_MR1)
private fun provideShortcutManager(): ShortcutManager? {
  return if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    null
  } else {
    getSystemService(MainApplication.context, ShortcutManager::class.java)
  }
}

Kotlin and Java conversion

Android apps run on a virtual machine called Dalvik, which is a modified version of Java virtual machine. In the past, developers had to use two tools to compile apps: javac and Dex. Now, the Android compiler has improved and a new tool called d8 allows for direct conversion of Java or Kotlin code to Dalvik bytecode.

We can use the decompile tools inside Android Studio to get the Java bytecode. Let us compare different kinds of Kotlin implementations. Then, since we know some implementations will cause app crashes, let us find out what the Java bytecode looks like.

For the first implementation:

// This will cause a crash on old devices
private val shortcutManager: ShortcutManager?
  // this is because ShortcutManager added in API 25
  @TargetApi(Build.VERSION_CODES.N_MR1)
  get() = if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    null
  } else getSystemService(MainApplication.context, ShortcutManager::class.java)

after converting to Java:

@TargetApi(25) 
private final ShortcutManager getShortcutManager() { 
  return VERSION.SDK_INT < 25 ? null : (ShortcutManager)ContextCompat.getSystemService(MainApplication.Companion.getContext(), ShortcutManager.class); 
}

For the second implementation:

// This works on old devices
private fun provideShortcutManager(): ShortcutManager? {
  if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    return null
  } else {
    return getSystemService(MainApplication.context, ShortcutManager::class.java)
  }
}

after converting to Java:

@TargetApi(25) 
private final ShortcutManager getShortcutManager() { 
  return VERSION.SDK_INT < 25 ? null : (ShortcutManager)ContextCompat.getSystemService(MainApplication.Companion.getContext(), ShortcutManager.class); 
}

For the third implementation:

// This will cause a crash on old devices
@TargetApi(Build.VERSION_CODES.N_MR1)
private fun provideShortcutManager(): ShortcutManager? {
  return if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    null
  } else {
    getSystemService(MainApplication.context, ShortcutManager::class.java)
  }
}

after converting to Java:

@TargetApi(25) 
private final ShortcutManager getShortcutManager() { 
  return VERSION.SDK_INT < 25 ? null : (ShortcutManager)ContextCompat.getSystemService(MainApplication.Companion.getContext(), ShortcutManager.class); 
}

They look exactly the same. It is hard to believe code that causes app crashes would have an identical implementation when we decompile the source from Kotlin to Java bytecode in IDE. There must be something different instructions have been executed.

Reverse engineering an APK

Let’s take a look at the source code inside the APK file. We can use tools to reverse engineer the APK and get the Java bytecode generated by the d8 compiler. There is a tool called jadx that can help us to reverse engineer the dex file to Java source code.

To install jadx: $ brew install jadx
To extract source code from apk: $ jadx -r [your_sample].apk -d [folder_name]

Here is the code we found in the reverse-engineered Java files from those three different implementations:

private final ShortcutManager getShortcutManager() { 
  if (Build.VERSION.SDK_INT < 25) { 
    return null; 
  } 
  return (ShortcutManager)ContextCompat.getSystemService(MainApplication.Companion.getContext(), ShortcutManager.class);
}

It is the same as the one we got from the IDE tools. However, if the app runs as the Java source code tells us, it won’t crash because it will enter a version check and do an early return. So how can it access the inexisted ShortcutManager in the legacy machine?

We should go even deeper into our final destination of the Dalvik virtual machine — the smali code.

Smali Code

The machine code is represented in hexadecimal sequences. The smali file is a dissembler version of machine code. It makes machine code easier to understand.

We extracted the Java bytecode from an APK file in the previous section. The apktool is one of the tools we need.

To install apktool: $ brew install apktool
To unarchive an apk file with smali source codes: $ apktool d -f <target_file>.apk

First, let’s see the code without a crash:

// This works on old devices
@TargetApi(Build.VERSION_CODES.N_MR1)
private fun provideShortcutManager(): ShortcutManager? {
  if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    return null
  } else {
    return getSystemService(MainApplication.context, ShortcutManager::class.java)
  }
}

Here is its smali code:

.method private final getShortcutManager()Landroid/content/pm/ShortcutManager;
  .locals 2 # 2 local register

  .line 40 # original line of Java source code for debugging
  # Reads android/os/Build$VERSION;->SDK_INT:I into v0
  sget v0, Landroid/os/Build$VERSION;->SDK_INT:I

  const/16 v1, 0x19 # Puts the 16 bit constant of 0x19 into v1.

  if-ge v0, v1, :cond_0 # Jumps to the position :cond_0 if v0 >= v1.

  .line 41 # original line of Java source code for debugging
  const/4 v0, 0x0 # Moves 0x0 (Expand to 32bit) into v0.
  return-object v0 # Returns with object reference value in v0

  .line 43 # original line of Java source code for debugging
  :cond_0
  # Reads MainApplication$Companion into v0.
  sget-object v0, Lexample/android/MainApplication;->Companion:Lexample/android/MainApplication$Companion;

  # Invokes getContext() method in the method table with the following arguments:v0
  invoke-virtual {v0}, Lexample/android/MainApplication$Companion;->getContext()Landroid/content/Context;

  move-result-object v0 # Move the result object reference of the previous method invocation into v0.

  # Moves the class object of a class identified by android/content/pm/ShortcutManager into v1.
  const-class v1, Landroid/content/pm/ShortcutManager;

  # Invokes a static method with parameters. The method is called with two parameter, v0, v1
  invoke-static {v0, v1}, Landroidx/core/content/ContextCompat;->getSystemService(Landroid/content/Context;Lja
va/lang/Class;)Ljava/lang/Object;

  move-result-object v0 # Move the result object reference of the previous method invocation into v0.

  # Checks whether the object reference in v0 can be cast to an instance of a class
  # referenced by android/content/pm/ShortcutManager.
  check-cast v0, Landroid/content/pm/ShortcutManager;

  return-object v0 # Return with v0 object reference value.
.end method

As we can see when we call getShortcutManager() method, it will check the Android version. If the Android version is below the value we got from the const/16 constant, it will do an early return.

Root cause

Let’s see the code which caused the app crash in the legacy Android version:

// This will cause a crash on old devices
private val shortcutManager: ShortcutManager?
  // this is because ShortcutManager added in API 25
  @TargetApi(Build.VERSION_CODES.N_MR1)
  get() = if (Build.VERSION.SDK_INT < BUILD.VERSION_CODES.N_MR1) {
    null
  } else getSystemService(MainApplication.context, ShortcutManager::class.java)

Let’s look at the difference between codes with and without syntactic sugar. Here is its smali code:

.method private final getShortcutManager()Landroid/content/pm/ShortcutManager;
  .locals 2 # 2 local register

  .line 40 # original line of Java source code for debugging
  # Reads the integer field android/os/Build$VERSION;->SDK_INT into v0.
  sget v0, Landroid/os/Build$VERSION;->SDK_INT:I

  const/16 v1, 0x19 # Puts the 16 bit constant into v1

  if-ge v0, v1, :cond_0 # Jumps to :cond_0 if v0 >= v1.

  .line 41 # original line of Java source code for debugging
  const/4 v0, 0x0 # Puts the 4 bit constant into v0, 0x0 is 0. 0 can also be a null reference

  move-object v1, v0 # Moves the object reference from v0 to v1.

  # Checks whether the object reference in v1 can be cast to an instance of a class
  # referenced by android/content/pm/ShortcutManager.
  check-cast v1, Landroid/content/pm/ShortcutManager; # < Crash Here

  goto :goto_0 # Unconditional jump to :goto_0

  .line 42 # original line of Java source code for debugging
  :cond_0
  # Reads the object reference field identified by MainApplication$Companion into v0.
  sget-object v0, Lexample/android/MainApplication;->Companion:Lexample/android/MainApplication$Companion;

  # Invokes getContext() method in the method table with the following arguments: v0
  invoke-virtual {v0}, Lexample/android/MainApplication$Companion;->getContext()Landroid/content/Context;

  move-result-object v0 # Move the result object reference of the previous method invocation into v0

  # Moves the class object of a class identified by android/content/pm/ShortcutManager into v1.
  const-class v1, Landroid/content/pm/ShortcutManager;

  # Invokes a static method with parameters. The method is called with two parameter, v0, v1
  invoke-static {v0, v1}, Landroidx/core/content/ContextCompat;->getSystemService(Landroid/content/Context;Ljava/lang/Class;)Ljava/lang/Object;

  move-result-object v0 # Move the result object reference of the previous method invocation into v0

  # Checks whether the object reference in v0 can be cast to an instance of a class
  # referenced by android/content/pm/ShortcutManager.
  check-cast v0, Landroid/content/pm/ShortcutManager;

  :goto_0
  return-object v0 # Return with v0 object reference value.
.end method

We may find the place causing the crash. The code didn’t simply do an early return.

If the Android version is below our target version, the jumping instruction didn’t execute. Instead, the compiler wants to use the same return-object instruction. So it sets the v0 register point to 0 (representing a false or null object). However, before it jumps to return the null value, it will do a check-cast to see whether the object in v1 can be casked to ShortcutManager. The legacy version of Android didn’t have ShortcutManager, so it crashed.

We can check the crash log again and see which line it reported.

java.lang.NoClassDefFoundError: Failed resolution of: 
Landroid/content/pm/ShortcutManager; 
  at ShortCutCompat.getShortcutManager(ShortCutCompat.kt:41) 
  at ShortCutCompat.isShortCutPinned(ShortCutCompat.kt:171) 
  at ShortCutCompat.isShortCutPinned(ShortCutCompat.kt:188)

The crash happened on line 41. So let’s see the smali code and find where line 41 executed. Bingo! It matches our presumption.

  .line 41 # original line of Java source code for debugging
  const/4 v0, 0x0 # Puts the 4 bit constant into v0, 0x0 is 0. 0 can also be a null reference

  move-object v1, v0 # Moves the object reference from v0 to v1.

  # Checks whether the object reference in v1 can be cast to an instance of a class
  # referenced by android/content/pm/ShortcutManager.
  check-cast v1, Landroid/content/pm/ShortcutManager; # < Crash Here

What we have learned from this issue

After the investigation, we learned that the compiler is smart. It tries to reduce duplicated work and make shortcuts. But, in some situations, The clever might cause issues like we’ve met. We need to know how an app has compiled to find the root cause. It can’t figure out by just reading Kotlin code, even Java bytecode (They would be the same). Another thing we learned is code with syntactic sugar might run differently from those without it.

To know how an app runs, we must reverse engineer the machine code to human readable smali code. The smali code represented how an app would run at the JVM level. On the flip side , It can be reassembled with modification so a hacker can inject malicious code into it. Installing an app from an unknown website might be harmful.

To our service provider, we should ensure encryption tools have protected our app like DexGuard, to prevent this kind of hacking.

Finally, This issue might be fixed in the future version of the d8 compiler, so it is important to keep our developing tools up to date.