Android proguard obfuscated code is causing NullPointerException when it really shouldn't be

Question

I have distributed an application on the Android Marketplace. I am getting error reports back in from a small handful of users (maybe 2%) where they are getting NullPointerExceptions where it doesn't make logical sense.

I have never been able to replicate this myself. The code is relatively straightforward and is a common code path that EVERY user has to follow. I've actually taken every separate line of code that could possibly be creating the NPE and wrapped it in a try-catch block and throw a custom runtime exception, but I'm still getting NullPointerException errors not caught.

At this point, the only thing I can imagine it would be is something related to my Proguard obfuscation. I have seen some other article talking about taking out the -overloadaggressively option if you notice odd behavior, but as far as I can tell, I'm not using that option.

Has anybody else experienced mysterious NPEs using android and proguard. Are there any other settings people can recommend to dial down the optimizations that might be causing this issue?

Any other ideas?

For reference, here is the unobfuscated function that is getting the NPE:

public MainMenuScreen(final HauntedCarnival game) {
    super(game);

    game.startMusic("data/music/intro.mp3");

    stage = new Stage(Screen.SCREEN_WIDTH, Screen.SCREEN_HEIGHT,true);
    stage.addActor(new Image("background", Assets.mainMenuBackground));
    Image title = new Image("title", Assets.mainMenuTitle);
    title.x = 0;
    title.y = 340;
    resetEyeBlink();
    stage.addActor(title);
    dispatcher.registerInputProcessor(stage);
    settings = game.getSettings();

    eyeBlinkImage = new Image("eyeBlink", Assets.eyeBlink);
    if (settings.getPlayers().isEmpty()) {
        settings.addPlayer("Player One");
        settings.save(game);
    }
    setupContinue();


}

So the only possibilities I can see are game, dispatcher and settings.

game gets set via this code in another class. game is a final variable in that other class:

game.setScreen(new MainMenuScreen(game));

dispatcher gets set within in the call to super above.

getSettings() returns a settings object that gets set at the very start of the application, is private and never gets unset. Its also used before this method several times.

There are not auto-boxing primitives.

here is the proguard config:

-optimizationpasses 5
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-dontpreverify
-verbose
-optimizations !code/simplification/arithmetic,!field/*,!class/merging/*

-keepattributes Signature

-keep public class com.alkilabs.hauntedcarnival.settings.Settings
-keep public class com.alkilabs.hauntedcarnival.settings.Settings {
    *;
}
-keep public class com.alkilabs.hauntedcarnival.settings.Player
-keep public class com.alkilabs.hauntedcarnival.settings.Player {
    *;
}
-keepnames public class com.alkilabs.hauntedcarnival.world.World
-keepnames public class * extends com.alkilabs.hauntedcarnival.world.upgrades.Upgrade
-keepnames public class * extends com.alkilabs.hauntedcarnival.world.achievments.Achievement

-keepnames public class com.alkilabs.hauntedcarnival.world.monsters.MonsterType
-keepclassmembers class * extends com.alkilabs.hauntedcarnival.world.monsters.Monster {
    public <init>(com.alkilabs.hauntedcarnival.world.monsters.MonsterType, java.lang.Integer, com.alkilabs.hauntedcarnival.world.World);
}

-keepnames public class com.alkilabs.hauntedcarnival.world.items.ItemType
-keepclassmembers class * extends com.alkilabs.hauntedcarnival.world.items.Item {
    public <init>(com.alkilabs.hauntedcarnival.world.World, java.lang.Integer, java.lang.Integer);
}


-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.app.backup.BackupAgentHelper
-keep public class * extends android.preference.Preference

-dontwarn com.badlogic.gdx.scenes.scene2d.ui.utils.DesktopClipboard
-dontwarn com.badlogic.gdx.utils.JsonWriter
-dontwarn com.badlogic.gdx.utils.XmlWriter

-keepclasseswithmembernames class * {
    native <methods>;
}

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet);
}

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet, int);
}

-keepclassmembers class * extends android.app.Activity {
   public void *(android.view.View);
}

-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}

-keep class * implements android.os.Parcelable {
  public static final android.os.Parcelable$Creator *;
}

Can you pls post the code which causes NPE and ProGuard settings you used? — Volo, Oct 02 '11 at 19:26
I added more detail about the specific code and my proguard config — Paul, Oct 02 '11 at 23:40

score 8 · Answer 1 · answered Oct 04 '11 at 17:24

8

OK, I think I got to the root of the issue/confusion.

One of the things proguard does is in-line some methods. Because of this, the entire contents of my setupContinue() function at the bottom of my constructor was added directly into the contents of my constructor. So now I have a bunch more code to review, and I do see some more possibilities for NPEs. I'm pretty sure I'll get to the bottom of my issue.

I figured this out by taking the obfuscated.jar that proguard produces, and running it through a decompiler. Its an interesting exercise as you get get little more insight into the inner workings of proguard. I highly recommend it for people wanting to better understand the impacts that proguard has on their code.

answered Oct 04 '11 at 17:24

Paul

181
1
9

Looking at the processed code can be very useful. You should make sure that you are using the latest version of ProGuard (4.6 or 4.7 beta, at this time). Notably, version 4.6 contains a fix related to static initializers not being run due to methods being moved around (inlined). Still, it's surprising that the problem doesn't happen consistently. – Eric Lafortune Oct 06 '11 at 20:17
The part about Proguard in-lining code was a big help. Now I realize why my user submitted crash/stack traces seem to end at a method signature instead of the actual line of code causing the NPE. It's too bad it doesn't point specifically to the line causing the problem though; it would really suck if the method contained a ton of code (like finding a needle in a haystack). – Tony Chan Dec 16 '13 at 20:11

score 2 · Answer 2 · answered Oct 02 '11 at 19:46

2

Your best bet would be to use the mapping.txt file and the retrace tool to find the exact location of the error. From there it would be easier to understand if it's indeed Proguard or some other end-case you didn't think of.

To do this, you need to copy the stack trace from the developer's console to another file, let's assume it's called

c:\trace.txt

Now, in your project, you'll find a Proguard folder with 4 files. Let's assume your project is in

c:\project

What you'll need to do is run the retrace tool (using the batch file for easier use) located at (change to the location of your Android Sdk folder) :

c:\android-sdk-windows\tools\proguard\bin\retrace.bat

Go to that folder and run:

retrace c:\project\proguard\mapping.txt c:\trace.txt

From there on, it would be much easier to figure our the exact line of the exception and to probably find the error.

From my experience, the only things that could get messed up are third party libraries. Normal Android code, for all my projects, was never harmed from obfuscation.

answered Oct 02 '11 at 19:46

IncrediApp

10,303
2
33
24

Yes I've already done that. I know the exact method and the entire stack trace that called that method. The only thing I don't know is the line number but this is because this is a production release without line numbers. Also, from what I understand, Proguard actually rewrites some of your code during it's optimization phase (which is one of the reasons why I suspect proguard). So i was relegated to old fashion debugging of each line that could produce an NPE, but alas, no dice. – Paul Oct 02 '11 at 19:53
So the only advice I have for you is to put an unobfuscated version up for a few days and see if you get additional error reports... I never encountered errors with Proguard in my code (from the most complex apps to the simplest ones) and I always obfuscate my projects. – IncrediApp Oct 02 '11 at 20:02
Yeah, I was worried that it may have to come to that, but unfortunately I'm just not going to let my code out into the wild for all to see. I have seen some other posts talk about issues with proguard code on mobile phones: http://stackoverflow.com/questions/93290/best-java-obfuscation-application-for-size-reduction So I don't think this is unprecedented. I was just hoping other people might have had similar circumstances. – Paul Oct 02 '11 at 21:48
This is from 2008.. Now Proguard is part of the Android Sdk with predefined settings. – IncrediApp Oct 03 '11 at 04:45
@IncrediApp I have the same problem in 2020 – Tamim Attafi Dec 06 '20 at 14:33

score 1 · Answer 3 · answered Oct 02 '11 at 22:06

Sorry I can't post comments yet (I'm new to SO).

This could be another problem I've encountered myself. On some phones there was a problem at some point with missing Android libraries like a JSon library.

I'd recommend you take a closer look at what phones that actually get the NPE - there might be some similarities.

In my case it was the HTC Desire Z, that was missing the JSon library and so the app force closed every time the JSon part was called. The problem was fixed by HTC later on with a hotfix to the Desire Z's rom.

Android proguard obfuscated code is causing NullPointerException when it really shouldn't be

3 Answers3

Linked