Android assets are too slow

Question

I have Android app with Assets. They contains 20,000+ files, most of them are simple text or png files separated into different folders and subfolders. Maximal size of 1 single file is 500kb, 90% of them are around 2kb.

Sample structure

Data
  ... subfolders with 20k+ files
StartData
  .... FolderA
  ........ 5 files *.txt
  .... FolderB 
  ........ 5 files *.txt
  .... a.xml
  .... b.xml

The problem is load time of app. During app loading, I need to open around 20 files from StartData directory and read settings from them. They are in 2 xml files and based on info in xml I open additional data from two subfolders FolderA / FoldeB.

The speed is, however, very poor and app starts at least 3-4x times longer than without assets.

In former version I have zip in raw directory, that was deocmpressed upon first run to documents directory. First run was very slow, but other were fast. The problem with this solution was the install spedd, since deocmpresing took up to 5 minutes on older phones and it cannot die during that process. If so, app was damaged and data only partially decompressed, so I dropped this solution.

With my former approach, if unzipping finished correctly, cold app start was about 5s. Now, with assets, it takes up to 20s. Any solution?

I have disabled assets compression (I have only *.txt, *.png and *.xml files) in gradle

aaptOptions{
       ignoreAssetsPattern  ''
       noCompress 'txt', 'png', 'xml'
       cruncherEnabled = false
}

but the improvement is only small.

Also, I am using NDK (JNI) and AssetManager to load my assets in C++. Solution for assets opening in C++ is taken from this: http://www.50ply.com/blog/2013/01/19/loading-compressed-android-assets-with-file-pointer/

Is the speed influenced by these 20.000? If you do a test with only 10 files in assets then what is the loading time? — greenapps, Dec 14 '17 at 13:15

score 1 · Accepted Answer · answered Dec 14 '17 at 22:29

I am not surprised that discovering offsets to thousands of files in the assets folder is slow, and that the file system is better for such task. I would say that maybe the data structure should be rewritten. For example, you could put this information in a database.

I am sceptical about the funopen() trick. I mean, it works reliably, but I am concerned about the overhead. Much more efficient approaches to deal with non-compressed assets are through AAsset_getBuffer() or AAsset_openFileDescriptor(). Check an example of using file descriptor. Note that in the end, there will be one file descriptor for all thousands of your 'files'.

By the way, I don't think that Java performance with these direct access methods is worse than C. You could use zlib to read your assets from the APK file (which is in ZIP format itself), but I doubt you will do it faster than the AssetManager.

Does the `funopen` trick imply that you lose the "memory mapping" you would get from `AAsset_getBuffer()`? I.e. buffers would need to be allocated and freed for simple reads. — l33t, Nov 25 '18 at 21:52
@l33t: I believe that it's possible to use the **funopen()** trick to feed an uncompressed asset to **ffmpeg** as efficiently as using [AAsset_openFileDescriptor()](https://developer.android.com/ndk/reference/group___asset.html#gaa34e35e5f3d087d2b4d1910932d7eb7a). But the gain of *memory mapping* via [AAsset_getBuffer()](https://developer.android.com/ndk/reference/group___asset.html#ga553a14512a98542306238c3ce70d344f) is lost with this approach. I am not sure how bad are buffer allocations, but even memcpy() is wasteful if you can use AAsset_GetBuffer() instead. — Alex Cohn, Nov 25 '18 at 22:58

Android assets are too slow

1 Answers1