8

I am trying to use the Android jobb tool to create a large OBB file for my application, but I am plagued by the "FAT Full" IOException:

java.io.IOException: FAT Full (XXXX, YYYY)
    at de.waldheinz.fs.fat.Fat.allocNew(Fat.java:298)
    at de.waldheinz.fs.fat.Fat.allocAppend(Fat.java:376)
    at de.waldheinz.fs.fat.ClusterChain.setChainLength(ClusterChain.java:175)
    at de.waldheinz.fs.fat.ClusterChain.setSize(ClusterChain.java:132)
    at de.waldheinz.fs.fat.FatFile.setLength(FatFile.java:91)
    at de.waldheinz.fs.fat.FatFile.write(FatFile.java:154)
    at com.android.jobb.Main$1.processFile(Main.java:495)
    at com.android.jobb.Main.processAllFiles(Main.java:604)
    at com.android.jobb.Main.processAllFiles(Main.java:600)
    at com.android.jobb.Main.main(Main.java:417)
Exception in thread "main" java.lang.RuntimeException: Error getting/writing file with name: LAST_PROCESSED_FILE
    at com.android.jobb.Main$1.processFile(Main.java:501)
    at com.android.jobb.Main.processAllFiles(Main.java:604)
    at com.android.jobb.Main.processAllFiles(Main.java:600)
    at com.android.jobb.Main.main(Main.java:417)

In the above error message, XXXX is always printed as exactly one integral value lower than YYYY, and represents the number of usable "clusters" (I'm not versed enough in storage jargon to know exactly what this means). YYYY represents the last successfully allocated cluster index, which in my experience is always the same as the last usable cluster index (the array is sized at XXXX + 2, so XXXX + 1 which is the same as YYYY is the last usable index).

The crash seems to appear at the point at which the total file size exceeds 511 MB (the actual limit is 536,193,820 bytes, a single byte more causes the overflow!), so LAST_PROCESSED_FILE is rather arbitrary, but it lists the file being processed when the crash occurred. Given that the storage format is FAT16 (from what I've been told), shouldn't the maximum file size then be 2 GB?

I have read through various sources that empty or small directories or files, small total file size, or individual files within the directory over 500 MB can cause this crash (though I have not been able to determine why). None of these reasons are applicable to my case (which again, is based on total file size).

My own review of the jobb tool source has not provided any insight. Can anyone please shed any light on this issue?

monkey0506
  • 2,489
  • 1
  • 21
  • 27
  • On **failure** (536,193,821 bytes total file size) I get the following output from the tool: `Filesystem Size: 536870912`, `Total Files: 536338432`, `FAT Overhead: 524320`, `Slop: 141763`, `Directory Overhead: 2848`. On **success** (536,193,821 bytes total file size) I get this instead: `Filesystem Size: 536862720`, `Total Files: 536330240`, `FAT Overhead: 524320`, `Slop: 133572`, `Directory Overhead: 2848`. – monkey0506 Sep 19 '13 at 22:56
  • http://stackoverflow.com/questions/15832039/jobb-crash-fat-full and http://stackoverflow.com/questions/13562617/using-jobb-tool-in-android describe a similar issue – dst Sep 19 '13 at 23:05
  • The full output for both failure and success is here: http://pastebin.com/A0hzBBmk – monkey0506 Sep 19 '13 at 23:10
  • @dst: It's true that they have similar exceptions, but they are not related to my issue (as I noted). The answers to those questions do not resolve my problem. – monkey0506 Sep 19 '13 at 23:12
  • I have discovered that de.waldheinz.fs.fat.SuperFloppyFormatter.fatTypeFromSize (begins at line 304) is determining the "appropriate" FAT type to be FAT32 for any system where the total file size in MB is >512. From what I've read, OBB files are expected to be FAT16 (which is congruent with the 2GB limitation on their size). – monkey0506 Sep 20 '13 at 19:27
  • I asked a potentially relevant question here: http://stackoverflow.com/questions/18924872/superfloppyformatter-returning-fat32-for-anything-over-512-mb This may explain why the jobb tool is failing for directories < 4 MB but > 511 MB. – monkey0506 Sep 20 '13 at 20:01

1 Answers1

11

It turns out that a lot of the issues with the jobb tool are related to the FAT Filesystem library it uses, which is incorrectly determining the maximum size of a FAT16 storage unit to be < 512 MB (while in reality it should be 2 GB).

By modifying the FAT library I am able to successfully build OBB files over 512 MB with the jobb tool. This is also relevant to the reason why OBB files under 4 MB are invalid. The jobb tool source should also be updated because the expected file system should always be FAT16. Small units should be fine, and it should only give issue if there are more than 2 GB worth of data.

I will be reporting this as a bug in the FAT library, and an issue in the jobb tool.

Edit: The modified source files and a GUI tool are available on my GitHub (though, regrettably, I didn't retain a proper version history of the changes).

monkey0506
  • 2,489
  • 1
  • 21
  • 27
  • 1
    I've written a simple GUI app as a replacement/wrapper for the console tool, which can be downloaded at https://github.com/monkey0506/jobbifier/tree/master/jObbifier/bin (just grab jObbifier.jar, which is a runnable JAR file). This contains a fix to allow up to 2 GB of data. – monkey0506 Sep 26 '13 at 09:40
  • > "By modifying the FAT library" - can you provide more details on what you did exactly and which file(s) were affected? – Mathias Conradt Feb 01 '15 at 12:24
  • @MathiasLin I didn't see your question before, but this other question I asked (http://stackoverflow.com/questions/18924872/superfloppyformatter-returning-fat32-for-anything-over-512-mb) explains which file was causing the problem. The modification I made was to return FAT16 if the total file size was under 2 GB. That change did not ever cause any problems for me, and fixed this crash. – monkey0506 Jul 19 '15 at 05:40
  • @monkey_05_06 thanks for the tip. Since I found nothing really straightforward, I've set up a patched version at https://github.com/idminnovates/android-sdk-fat32lib – cdelacroix Sep 02 '15 at 13:28
  • Thanks for this, saved me a lot of frustration. I ran into the issue when testing because I was using a small test archive < 4Mb and just added a few more dummy files in there to increase the size and then it worked fine. My final OBB file will be much larger. – samgak Feb 09 '17 at 23:23