How to store large array of integers in Android?

Question

I need to store Objects that contains 50,000 - 200,000 of integers. Everyday user could generate 0-10 such Objects. My idea was to use Realm, but Realm doesnt have built-in support for array/list of primitives. Using Realm there are 2 ways :

1)Create RealmInt - Realm object with only one field

2)Storing array of 100,000 ints like string : "1,20,41,"

So, is there are way that could called "best practice" for this case?

P.S. I am sorry if this question is out of stackoverflow "topic". If yes - say, and i will delete this question

Answer 1

You can convert the integer array into a byte array and store it as a blob.

The following code is a test SQliteOpenHelper subclass which defines a simple 1 column table along with two methods for inserting and retrieving the integer array:-

public class DBHelperObjectStore extends SQLiteOpenHelper {

    private static final String DATABASENAME = "objects";
    private static final int DATABASEVERSION = 1;

    private static final String MAINTABLE = "main";
    private static final String OBJCOL = "object";
    private static final String OBJCOLTYPE = " BLOB ";
    private static final String MAINTABLECREATE = "CREATE TABLE " +
            MAINTABLE + "(" +
            OBJCOL + OBJCOLTYPE +
            " );";

    private long lastinsertedrow;

    DBHelperObjectStore(Context context) {
        super(context,DATABASENAME,null,DATABASEVERSION);
    }

    @Override
    public  void onCreate(SQLiteDatabase db) {
        db.execSQL(MAINTABLECREATE);

    }
    @Override
    public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {

    }

    public void insertRow(int[] intarray) {

        ByteBuffer bb = ByteBuffer.allocate(4);

        byte[] objasbytes = new byte[intarray.length * 4];
        int byteidx = 0;
        for (int i: intarray) {
            objasbytes[byteidx++] = (byte) (i >>> 24);
            objasbytes[byteidx++] = (byte) (i >>> 16);
            objasbytes[byteidx++] = (byte) (i >>> 8);
            objasbytes[byteidx++] = (byte) i;
        }
        SQLiteDatabase db = this.getWritableDatabase();
        ContentValues cv = new ContentValues();
        cv.put(OBJCOL,objasbytes);
         lastinsertedrow = db.insert(MAINTABLE,null,cv);
        db.close();
    }

    public int[] getRow(long id) {
        int[] rv = new int[0];
        SQLiteDatabase db = this.getWritableDatabase();
        String[] columns = {"*"};
        String whereclause = "ROWID =?";
        String[] whereargs = {Long.toString(id)};
        Cursor csr = db.query(MAINTABLE,columns,whereclause,whereargs,null,null,null);
        while (csr.moveToNext()) {
            byte[] bytes = csr.getBlob(csr.getColumnIndex(OBJCOL));
            rv = new int[bytes.length / 4];
            for (int i=0; i < bytes.length; i = i + 4 ) {
                rv[i/4] = bytes[i] << 24 |
                        (bytes[i+1] & 0xff) << 16 |
                        (bytes[i+2] & 0xff) << 8 |
                        (bytes[i+3] & 0xff);
            }
        }
        csr.close();
        return rv;
    }
}

The following is code from the activity (with some testing code commented out) :-

        DBHelperObjectStore dbstorehlp = new DBHelperObjectStore(this);
        int arrayelements = 200000;
        Random rdm = new Random(System.currentTimeMillis());
        int[] arraytosave = new int[arrayelements];
        for (int i = 0; i < arraytosave.length; i++) {
            arraytosave[i] = rdm.nextInt(655555555);
        }
        dbstorehlp.insertRow(arraytosave);
        int retrievedarray[] = dbstorehlp.getRow(1);


        int errorcount = 0;
        int matchcount = 0;
        if (arraytosave.length != retrievedarray.length) {
            Log.d("RESULTS",
                    "Array MISMATCH the number of elements of the saved array is " +
                            Integer.toString(arraytosave.length) +
                            " whilst the retrieved array has " +
                            Integer.toString(retrievedarray.length) +
                            " elements."
            );
        } else {
            for (int i=0; i < arraytosave.length; i++) {
                if (arraytosave[i] != retrievedarray[i]) {

                    /*
                    Log.d("RESULTS","Elements with index of " +
                            Integer.toString(i) + " mismatch " +
                            Integer.toString(arraytosave[i]) +
                            " against " +
                            Integer.toString(retrievedarray[i])
                    );
                    */
                    errorcount++;
                } else {
                    /*
                    Log.d("RESULTS","Element at index " +
                            Integer.toString(i) +
                            " are both " +
                            Integer.toString(retrievedarray[i])
                     );
                    */
                    matchcount++;
                }
            }
            Log.d("RESULTS","Error Count   =" + Integer.toString(errorcount));
            Log.d("RESULTS","Matched Count =" + Integer.toString(matchcount));

The above creates the DBHelper, then generates an integer array of the specified size (200000) above populating it with random numbers up to 655555555 (close to the max).

The array is saved in the DB via dbstorehlp.insertRow(arraytosave);

The row with rowid 1 (obviously only suitable for testing) is retrieved using int retrievedarray[] = dbstorehlp.getRow(1);

The rest of the code compares the original against the retrieved (again for testing).

I've run this a number of times with 200000 (around 800 or so K per row). I tried at 2000000, it crashed (memory) and again when just reducing count to 200000 but ran OK again after deleting the app data.

Note! if you run the above a second time then you will likely get 100% mismatches as it would compare the first row against another row.