greendao - android.database.CursorWindowAllocationException: Could not allocate CursorWindow of size 104857600 due to error -12

Question

So I basically parse a JSON and store ~3000 rows (no multimedia) in the greendao generated table and I am getting the above error from while querying data in this function:

    @Synchronized
    private fun checkGroupVisibility(
        dependencyList: ArrayList<Long>,
        dependencyType: String,
        mInspectionId: Long,
        isFromPhilosophyFlow: Boolean,
        daoSession: DaoSession?,
        string: String?
    ): Boolean {
        var show = true
//        Log.d("TAG", "checkGroupVisibility: lock testing 2"+string)
        if (!DataSanityUtils.isListEmpty(dependencyList)) {
            //check further in database if dependency question is answered
            val inspectionReportCarParList:List<InspectionReportCarPart>? = daoSession?.inspectionReportCarPartDao?.queryBuilder()
                ?.where(InspectionReportCarPartDao.Properties.InspectionID.eq(mInspectionId))
                ?.where(InspectionReportCarPartDao.Properties.QuestionId.`in`(dependencyList))
                ?.list()

            val inspectionReportCheckpointList:List<InspectionReportCheckpoint>? = daoSession?.inspectionReportCheckpointDao?.queryBuilder()
                ?.where(InspectionReportCheckpointDao.Properties.InspectionID.eq(mInspectionId))
                ?.where(InspectionReportCheckpointDao.Properties.ReportCheckpointVerdict.eq(CommonConstants.CHECKPOINT_VERDICT_NOT_OKAY))
                ?.list()

            for (checkPointId in dependencyList) {
                if (!isFromPhilosophyFlow && philosophyIdCheckPointMap.containsKey(checkPointId)) {
                    show = false
                    break
                }
                val isCarPartFound:InspectionReportCarPart?= inspectionReportCarParList?.find {
                    it.questionId==checkPointId
                }

                if (isCarPartFound==null) {
                    val isFound= inspectionReportCheckpointList?.find {
                        it.optionId==checkPointId
                    }
                    if (isFound == null) {
                        show = false
                        if (CommonConstants.DEPENDENCY_RELATION_AND.equals(
                                dependencyType,
                                ignoreCase = true
                            )
                        ) {
                            break
                        }
                    } else if (CommonConstants.DEPENDENCY_RELATION_OR.equals(
                            dependencyType,
                            ignoreCase = true
                        )
                    ) {
                        show = true
                        break
                    }
                } else if (CommonConstants.DEPENDENCY_RELATION_OR.equals(
                        dependencyType,
                        ignoreCase = true
                    )
                ) {
                    show = true
                    break
                }
            }
        }
//        Log.d("TAG", "checkGroupVisibility: lock testing 3:"+string)
        return show
    }

The exact error is coming when evaluating inspectionReportCarParList (i.e. the first query in the code). Here is the full stacktrace :

android.database.CursorWindowAllocationException: Could not allocate CursorWindow '/data/user/0/com.example.visor/databases/report.json-db' of size 104857600 due to error -12.
    at android.database.CursorWindow.nativeCreate(CursorWindow.java)
    at android.database.CursorWindow.<init>(CursorWindow.java:139)
    at android.database.CursorWindow.<init>(CursorWindow.java:120)
    at android.database.AbstractWindowedCursor.clearOrCreateWindow(AbstractWindowedCursor.java:202)
    at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:149)
    at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:142)
    at de.greenrobot.dao.AbstractDao.loadAllFromCursor(AbstractDao.java:371)
    at de.greenrobot.dao.AbstractDao.loadAllAndCloseCursor(AbstractDao.java:184)
    at de.greenrobot.dao.InternalQueryDaoAccess.loadAllAndCloseCursor(InternalQueryDaoAccess.java:21)
    at de.greenrobot.dao.query.Query.list(Query.java:130)
    at de.greenrobot.dao.query.QueryBuilder.list(QueryBuilder.java:353)
    at com.example.inspectionreport.utils.ReportUtils.checkGroupVisibility(ReportUtils.kt:31)

Also I am using de.greenrobot:greendao:2.0.0 this version of greendao.

I thought greendao is supposed to handle all the cusrsor actions and it closes the cursor after querying itself. But still cursorWindow size is getting very huge and the application crashes. Please help!

MikeT · Accepted Answer · 2022-07-06T22:47:09.390

A Cursor Window has a restricted size (depends upon the Android Version but up to 4Mb). 104857600 is 100Mb, so way too large.

A CursorWindow has to be able to accommodate 1 or more rows, if it cannot hold 1 row then you will get such an error.

It is possible to store data that is too large to be retrieved (it is a common issue with images).

The recommended way of handling such large amounts of data is to store the data, such as an image, in a file and store the path or part of the path in the database.

Additional re the comment

I got it but in this case I do not store any multimedia directly here

The issue is not limited to multimedia, that is just an example of the data being too large. It could be String (TEXT) data.

Here's an example that shows the case (just using SQLite which greenDao is basically just a wrapper around), the example also includes an example of an alternative that does not experience the issue as it stores managable sections of the details of the report in a separate related table.

Consider the following code:-

public class MainActivity extends AppCompatActivity {

    DBHelper mDBHelper;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        mDBHelper = DBHelper.getInstance(this);
        /* Generate the base string i.e. 1000K */
        String current_report_detail = generateReportDetail(1000);

        /* Insert Rows with 1000K report detail */
        mDBHelper.insertReportV1(System.currentTimeMillis() / 1000,current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail);

        /* Insert Rows with 2000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail);

        /* Insert Rows with 3000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail);

        /* Insert Rows with 4000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail + current_report_detail);

        /* Insert Rows with 5000K eport detail */
        mDBHelper.insertReportV1(System.currentTimeMillis()/1000,current_report_detail + current_report_detail + current_report_detail + current_report_detail + current_report_detail);
        mDBHelper.insertReportV2(System.currentTimeMillis() / 1000, current_report_detail + current_report_detail + current_report_detail + current_report_detail + current_report_detail);

    }

    /* return a String whose length is the passed linecount * 1024 i.e. 1K per line */
    private String generateReportDetail(int linecount) {
        StringBuilder sb1 = new StringBuilder();
        /* Build base line  of 1K of characters */
        for(int i=0; i < 1024; i++) {
            sb1.append((char) ((i % 95) + 33));
        }
        StringBuilder sb2 = new StringBuilder();
        for (int i=0;i<linecount;i++) {
            sb2.append(sb1.toString());
        }
        return sb2.toString();
    }
}

So there are 5 stages each progressively inserting close to 1Mb of textual data along the lines of :-

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghi

....

Each stage inserts the same data (1Mb,2Mb .... 5Mb) twice.

The first into a single table with the detail (the String).
The second into two tables, the main table to hold the static report data (date in this case) and a table to hold sections of the String data (split into 1k chunks) i.e. a solution to the issue*.

When run, using App Inspection, then:-

The schema looks like
- V1 is the single table version that will have issues.
- V2 is an example of circumventing the CursorWindow size issues for EXACTLY THE SAME CORE DATA.
- the v2_details table holds chunks/sections of the lengthy report (1024 bytes in this example)

If an attempt is made to read all of the data in the v1_report table then the CursorWindow size occurs:-

Now for v2_report, there is no issue :-

As the long textual data has been split and placed into the v2_details table :-

The left most number is the row count as generated by App Inspection, showing 15000 rows exist in the detail. That is the details has been split into sections/chunks.

The issue can be clearly seeing (also proving that the data exists and has been inserted successfully into v1_report) by using the following:-

Although not accurate (the example has been quickly put together and it looks as though I've dropped a byte/character) the equivalent can be extracted from both the v2 tables as per:-

so instead of 1024000, the sectioned data is 1023000 (i.e. 1000 bytes dropped which is 1 byte per row).

I got it but in this case I do not store any multimedia directly here, but like 3000-3500 rows of data is stored in this db, even this does occur only in low end phones — suvodipMondal, Jul 06 '22 at 13:00
@suvodipMondal it's not the number of rows, it's the size of a single row that is the restriction/issue. In short a Cursor is a buffer but it has to be able to hold at least one row. The issue isn't limited to multimedia but that is the most common occurrence, perhaps the inspection report exceeds the size limitation, in which case you would want to look at splitting the report into manageable sections. — MikeT, Jul 06 '22 at 19:52
Thanks @MikeT. Yes this is one solution, let's see how can I split it — suvodipMondal, Jul 07 '22 at 05:38

greendao - android.database.CursorWindowAllocationException: Could not allocate CursorWindow of size 104857600 due to error -12

1 Answers1