0

I would like to load an external database in Android. I have used the approach from with tutorial Read External Database with SQLiteOpenHelper on Android. I created an folder called "assets" in src/main/assets and I inserted the database 'Test_DB' there. Unfortunately, when I start the app I get the error message:

Cannot open database 'C:/Users/user/Projects/AndroidStudioProjects/Bapp/Bapp_Projekt/app/src/main/assets/Test_DB.db': Directory C:/Users/user/Projects/AndroidStudioProjects/Bapp/Bapp_Projekt/app/src/main/assets doesn't exist

altough the database 'Test_DB.db' exists at the mentioned path. Do you have an idea what causes the problem and how I can read this external database file.

Here you can see the code of the 'DatabaseContext' class for reading the external database:

package com.example.td.bapp;


import android.content.Context;
import android.content.ContextWrapper;
import android.database.DatabaseErrorHandler;
import android.database.sqlite.SQLiteDatabase;
import android.os.Environment;
import android.util.Log;

import java.io.File;

public class DatabaseContext extends ContextWrapper {

    private static final String DEBUG_CONTEXT = "DatabaseContext";

    public DatabaseContext(Context base) {
        super(base);
    }

    @Override
    public File getDatabasePath(String name)  {

        String dbfile = "C:/Users/user/Projects/AndroidStudioProjects/Bapp/Bapp_Projekt/app/src/main/assets/" + name;
        if (!dbfile.endsWith(".db")) {
            dbfile += ".db" ;
        }

        File result = new File(dbfile);

        if (!result.getParentFile().exists()) {
            result.getParentFile().mkdirs();
        }

        if (Log.isLoggable(DEBUG_CONTEXT, Log.WARN)) {
            Log.w(DEBUG_CONTEXT, "getDatabasePath(" + name + ") = " + result.getAbsolutePath());
        }

        return result;
    }

    /* this version is called for android devices >= api-11. thank to @damccull for fixing this. */
    @Override
    public SQLiteDatabase openOrCreateDatabase(String name, int mode, SQLiteDatabase.CursorFactory factory, DatabaseErrorHandler errorHandler) {
        return openOrCreateDatabase(name,mode, factory);
    }

    /* this version is called for android devices < api-11 */
    @Override
    public SQLiteDatabase openOrCreateDatabase(String name, int mode, SQLiteDatabase.CursorFactory factory) {
        SQLiteDatabase result = SQLiteDatabase.openOrCreateDatabase(getDatabasePath(name), null);

        if (Log.isLoggable(DEBUG_CONTEXT, Log.WARN)) {
            Log.w(DEBUG_CONTEXT, "openOrCreateDatabase(" + name + ",,) = " + result.getPath());
        }
        return result;
    }
}

and this class is called in an other class 'DataBaseHelper' which extends SQLiteOpenHelper and that has the following constructor

public DataBaseHelper(@Nullable Context context, String name) {
   // super(context, DATABASE, null, 1);
    super(new DatabaseContext(context), name, null, 1);

}

I'll appreciate every copmment and will be quite thankful for your help.

I think the problem might be that I created the folder 'assets' on my own as it was not there before. How can I tell Android that there is this 'assets' folder that should be included in the App?

UPDATE: I did what MikeT adviced me and utilise SQLiteAssetHelper SQLiteAssetHelper. Now the database can be loaded. However, every query of the database does not return anything as if the database was empty. This is definitely not the case. For example the following query yiels a rowCount of 0 which is not true

public Cursor getDataDB_TableItemNamesByItemType(String itemType) {

    SQLiteDatabase db = this.getWritableDatabase();
    Cursor res = db.rawQuery("select * from " + TABLE_ITEM + " where "
            + ITEM_TYPE + " = '" + itemType+ "'", null);
    Log.e("LogTag", "res.getCount(): " + res.getCount());
    return res;

}

If I perform the exactly same query on the SQLdatabase I get positive row counts (depending on the argument itemType). Can you imagine what the problem is as to why my externally read database appears to be empty?

VanessaF
  • 515
  • 11
  • 36

2 Answers2

1

Your main issue is with this line

String dbfile = "C:/Users/user/Projects/AndroidStudioProjects/Bapp/Bapp_Projekt/app/src/main/assets/" + name;

You are basically trying to tell the device to look at your source code rather than to look for the asset file (the database) in the devices assets folder.

Android Studio (assuming you are using it) when it makes the package that is installed places the file in the assets folder not in c:/Users/user/Projects ....

In the tutorial (which doesn't use the assets file) but places the file on the SD card it uses:-

String dbfile = sdcard.getAbsolutePath() + File.separator+ "database" + File.separator + name;

However, as you have copied the file into the assets folder of the project then what you do is copy the file (the database) from the assets folder to a location (you cannot use the database from the assets folder directly as it is compressed).

Here's a quick quide that shows how the above can be done (note that step10 is for the tables/columns and would have to be changed accordingly) how to launch app with SQLite darabase on Android Studio emulator?

An alternative would be to utilise SQLiteAssetHelper here's a guide that uses SQLiteAssetHelper .How to copy files from project assets folder to device data folder in android studio?



EXAMPLE based upon your code

Stage 1 - Create the external database

Using Navicat as the SQLite tool (my preferred tool)

The database was created with a single table namely item The table has 2 columns namely item_name and item_type

4 rows were inserted 2 with an item_type of type1 and 2 with an item_type of type2 as per :-

enter image description here

The database connection was closed and therefore saved and then the connection was reopened to confirm that the database is in fact populated, the connection was then closed again.

The existence of the database file was then confirmed to be at the appropriate location as per :- enter image description here

Stage 2 - Copy the database file into the assets folder

In this case the project has been called SO66390748 (the question #).

The assets folder was created in the Apps src/main folder as by default it doesn't exist and then the file copied into that folder noting that the size is as expected. enter image description here

The project file structure was then tested in Android Studio to confirm the existence of the database in the asset folder :- enter image description here

Stage 3 - Create the DatabaseHelper class as per

class DataBaseHelper extends SQLiteOpenHelper {

    private static final String TAG = "DBHELPER";
    public static final String DBNAME = "TestDB.db"; //<< Name of the database file in the assets folder
    public static final int DBVERSION = 1;
    public static final String TABLE_ITEM = "item";
    public static final String ITEM_NAME = "item_name";
    public static final String ITEM_TYPE = "item_type";

    SQLiteDatabase mDB;

    public DataBaseHelper(Context context) {
        super(context,DBNAME,null,DBVERSION);
        if (!ifDBExists(context)) {
            if (!copyDBFromAssets(context)) {
                throw new RuntimeException("Failed to Copy Database From Assets Folder");
            }
        }
        mDB = this.getWritableDatabase();
    }

    @Override
    public void onCreate(SQLiteDatabase sqLiteDatabase) {
        // Do NOTHING in here as the database has been copied from the assets
        // if it did not already exist
        Log.d(TAG, "METHOD onCreate called");
    }

    @Override
    public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) {
        Log.d(TAG,"METHOD onUpgrade called");
    }

    public Cursor getDataDB_TableItemNamesByItemType(String itemType) {
        SQLiteDatabase db = this.getWritableDatabase();
        Cursor res = db.rawQuery("select * from " + TABLE_ITEM + " where "
                + ITEM_TYPE + " = '" + itemType+ "'", null);
        Log.e("LogTag", "res.getCount(): " + res.getCount());
        return res;
    }

    /*
        Copies the database from the assets folder to the apps database folder (with logging)
        note databases folder is typically data/data/the_package_name/database
             however using getDatabasePath method gets the actual path (should it not be as above)
        This method can be significantly reduced one happy that it works.
     */
    private boolean copyDBFromAssets(Context context) {
        Log.d("CPYDBINFO","Starting attemtpt to cop database from the assets file.");
        String DBPATH = context.getDatabasePath(DBNAME).getPath();
        InputStream is;
        OutputStream os;
        int buffer_size = 8192;
        int length = buffer_size;
        long bytes_read = 0;
        long bytes_written = 0;
        byte[] buffer = new byte[length];

        try {

            is = context.getAssets().open(DBNAME);
        } catch (IOException e) {
            Log.e("CPYDB FAIL - NO ASSET","Failed to open the Asset file " + DBNAME);
            e.printStackTrace();
            return false;
        }

        try {
            os = new FileOutputStream(DBPATH);
        } catch (IOException e) {
            Log.e("CPYDB FAIL - OPENDB","Failed to open the Database File at " + DBPATH);
            e.printStackTrace();
            return false;
        }
        Log.d("CPYDBINFO","Initiating copy from asset file" + DBNAME + " to " + DBPATH);
        while (length >= buffer_size) {
            try {
                length = is.read(buffer,0,buffer_size);
            } catch (IOException e) {
                Log.e("CPYDB FAIL - RD ASSET",
                        "Failed while reading in data from the Asset. " +
                                String.valueOf(bytes_read) +
                                " bytes read successfully."
                );
                e.printStackTrace();
                return false;
            }
            bytes_read = bytes_read + length;
            try {
                os.write(buffer,0,buffer_size);
            } catch (IOException e) {
                Log.e("CPYDB FAIL - WR ASSET","failed while writing Database File " +
                        DBPATH +
                        ". " +
                        String.valueOf(bytes_written) +
                        " bytes written successfully.");
                e.printStackTrace();
                return false;

            }
            bytes_written = bytes_written + length;
        }
        Log.d("CPYDBINFO",
                "Read " + String.valueOf(bytes_read) + " bytes. " +
                        "Wrote " + String.valueOf(bytes_written) + " bytes."
        );
        try {
            os.flush();
            is.close();
            os.close();
        } catch (IOException e ) {
            Log.e("CPYDB FAIL - FINALISING","Failed Finalising Database Copy. " +
                    String.valueOf(bytes_read) +
                    " bytes read." +
                    String.valueOf(bytes_written) +
                    " bytes written."
            );
            e.printStackTrace();
            return false;
        }
        return true;
    }
    /*
    Checks to see if the database exists if not will create the respective directory (database)
    Creating the directory overcomes the NOT FOUND error
 */
    private boolean ifDBExists(Context context) {
        String dbparent = context.getDatabasePath(DBNAME).getParent();
        File f = context.getDatabasePath(DBNAME);
        if (!f.exists()) {
            Log.d("NODB MKDIRS","Database file not found, making directories."); //<<<< remove before the App goes live.
            File d = new File(dbparent);
            d.mkdirs();
            //return false;
        }
        return f.exists();
    }
}
  • As you can see your original getDataDB_TableItemNamesByItemType has been included unchanged.
  • As per the comments (I'd suggest reading them) the above is a little long-winded BUT this enables you to see what is happening. Obviously remove the logging before distributing the App.

Stage 4 - Invoke the database helper and extract the data from the database

In this case the App's main activity is used to invoke.

The activity used is :-

public class MainActivity extends AppCompatActivity {

    DataBaseHelper myDBHlpr;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        // Get an instance of the DatabaseHelper class (copy will be done IF DB does not exist)
        myDBHlpr = new DataBaseHelper(this); 

        // Get some data from the database using your method
        Cursor csr = myDBHlpr.getDataDB_TableItemNamesByItemType("type2");
        while(csr.moveToNext()){
            Log.d("DB_ROWINFO",
                    "ITEM NAME is " + csr.getString(csr.getColumnIndex(DataBaseHelper.ITEM_NAME))
                    + "ITEM TYPE is "
                    + csr.getString((csr.getColumnIndex(DataBaseHelper.ITEM_TYPE))
                    )
            );
        }
        // ========================================
        // ALWAYS CLOSE CURSORS WHEN DONE WITH THEM
        // ========================================
        csr.close();
    }
}

Stage 5 - The Results (the log)

The App if it exists is uninstalled then run and produces:-

03-06 09:32:52.759 5341-5341/a.a.so66390748 I/art: Rejecting re-init on previously-failed class java.lang.Class<androidx.core.view.ViewCompat$2>
03-06 09:32:52.787 5341-5341/a.a.so66390748 D/NODB MKDIRS: Database file not found, making directories.
03-06 09:32:52.787 5341-5341/a.a.so66390748 D/CPYDBINFO: Starting attemtpt to cop database from the assets file.
03-06 09:32:52.787 5341-5341/a.a.so66390748 D/CPYDBINFO: Initiating copy from asset fileTestDB.db to /data/user/0/a.a.so66390748/databases/TestDB.db
03-06 09:32:52.787 5341-5341/a.a.so66390748 D/CPYDBINFO: Read 8191 bytes. Wrote 8191 bytes.
03-06 09:32:52.805 5341-5341/a.a.so66390748 D/DBHELPER: METHOD onCreate called
03-06 09:32:52.811 5341-5341/a.a.so66390748 E/LogTag: res.getCount(): 2
03-06 09:32:52.811 5341-5341/a.a.so66390748 D/DB_ROWINFO: ITEM NAME is Item2ITEM TYPE is type2
03-06 09:32:52.811 5341-5341/a.a.so66390748 D/DB_ROWINFO: ITEM NAME is Item4ITEM TYPE is type2
03-06 09:32:52.822 5341-5355/a.a.so66390748 D/OpenGLRenderer: Use EGL_SWAP_BEHAVIOR_PRESERVED: true

The App is rerun a second time (not uninstalled) and the log is :-

03-06 09:35:37.876 5465-5465/a.a.so66390748 I/art: Rejecting re-init on previously-failed class java.lang.Class<androidx.core.view.ViewCompat$2>
03-06 09:35:37.908 5465-5465/a.a.so66390748 E/LogTag: res.getCount(): 2
03-06 09:35:37.908 5465-5465/a.a.so66390748 D/DB_ROWINFO: ITEM NAME is Item2ITEM TYPE is type2
03-06 09:35:37.908 5465-5465/a.a.so66390748 D/DB_ROWINFO: ITEM NAME is Item4ITEM TYPE is type2
03-06 09:35:37.956 5465-5498/a.a.so66390748 D/OpenGLRenderer: Use EGL_SWAP_BEHAVIOR_PRESERVED: true

i.e. The database, as it already exists and the database thus exists, doesn't copy the database but still extracts the data as expected.

Note the above has been written at my convenience so just Main activity and the database helper are used. You will obviously have to adapt the code accordingly

It is assumed that you followed the advice given in the comments and tried SELECT * FROM the_table_name (i.e. no WHERE clause). I says this as the query you have used is case sensitive and if the argument passed to your getDataDB_TableItemNamesByItemType method doesn't exactly match, then you would extract nothing. (e.g. passing Type2 instead of type2 shows 0 from the count)

MikeT
  • 51,415
  • 16
  • 49
  • 68
  • Thanks MikeT for your answer. I tried your second suggested approach by utilising the SQLiteAssetHelper. Basically now I do not get an error message any more. HOWEVER: The externally loaded database seems to be empty altough it is definitely not empty. All queries yield a rowCount of 0 altough the same queries yiels positive rowCounts when executed directly in the database (by using an SQL Editor). Do you have an idea, what might lead to this problem? I have updated my question and I added the query that yields to a rowCount of 0 altough there are entries in the database for that query. – VanessaF Feb 28 '21 at 09:27
  • Did you re-install the app as opposed to just rerun the App? If so re-install the App and try again. If you did re-install then try changing the query to be `"SELECT * FROM " + TABLE_ITEM` i.e. without a where clause so that it extracts all rows from the table. Are you checking the log for all messages? If you use the first posted link rather than the 2nd Asset helper example it logs the various stages so you can easily see the size of the file copied. – MikeT Feb 28 '21 at 19:28
  • Thanks MikeT for your answer and effort. What do you exactly mean by "re-installing" the app? I use Android Studio with an Emulator and I just built the project and run the app. – VanessaF Mar 02 '21 at 18:47
  • I tried what you suggested with the modified query, but this also led to a rowCount of 0 altough there are entries in the database which I give the path to in the Java class. Somehow the app does not see the entries in the SQLite database. When I not read the database externally but rather create the entries internally, then they are correctly displayed but when reading the database externally with you suggested approach there is a problem as the entries (which are definitely there) are not seen. – VanessaF Mar 02 '21 at 18:52
  • @VanessaF re-installing, is a matter of uninstalling the App (from the emulated device via Settings) and then running it again. SQLite databases are permanent (they persist) as they are stored in the App's storage. Uninstalling the App deletes the database and it will be recopied. SQLiteAssetHelper WILL NOT override an existing database, so you may somehow have an empty version of the DB and it will stay empty. You may also wish to delete the asset file and re-copy it (note how I stress checking the source database). – MikeT Mar 02 '21 at 22:33
  • @VanessaF another test is, when the emulated device is running, use **Device File Explorer** in Studio. Look at the device and then select the **data** folder, when this opens then open the **data** folder (data folder inside the data folder). Open the folder that is the **same as the package name of the App**. Inside the package folder there should be a folder name **databases**. Inside this there should be the database file *(may be other files with the same name but extension such as .wal .shm or -journal)*. File should be the same as the source file size. – MikeT Mar 02 '21 at 22:52
  • Thanks Mike for your answer and effort. I really appreciate it. Basically I checked what you told me (I knew about this before) and noticed something strange. The database is available there and has the same size as the externally loaded database with 104 KB (and of course the same name). HOWEVER: When I save the database from the file explorer and open it, the database it completely empty. It has all the tables and columns, but - opposed to the externally loaded database - there are no entries at all. – VanessaF Mar 03 '21 at 18:46
  • Is database WAL (write-ahead logging)? Sounds like you may be omitting the .wal file. Changes are made to the .wal file and written to the actual database when it is checkpointed. If for some reason when you use the editor if the file isn't closed and it's WAL then the data may be lost if the file is copied at that stage. I'd suggest editing the database in the editor. see https://www.sqlite.org/pragma.html#pragma_wal_checkpoint and probably use the TRUNCATE option, the .wal file should be 0 bytes, if it isn't then you need to empty it or additionally copy the .wal file (not recommended). – MikeT Mar 04 '21 at 08:45
  • Thanks MikeT for your answer. Unfortunately I have to admit that I do not understand at all what you are suggesting? Basically - as said before - the externally loaded database, called "TestDB.db" is in the assets folder and is not empty. When I read this database using your suggested 2nd appraoch, there are no entries at all there. I do not read any WAL file (do not even know what this is) and yes, I edit the database in the SQLIte editor and created the entries there. So what shall I do now? – VanessaF Mar 04 '21 at 18:35
  • I'm saying look at wherever the original database is (not any copy) if there is a [`TestDB.db.wal`] file then this is the WAL file. If it's not 0 bytes then it contains some data (e.g. inserted rows) that have not been applied to the file [`TestDB.db`] so if you copy this file you've lost data. Id you copy the .wal file to the assets folder then you also need to have it copied from the package along with the database BEFORE opening the database in the App. Better to use whatever tool you used to create the database to ensure that it is checkpointed so the .wal file is empty. – MikeT Mar 05 '21 at 03:55
  • Another option is to use journal mode as using this updates the database but stores the updates in the journal file (TestDB.db-journal) allowing the updates to be undone so not copying the journal file doesn't cause such and issue. The only other way I could perhaps help you is if you allow me to access all of the source including the database, perhaps by using something like github . – MikeT Mar 05 '21 at 03:59
  • Thanks MikeT for your answer and effort. I really appreciate it. Unfortunately I am not allowed to share the full project. Can you think about something how I can solve the problem? As said before, the database is not empty but in Andorid it is empty (altough the size is 104 KB which is equal to the size in the datastorage). – VanessaF Mar 05 '21 at 18:25
  • Maybe there is something wrong with the DatabaseHelper Class that extends from SQLiteOpenHelper. I created this class thinking that I would not use an external database but just generate all the entries in Java. So this is why I have an insert method for all database tables and I have a public void onCreate(SQLiteDatabase sqLiteDatabase) and a public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) method. Might this somehow interfere with your suggested approach for reading an external database? – VanessaF Mar 05 '21 at 18:28
  • I checked whether those methods in the DatabaseHelper Class that extends from SQLiteOpenHelper are called but they are not called. So I guess the DatabaseHelper Class itself is not the problem. Further, I checked the 'Test_DB.db-journal' file but it has a size of 0 bytes and can't be opened using a SQL editor. So the basic problem and question is why does Android delete all entries when reading the external database and the very strange thing is why has the internal Test.db database with NO ENTRIES execatly the same size as the external database (104 KB) which HAS ENTRIES. – VanessaF Mar 05 '21 at 18:46
  • @VanessaF I've added a working example to the answer. – MikeT Mar 05 '21 at 22:48
  • Thanks MikeT for your tremendous answer. now understand what my mistake was thanks to your great help. In fact in the onCreate(SQLiteDatabase sqLiteDatabase) method of the DataBaseHelper I overwrote the external database by creating the same tables again (ideally SQLite should have sent a message somehow that those tables existed before, but this did not happen). I realized this, when I was using your item database and saw that all further tables with all columns had been inserted there. – VanessaF Mar 06 '21 at 08:54
  • Still I have a question to you code: You inserted some methods that I do not understand like "private boolean copyDBFromAssets(Context context) " and "private boolean ifDBExists(Context context)". Can I just delete them because as far as I understood they are only used for logging? – VanessaF Mar 06 '21 at 08:56
  • Another strange thing that I realized is that if I change some entries in the database in the asset folder, the app does not realized it and read the 'old' version of the database. So the changes are not updated. I always have to change the name of the database in the database folder and thre corresponding name in the Java class to 'update' the database such that the changes are visible. Is there not a more convenient way of doing this (instead of always changing the file name and the variable name in the Java class)? Changing the version number only did not help. – VanessaF Mar 06 '21 at 09:37
  • @VanessaF first things first. If you believe the answer has helped you then please tick the question as answered (I believe the original question has well and truly been answered). You should really be adding new questions as you encounter subsequent problems. OK first those two methods are required they copy the database from the assets folder BUT only once (the `ifDBExists` checking if the database has already been copied). That's how you would typically have the App in production. I'll answer the strange thing next. – MikeT Mar 06 '21 at 11:23
  • The strange thing is typically how an App would work. That is the asset db is copied once. If you want to manage new versions of the assets file then matters get a little more complex. However, if it's while you are developing the app before distributing it then just make the changes overwrite the asset file and uninstall the App. This effectively deletes the database so the copy is redone. Here's a link that you may find useful https://stackoverflow.com/questions/54822935/which-methods-can-be-used-to-manage-differing-versions-of-pre-existing-databases/54822936#54822936 – MikeT Mar 06 '21 at 11:35
  • @VanessaF Some more links re the updating via a changed/new asset;- https://stackoverflow.com/questions/53918372/handle-sqlite-database-upgrade-on-pre-populated-database/53918642#53918642 and https://stackoverflow.com/questions/53801149/update-db-sqlite-asset-helper-library/53827525#53827525 and https://stackoverflow.com/questions/59634325/createfromasset-migration-but-keep-specific-columns/59637092#59637092 and https://stackoverflow.com/questions/56873021/overwrite-existing-shipped-sqlite-db-with-the-new-db-version-on-my-next-android/56877939#56877939 – MikeT Mar 06 '21 at 11:44
  • Thanks for your answer and effort MikeT. Basically the database is changed all the time dynamically by the user as new items are added and removed based on the user iteraction. Further, the database should be changed from time to time externally. As stated before, I can easily archieve the update by just renaming the database and I was wondering whether a more convenient way exists for that (normally it should exist). However, when having a look at your suggested code it looks way more complex than just changing the database name. So I guess there is no easy solution for that – VanessaF Mar 07 '21 at 08:26
  • I upvoted and accepted your answer. Thanks for your tremendous help. I really appreciate it. – VanessaF Mar 07 '21 at 08:27
  • Hi Mike, quite some time has passed since I asked this question and had a conversation with you. I just have a follow-up question. Can I always use your suggested code to load the database in an app? So can I use it with every app? I am strongly wondering, why one has to use such a complex method that you implemented and why there is not built in function in SQLiteOpenHelper to read the database from the asset folder – VanessaF Jan 20 '22 at 21:14
  • @VanessaF There's SQliteAssetHelper but that's no longer supported. Also if you use Room then you have createFromAsset and createFromFile which will do the copy. However with Room one of the issues is matching the existing database with room as "Room Expects" (the message you get when room doesn't like the schema if Expected ...... Found ..... ) – MikeT Jan 21 '22 at 04:02
  • Thanks for your answer. Actually I don't use room but only SQLiteOpenHelper. So can I use your code for every App? For me it is indeed extremely strange that those libraries don't offer an option to integrate the database in your app and you have to write complex methods for that purpose on your own altough literally every app uses an internal database like SQLite. – VanessaF Jan 21 '22 at 17:16
  • 1
    @VanessaF yes you can. – MikeT Jan 21 '22 at 18:08
  • Thanks Mike for your great help. I really appreciate it. – VanessaF Jan 22 '22 at 08:23
  • Hi Mike. You wrote that I can always use your method. As far as I understand I have to use this method as there is no alternative? Is that correct? I am quite suprised that there is no method for Android to integrate a SQLite database into the app. I mean you said your implemented method is suitable for every application. Why do the developers of libraries for SQLite in Android or Google itself not just include a similar method into their libraries. For me this does not make sense. Why do they develop libraries for working with SQLite but not for integrating it into the app. – VanessaF Jan 22 '22 at 15:40
  • 1
    @VanessaF My guess is that Google want you to use either Room or Firebase. – MikeT Jan 22 '22 at 19:15
1

Supplementary

In regards to the comments :-

Basically the database is changed all the time dynamically by the user as new items are added and removed based on the user iteraction. Further, the database should be changed from time to time externally. As stated before, I can easily archieve the update by just renaming the database and I was wondering whether a more convenient way exists for that (normally it should exist). However, when having a look at your suggested code it looks way more complex than just changing the database name. So I guess there is no easy solution for that.

I believe you need to consider two separate functionalities.

That is 1) maintaining the App user's data and 2) changing the App's supplied data.

You could separate them by using separate but connected databases, another way could be to differentiate the two types according to entity naming (an entity being tables triggers views etc).

  • e.g. you have the App supplied entities be called as_ or the user's entities called user_ or both.

When a new asset file is detected you could rename the existing file to old_TestDB.db and then copy the new file to TestDB.db. You could then delete all rows from the user's entities (perhaps not needed if you never supply user's data) and then connect the old_TestDB.db copy the data from the old to the new drop the connection to old_TestDB.db, and finally delete the old_TestDB.db file (or perhaps keep it just in case there are issue so you could have something to restore from).

Detecting the new asset file using file names could be problematic as you might end up with many old files in the assets folder. As such it could be simpler and more favourable for the App size if the same filename was used BUT the sqlite user_version was utilised.

The sqlite user_version is a version number stored in the header data of the database. It's 4 byte at an offset of 60 bytes into the header. You can access this without the overheads of opening the database by using standard IO (like when the asset file is copied). You'd compare this to the already existing database and if different handle the new asset file.

You can change the user version using the SQL PRAGMA user_version = n (some editors allow you to change this via a window) https://www.sqlite.org/pragma.html#pragma_user_version

An alternative approach would to not try to detect a changed asset, nor utilise the SQLite user_version but to always assume a changed asset and thus when a new version of the App is introduced to additionally increment the DATABASE VERSION that is passed to the SQLiteOpenHelper and to then perform the copy of the asset file and applying the user's data in the onUpgrade method. However, this can be fraught with errors/issues as the database has been opened (to extract the database's user version the user_version will be changed to the App's DATABASE VERSION).

However, you should design the database accordingly first before starting to write code.

MikeT
  • 51,415
  • 16
  • 49
  • 68
  • @VanessaF I've added this supplementary in response to your later comments. P.S> thanks for marking my answer. – MikeT Mar 07 '21 at 09:49
  • Thanks Mike for your further comment and effort. I really appreciate your tremendous help. Basically what I want is also to use the databases in a client servers architecture on different devices. Both server and clients should have a SQLite database. Based on the actions of both the server and the client the databases should be updated (the clients should update their own database but also the one of the server; the server should also update the clients databases). You can think about this as an ordering system in a restaurant. The server is the ktichen and the clients are the tables. – VanessaF Mar 07 '21 at 10:00
  • You perhaps want to consider using firebase (see https://firebase.google.com/) which acts as the server. The client server model is not really suitable for SQLite (see https://sqlite.org/whentouse.html) as the server aspect is not built in. – MikeT Mar 07 '21 at 19:07
  • Thanks for your answer MikeT. I do not want to use the Firebase DB since it is not a SQL database and I use SQLite databases both in my clients and in my servers. I would like to use sockets to send data from clients to servers and vice versa. Is this not possible with SQLite? I thought that this should be managable. In your suggested link about SQLite it says "If there are many client programs sending SQL to the same database over a network, then use a client/server database engine instead of SQLite." What is meant by 'many'. I think in my case there are not more than 10 clients. – VanessaF Mar 10 '21 at 18:05
  • Do you maybe know a SQLite based client/server database engine? As I have already spent quite much time into SQLite database I do not want to learn a completely new type of database system. – VanessaF Mar 10 '21 at 18:40
  • @VanessaF many really means > 1. You could utilise SQLite, it's a lot of work to implement. You may wish to check out https://dba.stackexchange.com/questions/21/is-it-possible-to-use-sqlite-as-a-client-server-database. This mentions Valentina (https://www.valentina-db.com/en/sqlite-database-server). Code Project also has https://www.codeproject.com/Articles/22416/A-SQLite-Server-Client-Implementation. I'd also suggest having a look at https://stackoverflow.com/questions/62486948/using-sqlite-in-a-server-application-alternatives (the diagram). – MikeT Mar 10 '21 at 19:02
  • Thanks MikeT for your answer and effort . I really appreciate your tremendous help (I upvoted your second answer as well and I had already accepted your first answer). Why is it a lot of work to implement SQLite? Basically there should not be anything specific that I have to implement in SQLite other then the normal methods (insert, update, delete). The rest should be done by socket programming in Java. – VanessaF Mar 12 '21 at 18:15
  • Thanks MikeT for your comments. Any further comments on my last comment? I'd highly appreciate any further comments from you. – VanessaF Mar 14 '21 at 08:17
  • @VanessaF on each client/app you would probably have to cater for "should I update, insert, delete" and additionally perhaps also "where from or where to". i.e. the things that are typically left to the server in a server client system. Go into more depth and you may also have to include queuing/ordering changes and perhaps even extracts. – MikeT Mar 14 '21 at 10:30
  • Thanks MikeT for your comments. I really appreciate that you share your huge knowledge. I have to admit that unfortunately I still do not understand what the problem is when using SQLite in a client server architecture. The clients have all the same app. So I just have to implement the insert, delete, update and create method once for every table. Now the clients can receive a message (via sockets) from the server and based on that they just call one of those methods. And the clients can just send a message to the server (e.g. an order) and the server just updates its SQLite database – VanessaF Mar 17 '21 at 18:25
  • So I do not see any difference compared to normal database queries and methods using SQLite. The only difference is that the database queries are triggered by other devices via a message and the database methods (insert, update etc) remain exactly the same – VanessaF Mar 17 '21 at 18:28
  • Thanks MikeT for your comments. Any further comments on my last comment? I'd highly appreciate any further comments from you. – VanessaF Mar 19 '21 at 17:19
  • @VanessaF it sounds like you understand and have thought things out. Go for it. :) – MikeT Mar 19 '21 at 19:50
  • Thanks MikeT for your comment. So do you think my approach is okay because at the moment I am not really sure what to do. You can think about my application as a ordering system in a restaurant. Whenever one client orders something, it is written in the client's SQLite db and a message is sent to the server which updatedes one of his SQLite db tables. And the server can also adjust it SQLite db (for example regarding the available meals) and if that is done the server sends a message via sockets to the clients which update their SQLite database accordingly. What do you think about this? – VanessaF Mar 20 '21 at 08:30
  • Thanks for your answers and effort. I really appreciate it. Any comments on my last comment? I'd highly appreciate it if you can share your knowledge. – VanessaF Mar 21 '21 at 08:08
  • Hi MikeT. Thanks for your answer and effort. Any comments to my last comments? I'd be quite happy to get further comments from you about my question. – VanessaF Mar 26 '21 at 13:31
  • @VanessaF it sounds OK. You might want to consider a means of confirming actions done. So order sent by client, server sends update (flagged as pending) client updates sends sever confirmation server flags as done. Whenever client contacts server any pending requests are resent. – MikeT Mar 26 '21 at 20:41
  • Thanks MikeT for your comment and effort. I really appreciate your tremendous help. As I have never implemented a client server system I am not sure what you mean by "flagged as pending" and " server flags as done". Shall the server sent a message to the client (maybe a String or a boolean) to tell them whether it has received their message? Shall this be done in the SQLite database? And what do you mean by "Whenever client contacts server any pending requests are resent"? What is a pending request and do I have to do this in the SQLite database? – VanessaF Mar 27 '21 at 08:16
  • @VanessaF by pending the message is sent (whichever way) but not actioned when the receiver actions the message than it sends a done message and the pending status would be changed to complete. Until the action/message has been actioned it could be resent (this is where it can get difficult as you then may need to protect against duplicating actions). e.g. bad messaging could result in NO ORDER or MULTIPLE ORDERS instead of the single order. Perhaps do a search on *message handling between client an server*. – MikeT Mar 28 '21 at 03:29
  • Thanks MikeT for your comment and great help. I searched about this issue but I could not find much about message handling with SQLite and Java in Android. So I can for example just sent a boolean variable if a SQLite database entry is added or deleted from the server or the client to the others such that they know about this? My question would be how to assert that a SQLite method has been excecuted correctly sucht that I can send the boolean variable (or String message)? – VanessaF Mar 28 '21 at 08:17
  • @VanessaF to Assert, check the return e.g. for an insert if the row was inserted then the SQLidatabase convenience insert method returns the rowid or -1, if not inserted. Update method returns rows affected 0 if none, Likewise for Delete. Could be complicated if more than 1 action per message. Boolean would suffice BUT obviously an identifier so that what is being said yes/no can be determined. Especially considering the server may be handling many requests at a time. – MikeT Mar 28 '21 at 08:27
  • Thanks MikeT for your answers and effort. I really appreciate your tremendous help and knowlege about databases. So when the client orders something, it will be inserted into its SQLite db. Then the client should send the information from the order to the Server SQLite db via sockets. Shall the server now send a boolean to the client that it has received the order (maybe with an idetifier)? And vice verca: When the server has finished something it will update its SQLite database and try to infrom the client such that they can update their db. Shall the client send a boolean to the server now? – VanessaF Apr 01 '21 at 17:07
  • 1
    @VanessaF Yes to both so the pending requests become finalised. – MikeT Apr 01 '21 at 18:41
  • Thanks for your tremendous help MikeT. I will try to implement my application considering your great adivces. I really appreciate your effort. – VanessaF Apr 09 '21 at 16:14