File taking a long time to download in Android Studio

Question

When I start my application in Android Studio I have to convert a file from my assets folder to a File in order to read it as an RDF model. The issue is that my file is now getting larger and the App usually crashes or the file will take far too long to download. The file is around 170.384 MB, so it is a fairly big file, but I expect this file to grow to around 1GB before I am done. I have searched for answers on here as to how to get the file to download faster, but I haven't seen anything new that I haven't tried. For example, I started an AsyncTask in order to perform the "download" operation on a separate thread from the main UI thread. This stopped my application from crashing, but the file is still taking too long to download and usually times out. Here is an example of my code where I download the file in the AsyncTask:

 @Override
 public Model doInBackground(String... params){
        Model tdb = null;
        try {
            String filePath = context.getFilesDir() + File.separator + "my_turtle.ttl";
            File destinationFile = new File(filePath);
            FileOutputStream outputStream = new FileOutputStream(destinationFile);
            AssetManager assetManager = context.getAssets();
            InputStream inputStream = assetManager.open("sample_3.ttl");
            byte[] buffer = new byte[1024];
            int length = 0;
            int per = 0;
            while ((length = inputStream.read(buffer)) != -1) {
                outputStream.write(buffer, 0, length);
                publishProgress(per);
                per++;
            }
            inputStream.close();
            outputStream.close();


            Dataset dataset = TDBFactory.createDataset();
            tdb = dataset.getNamedModel("my_dataset.ttl");
            ///Model m = FileManager.get().loadModel(FileManager.get().readWholeFileAsUTF8(inputStream));
            TDBLoader.loadModel(tdb, filePath, false);
            model = tdb;
            MainActivity.presenter.setModel(model);
        }catch(java.io.FileNotFoundException e){
            e.printStackTrace(System.out);
        }
        catch(java.io.IOException e){
            e.printStackTrace(System.out);
        }
        return tdb;
    }

As you can see I must create a new filePath and create a new file, create an InputStream in order to read the file from assets, and write the InputStream to the file in order to load the RDF model in the TDB store. It is during this process that my app begins to "time out" within the AsyncTask.

Another issue that I am experiencing is updating my progress bar while the file is loading. I'm not sure how to get the length of an InputStream and I am doing something in my code like this:

while ((length = inputStream.read(buffer)) != -1) {
                outputStream.write(buffer, 0, length);
                publishProgress(per); //Publishing progress here...
                per++;
            }

Publishing the progress should trigger the onProgressUpdate method():

@Override
public void onProgressUpdate(Integer... progress){
        progressBar.setProgress(progress[0]);
        pd.setProgress(progress[0]);
    }

But for some reason my ProgressBar and ProgressDialog remain at 0% the entire upload time (even when I work with a smaller file that will upload). I am not sure why that would be the case.

Below is the full code for my AsyncTask class. I wanted to post the relevant parts of the code, but their may be something within the broader context of my code that is causing an issue:

 public class DownloadModel extends AsyncTask<String, Integer, Model> {
    Context context;
    Model model = null;
    ProgressDialog pd;

    public DownloadModel(Context context){
        this.context = context;
        this.pd = new ProgressDialog(context);
    }

    @Override
    public Model doInBackground(String... params){
        Model tdb = null;
        try {
            String filePath = context.getFilesDir() + File.separator + "my_turtle.ttl";
            File destinationFile = new File(filePath);
            FileOutputStream outputStream = new FileOutputStream(destinationFile);
            AssetManager assetManager = context.getAssets();
            InputStream inputStream = assetManager.open("sample_3.ttl");
            byte[] buffer = new byte[10000000];
            int length = 0;
            int per = 0;
            while ((length = inputStream.read(buffer)) != -1) {
                outputStream.write(buffer, 0, length);
                publishProgress(per);
                per++;
            }
            inputStream.close();
            outputStream.close();


            Dataset dataset = TDBFactory.createDataset();
            tdb = dataset.getNamedModel("my_dataset.ttl");
            ///Model m = FileManager.get().loadModel(FileManager.get().readWholeFileAsUTF8(inputStream));
            TDBLoader.loadModel(tdb, filePath, false);
            model = tdb;
            MainActivity.presenter.setModel(model);
        }catch(java.io.FileNotFoundException e){
            e.printStackTrace(System.out);
        }
        catch(java.io.IOException e){
            e.printStackTrace(System.out);
        }
        return tdb;
    }

    @Override
    public void onProgressUpdate(Integer... progress){
        progressBar.setProgress(progress[0]);
        pd.setProgress(progress[0]);
    }

    @Override
    public void onPreExecute(){
        pd.setMessage("Loading Knowledgebase...");
        pd.setProgressStyle(ProgressDialog.STYLE_HORIZONTAL);
        pd.setMax(100);
        pd.show();
    }

    @Override
    public void onPostExecute(Model i){
        progressBar.setVisibility(View.GONE);
        pd.dismiss();
    }
  }

Thank you for your help!

Answer 1

How to show progressbar

The length of the inputstream is same as size of your file in bytes. as you know your file's size, simply convert it to bytes from Mb by multiplying 1024*1024 and you can use this hard-coded value.

Update while loop inside your doInBackground method as follows

int size = 0;
        while ((length = inputStream.read(buffer)) != -1) {
            outputStream.write(buffer, 0, length);
            size += length;
            publishProgress(size*100/your_file_size_in_bytes);
        }

There is no way to accelerate the process if you are dealing with the single file.

However, if you have multiple files you can run multiple threads in parallel using ThreadPoolExecutor.

Answer 2

I ended up finding a very different solution. If anyone encounters a similar issue of uploading a large file for an Android Studio application, simply consider using a back-end server to store the data. That is ultimately the decision that I made because it will take a long time to Download a large file directly into the Application because a mobile device has limited memory. It makes much more sense to run a server from a computer and connecting a client on the phone to retrieve the relevant information at appropriate times. If not, you will ultimately have a file that eats up storage on the mobile device which will make performance considerably worse for your end-users. I ultimately decided to store the RDF file on a computer, upload the file on the computer, run a Java Server Socket in order for a user to connect a client from the application, and then simply query the file on the computer and return the appropriate information to the user. This improved my applications performance and ended the 10 minute download period which is unacceptable for an application.