2

I am trying to transfer some files in Android.

I am having the list of files and then I transfer them to the desired location.

The type of file may be anything Images, Videos, Audios, Gifs, PowerPoint, Word etc.

The size of the list of files is around 100.

This is the code which I am using to transfer files

public static boolean copy(File copy, String directory, Context con) {
   static FileInputStream inStream = null;
   static OutputStream outStream = null;
   DocumentFile dir = getDocumentFileIfAllowedToWrite(new File(directory), con);
   String mime = "";
   DocumentFile copy1 = dir.createFile(mime, copy.getName());

   try {
      inStream = new FileInputStream(copy);
      outStream = con.getContentResolver().openOutputStream(copy1.getUri());
      byte[] buffer = new byte[16384];
      int bytesRead;
      while ((bytesRead = inStream.read(buffer)) != -1) {
         outStream.write(buffer, 0, bytesRead);
      }
   }
   catch (FileNotFoundException e) {
      e.printStackTrace();
   } catch (IOException e) {
      e.printStackTrace();
   } finally {
      try {
         inStream.close();
         outStream.close();
         return true;
      } catch (IOException e) {
         e.printStackTrace();
      }
   }
   return false;
}

Can any one please explain the reason of file being corrupted after transferring and the precautions which could be taken.

EDIT: This is how I use the function

String from = "/storage/emulated/0/Pictures/IMG.jpg";
String to = "/storage/3096-13FF/Pictures";

Class.copy(from, to, this);

EDIT 2: Original file size is around 1-2 MB on an average. Copied file size = Unknown as this issue has been raised by an user.

This is the method which return a DocumentFile by inputting an path.

 public static DocumentFile getDocumentFileIfAllowedToWrite(File file, Context con) {

List<UriPermission> permissionUris = con.getContentResolver().getPersistedUriPermissions();

for (UriPermission permissionUri : permissionUris) {

    Uri treeUri = permissionUri.getUri();
    DocumentFile rootDocFile = DocumentFile.fromTreeUri(con, treeUri);
    String rootDocFilePath = "SD CARD PATH";
    if (file.getAbsolutePath().startsWith(rootDocFilePath)) {

        ArrayList<String> pathInRootDocParts = new ArrayList<String>();
        while (!rootDocFilePath.equals(file.getAbsolutePath())) {
            pathInRootDocParts.add(file.getName());
            file = file.getParentFile();
        }

        DocumentFile docFile = null;

        if (pathInRootDocParts.size() == 0) {
            docFile = DocumentFile.fromTreeUri(con, rootDocFile.getUri());
        } else {
            for (int i = pathInRootDocParts.size() - 1; i >= 0; i--) {
                if (docFile == null) {
                    docFile = rootDocFile.findFile(pathInRootDocParts.get(i));
                } else {
                    docFile = docFile.findFile(pathInRootDocParts.get(i));
                }
            }
        }
        if (docFile != null && docFile.canWrite()) {
            return docFile;
        } else {
            return null;
        }

    }
}
return null;
 }

I am making the user choose the SD Card Directory using the storage access framework so that I can write to any directory in the SD Card. And then when ever the file has to be transferred I am making the user choose the path using an custom dialog box.

Rajesh K
  • 683
  • 2
  • 9
  • 35
  • Why don't you use `Files.copy(source, target, REPLACE_EXISTING);`? – sarkasronie Mar 16 '18 at 11:56
  • @sarkasronie The reason for not using this method is The app might not only transfer data to the same storage but also to Removable SD Card. And from Android 5.0 and above Storage Access Framework has to be used to transfer data to Removable Storage. – Rajesh K Mar 16 '18 at 12:04
  • `public static boolean copy(File copy, String directory,` Please start with an example where you call this function. We wanna see all full paths. Put the example in your post please. – greenapps Mar 16 '18 at 12:55
  • @greenapps Sorry. I have added the info. Can you please have a look. – Rajesh K Mar 16 '18 at 13:30
  • Not doing much with Android, but `mime="application/octet-stream";` (binary data) I would have expected. – Joop Eggen Mar 16 '18 at 13:40
  • Thank You @JoopEggen for the suggestion. Can you please clarify whether generally this mime type can also be used for copying an Image/Video or any other file? – Rajesh K Mar 16 '18 at 14:02
  • It is for binary data, and so no change happens. Other than `text/plain` that transferred by FTP could have swap line endings between Unix `\n` and Windows `\r\n` format. – Joop Eggen Mar 16 '18 at 14:16
  • You said the files are corrupted. Please start telling the file size of original and copy. Every byte counts. – greenapps Mar 16 '18 at 15:04
  • `getDocumentFileIfAllowedToWrite()` ?? Dont know that function. Pease post code. I wonder how you ever could get a DocumentFile for `"/storage/3096-13FF/Pictures";` It looks like a very bad approach. – greenapps Mar 16 '18 at 15:05
  • I always use `"*/*"` for mime there. – greenapps Mar 16 '18 at 15:06
  • `"/storage/3096-13FF/Pictures";` How did you create the Pictures dir on the SD card? How did you let the user choose that directory? You are not using SAF in the right way. I think you made it very complicated. All can be done much easier. – greenapps Mar 16 '18 at 15:11
  • @greenapps I have edited the question and added the necessary info. Can you please have a look. I am not aware of the file original file size and copied file size as this issue is complained by the user. I have added the `getDocumentFileIfAllowedToWrite()` method. Using `"*/*"` sounds good. But can you clearify that will that work fine with Image, Video etc. Either the user would have created `picture` directory using my app or it would have been present there before hand. – Rajesh K Mar 16 '18 at 16:49
  • @JoopEggen Thank You very much for the suggestions. But I am not very much knowledged about coding hence your info seems to bounce over my head. – Rajesh K Mar 16 '18 at 16:51
  • `I am making the user choose the SD Card Directory using the storage access framework so that I can write to any directory in the SD Card.` Very good. `And then when ever the file has to be transferred I am making the user choose the path `. I wonder what the user can choose. A subdirectory like `Pictures`? Or what happens? For the rest you should get rid of that function getDocumentFileIfAllowedToWrite(). – greenapps Mar 16 '18 at 17:14
  • `. I am not aware of the file original file size and copied file size as this issue is complained by the user.`. Well ask the user. File sizes is the first thing you should check. – greenapps Mar 16 '18 at 17:18
  • You use a terrible function. You are not using the saf rigth. But then it would not be the reason that the copy would fail. Is this only one user? You cannot reproduce this yourself? All unclear. – greenapps Mar 16 '18 at 17:20
  • @greenapps Can you pls suggest a way to get rid of `getDocumentFileIfAllowedToWrite()`. As you are already aware of starting from Android 5.0 and above SAF has to be used to transfer files. Will surely ask the user about the file size info and update you soon. – Rajesh K Mar 16 '18 at 17:24
  • @greenapps No. This users is not the first user I have two or three more test users complaining about this. No I cannot reproduce this error. – Rajesh K Mar 16 '18 at 17:26
  • You have obtained a content scheme of the root tree of the sd card when you let the user choose it with ACTION_OPEN_DOCUMENT_TREE? And in onActivityResult you obtained an Uri uri = data.getData()? And you saved the value of uri.toString() so you could use it later? – greenapps Mar 16 '18 at 17:27
  • Wel ask those users. Phone them. App them. Mail them! – greenapps Mar 16 '18 at 17:27
  • You informed us wrong. You presented your problem in such a way that we assumed the files were always corrupted. Pretty strange you did not directly tell you could not reproduce it yourself. Then how would we? – greenapps Mar 16 '18 at 17:30
  • @greenapps Extremely Sorry. Never meant to do so. Will surely try to contact the user and ask for the info. – Rajesh K Mar 16 '18 at 17:39
  • @greenapps Tried contacting the user. But only one use was knowing the info. He told me that about more than 90% file had been transferred. Suppose the original file size is 100 MB then the copied file size was more than 90 MB. But he wasn't able to tell the exact size. – Rajesh K Mar 22 '18 at 13:36

1 Answers1

4

Data is buffered by OtputStream before actual write, and not directly writen to file. And some of data might still remain in this buffer at time you close the file. This is optimisation that improves performance of small writes, it removes lot of latency caused by slow persistent storage device, like hard disk, SD card ....

You need to flush output stream buffer, before closing the file. I think that all data didn’t sync to file. So before closing output, try to flush buffer.

outStream.flush();

And to make sure data is writen to storage device (SD card) from file system cache, (if not implicit done by close() )

outStream.getFD().sync();

By defult, file system writes and reads are cached by operating system. This should not affect you except you remove battery at exact moment after copy()

More about is already explaeined here

I/O concept flush vs sync

user9608780
  • 106
  • 2
  • Thanks for the Answer. But this issue with almost the same code as posted by the Original poster copies the files corrupted very rarely. Is there any reason for that? – Rahulrr2602 Apr 06 '18 at 17:22
  • Thanks @Rahulrr2602 for offering that Bounty. – Rajesh K Apr 06 '18 at 17:29
  • What do you exactly mean by `data is commited to storage from file system cache`? – Rajesh K Apr 06 '18 at 17:30
  • 1
    I updated answer, hope it is better explanation now. Thanks for feedback. – user9608780 Apr 06 '18 at 17:50
  • @user9608780 Thanks. In my case I am writing from one Storage to another, suppose from Internal Storage to Removable SD Card. In my case do I need to use `getFD().sync();`. Sorry for the doubt but I am pretty much new to Android development. – Rajesh K Apr 06 '18 at 18:29
  • 1
    Rajesh K , no you don’t have to, only flush is is required. Ther are use cases when you want to force file system sync, but i don’t think this is such case. Also this is not just specific to java implementation of output/input stream, its more general practice. – user9608780 Apr 06 '18 at 19:16
  • 1
    Rajesh K, how ever when I think again, it should be required. Because you are doing copy to removable SD card. So imagine scenario when your application indicates that copy is completed and user removes the card immediatly after. Simple because one might think copy is completed, sync should save you here. This is like windows “safe remove of removable drive”. But what it does is that iz makes sure tha file system cache is synchronised. – user9608780 Apr 06 '18 at 19:23
  • @user9608780 Thank You for the excellent explanation and answer. Awarding the Bounty. Even though I cannot retrace the issue but surely believe that this will solve the issue. Thanks once again. – Rahulrr2602 Apr 07 '18 at 09:25
  • @user9608780 It says that I can award the Bounty only after 7 hours. Will do it after 7 hours. – Rahulrr2602 Apr 07 '18 at 09:26
  • @user9608780 Thanks. But will there be any side effect of using `getFD().sync()` . Also is there any other precaution which has to be taken to avoid the corruption of data. – Rajesh K Apr 07 '18 at 09:51
  • Rajesh K, all I know is that after this it is usualy below application level. It is more filesystem responsibility. If I would go further, I would probably make checksum of original data and verify it against copy. That mean program would have to read a copy again amd compare. This would only help me to detect issue with sd card or something else, not to correct it. – user9608780 Apr 07 '18 at 19:56
  • @user9608780 Thanks for all the help. – Rajesh K Apr 08 '18 at 08:50
  • @user9608780 I was just reviewing this, will it be fine if I compare the length of the original file and the new file before deleting the original file so that even if the new file is corrupted I will not delete the original file. Please help. – Rajesh K May 23 '18 at 22:18
  • @RajeshK Yeah, but thats not 100%, maybe you can check this thread and similar for how to do this things : https://stackoverflow.com/questions/304268/getting-a-files-md5-checksum-in-java – user9608780 Jul 03 '18 at 06:30
  • Thanks very much for the answer. I have seen many threads online complaining about ES File browser Deleting their files instead of moving. So I assume there is not way to efficiently guarantee 100%. As suggested in the link I can do that but will that take time to build MD5? Please help. – Rajesh K Jul 03 '18 at 17:22