How to handle SAF when I can only handle File or file-path?

Question

Background

Up until Android Q, if we wanted to get information about an APK file, we could use WRITE_EXTERNAL_STORAGE and READ_EXTERNAL_STORAGE to get access to the storage, and then use PackageManager.getPackageArchiveInfo function on the file-path.

Similar cases exist, such as using ZipFile class on a compressed file, and probably countless framework APIs and third party libraries.

The problem

Google announced a huge amount of restrictions recently on Android Q.

One of them is called Scoped Storage, which ruins storage permission when it comes to accessing all files the device has. It lets you either handle media files, or use the very restricted Storage-Access-Framework (SAF) which can't allow apps to reach and use files using File API and file-paths.

When Android Q Beta 2 was published, it broke a lot of apps because of it, including of Google. The reason was that it was turned on by default, affecting all apps, whether they target Android Q or not.

The reason is that many apps, SDKs and Android framework itself - all use File API quite often. On many cases, they also don't support InputStream or SAF-related solutions. An example for this is exactly the APK parsing example I wrote about (PackageManager.getPackageArchiveInfo).

On Q beta 3, however, things changed a bit, so that app that target Q will have the scoped storage, and there is a flag to disable it and still use the normal storage permissions and File API as usual. Sadly the flag is only temporary (read here), so it's delaying the inevitable .

What I've tried

I've tried and found the next things:

Using the storage permission indeed didn't let me read any file that's not media file (I wanted to find APK files). It's as if the files don't exist.
Using SAF, I could find the APK file, and with some workaround to find its real path (link here), I've noticed that File API can tell me that indeed the file exist, but it couldn't get its size, and the framework failed to use its path using getPackageArchiveInfo . Wrote about this here
I tried to make a symlink to the file (link here), and then read from the symlink. It didn't help.
For the case of parsing APK files, I tried to search for alternative solutions. I've found 2 github repositories that handle the APK using a File class (here and here), and one that uses InputStream instead ( here). Sadly the one that uses InputStream is very old, missing various features (such as getting the app's name and icon) and isn't going to be updated anytime soon. Besides, having a library requires maintenance to keep up with future versions of Android, otherwise it might have issues in the future, or even crash.

The questions

Generally, is there a way to still use File API when using SAF ? I'm not talking about root solutions or just copying the file to somewhere else. I'm talking about a more solid solution.
For the case of APK parsing, is there a way to overcome this issue that the framework only provides file-path as a parameter? Any workaround or a way to use InputStream perhaps?

"Generally, is there a way to still use File API when using SAF ?" -- no, because there is no requirement that the `Uri` points to a plain file on the filesystem, let alone one that you can access. For example, the `DocumentsProvider` might need to decrypt the file on the fly, as seen in [this sample](https://android.googlesource.com/platform/development/+/android-5.0.0_r2/samples/Vault/src/com/example/android/vault/VaultProvider.java) that [you pointed to 4.5 years ago](https://stackoverflow.com/q/26744842/115145). All you can do is copy the content to a file that you control. — CommonsWare, May 18 '19 at 13:57
@CommonsWare I don't remember the first link you've provided and don't know what it's about, but the second link I remember. Thing is that it was about SD-card, and now this extends to the normal storage too. But even for SD-card, it was still possible to read from it using File API (using the workarounds I've mentioned), and I could parse APK files fine. I don't know if it would have worked for encrypted storage, but if it didn't, at least I could try. Isn't there any workaround you can think of? Copying files for this is bad... — android developer, May 18 '19 at 16:54
"I don't remember the first link you've provided and don't know what it's about" -- you linked to it from your 4.5-year-old question. It is an example of a `DocumentsProvider`. Any independently-written `DocumentsProvider` would break your `getFullPathFromTreeUri()` method, as you blindly assume that the `Uri` is from a specific `DocumentsProvider` (one that ideally would not support your hack and could conceivably stop supporting it in some future Android version). — CommonsWare, May 18 '19 at 17:09
"Isn't there any workaround you can think of?" -- by definition, there can't be a reliable workaround. **There is no requirement for a `Uri` to point to a file on the filesystem**, let alone one that you can access directly by any means. The user could use SAF to pick a location on Google Drive, or the Vault sample that I linked to, or a SMB/CIFS network share, or any other `DocumentsProvider` that they have access to. The `Uri` might point to a location on a server, or in a database `BLOB` column, or whatever else the `DocumentsProvider` author wants. — CommonsWare, May 18 '19 at 17:11
@CommonsWare I have Google Drive, and when I use the picker to choose a folder, it didn't offer it to me. How come? And, what if I assume the user has chosen a real path from the real file system? Would it still work? The problem here is not the assumption (because in my test I do choose the path from the file system), but the usage itself. The API that I have needs file-path or File API. Does Google really intends to break so many apps ? What can I do to scan for APK files in entire file system? — android developer, May 18 '19 at 18:08
"How come?" -- I don't know. I do not use Google Drive personally, but it is the pre-eminent example of an independent `DocumentsProvider`. "Would it still work?" -- I don't know what "it" is. "Does Google really intends to break so many apps ?" -- Google broke them five years ago. Rather than working on alternative solutions for your problems (e.g., an APK parsing library that does not depend on files), you and others went with hacks. Had you used those hacks as a short-term stopgap and invested effort in eliminating the need for them, you would be in better shape today. — CommonsWare, May 18 '19 at 18:19
"What can I do to scan for APK files in entire file system?" -- I don't think that there is an option for that. There *should* be an option for that. You have complained loudly about SAF's shortcomings, and I agree with some. One is that there should be a "give me a list of all APKs" (or PDFs, for a more conventional scenario), and I don't think there's an option for that at the moment. — CommonsWare, May 18 '19 at 18:23
Well, Google said it's ok to use the storage permission, and that SD-card can still be read using it, so all should have stayed. Also, some of the functions aren't quite hacks, but more like workarounds, but ok... About "Would it still work?", I mean getting the file content correctly from SAF using other means. So far it worked fine using File API, but now it doesn't seem it's possible to even reach the file using File API. The SAF API is lacking in so many ways. It got a way to move a file only on API 24 (`moveDocument`). Till then you had to copy&delete. And that's just one example. — android developer, May 18 '19 at 19:26
Apps can't just adjust to these changes. Many libraries (especially in C/C++) and even the framework aren't ready for SAF. Not always we have a function that has InputStream as a parameter. Sometimes all we have is File or file-path. And a modern OS should have files API, to access all files as they are, and File API is a very known standard that shouldn't be deprecated without a sufficient replacement that has backward compatibility in mind. Not all files on the file system are media files. Apps should be able to reach all files, if the user allows it. — android developer, May 18 '19 at 19:29
And it will probably stay this way, having libraries that can't handle what SAF has to offer, making copying files the only workaround to be able to use them, which is a huge disadvantage.Even the docs say that it's inefficient to use SAF: ""Representation of a document backed by either a DocumentsProvider or a raw file on disk. This is a utility class designed to emulate the traditional File interface. It offers a simplified view of a tree of documents, but it has substantial overhead. For optimal performance and a richer feature set, use the DocumentsContract methods and constants directly" — android developer, May 18 '19 at 20:05

score 1 · Accepted Answer · answered Oct 12 '19 at 12:48

1

How to handle SAF when I can only handle File or file-path? It is possible, even if you can send only a Java File object, or path string to a library function which you cannot modify:

First, obtain a Uri to a file you need to handle (in String form it would be like "content://..."), then:

    try {
        ParcelFileDescriptor parcelFileDescriptor =
                getContentResolver().openFileDescriptor(uri, "r"); // may get FileNotFoundException here
        // Obtain file descriptor:
        int fd = parcelFileDescriptor.getFd(); // or detachFd() if we want to close file in native code
        String linkFileName = "/proc/self/fd/" + fd;
        // Call library function with path/file string:
        someFunc(/*file name*/ linkFileName);
        // or with File parameter
        otherFunc(new File(linkFileName));
        // Finally, if you did not call detachFd() to obtain the file descriptor, call:
        parcelFileDescriptor.close();
        // Otherwise your library function should close file/stream...
    } catch (FileNotFoundException fnf) {
        fnf.printStackTrace(); // or whatever
    }

answered Oct 12 '19 at 12:48

gregko

5,642
9
49
76

Sadly I've found this solution too and wrote it somewhere else, and even though it seems to work, it doesn't always work. For example, you can use getPackageArchiveInfo on the file, but when you try to get its icon or label, you will fail. Please correct me if I'm wrong on this, and if you have a workaround for this or at least an explanation, please let me know this too. I've upvoted your answer just because you are trying to help and did it fine. – android developer Oct 12 '19 at 17:06
By the label and icon, do you mean app package label and icon? Can't you get them from PackageManager, e.g. getApplicationIcon(String packageName) and similar for the label? No file is needed there, just package name... – gregko Oct 12 '19 at 23:08
I just tried these calls - pm.getApplicationLabel() and pm.getApplicationIcon() on Android 10, legacy storage access not enabled, work fine... – gregko Oct 12 '19 at 23:22
`getApplicationIcon` returns me the default icon so I didn't use it and using `ResourcesCompat.getDrawable` causes an exception. As for app-name, I always get something weird: package name with some string , but never the app name. Tested on the APK of F-Droid. If somehow you managed to make those work, can you please share your code for it, maybe put a full sample here: https://stackoverflow.com/q/56309165/878126 . I will even set a new bounty. As I wrote, I actually found the solution you wrote (someone else wrote me this somewhere), but app-icon and app-label are those that never work. – android developer Oct 13 '19 at 19:55
I'm sorry, I don't have solutions beyond what I wrote so far, package management is not my area of interest... – gregko Oct 14 '19 at 01:16
But you said it works for you. Does it work or not? – android developer Oct 14 '19 at 08:59
What worked fine for me, at least on two apps that I tried, was calling packageManager.getApplicationIcon(packageName), and packageManager.getApplicationLabel(appInfo). – gregko Oct 14 '19 at 21:28
If you'd really like to use packageManager.getPackageArchiveInfo (String archiveFilePath, int flags); I really don't see how to do this, unless you had an APK file copied somewhere on user visible internal storage or SD card, and asked the user for permission to read it with Scoped Storage...But even in earlier Android versions, could you really access APK files in /data/app directory, without rooting? – gregko Oct 14 '19 at 21:37
APK files can exist on the storage that the user sees. The user can copy APK files via USB or download them. That's why I use getPackageArchiveInfo, as it's the only one that can get information about them. Before Android Q I just used storage permission and it all worked fine. As for "/data/app", of course this folder's content of APK files is publicly available for all apps (without root), but that's not what I'm asking about. I just take it as an easy way to check if it works. Not everyone have APK file on the downloads folder, but all have APK files of installed apps. – android developer Oct 15 '19 at 05:28
OK, so I called packageManager.getPackageArchiveInfo (linkFileName, 0); on Android 10, no legacy storage access, on an APK file copied to Download directory. I got PackageInfo object fine. What exactly do you need to do with this PackageInfo that does not work? I could try your exact code to see what happens. – gregko Oct 15 '19 at 12:17
Already said: icon and app name. For icon, I somehow get either an exception or the default one. For app name (label), I somehow get a weird combination of package name and something else. If you have a solution, please write it on the other thread. Here you've already answered and even though I got this solution at some point, it's unfair not to accept your answer here because I didn't answer it myself. So you get V now. :) – android developer Oct 15 '19 at 13:16
I see that there is a difference if you do getPackageArchiveInfo() of an installed app, with its package name, and if you get the same info by calling getPackageArchiveInfo() giving it a file name to an APK file. Calling then packageManager.getApplicationLabel(appInfo) on both types of applicationInfo object on Android 10 gives me different results. Probably the API call getApplicationLabel(appInfo) cannot open the APK file, no permission. You could however open the APK as a ZIP file and read values from it (e.g. icon resource, app name from string tables etc.) – gregko Oct 15 '19 at 15:00
Of course there is a difference. There is a full access to the installed app , using the normal File API. And outside it's using the terrible SAF with workarounds everywhere... :( – android developer Oct 15 '19 at 16:39

score 0 · Answer 2 · answered Oct 15 '19 at 15:29

0

Posting another answer just to have more room and let me insert code snipes. Given the file descriptor as explained in my previous answer, I tried using net.dongliu:apk-parser package mentioned by @androiddeveloper in the original question, as follows (Lt.d is my shorthand to using Log.d(SOME_TAG, string...)):

            String linkFileName = "/proc/self/fd/" + fd;
            try (ApkFile apkFile = new ApkFile(new File(linkFileName))) {
                ApkMeta apkMeta = apkFile.getApkMeta();
                Lt.d("ApkFile Label: ", apkMeta.getLabel());
                Lt.d("ApkFile pkg name: ", apkMeta.getPackageName());
                Lt.d("ApkFile version code: ", apkMeta.getVersionCode());
                String iconStr = apkMeta.getIcon();
                Lt.d("ApkFile icon str: ", iconStr);
                for (UseFeature feature : apkMeta.getUsesFeatures()) {
                    Lt.d(feature.getName());
                }
            }
            catch (Exception ex) {
                Lt.e("Exception in ApkFile code: ", ex);
                ex.printStackTrace();
            }
        }

It gives me the correct app label, for the icon it gives me only a string to the resource directory (like "res/drawable-mdpi-v4/fex.png"), so again raw ZIP reading functions would have to be applied to read the actual icon bits. Specifically I was testing ES File Explorer Pro APK (bought this product and saved APK for my own backup, got the following output:

 I/StorageTest: ApkFile Label: ES File Explorer Pro
 I/StorageTest: ApkFile pkg name: com.estrongs.android.pop.pro
 I/StorageTest: ApkFile version code: 1010
 I/StorageTest: ApkFile icon str: res/drawable-mdpi-v4/fex.png
 I/StorageTest: android.hardware.bluetooth
 I/StorageTest: android.hardware.touchscreen
 I/StorageTest: android.hardware.wifi
 I/StorageTest: android.software.leanback
 I/StorageTest: android.hardware.screen.portrait

answered Oct 15 '19 at 15:29