Google Keyboard - insert image "send" intent weirdness

Question

I'm investigating adding Gboard image support into our app.

I have three specific questions / issues below.

Have seen the official docs at https://developer.android.com/guide/topics/text/image-keyboard.html

There is something weird going on though.

When the user taps on an EditText, the editor sends a list of MIME content types that it accepts in EditorInfo.contentMimeTypes.

The IME reads the list of supported types and displays content in the soft keyboard that the editor can accept.

I'm reading this to also mean that if an app does not set EditorInfo.contentMimeType, then the keyboard will not enable the UI for inserting an image.

That's not how it works in practice though.

First, what I'm seeing is that GBoard shows the insert image UI for almost any EditText, presumably based on its inputType.

I tried making an EditText subclass, overriding onCreateInputConnection and making sure that EditorInfo.contentMimeType -> the insert image UI does show anyway.

There are dozens of EditText input fields in our app (probably more) on different screens. Inserting animated GIF's does not make sense for the vast majority of them.

Question #1 - how can this be stopped (if it can be)?

By the way this is easy to check in almost any app, almost any input field. For example, in Gmail app in the search bar. Or in Google Chrome in its URL input bar.

This is harmless for most apps - if you select a GIF in an input field that doesn't support image input (such as the examples above, Gmail and Chrome), GBoard shows a "toast-like" message that "this text field doesn't support GIF insertion from the keyboard".

OK, but:

As fallback, when GBoard cannot send the selected GIF through InputConneciton -> it then tries to launch an ACTION_SEND intent restricted to the app's package with an URL that represents the image.

Our app (an email app) does have an intent filter in its manifest for ACTION_SEND to enable users to share "stuff" with an email message. Things like Gallery images, files from a file manager, anything really.

And so, GBoard ends up launching our "email compose screen" with ACTION_SEND and the image's URL.

This results in two more issues:

First, it's confusing for the user

Let's say he/she tries to insert a GIF into "some" EditText in the app.

He/she will then be taken to the app's "message compose" screen because of ACTION_SEND - "whoa, what happened".

Question #2 - how can this be stopped (if it can be)?

Second, the URI for the image cannot be opened with any method that I'm aware of.

I've tried:

cr.openFileDescriptor(uri, "r");
cr.openAssetFileDescriptor(uri, "r");
cr.openInputStream(uri);
cr.query(uri, null, null, null, null);

All of these fail with variations of "invalid URI" and so on.

I do understand that by implementing an InputConnectionWrapper for the one or a few input fields where we want to support GIFs we can request the permission directly from inputContentInfo as explained in the docs.

I'm talking about a different case - I'll reiterate - when the user tries to insert a GIF into "some other" EditText in our app and GBoard launches our "message compose" activity with the GIF's URI and ACTION_SEND.

Currently our code tries to handle this URI the same way as any other (such as when the user shares a photo from Google Photos into our app) - but opening this URI fails, so the user not only ends up quite unexpectedly on the compose screen, but then also gets an error that we were not able to open / copy the "attachment".

Question #3 - can GBoard image URI's sent with ACTION_SEND be opened at all using "standard" ContentResolver methods?

April 12, update on "Question #3" (failures to open the stream).

Our app uses sharedUserId in its manifest.

Removing sharedUserId and making no other changes made it so that those animated image URI's now open just fine with cr.openInputStream, both in input connection callback and in ACTION_SEND.

While there may be reasons to not use sharedUserId, our app does since 2012 or so, and removing it for just this feature isn't possible (as that would prevent updates).

The URI's look like this, with the app's package name encoded in a parameter:

content://com.google.android.inputmethod.latin.inputcontent/inputContent?fileName=%2Fdata%2Fuser_de%2F0%2Fcom.google.android.inputmethod.latin%2Ffiles%2Fgif20152912710254894520&packageName=org.kman.AquaMail&mimeType=image%2Fgif

So question #3 is now replaced by

Question #4 - how can we report this bug (sharedUserId makes it so that image URIs can't be opened) to Google?

Nope. Don't know if there is a way to report this to Google or to draw their attention to this post. And so here it is with no new info. — Kostya Vasilyev, Aug 22 '18 at 18:06

Pedro Oliveira · Answer 1 · 2019-04-05T17:48:31.213

Like you said, Gboard is sending a SEND intent when the view doesn't accept Gifs. The thing is, the same is happening for other keyboards I tested so it doesn't seem to be a bug with Gboard.

So the solution I came up with, was to ignore SEND intents from the current keyboard.

When I receive an intent in my receiver activity, I check if the data came from the keyboard and for that I use the following method which might be able to help you too:

private boolean isClipDataAuthorityValid(@NonNull ClipData clipData) {
    if (clipData.getItemCount() == 0) {
        return true;
    }

    Uri uri = clipData.getItemAt(0).getUri();

    if (uri == null) {
        return true;
    }

    String authority = uri.getAuthority();
    if (TextUtils.isEmpty(authority)) {
        return true;
    }

    String defaultInputMethod = Settings.Secure.getString(getContentResolver(), "default_input_method");
    if (TextUtils.isEmpty(defaultInputMethod)) {
        return true;
    }

    String keyboardPackage = defaultInputMethod.split("/")[0];
    try {
        ProviderInfo[] providers = getPackageManager().getPackageInfo(keyboardPackage, PackageManager.GET_PROVIDERS).providers;
        if (providers == null || providers.length == 0) {
            return true;
        }

        //Check if the authority of the given clipdata's uri matches any of the keyboards's provider authority
        for (ProviderInfo provider : providers) {
            if (TextUtils.equals(authority, provider.authority)) {
                return false;
            }
        }

    } catch (PackageManager.NameNotFoundException e) {
        //Do nothing
    }

    return true;
}

The method receives a clipdata which can be easily extracted from the intent using intent.getClipData(). It fetches the current keyboard, check the providers for it, matches their authority with the authority of the Uri of the clipdata and returns false if the clipdata is from the keyboard.

Currently, I was unable to find another generic solution as the keyboards I tested displayed different results.

I hope this helps :)

Google Keyboard - insert image "send" intent weirdness

1 Answers1