7

I want to use a QR code scanner within a PWA over Android WebView.

  • The PWA works as intended over the chrome browser.
  • I do have permission for camera set under Apps -> App name -> Permissions
  • minSdkVersion 26 and targetSdkVersion 28
  • Tested on Huawei phone with Android 9

The problem is that the permission seems to be not granted. Additionally, the permission request is repeated multiple times.

Android Code

Manifest

    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-feature
            android:name="android.hardware.camera"
            android:required="true" />

According to answers from previous question I have this code in class WebViewHelper

lateinit var webkitPermissionRequest : PermissionRequest
...
webView.webChromeClient = object : WebChromeClient() {

    override fun onPermissionRequest(request: PermissionRequest) {
        webkitPermissionRequest = request
        val requestedResources = request.resources
        for (r in requestedResources) {
            if (r == PermissionRequest.RESOURCE_VIDEO_CAPTURE) {
                // In this sample, we only accept video capture request.
                val alertDialogBuilder = AlertDialog.Builder(activity)
                    .setTitle("Allow Permission to camera")
                    .setPositiveButton("Allow") { dialog, which ->
                        dialog.dismiss()
                        webkitPermissionRequest.grant(arrayOf(PermissionRequest.RESOURCE_VIDEO_CAPTURE))
                        Log.d(TAG, "Granted")
                    }
                    .setNegativeButton("Deny") { dialog, which ->
                        dialog.dismiss()
                        webkitPermissionRequest.deny()
                        Log.d(TAG, "Denied")
                    }
                val alertDialog = alertDialogBuilder.create()
                alertDialog.show()
                break
            }
        }
    }
    ...
}

Logs: After the permission is granted, it is requested again (multiple times)

D/WebViewHelper: Granted
V/InputMethodManager: Reporting focus gain, without startInput
I/PermissionManager: camera remind result:true
I/CameraManager: open camera: 1, package name: "myApp"
I/BackgroundPermManager: pkgName: "myApp", pid: 31365 ,uidOf3RdApk: 10197 ,permType: 0 ,permCfg: 1
I/HwCameraUtil: notifySurfaceFlingerCameraStatus : isFront = true , isOpend = true
I/HwCameraUtil: notifySurfaceFlingerFrontCameraStatus 8011 transact success!
E/cr_VideoCapture: CameraDevice.StateCallback onOpened
I/WebViewHelper: onPermissionRequest
    onPermissionRequest

This seems to be the problem

I/GRALLOC: LockFlexLayout: baseFormat: 11, yStride: 640, ySize: 307200, uOffset: 307200,  uStride: 640
E/ion: ioctl c0044901 failed with code -1: Invalid argument
I/chromium: "Unhandled rejection", source: "PWA
            "Uncaught (in promise) NotAllowedError: play() can only be initiated by a user gesture.", source: "PWA"

Finally, this error is repeated (indefinitely)

I/GRALLOC: LockFlexLayout: baseFormat: 11, yStride: 640, ySize: 307200, uOffset: 307200,  uStride: 640

JavaScript Code

The code on the JS side works perfectly fine until opened in WebView.

    app.ports.scanQR.subscribe(() => {
        // Delay until page loaded
        setTimeout(function(){
            const video = document.getElementById('media-video');
            function returnResult(result) {
                app.ports.onGotQR.send(result);
                scanner.destroy();
            }
        const scanner = new QrScanner(video, result => returnResult(result));
        scanner.start();
        }, 50);
    });
}

Issue persist if I call getUserMedia directly

if (navigator.mediaDevices.getUserMedia) {
    navigator.mediaDevices.getUserMedia({ video: true })
    .then(function (stream) {
        video.srcObject = stream;
    })
    .catch(function (err0r) {
        console.log("Something went wrong!");
    });
}

Getting the App permission for the camera:

In WebviewHelper.kt

webView.webChromeClient = object : WebChromeClient() {
    override fun onPermissionRequest(request: PermissionRequest) {
        Log.i(TAG, "onPermissionRequest")

        // grants permission for app. video not showing
        if (ContextCompat.checkSelfPermission(activity, Manifest.permission.CAMERA)
            != PackageManager.PERMISSION_GRANTED
        ) {
            Log.i(TAG, "Request Permission")
            requestPermissions(activity, arrayOf(Manifest.permission.CAMERA), 1010)
        } else {
            Log.i(TAG, "Permission already granted")
        }
    ...
}

In MainActivity.kt

override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) {
        when (requestCode) {
            cameraRequestCode -> {
                Log.d("MainActivity", "onRequestPermissionsResult: Camera Request")
                if ((grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED)) {
                    Log.d("MainActivity", "Camera Request: Permission granted")
                    // permission was granted, yay!
                } else {
                    // permission denied, boo!
                    Log.d("MainActivity", "Camera Request: Permission denied")
                }
                return
            }
...

Which result in the following log as expected, when App permission for the camera is granted:

D/MainActivity: onRequestPermissionsResult: Camera Request
    Camera Request: Permission granted
Noah Studach
  • 405
  • 4
  • 14
  • "play() can only be initiated by a user gesture" this is for sure in web control (chrome) error. It is ok for your permission in android env. You must use touch(click/user request) to initial/run getUserMedia method! You are on good way. – Nikola Lukic Apr 25 '19 at 12:41
  • Am I not currently doing this with the `alertDialog`? If not how can I get the user gesture? – Noah Studach Apr 25 '19 at 17:36
  • You are right for android grand permission. Any way try to make html button and js code for getUserMedia. When you use implementation of browser/chrome this control has still same behavior like real chrome. Even worst is try to access camera on onload event. I found "Asking for access to the camera on page load will result in most of your users rejecting access to it.". – Nikola Lukic Apr 25 '19 at 21:52
  • First try with "onRequestPermissionsResult(int, java.lang.String[], int[]) " to see what happen in native android. – Nikola Lukic Apr 25 '19 at 22:07
  • I have added code to onRequestPermissionsResult (see original question). As far as I understand, the permission in the scope of the App is different than the permission granted for an explicit website, which is what I am trying to achieve here. – Noah Studach Apr 26 '19 at 14:53
  • Did you solve this? – user257980 Jun 26 '19 at 12:51
  • @user257980 I think I found a [solution](https://stackoverflow.com/a/56790876/10216108), but right now there is no need for me to test it. – Noah Studach Jun 27 '19 at 12:23

3 Answers3

2

All of your original code is correct inside of the onPermissionRequest(...). The key line of code you're missing was webView.settings.mediaPlaybackRequiresUserGesture = false.

dannyskim
  • 165
  • 2
  • 8
0

I can make it work on stackoverflow because cross origin role. Try this, must work.

const captureVideoButton = document.querySelector('#capture-button');
const screenshotButton = document.querySelector('#screenshot-button');
const img = document.querySelector('#image');
const video = document.querySelector('#video');
const canvas = document.querySelector('#canvas');
const constraints = {
  audio: true,
  video: true
};

captureVideoButton.onclick = function() {
  navigator.mediaDevices.getUserMedia(constraints).
    then(handleSuccess).catch(handleError);
};

screenshotButton.onclick = video.onclick = function() {
  canvas.width = video.videoWidth;
  canvas.height = video.videoHeight;
  canvas.getContext('2d').drawImage(video, 0, 0);
  // Other browsers will fall back to image/png
  img.src = canvas.toDataURL('image/webp');
};

function handleError(error) {
  console.error('Error: ', error);
}

function handleSuccess(stream) {
  screenshotButton.disabled = false;
  video.srcObject = stream;
}
<meta http-equiv="Content-Security-Policy" content="default-src * gap:; script-src * 'unsafe-inline' 'unsafe-eval'; connect-src *; img-src * data: blob: android-webview-video-poster:; style-src * 'unsafe-inline';">

<video id="video" autoplay crossorigin="anonymous"></video>
<img id="image" src="" crossorigin="anonymous">
<canvas id="canvas" style="display:none;"></canvas>
<button id="capture-button">Capture video</button>
<button id="screenshot-button">Capture video</button>

Try to add meta http-equiv="Content-Security-Policy" tag also

Nikola Lukic
  • 4,001
  • 6
  • 44
  • 75
  • First of all, thank you for your support. I have set up your web page and it works when using the browser, but not in WebView. This is the log I get in Android Studio: `D/WebViewHelper: Granted V/InputMethodManager: Reporting focus gain, without startInput I/chromium: [INFO:CONSOLE(40)] "Error: "` Console Log: `Access to image at 'android-webview-video-poster:default_video_poster/-3437894000281861935' from origin 'site' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.` – Noah Studach Apr 28 '19 at 09:38
0

I worked around this issue using a native QR Scanner module.

However, I eventually needed also access to Geolocation. So I found this solution: WebViewActivity with permission request

I only needed the following parts for requesting permissions:

  • fuckMarshMallow
  • onGeolocationPermissionsShowPrompt
  • onRequestPermissionsResult

Additionally, you need some of the prompt functions and variables and a manifest permission entry.

Since this works with Geolocation I assume it should also work for Camera if adapted accordingly.

Kotlin Version (shortend):


// in webview settings add
webSettings.apply {
            setGeolocationEnabled(true)
        }

webView.webChromeClient = object : WebChromeClient() {

            override fun onGeolocationPermissionsShowPrompt(
                origin: String,
                callback: GeolocationPermissions.Callback
            ) {
                // Always grant permission since the app itself requires location
                // permission and the user has therefore already granted it
                callback.invoke(origin, true, false)
            }
}

override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) {
        when (requestCode) {
            REQUEST_CODE_ASK_MULTIPLE_PERMISSIONS -> {
                val perms = HashMap<String, Int>()
                // Initial
                perms[Manifest.permission.ACCESS_FINE_LOCATION] = PackageManager.PERMISSION_GRANTED

                // Fill with results
                for (i in permissions.indices)
                    perms[permissions[i]] = grantResults[i]

                // Check for ACCESS_FINE_LOCATION
                if (perms[Manifest.permission.ACCESS_FINE_LOCATION] == PackageManager.PERMISSION_GRANTED) {
                    // All Permissions Granted
                } else {
                    // Permission Denied
                    finish()
                }
            }
            else -> super.onRequestPermissionsResult(requestCode, permissions, grantResults)
        }
    }

@TargetApi(Build.VERSION_CODES.M)
internal fun fuckMarshMallow() {
        val permissionsNeeded = ArrayList<String>()

        val permissionsList = ArrayList<String>()
        if (!addPermission(permissionsList, Manifest.permission.ACCESS_FINE_LOCATION))
            permissionsNeeded.add("Show Location")

        if (permissionsList.size > 0) {
            if (permissionsNeeded.size > 0) {

                // Need Rationale
                var message = "App need access to " + permissionsNeeded[0]

                for (i in 1 until permissionsNeeded.size)
                    message = message + ", " + permissionsNeeded[i]

                showMessageOKCancel(message,
                    DialogInterface.OnClickListener { _, _ ->
                        requestPermissions(
                            permissionsList.toTypedArray(),
                            REQUEST_CODE_ASK_MULTIPLE_PERMISSIONS
                        )
                    })
                return
            }
            requestPermissions(
                permissionsList.toTypedArray(),
                REQUEST_CODE_ASK_MULTIPLE_PERMISSIONS
            )
            return
        }
    }

private fun showMessageOKCancel(message: String, okListener: DialogInterface.OnClickListener) {
        AlertDialog.Builder(this@WebViewActivity)
            .setMessage(message)
            .setPositiveButton("OK", okListener)
            .setNegativeButton("Cancel", null)
            .create()
            .show()
    }

@TargetApi(Build.VERSION_CODES.M)
private fun addPermission(permissionsList: MutableList<String>, permission: String): Boolean {

        if (checkSelfPermission(permission) != PackageManager.PERMISSION_GRANTED) {
            permissionsList.add(permission)
            // Check for Rationale Option
            if (!shouldShowRequestPermissionRationale(permission))
                return false
        }
        return true
    }

Dont forget manifest:

 <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
Noah Studach
  • 405
  • 4
  • 14