How to select proper backfacing camera in Javascript?

Question

I am using navigator.mediaDevices.getUserMedia to open MediaStream from camera device within a web browser. My app wants to do some realtime image processing in WebAssembly and for that, I need to provide a live stream of images directly from the camera.

My solution works pretty well on most devices, however, I have a problem on devices with multiple back-facing cameras, such as Samsung Galaxy S10 on Google Chrome for Android. The problem is that the following snippet:

const constraints = {
    audio: false,
    video: {
        width: { min: 640, ideal: 1280, max: 1920 },
        height: { min: 480, ideal: 720, max: 1080 },
        facingMode: { ideal: 'environment' },
    }
};
const stream = await navigator.mediaDevices.getUserMedia( constraints );

always opens a wrong camera - the wide-lens camera which does not support autofocus and provides images that are too distorted for my code. The wide-lens camera is good for landscape photography but it is terrible for barcode and text scanning.

How can I select the correct camera by using MediaTrackConstraints? I've tried adding also

focusMode: { ideal: 'continuous' }

to the constraints (according to MDN documentation, this should be a possible constraint for image tracks) but it does not seem to work.

I've also tried enumerating all devices (from this SO answer), but I don't know how to properly select the correct camera.

Notably, this code snippet:

const devices = await navigator.mediaDevices.enumerateDevices();
devices.forEach( ( device: MediaDeviceInfo ) => {
    console.log( "Found device: " + JSON.stringify( device ) );
});

produces the following console output:

Found device: {"deviceId":"default","kind":"audioinput","label":"","groupId":"4852f187ff6a41e6d3fb3ba41c4897f46bd8ff153579da6fcb8f485432a32f66"}
Found device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"audioinput","label":"","groupId":"c2cfc78763f7668263b0033c44d0f906ca0f33264ebfa6b96e9846265a21ff09"}
Found device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"audioinput","label":"","groupId":"7a86866423279d7b1e12dbe585a14a677a2f2df4e41ec5d388b6c90f7319e88d"}
Found device: {"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","kind":"videoinput","label":"camera2 1, facing front","groupId":"b1bd1a6ed8a87cd07ca0fa84744ae515b1ab2bed61cc257765c37d3426269af7"}
Found device: {"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","kind":"videoinput","label":"camera2 3, facing front","groupId":"a23c2f0e311c0ca0c80a56a8b5ff7c1f8aa093df4f6ac080b051c1d95f60a94e"}
Found device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"videoinput","label":"camera2 2, facing back","groupId":"9b6b1a429e0db2d5094ddebe205d23309464650d8bcd585b2fe4ae8196b86f1c"}
Found device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"videoinput","label":"camera2 0, facing back","groupId":"0de9556e1253763d7203b3b9e5db313cf89e05dd4bdd4ea0c5aff52d2952cf11"}
Found device: {"deviceId":"default","kind":"audiooutput","label":"","groupId":"default"}

The correct camera has the label camera2 0, facing back and for some reason Chrome always selects camera2 2, facing back.

Of course, I could hardcode the selection of camera by its label, but that would work only on Samsung Galaxy S10 and I would like my code to work on any device having multiple back-facing cameras.

I still haven't tried running my page on iPhone 11 Pro (which has 3 back-facing cameras), but it works correctly on Huawei Mate 30 Pro (4 back-facing cameras) and Oppo Reno 9. Also, the problem seems to related to Google Chrome on Android. When I open my page on Firefox for Android, the browser asks me to select which camera should be used just after closing the camera permission dialog and only if multiple cameras satisfy the given constraints. That is quite fair, as it makes it possible to select the correct camera for performing the scan. I haven't tried opening my page on Samsung Internet and Opera browsers (yet).

Since Google Chrome is the most popular web browser on Android, I would be even satisfied with Chrome-specific solution, but of course, the best would be to get answer that works everywhere.

EDIT

Following the comment from jib, I've tried also using focusDistance with

focusDistance: { min: 0.05, ideal: 0.12, max: 0.3 }

and it didn't help.

I've also tried logging the outputs of getSettings and getCapabilities with the following snippet:

const devices = await navigator.mediaDevices.enumerateDevices();
let videoDevices: Array< MediaDeviceInfo > = [];
devices.forEach( ( device: MediaDeviceInfo ) => {
    if ( device.kind == 'videoinput' ) {
        console.log( "Found video device: " + JSON.stringify( device ) );
        videoDevices.push( device );
    }
});

console.log( '' );

// open every video device and dump its characteristics
for ( let i in videoDevices ) {
    const device = videoDevices[ i ];
    console.log( "Opening video device " + device.deviceId + " (" + device.label + ")" );
    const stream = await navigator.mediaDevices.getUserMedia( { video: { deviceId: { exact: device.deviceId } } } );
    stream.getVideoTracks().forEach( track => {
            const capabilities = track.getCapabilities();
            console.log( "Track capabilities: " + JSON.stringify( capabilities ) );
            const settings = track.getSettings();
            console.log( "Track settings: " + JSON.stringify( settings ) );
            console.log( '' );
        }
    )

    stream.getTracks().forEach( track => track.stop() );
}

The output is the following:

Found video device: {"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","kind":"videoinput","label":"camera2 1, facing front","groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868"}
Found video device: {"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","kind":"videoinput","label":"camera2 3, facing front","groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe"}
Found video device: {"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","kind":"videoinput","label":"camera2 2, facing back","groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af"}
Found video device: {"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","kind":"videoinput","label":"camera2 0, facing back","groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987"}

Opening video device b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978 (camera2 1, facing front)
Track capabilities: {"aspectRatio":{"max":3216,"min":0.0004528985507246377},"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","facingMode":["user"],"frameRate":{"max":30,"min":0},"groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868","height":{"max":2208,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":3216,"min":1}}
Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"b46cd34041256d2cf72ed6e8500f71beb698a01b8c47c7c04801c20c47630978","facingMode":"user","frameRate":30,"groupId":"500dd57c6795399100a5ca8bf7f0cc4d7ed8b1bcb0877101d1bef7eb74921868","height":480,"resizeMode":"none","width":640}

Opening video device 39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6 (camera2 3, facing front)
Track capabilities: {"aspectRatio":{"max":3968,"min":0.0003654970760233918},"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","facingMode":["user"],"frameRate":{"max":30,"min":0},"groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe","height":{"max":2736,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":3968,"min":1}}
Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"39d63e8a9764261b73785c90beb58399997a5a4de56b3238fff6676c738331a6","facingMode":"user","frameRate":30,"groupId":"245693c8d34be77fe2f15be31b6054a19edb8ea9ed4116d966d2a03695bebebe","height":480,"resizeMode":"none","width":640}

Opening video device 4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c (camera2 2, facing back)
Track capabilities: {"aspectRatio":{"max":4608,"min":0.00028935185185185184},"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","facingMode":["environment"],"frameRate":{"max":60,"min":0},"groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af","height":{"max":3456,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":4608,"min":1}}
Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"4d5fecf5a3eee5d41812bb6c34efe6d25342af9448628b006561c7385a22ca6c","facingMode":"environment","frameRate":60,"groupId":"c219595df2c2a430aea7007f64e6ce8fbfa783b038cf53069336361cc07e71af","height":480,"resizeMode":"none","width":640}

Opening video device 86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d (camera2 0, facing back)
Track capabilities: {"aspectRatio":{"max":4032,"min":0.00033068783068783067},"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","facingMode":["environment"],"frameRate":{"max":60,"min":0},"groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987","height":{"max":3024,"min":1},"resizeMode":["none","crop-and-scale"],"width":{"max":4032,"min":1}}
Track settings: {"aspectRatio":1.3333333333333333,"deviceId":"86d4706b0bf160ff12fa75535173edcc68d4fa7ad5e00ec186cb1285ff22869d","facingMode":"environment","frameRate":60,"groupId":"96a68b6d6429786317e3b2c4773082604c5ab9a5cdaaf49c481878e40d67e987","height":480,"resizeMode":"none","width":640}

So, still, nothing to distinguish between camera2 0, facing back and camera2 2, facing back, except different maximum available resolution.

I've also tried with Samsung Internet browser and it behaves the same as Google Chrome.

Any other ideas (except iterating over all back-facing camera and selecting the one with least resolution)?

Good question. Maybe try looking at `deviceInfo.getCapabilities()` and `track.getSettings()` from the two cameras and look for differences. Have you tried `focusDistance`? — jib, Jan 08 '20 at 01:33
Thank you for the suggestion. Unfortunately, it does not help. See the updated question for details. — DoDo, Jan 08 '20 at 11:25
Just found [this little gem](https://unpkg.com/browse/scandit-sdk@4.6.1/src/lib/cameraAccess.ts). It's hacky but it works. — DoDo, Jan 08 '20 at 15:33
I've filed an [issue with the spec](https://github.com/w3c/mediacapture-main/issues/655) over this. — jib, Jan 13 '20 at 03:19
Hey @DoDo Did you find a reliable solution? I have the same problem... — King Julien, Mar 02 '20 at 13:49
@KingJulien, the solution from the [link above](https://unpkg.com/browse/scandit-sdk@4.6.1/src/lib/cameraAccess.ts) works well for me (so far). — DoDo, Mar 03 '20 at 14:12
@Frogger, [have fun](https://github.com/BlinkID/blinkid-in-browser/blob/master/src/MicroblinkSDK/VideoRecognizer.ts). — DoDo, Sep 09 '20 at 08:57
Why would you filter through `backCameraKeywords` when there is `environment` facingMode in capabilities though? — Brackets, Aug 20 '21 at 07:36
@Brackets, please read the original question. In cases when there are multiple cameras with the same `facingMode`, it's not possible to define which specific camera you wish to select and there are use cases when a randomly assigned camera is not good enough. — DoDo, Aug 20 '21 at 11:58
Yes but from what I see, camera having word "back" in its label will always have `environment` facingMode. I understand the problem, I only see facingMode as more reliable indicator whether it's front or back — Brackets, Aug 20 '21 at 12:29
But `facingMode` is property of the `constraint` object, and the resulting `device` object does not have it. Check the console output given above. — DoDo, Aug 20 '21 at 16:03
@DoDo recently I came across labels like 'Video device 1' instead of 'camera2 0, facing back'. Have you come across the same? — pix1289, Oct 27 '21 at 02:47
@pix1289, yes, I did on some devices. Fortunately for me, those devices didn't have multiple back-facing cameras so I could fall back simply using `facingMode` to detect the correct camera. — DoDo, Oct 27 '21 at 08:03
@DoDo how do I achieve the same using ```facingMode``` ? Can you please provide me with some example? I am currently stuck on the same. I am using ```navigator.enumerateDevices``` method only in my application. — pix1289, Oct 27 '21 at 10:04
@pix1289, I suggest that you take a look at [our camera selection code](https://github.com/BlinkID/blinkid-in-browser/blob/c59900e573e7a40e2c3a23f06f05309a4b25cd9e/src/MicroblinkSDK/CameraUtils.ts#L139). This is used in our production software (the camera part is open source). — DoDo, Oct 28 '21 at 09:32
To summarize the practices outlined in the links above, requesting cameras with facingMode: environment are further filtered by known labels indicating "back" and crucially sorting them by name, so something like `camera 2 0, facing back` appears before `camera 2 2, facing back`. What a mess! — Alex Suzuki, Dec 03 '21 at 13:06
https://stackoverflow.com/questions/63152117/issue-with-huawei-devices-using-rear-cameras-by-web-rtc-engine — efan, Jun 20 '23 at 08:33

score 3 · Answer 1 · answered Dec 03 '21 at 16:47

as this problem is somehow still relevant today, currently the best way of detecting "non-telephoto" / "non-wide-lens" camera is IMHO simply to check torch parameter.

(As this is the literally only parameter, which on some devices different “standard” camera and the others.)

I am doing this:

open default camera stream( = facingMode: { ideal: 'environment' },, find out, if torch is present
if not, close this camera stream and iterate for each device; try to detect torch
if nothing is found, fall back to the first camera; or possibly get better by some combination of other parameters - eg. focusDistance.
(save selected camera id to eg. cookie, so next time is for this user quicker)

Which object would have this `torch` property, though. – Qasim Jul 25 '23 at 05:37 — Qasim, Jul 25 '23 at 05:37

score 0 · Answer 2 · answered Nov 10 '20 at 09:52

I placed all the cameras by id into array like that

      navigator.mediaDevices.enumerateDevices()
        .then(function(devices) {
        
        for(;devices[i];){
        if(devices[i].kind == "videoinput"){
            that.aCameras.push(   [devices[i].deviceId , devices[i].label]   )
            j++;            
            }
        i++;
        }
    });

Than on the event that flip the camera by pressing the button i did this:

      var defaultsOpts = { audio: false, video: true };
      defaultsOpts.video = { 
              deviceId: that.aCameras[that.currentCamera][0]
      };
      if ( that.aCameras.length-1 != that.currentCamera ){
          that.currentCamera++;
      }
      else{
          that.currentCamera = 0;
      }      
      navigator.mediaDevices.getUserMedia(defaultsOpts)
            .then(function (stream) {
                  vid.srcObject = stream;
                  localstream = stream;
                  vid.play();
               });
          
       });

like that instead of using user/enviroment, my problem was sort of solved.

hope will help you too.

Regards, Avi.

This does not look correct. It assumes that n+1-th camera in your array is of opposite orientation than n-th camera, which is incorrect. See the question - Samsung S10 lists both its front and back cameras consecutively and there is no way to distinguish between them. For my case it's relevant which one of them gets selected. — DoDo, Nov 10 '20 at 13:07

score 0 · Answer 3 · answered Mar 21 '23 at 02:14

0

I just naively select the last videoinput device. For unknown reason, the phone manufacturers seem to all put the environment (back) normal lens camera as the last videoinput device. See https://www.reddit.com/r/javascript/comments/8eg8w5/choosing_cameras_in_javascript_with_the/

answered Mar 21 '23 at 02:14

Martin

1

1

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 26 '23 at 11:58

How to select proper backfacing camera in Javascript?

EDIT

3 Answers3

Linked