I have built an Android plugin (.AAR file) for a Unity3d Augmented Reality app. The Augmented Reality Unity3d app is the standard HelloAR demo app. This AR app uses Google AR Core. Information on developing native Andriod plugins for Unity3D can be found here.
The Unity3D AR app loads the .AAR plugin and tries to open the camera, take a picture and save it to file, without any input from the user: no UI, no button press and no preview of the capture to the user.
When my plugin attempts to take a picture, I get the following exception:
2020/08/06 18:47:15.857 19906 21180 Info CameraManagerGlobal Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_OPEN for client com.cambridge.helloar API Level 1
2020/08/06 18:47:15.877 19906 19906 Warn System.err java.io.IOException: setPreviewTexture failed
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.hardware.Camera.setPreviewTexture(Native Method)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager.takePhoto(PluginManager.java:242)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager.access$100(PluginManager.java:38)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager$4.run(PluginManager.java:351)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.os.Handler.handleCallback(Handler.java:883)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.os.Handler.dispatchMessage(Handler.java:100)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.os.Looper.loop(Looper.java:237)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.app.ActivityThread.main(ActivityThread.java:8107)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at java.lang.reflect.Method.invoke(Native Method)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
2020/08/06 18:47:15.877 19906 19906 Warn System.err at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
2020/08/06 18:47:15.877 19906 19906 Debug TAKEPHOTO Opened camera
2020/08/06 18:47:15.877 19906 19906 Debug TAKEPHOTO Started preview
2020/08/06 18:47:15.877 19906 19906 Warn System.err java.lang.RuntimeException: takePicture failed
2020/08/06 18:47:15.877 19906 19906 Warn System.err at android.hardware.Camera.native_takePicture(Native Method)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.hardware.Camera.takePicture(Camera.java:1564)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.hardware.Camera.takePicture(Camera.java:1506)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager.takePhoto(PluginManager.java:250)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager.access$100(PluginManager.java:38)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at com.cambridge.helloar.PluginManager$4.run(PluginManager.java:351)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.os.Handler.handleCallback(Handler.java:883)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.os.Handler.dispatchMessage(Handler.java:100)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.os.Looper.loop(Looper.java:237)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at android.app.ActivityThread.main(ActivityThread.java:8107)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at java.lang.reflect.Method.invoke(Native Method)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
2020/08/06 18:47:15.878 19906 19906 Warn System.err at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
I've tried both the front and back camera but I get this exception in both cases.
I have also noticed that I get a warning on my AR app saying something like "another app is using the camera. Tap this app or try closing the other app", so it's definitely something to do with the camera being blocked by ARCore. I do not want to prevent the Unity3D AR app from using the camera... of course. Instead, I want the AR app and my plugin to BOTH have access to the camera.
I know my Plugin code works because doing this in a non-AR app works and a picture is taken in the background.
Here is the code:
@SuppressWarnings("deprecation")
private static void takePhoto() {
Log.d("TAKEPHOTO", "Surface created");
SurfaceTexture surfaceTexture = new SurfaceTexture(10);
Camera camera = Camera.open(0);
camera.getParameters().setPreviewSize(1, 1);
try {
camera.setPreviewTexture(surfaceTexture);
} catch (IOException e) {
e.printStackTrace();
}
Log.d("TAKEPHOTO", "Opened camera");
Log.d("TAKEPHOTO", "Started preview");
camera.takePicture(null, null, new Camera.PictureCallback() {
@Override
public void onPictureTaken(byte[] data, Camera camera) {
Log.d("TAKEPHOTO", "Took picture");
File pictureFileDir = new File("/sdcard/CaptureByService");
if (!pictureFileDir.exists() && !pictureFileDir.mkdirs()) {
return;
}
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyymmddhhmmss");
String date = dateFormat.format(new Date());
String photoFile = "PictureFront_" + "_" + date + ".jpg";
String filename = pictureFileDir.getPath() + File.separator + photoFile;
File mainPicture = new File(filename);
try {
FileOutputStream fos = new FileOutputStream(mainPicture);
fos.write(data);
fos.close();
Log.d("TAKEPHOTO", "image saved");
} catch (Exception error) {
Log.d("TAKEPHOTO", "Image could not be saved");
error.printStackTrace();
}
camera.release();
}
});
}
Does anyone have any suggestions/solutions?
