How can I orient an object on a tracked plane towards the camera in SparkAR Studio 98?

Question

I am looking for a way to orient a sceneobject on a tracked plane towards the camera via scripting in SparkAR Studio 98. This sounds trivial, but I am having a very hard time finding the world position of the camera, and then from that generating a rotation that I can apply to the object looking at the camera.

I got it to work WITHIN the tracked plane hierarchy, so that one object would always look at the other, but as the camera is living outside of that hierarchy and coordinate system, it doesn't help me.

I am, as always, struggeling with the reactive nature of the SparkAR API, as well as the fact that things are spread out over several coordinate systems and information is hard to find on the net and quickly outdated. My code looks like this:

const Scene = require('Scene');
const R = require('Reactive');
export const Diagnostics = require('Diagnostics');

(async function () {
    Diagnostics.log("Filter activated");

    const objectToOrient = await Scene.root.findFirst('object');
    const objectParent= await Scene.root.findFirst('objectParent');
    const target = await Scene.root.findFirst('target');

    const lookAtPt = R.point(
        target.worldTransform.x,
        target.worldTransform.y,
        target.worldTransform.z);

    const lookAtTransform = objectParent.transform.lookAt(lookAtPt);
})();

To illustrate the concept for the reader, I have set up this scene:

The "S" appears, as soon as the underlying plane is tracked. It then stays in place, along with the tracked plane. It should then always rotate towards the camera/device/user. Thank you!

Answer 1

Actually, there is another simple way to do it with lookAt method. It seems like it works a little bit different — it always rotates the object towards the camera position in the world space, instead of binding the camera rotation to the object rotation.

const Scene = require('Scene');

(async function () {

    const camera = await Scene.root.findFirst('Camera');
    const tracker = await Scene.root.findFirst('planeTracker');
    const plane = await Scene.root.findFirst('plane');

    // Get the camera world space position
    const cameraPositionSignal = tracker.worldTransform.inverse().applyToPoint(camera.worldTransform.position);

    // Apply the rotation of the relative transform between plane tracker and camera world position
    plane.transform.rotation = tracker.transform.lookAt(cameraPositionSignal).rotation;

    // Bring object back to the plane tracker origin
    plane.worldTransform.position = tracker.worldTransform.position;

})();

Probably, the lookAt method was not working properly at the time of Spark AR Studio 98 was active. This works in the currently latest 120 version.

Answer 2

WHAT WORKED

Having wasted a weekend, and ten minutes after giving up, I finally found out how to do it. The actual solution is so simple that it pains me to post it, but here it goes.

Using script, I just bound the rotation of the objectToOrient to the rotation of the camera. That's it.

const S = require('Scene');
const R = require('Reactive');
const D = require('Diagnostics');

// Enter look-at-target and object that needs to look at the target here
const objectName = 'lookAtObject';
const pivotName = 'pivot';
const targetName = 'Camera';
// const targetName = 'target';

(async function () {
    D.log("Filter activated");

    const [object, pivot, target] = await Promise.all([
        S.root.findFirst(objectName),
        S.root.findFirst(pivotName),
        S.root.findFirst(targetName)])
        .catch((error) => D.log(error.message));

    const deviceTransform = target.worldTransform;
    const pivotTransform = pivot.worldTransform;

    object.transform.rotation = deviceTransform.rotation;
    object.transform.position = pivotTransform.position;

    D.log("End of Log");
})();

After deleting two pages of my old code, I also tried to replicate this using patches, but plugging the DeviceMotion patch into the 3D rotation of the object actually produces different and counterintuitive behaviour on my android phone. Maybe the rotation in the patch and the one actually generated by the device don't line up, I'd be very interested in comments on this topic.

WHAT DIDN'T WORK

After having thought I solved the problem through angular calculations and world transforms like I posted here before, it actually wasn't working and I had to waste another two days of my life on this, troubleshooting and looking for problems in my calculations using either manually calculated angles between vectors, quaternions or all variations of lookAt that SparkAR offers.

On the way I learned about the DeviceMotion-module, which I hoped would fix the problem that I couldn't get an updated world-position of the camera into my scene. I wanted to use this position to calculate the angular difference between two vectors, and then from that rotate the object by that angle. Like most of the solutions, this worked perfectly in the editor, but not on my phone. Unfortunately it wouldn't update once in the scene. After some debugging I found out that DeviceMotion.worldTransform ACTUALLY ONLY CONTAINS A ROTATION, but doesn't throw an error when its positional coordinates are accessed. I find this package EXCESSIVELY frustrating to work with.