3

Edit: This is closed because it needs more details, yet people keep upvoting it. So I'll do my best to add some. This question was poorly formatted, as it was really 2 questions. 1) How do I fix this code from the documentation? 2) How do I align the unity scene to the player's physical space?

I feel like #1 was focused on the answer, but #2 was the title question. My main issues were that the occlusion mesh being created was bumpy/not flat, making collisions between holograms and the floor very wonky. I ended up creating a basic menu with the hologram of a cube in front of the player. The player was then instructed to lower the cube until it sat evenly on the floor. I then made sure all holograms didn't fall lower than that set value, either with collision planes, or via code. Hope that idea can help anyone. Thanks!

When the player loads into a unity scene in the Hololens2, the Unity Floor plane does not match the physical floor. With the hololens2 and MRTK, the Unity scene origin is locked to the players head being 0,0,0.

I am trying to use the Microsoft Scene Understanding SDK to set the Unity Scene environment Y position to the floor in the physical room. I am currently able to access the floor scene object, but when I try to do the SpatialCoordinateSystem portion, I am unable to use the .ToUnity() method to convert the 4x4 Matrix. I had to change the vector3 .ToUnity() call to .ToUnityVector3(); but I am unable to find a similar method for the matrix.

Are the .ToUnity() methods mentioned in this documentation deprecated? Am I missing a reference to something? (see images for references)

I greatly appreciate any assistance here, either in this specific issue, or the overall challenge of aligning a unity scene to a Hololens2 user's physical environment.

I'm following the information provided here https://learn.microsoft.com/en-us/windows/mixed-reality/develop/platform-capabilities-and-apis/scene-understanding-sdk

Code Snippet

I attempted the suggestion by Hernando - MSFT

Usings

NCarlson
  • 330
  • 2
  • 13
  • Please do not post images of code/errors/data. Instead post the code/errors/data as text in a code block. See [How do I ask a good question?](https://stackoverflow.com/help/how-to-ask) – Scratte Nov 23 '20 at 23:03
  • Thanks for your input. Have a great day. – NCarlson Nov 24 '20 at 15:58

2 Answers2

2

Compared with the documentation, you missed the Value property and invoked the ToUnity() method directly on the sceneToWorld object. This may be the cause of the issue.

var sceneToWorldUnity = sceneToWorld.ToUnity();

=>

var sceneToWorldUnity = sceneToWorld.Value.ToUnity();
Hernando - MSFT
  • 2,895
  • 1
  • 5
  • 11
  • Thank you so much for the help! I tried this and I'm still not able to call ToUnity() Please see my edit to the post. – NCarlson Nov 23 '20 at 13:57
  • 1
    I noticed that your `using` directive list does not include `NumericsConversionExtensions` class, it seems that you did not add this helper class to the project. Actually, according to the documentation, you should first add this helper class for converting between these two conventions, you can use the following code:[Converting between coordinate systems](https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/unity-xrdevice-advanced#converting-between-coordinate-systems) . – Hernando - MSFT Nov 24 '20 at 07:33
  • This worked! Thank you so much! – NCarlson Nov 24 '20 at 15:50
1

Hernando - MSFT's suggestions worked for me the missing ToUnity() references. The challenge of aligning all the scene holograms to the floor still remains, but this is progress! Thank you so much Hernando!

I basically just had to add .value to access the matrix data, and then implement this namespace :

namespace NumericsConversion
{
    public static class NumericsConversionExtensions
    {
        public static UnityEngine.Vector3 ToUnity(this System.Numerics.Vector3 v) => new UnityEngine.Vector3(v.X, v.Y, -v.Z);
        public static UnityEngine.Quaternion ToUnity(this System.Numerics.Quaternion q) => new UnityEngine.Quaternion(-q.X, -q.Y, q.Z, q.W);
        public static UnityEngine.Matrix4x4 ToUnity(this System.Numerics.Matrix4x4 m) => new UnityEngine.Matrix4x4(
            new Vector4(m.M11, m.M12, -m.M13, m.M14),
            new Vector4(m.M21, m.M22, -m.M23, m.M24),
            new Vector4(-m.M31, -m.M32, m.M33, -m.M34),
            new Vector4(m.M41, m.M42, -m.M43, m.M44));

        public static System.Numerics.Vector3 ToSystem(this UnityEngine.Vector3 v) => new System.Numerics.Vector3(v.x, v.y, -v.z);
        public static System.Numerics.Quaternion ToSystem(this UnityEngine.Quaternion q) => new System.Numerics.Quaternion(-q.x, -q.y, q.z, q.w);
        public static System.Numerics.Matrix4x4 ToSystem(this UnityEngine.Matrix4x4 m) => new System.Numerics.Matrix4x4(
            m.m00, m.m10, -m.m20, m.m30,
            m.m01, m.m11, -m.m21, m.m31,
           -m.m02, -m.m12, m.m22, -m.m32,
            m.m03, m.m13, -m.m23, m.m33);
    }
}

I also tried dropping a cube with a rigid-body and collider at run time to determine the Y value of the spacial data floor. This actually seems to work relatively well, but its definitely not the most precise solution.

NCarlson
  • 330
  • 2
  • 13