0

On one hand i have the image-based AR that Unity provides me, which is amazing in making things seem as if they are actually around my in the real world.
Problem is - this is only localized based on the imagery of my surrounding, disregarding my location on the latitude-longitude coordinates and my Azimuth (compass).
However, if i use the compass and GPS sensors on my Android i get a fidgety readings, and that causes the virtual objects around my to be jumpy.
I don't want to use Vuforia / Wikitude / Mapbox. I want to build this myself using only UNITY.
Is this possible? what should be the algorithm for combining image-based spatial-localization with GPS and compass data?
How did Niantic solve this in 2016 with Pokemon GO?

forstack overflowizi
  • 324
  • 1
  • 3
  • 15

2 Answers2

0

The short answer is "No", neither compass or GPS data is accurate enough for AR. You will never get a stable AR experience based on these two sensors, especially you are using mobile devices.

But it's not impossible, you can instantiate your starting position based on GPS reference, and compass if possible. But, the compass sensor quality is terribly bad on most of Android devices. You will have very high chance of getting wrong direction on mobile nowadays.

If you are in city with many tall buildings, you will end-up with ~10+ meters off from reality. And the compass sensor would be interfered a lot.

Once you initialised your starting position, then you have to switch to marker-less tracking. It will try to trace all the feature points from your webcam. It means, it only works well in day time environment as none of mobile webcam works well at night.

imleoson
  • 142
  • 11
  • Yes of course i want to use the markerless tracking as the main positioning system, I just want to ALSO integrate the GPS and compass so the space around me is aligned with the real world coordinate system (NORTH, EAST, etc). The initial position is not enough, i want it to constantly update based on continous readings – forstack overflowizi Aug 04 '21 at 09:53
  • I've tried similar workflow before. We ended up with putting markers in different city locations for initial position. The compass has 50%+ wrong direction, means you will never align buildings in AR. not just compass issue, gyro sensor also have problem in most of Android devices. According to my testings among devices, iPhone's sensors are more reliable than any of Android devices. – imleoson Aug 04 '21 at 09:58
  • Yes the sensors are bad, but yet somehow still usable when i use PokemonGO or SkyView. So Sensors can't have all the blame. There's probably an algorithm we are missing to make them less jumpy – forstack overflowizi Aug 04 '21 at 10:10
  • 1
    It really depends on what devices you have. The result would be far different from $100USD phones to $1000 USD phones. And it also depends on your site and signal. The general idea is lerping all the values, and filtering the noise with some maths. In some case, for example we noticed that the sensors will drift when a train pass-by..etc. – imleoson Aug 05 '21 at 11:23
0

Good news - Niantic just released the Lightship ARDK in Unity. It is the basis for Pokemon Go. They are slowly releasing more capabilities in the ADRK, so the features you're asking for (called VPS) are not yet out in the wild but are due very soon (Q2, 2022 - possibly May 24). I've been working with it for a few weeks now and it's a powerful and exciting platform in its early stages.

UPDATE 2023: Google just released their Geospatial Creator tools for ARCore in Unity, so now you have even more options.

gromiczek
  • 2,970
  • 5
  • 28
  • 49