5

I would like to get the colour of the detected world object at a specific feature point in the sceneView. For example, I have a feature point detected at (x:10, y:10, z:10).

How do I get the colour of the object/ surface at this position?

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Martin Ma
  • 69
  • 6
  • could you include in your question the desired behavior, a specific problem or error and the shortest code necessary to reproduce it in the question itself. Questions without a clear problem statement are not useful to other readers. See: [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve). – Artem Oct 28 '18 at 17:24

1 Answers1

2

At the moment it's not possible to get a colour of a real-world object under feature point using ARKit methods (the same way like you saw in many compositing apps). There's no ARKit method allowing you multiply an Alpha of a feature point by RGB value of corresponding pixel in a video stream.

enter image description here

.showFeaturePoints is an extended debug option ARSCNDebugOptions for an ARSCNView. This option just allow you to show detected 3D feature points in the world.

@available(iOS 11.0, *)
public static let showFeaturePoints: SCNDebugOptions

But I'm sure that you can try to apply a CIFilter to ARKit camera feed containing feature points.

Feature points in your scene are yellow, so you can use Chroma Key Effect to extract an Alpha channel. Then you need to multiply this Alpha by RGB from camera. So you'll get color-coded feature points.

enter image description here

You can alternatively use a CIDifferenceBlendMode op from Core Image Compositing Operations. You need two sources – one with feature points and another without them. Then you have to modify this result of Difference op and assign it to Alpha channel before multiplication.

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • 1
    Thanks for the help! I am trying some photogrammetric applications using ARKit like your first image shown. Gonna have some researches on your method. – Martin Ma Nov 01 '18 at 07:40