How to add a semi-transparent background on top of camera ARKit

Question

I'm trying to add a semi transparent background that cover my camera view from ARKit.

I try different things :

Add background to sceneView.scene but that not support transparency
Add an overlaySKScene but nodes on my scene are overlayed too.
Use CIImage from session > capturedImage but too slow.
Use this post : Reliable access and modify captured camera frames under SceneKit, it's OK for transform to Black And White but I don't understand how I can keep colors and blend gray color.
Search on OpenGL or Metal but I'm a noob !

So, do you have an idea to realize that operation in ARKit ? Thanks in advance.

Andy Jazz · Accepted Answer · 2019-10-07T14:03:13.547

You can do it this way:

import ARKit

class ViewController: UIViewController,
                      ARSCNViewDelegate {

    @IBOutlet var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()
        sceneView.delegate = self
        sceneView.session.delegate = self
        let scene = SCNScene()
        sceneView.scene = scene

        let planeNode = SCNNode()

        planeNode.geometry = SCNPlane(width: 100, 
                                     height: 100)

        planeNode.geometry?.firstMaterial?.diffuse.contents = UIColor(white: 0,
                                                                      alpha: 0.9)
        planeNode.position.z = -5     // 5 meters away

        sceneView.pointOfView?.addChildNode(planeNode)      // PINNING TO CAMERA
    }

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
        let configuration = ARWorldTrackingConfiguration()
        sceneView.session.run(configuration)
    }
    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)
        sceneView.session.pause()
    }
}

... or in extension:

extension ViewController: ARSessionDelegate {

    func session(_ session: ARSession,
           didUpdate frame: ARFrame) {

        let planeNode = SCNNode()

        planeNode.geometry = SCNPlane(width: 100, 
                                     height: 100)

        planeNode.position.z = -5

        planeNode.geometry?.firstMaterial?.diffuse.contents = UIColor(white: 0, 
                                                                      alpha: 0.9)

        // var translation = matrix_identity_float4x4
        // translation.columns.3.z = -5
        // planeNode.simdTransform = matrix_multiply(translation,
        //                                           frame.camera.transform)

        sceneView.pointOfView?.addChildNode(planeNode)
    }
}

Hello, Thanks for your response but I need to have a background that always cover the camera. In fact like a 2D static plane. This plane can't move and it must placed between camera view and scene nodes. — kopacabana, Apr 04 '19 at 12:08
It's simple. Pin this plane to ARCamera's transform. So it can't be tracked. — Andy Jazz, Apr 04 '19 at 12:10
Yes, it's work ! Great solution. If someone have an idea for the same trick in pure OpenGL or Metal, I'm interesting. Thanks @ARGeo — kopacabana, Apr 04 '19 at 12:44
@ARGeo, can you explain what you mean by "Pin this plane to ARCamera's transform" with some sample code? — elprl, Oct 07 '19 at 00:05
Do you mean effectively add the last 3 lines to `func session(_ session: ARSession, didUpdate frame: ARFrame)` ? — elprl, Oct 07 '19 at 00:18
@AndyJazz Is there a way to fit screen height with this SCNPlane overlay so that it matches point of view height size? — Ozgur Sahin, Jun 20 '22 at 20:19

How to add a semi-transparent background on top of camera ARKit

1 Answers1