Using a MTLTexture as the environment map of a SCNScene

Question

I want to set a MTLTexture object as the environment map of a scene, as it seems to be possible according to the documentation. I can set the environment map to be a UIImage with the following code:

let roomImage = UIImage(named: "room")
scene.lightingEnvironment.contents = roomImage

This works and I see the reflection of the image on my metallic objects. I tried converting the image to a MTLTexture and setting it as the environment map with the following code:

let roomImage = UIImage(named: "room")
let loader = MTKTextureLoader(device: MTLCreateSystemDefaultDevice()!)
let envMap = try? loader.newTexture(cgImage: (roomImage?.cgImage)!, options: nil)
scene.lightingEnvironment.contents = envMap

However this does not work and I end up with a blank environment map with no reflection on my objects.

Also, instead of setting the options as nil, I tried setting the MTKTextureLoader.Option.textureUsage key with every possible value it can get, but that didn't work either.

Edit: You can have a look at the example project in this repo and use it to reproduce this use case.

I believe this is a supported use case. What are the dimensions of your image? What format is it in (I assume PNG)? Does `envMap` contain a non-`nil` value after you finish loading? Can you QuickLook it (visually inspect its contents) in Xcode if you set a breakpoint right after the line where you load it? — warrenm, Dec 10 '17 at 23:21
@warrenm The image is a 1024x512 PNG file. `envMap` seems to be successfully created and Quick Look shows the same image as `roomImage`. — halileohalilei, Dec 11 '17 at 06:32
@AndyJazz It's been a long time since I abandoned this project and this platform, so I'll take your word for it and accept your answer since you seem to have put an incredibly detailed solution out there. Thank you for your efforts! — halileohalilei, Jan 31 '22 at 12:13
Hi @halileohalilei, It's never too late to answer a question with +31 upvotes. ))) — Andy Jazz, Jan 31 '22 at 12:36

Andy Jazz · Accepted Answer · 2022-07-25T07:14:50.150

Lighting SCN Environment with an MTK texture

Using Xcode 13.3.1 on macOS 12.3.1 for iOS 15.4 app.

The trick is, the environment lighting requires a cube texture, not a flat image.

Create 6 square images for MetalKit cube texture

in Xcode Assets folder create Cube Texture Set

place textures to their corresponding slots

mirror images horizontally and vertically, if needed

Paste the code:

import ARKit
import MetalKit

class ViewController: UIViewController {

    @IBOutlet var sceneView: ARSCNView!
    
    override func viewDidLoad() {
        super.viewDidLoad()    
        let scene = SCNScene()
        
        let imageName = "CubeTextureSet"
        let textureLoader = MTKTextureLoader(device: sceneView.device!)

        let environmentMap = try! textureLoader.newTexture(name: imageName, 
                                                    scaleFactor: 2, 
                                                         bundle: .main, 
                                                        options: nil)
        
        let daeScene = SCNScene(named: "art.scnassets/testCube.dae")!

        let model = daeScene.rootNode.childNode(withName: "polyCube", 
                                             recursively: true)!
        
        scene.lightingEnvironment.contents = environmentMap
        scene.lightingEnvironment.intensity = 2.5
        scene.background.contents = environmentMap

        sceneView.scene = scene
        sceneView.allowsCameraControl = true
        scene.rootNode.addChildNode(model)
    }
}

Apply metallic materials to models. Now MTL environment lighting is On.

If you need a procedural skybox texture – use MDLSkyCubeTexture class.

Also, this post may be useful for you.

Using a MTLTexture as the environment map of a SCNScene

1 Answers1

Lighting SCN Environment with an MTK texture

Linked