10

I was trying out the new RealityKit framework in swift and I want to place a 2D image in on an ARAnchor. I managed to create a mesh plane and I tried to add the image as a texture to this plane. I found out that I can give a local image to a material. How can I use a UIImage instead?

let resource = try? TextureResource.load(named: "restart")
var material = UnlitMaterial()
material.baseColor = MaterialColorParameter.texture(resource!)
material.tintColor = UIColor.white.withAlphaComponent(0.99)

let imagePlane = ModelEntity(mesh: MeshResource.generatePlane(width: 0.25, height: 0.25), materials: [SimpleMaterial(color: color, isMetallic: true)])
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
batabek
  • 103
  • 3
  • 7
  • May I ask why you want to use a UIImage, what is the goal? Is the content of the UIImage just a local image, remote image, or something else? – iicaptain Jun 06 '20 at 20:24

2 Answers2

13

Currently, you cannot use UIImage or CIImage as a shader's texture in RealityKit 2.0. In both versions of RealityKit, the texture must be loaded via the String type parameter of the load() method.

RealityKit 2.0

To assign a texture to a shader in RealityKit 2.0 use the following approach:

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
        
var material = SimpleMaterial()
material.color = .init(tint: .white.withAlphaComponent(0.999),
                    texture: .init(try! .load(named: "texture.png")))
material.metallic = .float(1.0)
material.roughness = .float(0.0)

let model = ModelEntity(mesh: mesh, materials: [material])

RealityKit 1.0

To assign a texture to a shader in RealityKit 1.0 use this approach:

let scene = try! Experience.loadMyScene()

var material = SimpleMaterial()
material.baseColor = try! .texture(.load(named: "texture.png"))
material.metallic = MaterialScalarParameter(floatLiteral: 1.0)
material.roughness = MaterialScalarParameter(floatLiteral: 0.0)
material.tintColor = UIColor.white

let mesh: MeshResource = .generatePlane(width: 0.45, depth: 0.45)
let component = ModelComponent(mesh: mesh, materials: [material])

scene.myFavoriteScene?.children[0].components.set(component)
arView.scene.anchors.append(scene)

CGImage

Nonetheless, you can create a texture resource from an in-memory Core Graphics image:

static func generate(from: CGImage, 
                 withName: String?, 
                  options: TextureResource.CreateOptions) -> TextureResource

Also, you can use a URL parameter:

material.color.texture = .init(try! .load(contentsOf: url))  // RealityKit 2.0
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • 1
    Just wanted to note for others that the image has to be added to a `Texture Set` inside of `Assets.xcassets`. The name of the asset as it appears in `Assets.xcassets` should be used. – tyirvine Apr 18 '23 at 06:50
1

There is not currently a way to take a UIImage and apply it to a Material directly (as of June 2020).

You could first write the UIImage to disk, then load it by name in a similar way you have done with your "restart" texture.

maxxfrazer
  • 1,173
  • 6
  • 15