physically based lighting on custom SCNGeometry Node

Question

Question

How do you define the material on a custom geometry from vertex data, so that it renders the same as 'typical' SCNNodes?

Details

In this scene there are

A directional light
A red sphere using physicallybased lighting model
A blue sphere using physicallybased lighting model
A custom SCNGeometry using vertex data, using a physicallybased lighting model

The red and blue spheres render as I would expect. The two points / spheres in the custom geometry are black.

Why?

Here is the playgrond code:

Setting the scene


import UIKit
import SceneKit
import PlaygroundSupport


// create a scene view with an empty scene
var sceneView = SCNView(frame: CGRect(x: 0, y: 0, width: 600, height: 600))
var scene = SCNScene()
sceneView.scene = scene
sceneView.backgroundColor = UIColor(white: 0.75, alpha: 1.0)
sceneView.allowsCameraControl = true
PlaygroundPage.current.liveView = sceneView

let directionalLightNode: SCNNode = {
    let n = SCNNode()
    n.light = SCNLight()
    n.light!.type = SCNLight.LightType.directional
    n.light!.color = UIColor(white: 0.75, alpha: 1.0)
    return n
}()

directionalLightNode.simdPosition = simd_float3(0,5,0) // Above the scene
directionalLightNode.simdOrientation = simd_quatf(angle: -90 * Float.pi / 180.0, axis: simd_float3(1,0,0)) // pointing down

scene.rootNode.addChildNode(directionalLightNode)

// a camera
var cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.simdPosition = simd_float3(0,0,5)
scene.rootNode.addChildNode(cameraNode)

Adding the blue and red spheres

// ----------------------------------------------------
// Example creating SCNSphere Nodes directly

// Sphere 1
let sphere1 = SCNSphere(radius: 0.3)
let sphere1Material = SCNMaterial()
sphere1Material.diffuse.contents = UIColor.red
sphere1Material.lightingModel = .physicallyBased
sphere1.materials = [sphere1Material]

let sphere1Node = SCNNode(geometry: sphere1)
sphere1Node.simdPosition = simd_float3(-2,0,0)

// Sphere2
let sphere2 = SCNSphere(radius: 0.3)
let sphere2Material = SCNMaterial()
sphere2Material.diffuse.contents = UIColor.blue
sphere2Material.lightingModel = .physicallyBased
sphere2.materials = [sphere2Material]

let sphere2Node = SCNNode(geometry: sphere2)
sphere2Node.simdPosition = simd_float3(-1,0,0)

scene.rootNode.addChildNode(sphere1Node)
scene.rootNode.addChildNode(sphere2Node)

Adding the custom SCNGeometry

// ----------------------------------------------------
// Example creating SCNGeometry using vertex data
struct Vertex {
    let x: Float
    let y: Float
    let z: Float
    let r: Float
    let g: Float
    let b: Float
}

let vertices: [Vertex] = [
    Vertex(x: 0.0, y: 0.0, z: 0.0, r: 1.0, g: 0.0, b: 0.0),
    Vertex(x: 1.0, y: 0.0, z: 0.0, r: 0.0, g: 0.0, b: 1.0)
]

let vertexData = Data(
    bytes: vertices,
    count: MemoryLayout<Vertex>.size * vertices.count
)

let positionSource = SCNGeometrySource(
    data: vertexData,
    semantic: SCNGeometrySource.Semantic.vertex,
    vectorCount: vertices.count,
    usesFloatComponents: true,
    componentsPerVector: 3,
    bytesPerComponent: MemoryLayout<Float>.size,
    dataOffset: 0,
    dataStride: MemoryLayout<Vertex>.size
)
let colorSource = SCNGeometrySource(
    data: vertexData,
    semantic: SCNGeometrySource.Semantic.color,
    vectorCount: vertices.count,
    usesFloatComponents: true,
    componentsPerVector: 3,
    bytesPerComponent: MemoryLayout<Float>.size,
    dataOffset: MemoryLayout<Float>.size * 3,
    dataStride: MemoryLayout<Vertex>.size
)

let elements = SCNGeometryElement(
    data: nil,
    primitiveType: .point,
    primitiveCount: vertices.count,
    bytesPerIndex: MemoryLayout<Int>.size
)
        
elements.pointSize = 100
elements.minimumPointScreenSpaceRadius = 100
elements.maximumPointScreenSpaceRadius = 100

let spheres = SCNGeometry(sources: [positionSource, colorSource], elements: [elements])
let sphereNode = SCNNode(geometry: spheres)

let sphereMaterial = SCNMaterial()
sphereMaterial.lightingModel = .physicallyBased

spheres.materials = [sphereMaterial]

sphereNode.simdPosition = simd_float3(0,0,0)
scene.rootNode.addChildNode(sphereNode)

Some Exploration

Adding normals now shows the colours, but in all directions (i.e, there's no shadow).

And I've added a black SCNSphere() and a 3rd point to my VertexData, both using the same RGB values, but the black in the VertexData object appears too 'light'

let vertices: [Vertex] = [
    Vertex(x: 0.0, y: 0.0, z: 0.0, r: 1.0, g: 0.0, b: 0.0),
    Vertex(x: 1.0, y: 0.0, z: 0.0, r: 0.0, g: 0.0, b: 1.0),
    Vertex(x: 0.0, y: 1.0, z: 0.0, r: 0.07, g: 0.11, b: 0.12)
]

let vertexData = Data(
    bytes: vertices,
    count: MemoryLayout<Vertex>.size * vertices.count
)

let normals = Array(repeating: SCNVector3(1,1,1), count: vertices.count)
let normalSource = SCNGeometrySource(normals: normals)

///
///

let spheres = SCNGeometry(
    sources: [
        positionSource,
        normalSource,
        colorSource
    ],
    elements: [elements]
)

aheze · Accepted Answer · 2021-09-24T21:36:55.473

According to the documentation, making a custom geometry takes 3 steps.

Create a SCNGeometrySource that contains the 3D shape's vertices.
Create a SCNGeometryElement that contains an array of indices, showing how the vertices connect.
Combine the SCNGeometrySource source and SCNGeometryElement into a SCNGeometry.

Let's start from step 1. You want your custom geometry to be a 3D shape, right? You only have 2 vertices, though.

let vertices: [Vertex] = [         /// what's `r`, `g`, `b` for btw? 
    Vertex(x: 0.0, y: 0.0, z: 0.0, r: 1.0, g: 0.0, b: 0.0),
    Vertex(x: 1.0, y: 0.0, z: 0.0, r: 0.0, g: 0.0, b: 1.0)
]

This will form a line...

A common way of making 3D shapes is from triangles. Let's add 2 more vertices to make a pyramid.

let vertices: [Vertex] = [
    Vertex(x: 0.0, y: 0.0, z: 0.0, r: 1.0, g: 0.0, b: 0.0), /// vertex 0
    Vertex(x: 1.0, y: 0.0, z: 0.0, r: 0.0, g: 0.0, b: 1.0), /// vertex 1
    Vertex(x: 1.0, y: 0.0, z: -0.5, r: 0.0, g: 0.0, b: 1.0), /// vertex 2
    Vertex(x: 0.0, y: 1.0, z: 0.0, r: 0.0, g: 0.0, b: 1.0), /// vertex 3
]

Now, we need to convert the vertices into something that SceneKit can handle. In your current code, you convert vertices into Data, then use the init(data:semantic:vectorCount:usesFloatComponents:componentsPerVector:bytesPerComponent:dataOffset:dataStride:) initializer.

let vertexData = Data(
    bytes: vertices,
    count: MemoryLayout<Vertex>.size * vertices.count
)
let positionSource = SCNGeometrySource(
    data: vertexData,
    semantic: SCNGeometrySource.Semantic.vertex,
    vectorCount: vertices.count,
    usesFloatComponents: true,
    componentsPerVector: 3,
    bytesPerComponent: MemoryLayout<Float>.size,
    dataOffset: 0,
    dataStride: MemoryLayout<Vertex>.size
)

This is very advanced and complicated. It's way easier with init(vertices:).

let verticesConverted = vertices.map { SCNVector3($0.x, $0.y, $0.z) } /// convert to `[SCNVector3]`
let positionSource = SCNGeometrySource(vertices: verticesConverted)

Now that you've got the SCNGeometrySource, it's time for step 2 — connecting the vertices via SCNGeometryElement. In your current code, you use init(data:primitiveType:primitiveCount:bytesPerIndex:), then pass in nil...

let elements = SCNGeometryElement(
    data: nil,
    primitiveType: .point,
    primitiveCount: vertices.count,
    bytesPerIndex: MemoryLayout<Int>.size
)

If the data itself is nil, how will SceneKit know how to connect your vertices? But anyway, there's once again an easier initializer: init(indices:primitiveType:). This takes in an array of FixedWidthInteger, each representing a vertex back in your positionSource.

So how is each vertex represented by a FixedWidthInteger? Well, remember how you passed in verticesConverted, an array of SCNVector3, to positionSource? SceneKit sees each FixedWidthInteger as an index and uses it access verticesConverted.

Since indices are always integers and positive, UInt16 should do fine (it conforms to FixedWidthInteger).

/// pairs of 3 indices, each representing a vertex
let indices: [UInt16] = [
   0, 1, 3, /// front triangle
   1, 2, 3, /// right triangle
   2, 0, 3, /// back triangle
   3, 0, 2, /// left triangle
   0, 2, 1 /// bottom triangle
]
let element = SCNGeometryElement(indices: indices, primitiveType: .triangles)

The order here is very specific. By default, SceneKit only renders the front face of triangles, and in order to distinguish between the front and back, it relies on your ordering. The basic rule is: counterclockwise means front.

So to refer to the first triangle, you could say:

0, 1, 3
1, 3, 0
3, 0, 1

All are fine. Finally, step 3 is super simple. Just combine the SCNGeometrySource and SCNGeometryElement.

let geometry = SCNGeometry(sources: [positionSource], elements: [element])

And that's it! Now that both your SCNGeometrySource and SCNGeometryElement are set up correctly, lightingModel will work properly.

/// add some color
let material = SCNMaterial()
material.diffuse.contents = UIColor.orange
material.lightingModel = .physicallyBased
geometry.materials = [material]

/// add the node
let node = SCNNode(geometry: geometry)
scene.rootNode.addChildNode(node)

Notes:

I noticed that you were trying to use 2 SCNGeometrySources. The second one was to add color with SCNGeometrySource.Semantic.color, right? The simpler initializer that I used, init(vertices:), defaults to .vertex. If you want per-vertex color or something, you'll probably need to go back to init(data:semantic:vectorCount:usesFloatComponents:componentsPerVector:bytesPerComponent:dataOffset:dataStride:).
Try sceneView.autoenablesDefaultLighting = true for some better lighting
Full demo playground here

Edit: Single Vertex Sphere?

You shouldn't be using a single point to make a sphere. If you're going to do...

elements.pointSize = 100
elements.minimumPointScreenSpaceRadius = 100
elements.maximumPointScreenSpaceRadius = 100

... then a 2D Circle is going to be the best you can get.

That's because, according to the pointSize documentation:

SceneKit can render each point as a small 2D surface that always faces the camera. By applying a texture or custom shader to that surface, you can efficiently render many small objects at once.

Since what's rendered is really just a circle that rotates to face you, .physicallyBased lighting won't work (.constant will, but that's it). It's better to make your sphere with many small triangles, like the pyramid in the above answer. This is also what Apple does with their built in geometry, including SCNSphere.

let sphere = SCNSphere(radius: 1)
let sphereMaterial = SCNMaterial()
sphereMaterial.diffuse.contents = UIColor.purple
sphereMaterial.fillMode = .lines /// add this to see the triangles
sphereMaterial.lightingModel = .physicallyBased
sphere.materials = [sphereMaterial]

let sphereNode = SCNNode(geometry: sphere)
scene.rootNode.addChildNode(sphereNode)

I don't think this answers my question. I am using `primitiveType: .point`, because I want 'points', not triangles. My code correctly draws the points, but I don't know how to set the materials and lighting so they look like the two seprate `SCNSphere()` objects. — tospig, Sep 22 '21 at 09:38
and in your comment you ask `/// what's `r`, `g`, `b` for btw? ` - those are the colours, used in the `.color semantic`. — tospig, Sep 22 '21 at 09:49
@tospig edited my answer. Points aren't going to work for 3D shapes. — aheze, Sep 22 '21 at 14:56
Thanks for the updated. The key, as you point out, is in the documentation to `pointSize`, where it only renders 2D surfaces. — tospig, Sep 22 '21 at 21:51

physically based lighting on custom SCNGeometry Node

Question

Details

Some Exploration

1 Answers1

Edit: Single Vertex Sphere?