4

I'm taking images with AVCapturePhotoOutput and then using their JPEG representation as the texture on a SceneKit SCNPlane that is the same aspect ratio as the image:

let image = UIImage(data: dataImage!)
let rectangle = SCNPlane(width:9, height:12)
let rectmaterial = SCNMaterial()
rectmaterial.diffuse.contents = image
rectmaterial.isDoubleSided = true
rectangle.materials = [rectmaterial]
let rectnode = SCNNode(geometry: rectangle)
let pos = sceneSpacePosition(inFrontOf: self.pictCamera, atDistance: 16.5)  // 16.5 is arbitrary, but makes the rectangle the same size as the camera
rectnode.position = pos
rectnode.orientation = self.pictCamera.orientation
pictView.scene?.rootNode.addChildNode(rectnode)

sceneSpacePosition is a bit of code that can be found here on SO that maps CoreMotion into SceneKit orientation. It is used to place the rectangle, which does indeed appear at the right location with the right size. All very cool.

The problem is that the image is rotated 90 degrees to the rectangle. So I did the obvious:

rectmaterial.diffuse.contentsTransform = SCNMatrix4MakeRotation(Float.pi / 2, 0, 0, 1)

This does not work property; the resulting image is unrecognizable. It appears that one small part of the image has been stretched to a huge size. I thought it might be the axis, but I tried all three with the same result.

Any ideas?

Community
  • 1
  • 1
Maury Markowitz
  • 9,082
  • 11
  • 46
  • 98
  • I'm not familiar with SceneKit but could it be that the image(texture) is not rotated on its center but on one of its corners and, unless your image is a perfect square, it could be rendered with stretching/tiling once rotated which could give strange results ? – Alain T. Mar 10 '17 at 00:05

1 Answers1

15

You are rotating on the upper left corner as suggested by Alain T. If you move your image down, you may get the rotation you were expecting.

Try this:

let translation = SCNMatrix4MakeTranslation(0, -1, 0)
let rotation = SCNMatrix4MakeRotation(Float.pi / 2, 0, 0, 1)
let transform = SCNMatrix4Mult(translation, rotation)
rectmaterial.diffuse.contentsTransform = transform
Jon Allee
  • 316
  • 2
  • 4
  • 1
    I implemented this and found that the image was upside down - the correct rotation would be -pi/2 . So I changed the rotation to -pi/2, at which point it was off in space again. – Maury Markowitz Mar 11 '17 at 13:30
  • 2
    Sorry, I wanted to edit the previous message but took to long. After thinking about it a bit I saw the issue - the translation is -1,0,0 in that case. It's working, thank you! – Maury Markowitz Mar 11 '17 at 13:41
  • 1
    For anyone trying to do this on an Apple Silicon M1, this causes a crash. Please report through Feedback Assistant. A workaround that worked for me was to not do the translation, but just set the transform to the rotation as shown above. – Morten J Dec 02 '20 at 14:04
  • Very interesting... has this been fixed with M1? I've been wondering about how compatible SceneKit. Are you not able reproduce the error on non-M1 devices? I assuming you were using an M1 iPad, or was this for macOS? – Dan M Jan 16 '22 at 22:36
  • I am using an M1 and I am having several issues with SceneKit. – user3069232 Mar 02 '22 at 07:01