4

I am adding a 3D model containing animations to the scene that I previously download from the internet. Before adding this node I use prepare function on it because I wan't to avoid frame drop. But still I get a very short frame drop to about 47 fps. This is caused by executing this prepare function. I also tried using prepare(_:, shouldAbortBlock:) on other dispatch queue, but this still didn't help. Can someone help me resolve this or tell me why there is this happening?

arView.sceneView.prepare([mediaNode]) { [mediaNode, weak self] (success) in
    guard let `self` = self else { return }
    guard
        let currentMediaNode = self.mediaNode as? SCNNode,
        currentMediaNode === mediaNode,
        !self.mainNode.childNodes.contains(mediaNode)
        else { return }
    self.mainNode.addChildNode(mediaNode)
}

By the way this is a list of files I'm using to load this model:

https://www.dropbox.com/s/7968fe5wfdcxbyu/Serah-iOS.dae?dl=1
https://www.dropbox.com/s/zqb6b6rxynnvc5e/0001.png?dl=1
https://www.dropbox.com/s/hy9y8qyazkcnvef/0002.tga?dl=1
https://www.dropbox.com/s/fll9jbjud7zjlsq/0004.tga?dl=1
https://www.dropbox.com/s/4niq12mezlvi5oz/0005.png?dl=1
https://www.dropbox.com/s/wikqgd46643327i/0007.png?dl=1
https://www.dropbox.com/s/fioj9bqt90vq70c/0008.tga?dl=1
https://www.dropbox.com/s/4a5jtmccyx413j7/0010.png?dl=1

DAE file is already compiled by Xcode tools so that it can be loaded after being downloaded from the internet. And this is the code I use to load it after it's downloaded:

class func loadModel(fromURL url: URL) -> SCNNode? {
    let options = [SCNSceneSource.LoadingOption.animationImportPolicy : SCNSceneSource.AnimationImportPolicy.playRepeatedly]
    let sceneSource = SCNSceneSource(url: url, options: options)
    let node = sceneSource?.entryWithIdentifier("MDL_Obj", withClass: SCNNode.self)
    return node
}
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
Damian Dudycz
  • 2,622
  • 19
  • 38

2 Answers2

0

Firstly

Get 3D model for AR app with no more than 10K polygons and a texture of 1K x 1K. The best result can be accomplished with 5K...7K polygons per each model. Totally, SceneKit's scene may contain not more than 100K polygons. This recommendation helps you considerably improve rendering performance and, I suppose, you'll have a minimal drop frame.

Secondly

The simplest way to get rid of drop frame in ARKit/SceneKit/AVKit is to use Metal framework. Just imagine: a simple image filter can be more than a hundred times faster to perform on the GPU than an equivalent CPU-based filter. The same things I could say about realtime AV-video and 3D animation – they perform much better on GPU.

For instance, you can read this useful post about using Metal rendering for AVCaptureSession. There's awesome workflow how to use Metal.

P.S. Check your animated object/scene in 3D authoring tool (if it's OK) before writing a code.

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • I'm really hoping to keep using SceneKit renderer. The framerate is back to 60 after fraction of a second. I was thinking that using prepare before adding the node would resolve this, maybe I'm using it wrong somehow or there is something I am missing. Both adding node and preparing it are causing this short stutter. – Damian Dudycz Jun 11 '18 at 12:14
  • 1
    @Andy, Damian - I have the same issue, regardless, if I use prepare method to add nodes or not. The framerate stuttering always happens when the model gets visible for the very first time. I read somewhere about this is related to shader precompilation. Is there really no way to tell skenekit to precompile/preload all the required (built-in) shaders that will be required to render the target node. It seems that if one uses a SCNProgram instead this would do the trick - yeah - but how to obtain the Apple PBR Shader as a SCNProgram? Any new good ideas on this? – ZAY Feb 01 '21 at 09:57
  • 1
    @Andy - in my actual case I have a scene containing a Model, which was originally a DAE file, that I converted to SCN. the Model is very simple and the original File was not even bigger than 100 kB in size. There are some materials attached to it (diffuse, metallic, normal, roughness and AO). Each texture file hase a size of 512x512 8-Bit RGB, or Grayscale. This model freezes the renderer for like 0.75 secoonds as soon as it enters the camera frustum. I am just out of ideas, what I could do, to not get this framedrops. The framedrops happens jutt the first time the model is shown. – ZAY Feb 01 '21 at 14:15
  • @ZAY, does freezing happen in app compiled via Xcode (with connected device) or when you're launched app on iPhone purely? – Andy Jazz Feb 02 '21 at 18:14
  • @Andy - in happens in both cases exactly the same, and regardless if I choose Debug or Release Build... It's really tricky... – ZAY Feb 03 '21 at 07:49
  • 1
    @Andy - I took one of the first backups of my current project, stripped it down and added the Demonstration object. If you want to play around with it, just install the App, launch and walk to the right side with the character. The demonstration Object enters the camera frustum and the freeze happens. Search for ANDY within the code, then you will immediatly see where the stuff is. (Ignore the other code, its not organized at all in this example) You can download it here: https://drive.google.com/file/d/1ciu-oN-_FCwodj6WHEBmnr-UCwNrv4fM/view?usp=sharing – ZAY Feb 03 '21 at 09:04
  • @ZAY, First of all – your textures are 2048 x 2048. Make a resolution for them `1024 x 1024` or even smaller – `512 x 512`. The scene will be considerably quicker. Secondly, make them `jpg`, instead of `png`. Thirdly, number of separate meshes in your scene must be around 50, not more. And fourth – use Blinn, Lambert or Phong shaders, not PBR. – Andy Jazz Feb 03 '21 at 10:45
  • @Andy - yeah, I know, the floor texture is way too big in that example, but its no longer the case. I forgot to replace it. BTW, what is the advantage of jpg? I always thought that png would be better, because jpg is sometimes heavily compressed and ugly, and has no alpha channel. – ZAY Feb 03 '21 at 10:49
  • Its size in MB is smaller. And JPEG is a 3-channel, not 4-channel as PNG. P.S.All your textures are 2K. Use a minimal compression for JPEG – and it'll be not so ugly. – Andy Jazz Feb 03 '21 at 10:51
  • I meant, all textures from the demonstration object - the house - are 512x512, It's an old example and there were some textures 2048x2048 in size left... – ZAY Feb 03 '21 at 10:55
  • Ok, I've got it. – Andy Jazz Feb 03 '21 at 10:55
0

I was experiencing the same issue. My nodes were all taking advantage of physically-based rendering (PBR) and the first time I added a node to the scene, the frame rate dropped significantly, but was fine after that. I could add as many other nodes without a frame rate drop.

I figured out a work around to this issue. What I do is after I create my ARConfiguration and before I call session.run(configuration) I add a test node with PBR to the scene. In order for that node to not appear, I set the node's material's colorBufferWriteMask to an empty array (see this answer: ARKit hide objects behind walls) Then before I add my content I remove that node. Adding and removing this test node does the trick for me.

Here is an example:

var pbrTestNode: SCNNode!

func addPBRTestNode() {
        let testGeometrie = SCNBox(width: 0.5, height: 0.5, length: 0.5, chamferRadius: 0)
        testGeometrie.materials.first?.diffuse.contents = UIColor.blue
        testGeometrie.materials.first?.colorBufferWriteMask = []
        testGeometrie.materials.first?.lightingModel = .physicallyBased
        pbrTestNode = SCNNode(geometry: testGeometrie)

        scene.rootNode.addChildNode(pbrTestNode)
    }

func removePBRTestNode() {
        pbrTestNode.removeFromParentNode()
    }

func startSessionWithPlaneDetection() {
        // Create a session configuration
        let configuration = ARWorldTrackingConfiguration()
        if #available(iOS 11.3, *) {
            configuration.planeDetection = [.horizontal, .vertical]
        } else {
            configuration.planeDetection = .horizontal
        }
        configuration.isLightEstimationEnabled = true

        // this prevents the delay when adding any nodes with PBR later
        sceneController.addPBRTestNode()

        // Run the view's session
        sceneView.session.run(configuration)
    }

Call removePBRTestNode() when you add your content to the scene.

KnutKnatter
  • 61
  • 1
  • 2
  • This does help a little, but not enough. I tried to have such nodes added before, and this eliminates the stutter in `addChildNode`, but `prepare` still causes stutter. I tested this by delaying `addChildNode` by 5 seconds after `prepare`. When I use prepare on new kind of model the first time, there is always this short frame drop. How can I test what is happening when I execute `prepare`? Maybe there is something wrong with this model what is causing this? Shouldn't prepare not ever cause stutter as it is running on other thread? – Damian Dudycz Jun 12 '18 at 10:46
  • Did you find any more information on this? I’m seeing the same problem when loading a plane with an image as the diffuse contents. I’ve tried using prepare but it doesn’t help. I’ve read many accounts of this short frame pause. Many of them like mine only happen the first time adding the node. But for me I can’t find a workaround by adding before hand... this theory is because the node has to show up in a camera to load... that’s what I’ve read and that’s what I’m experiencing :) – The Way Mar 21 '19 at 13:28