8

I'm developing ARKit app along with Vision/AVKit frameworks. I'm using MLModel for classification of my hand gestures. My app recognizes Victory, Okey and ¡No pasarán! hand gestures for controlling a video.

The app works fine but view's content is rendered at 120 fps. I do not need such a frame rate. It's too much for my app and it's a heavy burden for CPU. I tried to reduce a frame rate to 30 fps using SceneKit's instance property:

var preferredFramesPerSecond: Int { get set }

but my frame rate is still the same – 120 fps.

enter image description here

Here's how I made it:

import UIKit
import AVKit
import SceneKit
import ARKit
import Vision

class ViewController: UIViewController, ARSCNViewDelegate {

    @IBOutlet var sceneView: ARSCNView! 
    var avPlayerView = AVPlayerViewController()
    private let player = AVQueuePlayer()
    let clips = ["AA", "BB", "CC"]
    private var token: NSKeyValueObservation?
    var number: Int = 0
    var once: Bool = true    
    let dispatchQueueML = DispatchQueue(label: "net.aaa.iii") 
    var visionRequests = [VNRequest]()

    override func viewDidLoad() {
        super.viewDidLoad()
        sceneView.delegate = self
        sceneView.showsStatistics = true
        sceneView.preferredFramesPerSecond = 30     // HERE IT GOES
        sceneView.rendersContinuously = true
    
        let scene = SCNScene()
        sceneView.scene = scene
        sceneView.scene.background.contents = UIColor.black.withAlphaComponent(0)
    
        guard let selectedModel = try? VNCoreMLModel(for: handsModel().model) else {
            fatalError("Couldn't load a model.")
        }
    
        let classificationRequest = VNCoreMLRequest(model: selectedModel, 
                                        completionHandler: classificationCompleteHandler)
        classificationRequest.imageCropAndScaleOption = VNImageCropAndScaleOption.centerCrop
        visionRequests = [classificationRequest]       
    }

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        self.addAllVideosToPlayer()
        present(avPlayerView, animated: true,
                            completion: { self.player.play() })
    
        let configuration = ARWorldTrackingConfiguration()
        configuration.isAutoFocusEnabled = false
        sceneView.session.run(configuration)
    }

    func addAllVideosToPlayer() {
        avPlayerView.player = player
        // .........................
    }
    func toggleNextVideo() {
        // ..............
    }
    func togglePreviousVideo() {
        // ..............
    }

    // ..............................
    // ..............................

    DispatchQueue.main.async {
        if (topPredictionScore != nil && topPredictionScore! > 0.01) {  
            if (topPredictionName == "FistGesture") && (self.once == false) {
                self.once = true
            }
            if (topPredictionName == "OkeyGesture") && (self.once == true) {
                self.toggleNextVideo()
                self.once = false
            }
        }
    }
}

Here's what Apple says about it:

SceneKit chooses an actual frame rate that is as close as possible to your preferred frame rate based on the capabilities of the screen the view is displayed on. The actual frame rate is usually a factor of the maximum refresh rate of the screen to provide a consistent frame rate.

For example, if the maximum refresh rate of the screen is 60 fps, that is also the highest frame rate the view sets as the actual frame rate. However, if you ask for a lower frame rate, SceneKit might choose 30, 20, 15 or some other factor to be the actual frame rate. For this reason, you want to choose a frame rate that your app can consistently maintain. The default value is 60 fps.

How to lower a View's frame rate to 30 fps?

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220

1 Answers1

4

SceneKit + SwiftUI

When working with SwiftUI interfaces (iOS 14+ and macOS 11+), you have the option to run a simplified config of SceneKit's view for SwiftUI apps. It allows us to change a frame rate.

SceneView(scene: SCNScene? = nil, pointOfView: SCNNode? = nil,
                         options: SceneView.Options = [],
        preferredFramesPerSecond: Int = 30,
                antialiasingMode: SCNAntialiasingMode = .multisampling4X,
                        delegate: SCNSceneRendererDelegate? = nil,
                       technique: SCNTechnique? = nil)

enter image description here


SpriteKit + UIKit

When working with UIKit storyboards, SceneKit doesn't allow us to change a frame rate – it has a fixed 60 fps or 120 fps. So we could use the SpriteKit framework instead – it allows us set 30 fps.

enter image description here

Here is the code:

import SpriteKit

class ViewController: UIViewController, ARSKViewDelegate {

    @IBOutlet weak var skSceneView: ARSKView!

    override func viewDidLoad() {
        super.viewDidLoad()

        skSceneView.delegate = self
        skSceneView.preferredFramesPerSecond = 30 
    
        let scene = SKScene()
        skSceneView.presentScene(scene)
    }
}
Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
  • But the CPU says the same 74% in the gauges and the battery says Very High too, are there any real changes? also the frame rate is now limited by your GPU for what I can tell in the images it is fixed to 16ms that is 60 fps needed speed. thanks. – Juan Boero Jun 14 '19 at 14:23
  • Are you using ARKIt and/or CoreML as well? Or just only SpriteKit or SceneKit? – Andy Jazz Jun 14 '19 at 14:26
  • 2
    I am using SceneKit, and when I set the preferred frame rate it works for my case: `[self.sceneView setPreferredFramesPerSecond:30];` Maybe you have more than one Metal configuration so the different frame rates of each pile up in the Xcode gauges, try this, set your Metal stuff to 30, and your scene to 30 too, my agues say 30 with only SceneKit and as soon as we add another metal config/instance it goes to 60. I also recommend using the ScenKit debug views, they show up in the device the real FPS they are running. – Juan Boero Jun 21 '19 at 14:05