Pre-heat CoreML MLModel

Question

I'm seeing up to 3x-4x slower results on my first prediction on my MLModel. Seems like once you run the first prediction it's heating the model. Couldn't find any official information about it.

What is the right way to pre-heat my model? Call it on an async thread on app launch with dummy data?

It seems that it is lazy loading the model (implicitly) upon the first request. iOS 14+ includes an explicit async [loading function](https://developer.apple.com/documentation/coreml/mlmodel/3600218-load) that notifies you via a callback when loading finishes, so if you target pre iOS 14 your workaround sounds like a valid choice. — Alladinian, May 30 '21 at 14:37

score 4 · Accepted Answer · answered May 31 '21 at 15:22

As @Alladinian mentioned, starting from iOS 14 you can use the MLModel.load(contentsOf:...) function.
Here is a usage example for pre-loading a local .mlmodel file:

if let url = Bundle.main.url(forResource: "myModel", withExtension: "mlmodelc") {
    let config = MLModelConfiguration()
    MLModel.load(contentsOf: url, configuration: config) { [weak self] result in
        switch result {
            case .success(let model):
                print("Model loaded and ready.")
                let modelWrapper = MyModelWrapper(model: model)
            case .failure(let error):
                print("Error loading model: \(error)")
        }
    }
}

If you want to pre-load an externally fetched model, make sure it's compiled using MLModel.compileModel(at:).

Pre-heat CoreML MLModel

1 Answers1