How to benchmark Swift code execution?

Question

Is there a way/software to give precise time needed to execute a block of code written in Swift, other than the following?

let date_start = NSDate()

// Code to be executed 

println("\(-date_start.timeIntervalSinceNow)")

Answer 1

If you just want a standalone timing function for a block of code, I use the following Swift helper functions:

func printTimeElapsedWhenRunningCode(title:String, operation:()->()) {
    let startTime = CFAbsoluteTimeGetCurrent()
    operation()
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    print("Time elapsed for \(title): \(timeElapsed) s.")
}

func timeElapsedInSecondsWhenRunningCode(operation: ()->()) -> Double {
    let startTime = CFAbsoluteTimeGetCurrent()
    operation()
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    return Double(timeElapsed)
}

The former will log out the time required for a given section of code, with the latter returning that as a float. As an example of the first variant:

printTimeElapsedWhenRunningCode(title:"map()") {
    let resultArray1 = randoms.map { pow(sin(CGFloat($0)), 10.0) }
}

will log out something like:

Time elapsed for map(): 0.0617449879646301 s

Be aware that Swift benchmarks will vary heavily based on the level of optimization you select, so this may only be useful for relative comparisons of Swift execution time. Even that may change on a per-beta-version basis.

Answer 2

If you want to get insight into performance of a certain block of code and make sure performance doesn't hurt when you make edits, best thing would be using XCTest's measuring performance functions, like measure(_ block: () -> Void).

Write a unit test that executes method you want to benchmark, and that unit test will run it multiple times giving you time needed and deviation of results

func testExample() {

    self.measure {
        //do something you want to measure
    }
}

You can find more info in apple docs under Testing with Xcode -> Performance Testing

Answer 3

You can use this function to measure asynchronous as well as synchronous code:

import Foundation

func measure(_ title: String, block: (@escaping () -> ()) -> ()) {

    let startTime = CFAbsoluteTimeGetCurrent()

    block {
        let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
        print("\(title):: Time: \(timeElapsed)")
    }
}

So basically you pass a block that accepts a function as a parameter, which you use to tell measure when to finish.

For example to measure how long some call called "myAsyncCall" takes, you would call it like this:

measure("some title") { finish in
    myAsyncCall {
        finish()
    }
    // ...
}

For synchronous code:

measure("some title") { finish in
     // code to benchmark
     finish()
     // ...
}

This should be similar to measureBlock from XCTest, though I don't know how exactly it's implemented there.

Answer 4

Benchmarking Function - Swift 4.2

This is an incredibly versatile benchmarking function that allows for the labelling of tests, performing many tests and averaging their execution times, a setup block to be called between tests (i.e. shuffling an array between measuring a sorting algorithm on it), clear printing of benchmarking results, and it also returns the average execution time as a Double.

Try the following:

@_transparent @discardableResult public func measure(label: String? = nil, tests: Int = 1, printResults output: Bool = true, setup: @escaping () -> Void = { return }, _ block: @escaping () -> Void) -> Double {
    
    guard tests > 0 else { fatalError("Number of tests must be greater than 0") }
    
    var avgExecutionTime : CFAbsoluteTime = 0
    for _ in 1...tests {
        setup()
        let start = CFAbsoluteTimeGetCurrent()
        block()
        let end = CFAbsoluteTimeGetCurrent()
        avgExecutionTime += end - start
    }
    
    avgExecutionTime /= CFAbsoluteTime(tests)
    
    if output {
        let avgTimeStr = "\(avgExecutionTime)".replacingOccurrences(of: "e|E", with: " × 10^", options: .regularExpression, range: nil)
        
        if let label = label {
            print(label, "▿")
            print("\tExecution time: \(avgTimeStr)s")
            print("\tNumber of tests: \(tests)\n")
        } else {
            print("Execution time: \(avgTimeStr)s")
            print("Number of tests: \(tests)\n")
        }
    }
    
    return avgExecutionTime
}

---

Usage

var arr = Array(1...1000).shuffled()

measure(label: "Map to String") {
    let _ = arr.map { String($0) }
}

measure(label: "Apple's Sorting Method", tests: 1000, setup: { arr.shuffle() }) {
    arr.sort()
}

measure {
    let _ = Int.random(in: 1...10000)
}

let mathExecutionTime = measure(printResults: false) {
    let _ = 219 * 354
}

print("Math Execution Time: \(mathExecutionTime * 1000)ms")


// Prints:
//
// Map to String ▿
//     Execution time: 0.021643996238708496s
//     Number of tests: 1
//
// Apple's Sorting Method ▿
//     Execution time: 0.0010601345300674438s
//     Number of tests: 1000
//
// Execution time: 6.198883056640625 × 10^-05s
// Number of tests: 1
//
// Math Execution Time: 0.016927719116210938ms
//

Note: measure also returns the execution time. The label, tests, and setup arguments are optional. The printResults argument is set to true by default.

Answer 5

I like Brad Larson's answer for a simple test that you can even run in a Playground. For my own needs I've tweaked it a bit:

1. I've wrapped the call to the function I want to test in a testing function, which gives me space to play around with different arguments if I want to.
2. The testing function returns its name, so I don't have to include the 'title' parameter in the averageTimeTo() benchmarking function.
3. The benchmarking function allows you to optionally specify how many repetitions to perform (with a default of 10). It then reports both the total and average times.
4. The benchmarking function prints to the console AND returns the averageTime (which you might expect from a function called 'averageTimeTo'), so there's no need for two separate functions that have almost identical functionality.

For example:

func myFunction(args: Int...) {
    // Do something
}

func testMyFunction() -> String {
    // Wrap the call to myFunction here, and optionally test with different arguments
    myFunction(args: 1, 2, 3)
    return #function
}

// Measure average time to complete test
func averageTimeTo(_ testFunction: () -> String, repeated reps: UInt = 10) -> Double {
    let functionName = testFunction()
    var totalTime = 0.0
    for _ in 0..<reps {
        let startTime = CFAbsoluteTimeGetCurrent()
        testFunction()
        let elapsedTime = CFAbsoluteTimeGetCurrent() - startTime
        totalTime += elapsedTime
    }
    let averageTime = totalTime / Double(reps)
    print("Total time to \(functionName) \(reps) times: \(totalTime) seconds")
    print("Average time to \(functionName): \(averageTime) seconds\n")
    return averageTime
}

averageTimeTo(testMyFunction)

// Total time to testMyFunction() 10 times: 0.000253915786743164 seconds
// Average time to testMyFunction(): 2.53915786743164e-05 seconds

averageTimeTo(testMyFunction, repeated: 1000)

// Total time to testMyFunction() 1000 times: 0.027538537979126 seconds
// Average time to testMyFunction(): 2.7538537979126e-05 seconds

Answer 6

Benchmark Swift code execution

Please make your benchmark on Production build instead of Debug since it contains more optimizations

you have several possibilities

• DispatchTime.now().uptimeNanoseconds (and div on 1_000_000)

  Based on the Mach absolute time unit

Wall Clock(is not recommended because of leap second or other micro correction)

• Date().timeIntervalSince1970
• CFAbsoluteTimeGetCurrent

Test

• XCTest

let metrics: [XCTMetric] = [XCTMemoryMetric(), XCTStorageMetric(), XCTClockMetric()]

let measureOptions = XCTMeasureOptions.default
measureOptions.iterationCount = 3

measure(metrics: metrics, options: measureOptions) {
    //block of code
}

*I would not suggest you to create a general function with measure block because Xcode sometimes can not handle it properly. That is why use measure block for every performance test

Answer 7

Small improvement of @Brad Larsons's answer (the accepted answer) to allow the function to return the operation's result if needed

@discardableResult func printTimeElapsedWhenRunningCode<T>(title: String, operation: () -> T) -> T {
    let startTime = CFAbsoluteTimeGetCurrent()
    let result = operation()
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    print("Time elapsed for \(title): \(timeElapsed) s.")
    
    return result
}

Answer 8

class TimeMeasurement {
static var shared: TimeMeasurement = TimeMeasurement()
private var start = CFAbsoluteTimeGetCurrent()
func begin() {
    start = CFAbsoluteTimeGetCurrent()
}
@discardableResult func end() -> Double {
    let diff = CFAbsoluteTimeGetCurrent() - start
    print("Took \(diff) seconds")
    return diff
}

}

Answer 9

A made a tiny spm package to measure code execution time:

https://github.com/mezhevikin/Measure.git

Example of use

Measure.start("create-user")
let user = User()
Measure.finish("create-user")

Console output:

⏲ Measure [create-user]: 0.00521 sec.

Measure async request

Measure.start("request")
let url = URL(string: "https://httpbin.org/get")!
URLSession.shared.dataTask(with: url) { _, _, _ in
    let time = Measure.finish("request")
    Analytics.send(event: "Request", ["time" => time])
}.resume()

Full code of package:

// Mezhevikin Alexey: https://github.com/mezhevikin/Measure
import Foundation

public class Measure {
    static private var starts = [String: Double]()
    
    static public func start(_ key: String) {
        starts[key] = CFAbsoluteTimeGetCurrent()
    }

    @discardableResult
    static public func finish(_ key: String) -> Double {
        guard let start = starts[key] else {
            print(" Key [\(key)] not found")
            return 0
        }
        let time = CFAbsoluteTimeGetCurrent() - start
        print(String(format: "⏲ Measure [\(key)]: %.5f sec.", time))
        starts.removeValue(forKey: key)
        return time
    }
}