Can asynchronous operations be used with `progress` on OperationQueue?

Question

Starting in iOS13, one can monitor the progress of an OperationQueue using the progress property. The documentation states that only operations that do not override start() count when tracking progress. However, asynchronous operations must override start() and not call super() according to the documentation.

Does this mean asynchronous operations and progress are mutually exclusive (i.e. only synchronous operations can be used with progress)? This seems like a massive limitation if this is the case.

In my own project, I removed my override of start() and everything appears to work okay (e.g. dependencies are only started when isFinished is set to true on the dependent operation internally in my async operation base class). BUT, this seems risky since Operation explicitly states to override start().

Thoughts?

Documentaiton references:

https://developer.apple.com/documentation/foundation/operationqueue/3172535-progress

By default, OperationQueue doesn’t report progress until totalUnitCount is set. When totalUnitCount is set, the queue begins reporting progress. Each operation in the queue contributes one unit of completion to the overall progress of the queue for operations that are finished by the end of main(). Operations that override start() and don’t invoke super don’t contribute to the queue’s progress.

https://developer.apple.com/documentation/foundation/operation/1416837-start

If you are implementing a concurrent operation, you must override this method and use it to initiate your operation. Your custom implementation must not call super at any time. In addition to configuring the execution environment for your task, your implementation of this method must also track the state of the operation and provide appropriate state transitions.

Update: I ended up ditching my AysncOperation for a simple SyncOperation that waits until finish() is called (using a semaphore).

/// A synchronous operation that automatically waits until `finish()` is called.
open class SyncOperation: Operation {

    private let waiter = DispatchSemaphore(value: 0)

    /// Calls `work()` and waits until `finish()` is called.
    public final override func main() {
        work()
        waiter.wait()
    }

    /// The work of the operation. Subclasses must override this function and call `finish()` when their work is done.
    open func work() {
        preconditionFailure("Subclasses must override `work()` and call `finish()`")
    }

    /// Finishes the operation.
    ///
    /// The work of the operation must be completed when called. Failing to call `finish()` is a programmer error.
    final public func finish() {
        waiter.signal()
    }
}

Answer 1

You ask:

Does this mean asynchronous operations and progress are mutually exclusive (i.e. only synchronous operations can be used with progress)? This seems like a massive limitation if this is the case.

Yes, if you implement start, you have to add the operation’s child Progress to the queue’s parent progress yourself. (It is a little surprising that they did not have the base operation update the progress by observing the isFinished KVO, but it is what it is. Or they could have used the becomeCurrent(withPendingUnitCount:)-resignCurrent pattern, and then this fragile behavior would not exist.)

But I would not abandon asynchronous operations solely because you want their Progress. By making your operation synchronous, you would unnecessarily tie up one of the very limited number of worker threads for the duration of the operation. That is the sort of decision that seems very convenient, might not introduce immediate problems, but longer-term might introduce problems that are exceedingly hard to identify when you unexpectedly exhaust your worker thread pool.

Fortunately, adding our own child Progress is exceedingly simple. Consider a custom operation with its own child Progress:

class TestOperation: AsynchronousOperation {
    let progress = Progress(totalUnitCount: 1)

    override func main() {
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) { [self] in
            progress.completedUnitCount = 1
            finish()
        }
    }
}

And then, while adding them to your queue, add the operation’s progress as a child of the operation queue’s Progress:

class ViewController: UIViewController {
    @IBOutlet weak var progressView: UIProgressView!

    let queue: OperationQueue = ...

    override func viewDidLoad() {
        super.viewDidLoad()

        queue.progress.totalUnitCount = 10
        progressView.observedProgress = queue.progress

        for _ in 0 ..< 10 {
            queue.progress.becomeCurrent(withPendingUnitCount: 1)
            queue.addOperation(TestOperation())
            queue.progress.resignCurrent()
        }
    }
}

It is trivial to add the Progress of your own, custom, asynchronous, Operation subclasses to the operation queue’s Progress. Or, you might just create your own parent Progress and bypass the progress of the OperationQueue entirely. But either way, it is exceedingly simple and there is no point in throwing the baby (asynchronous custom Operation subclass) away with the bathwater.

---

If you want, you could simplify the calling point even further, e.g., define typealias for operations with Progress:

typealias ProgressOperation = Operation & ProgressReporting

extension OperationQueue {
    func addOperation(progressOperation: ProgressOperation, pendingUnitCount: Int64 = 1) {
        progress.addChild(progressOperation.progress, withPendingUnitCount: pendingUnitCount)
        addOperation(progressOperation)
    }
}

class TestOperation: AsynchronousOperation, ProgressReporting {
    let progress = Progress(totalUnitCount: 1)

    override func main() { ... }
}

And then when adding operations:

queue.progress.totalUnitCount = 10
progressView.observedProgress = queue.progress

for _ in 0 ..< 10 {
    queue.addOperation(progressOperation: TestOperation())
}

Answer 2

You are combining two different but related concepts; asynchronous and concurrency.

An OperationQueue always dispatches Operations onto a separate thread so you do not need to make them explicitly make them asynchronous and there is no need to override start(). You should ensure that your main() does not return until the operation is complete. This means blocking if you perform asynchronous tasks such as network operations.

It is possible to execute an Operation directly. In the case where you want concurrent execution of those operations you need to make them asynchronous. It is in this situation that you would override start()

If you want to implement a concurrent operation—that is, one that runs asynchronously with respect to the calling thread—you must write additional code to start the operation asynchronously. For example, you might spawn a separate thread, call an asynchronous system function, or do anything else to ensure that the start method starts the task and returns immediately and, in all likelihood, before the task is finished.

Most developers should never need to implement concurrent operation objects. If you always add your operations to an operation queue, you do not need to implement concurrent operations. When you submit a nonconcurrent operation to an operation queue, the queue itself creates a thread on which to run your operation. Thus, adding a nonconcurrent operation to an operation queue still results in the asynchronous execution of your operation object code. The ability to define concurrent operations is only necessary in cases where you need to execute the operation asynchronously without adding it to an operation queue.

In summary, make sure your operations are synchronous and do not override start if you want to take advantage of progress

Update

While the normal advice is not to try and make asynchronous tasks synchronous, in this case it is the only thing you can do if you want to take advantage of progress. The problem is that if you have an asynchronous operation, the queue cannot tell when it is actually complete. If the queue can't tell when an operation is complete then it can't update progress accurately for that operation.

You do need to consider the impact on the thread pool of doing this.

The alternative is not to use the inbuilt progress feature and create your own property that you update from your tasks.